How to Recruit User Research Participants from Craigslist

IMG_9852
Consistent communications with users is critical for successful products

Talking to your users should be a consistent part of your product development process, no matter what the stage.  Don’t ever assume you know how people will respond to what you’re doing – humans are complex creatures.  When vetting an idea, it’s a great way to confirm that your solution will be different, and that the problem you’re trying to solve really exists in the world.  When launching, it’s a great way to gauge reactions to user acquisition campaigns/ads as well as your first-time user experience.  When growing, it’s a critical way to ensure your tactics will scale to large audiences.

So how do you recruit people to interview for their insights and feedback?  There are a lot of options, such as UserTesting.com and respondent.io.

But in this post, I’m going to describe a low-cost and perhaps the fastest way to recruit: Craigslist.

Posting a Gig

  1. Pick a city to post in.  If you’re looking to recruit for an in-person interview, post locally or in the cities where you’re willing to go to.
  2. Create a post in the “gigs” section.  I typically choose the “computer gigs” category but you can experiment with others.
  3. Describe who you’re looking to talk to and what you’re asking them to do.
  4. Provide an incentive in both the description and the “pay” input field on the Craigslist form.

Here’s an example from a recent post I made:

Screen Shot 2017-08-31 at 9.37.25 AM

Some Things to Keep In Mind

  1. Keep in mind some biases that Craigslist will introduce to your participant pool: location (because you had to pick a city or multiple cities to post this to) and where you posted this.  For example, I posted the above in the “computer
  2. I find that more people respond when I post the gig to “computer gigs” (vs other categories like creative or writing).
  3. I find that most people respond to these gigs at night, so expect at least one day turnaround for starting the recruiting process.
  4. Craigslist doesn’t allow  you post links to screener surveys in the post (I used to use a Google Form to filter out people who didn’t meet the criteria of who I wanted to talk to) so make sure you’re clear in your posting what the requirements are.  You might get some fakers still email you, so you might want to think of ways to filter them out before scheduling an interview.
  5. If you feel like you’re not getting enough responses, consider increasing your incentive.  A high level of compensation is $1 per minute of time you’re asking them for, but I’d suggest starting a little lower at first as very few people make $60/hour.
  6. Don’t forget to factor in no-shows.  For example, I posted a gig like the one below in 5-6 cities and in the course of a week, I had 20 interviews scheduled. Only 6 people (30%) actually showed up, even after accepting the meeting invite.  I’d recommend following up the day before or the morning of to confirm they’re coming and remind them of the incentive.

Some Example Results

I posted this ad in both Austin and Craigslist as I’m trying some new meeting scheduling tools.  I had 7 interviews on my calendar within 2 hours!

Screen Shot 2017-10-03 at 5.25.41 PM

Want help recruiting user research participants, or any other parts of the user research process?

Email me: startupproductcoach@gmail.com

Advertisements

Agile 101: 3 Tips for Using Agile to Unlock Business Value

bml.png

Agile is not a process – it’s a mindset rooted in continuous learning by observing real world human behavior.

For a long time, software was built using waterfall, a long, sequential process that requires many handoffs between product, design, engineering, and QA.  A project goes from inception to requirement gathering to design to build to test and then deployment. All that could take months, and by the time the software is released, business priorities might have changed, or users might not like what they got.

Agile aims to remedy this by putting software into the hands of users faster and then measuring whether the changes are delivering the expected business outcomes.  If so, great, keep improving on what you have.  If not, no worries – you learned something about why not, and you only invested a few weeks to gain that insight.

Switching from waterfall to Agile isn’t easy.  But here are some tips to keep in mind.

Tip 1: Know the Roles

This is your “squad”:

Product Owner

The person responsible for ensuring the squad is constantly delivering business value through the product. This person often translates and prioritizes ideas from “the business” (CEO, strategy team, sales, marketing, etc) and customers for the squad who is actually producing the software.

Engineering

The folks who do most of the building – focused on how to deliver business value from a technical implementation perspective.

QA

The people who test changes before they go to users.  Testing strategies fall into 2 main categories: manual and automated, with much emphasis on automation as a more scalable/risk-reducing strategy.

Design

These folks translate stories into user journeys/experiences.  Typically they’re sharing their ideas via wireframes (for User Experience or UX designers) or visual comps (for User Interface or UI designers).

Scrum Master

This person’s role is to ensure the squad is efficiently and consistently delivering business value.  He or she will schedule all ceremonies (see below).

Subject Matter Experts

Not required, but sometimes it’s helpful to have SMEs on the squad to answer complex questions (for example, if you operate in a heavily regulated industry, you might want someone from compliance on the squad to approve changes quickly).

Tip #2: Commit to Frequent Production Releases

A key output of the process is shippable software.  It’s important to make always ship to production at the end of a sprint because:

  • Go/no-go meetings to decide whether to ship are painful and time-consuming.  More often than not, you’ll feel like you’re not ready and postpone, never knowing if you’re spending more time on something that’s not even going to deliver the business outcomes you’re expecting it to.  Shipping often lets you get feedback (qualitative or quantitative) from real users and eliminates the time needed for release planning (you always release).
  • The deadline creates a powerful forcing function.  As the sprint end date nears, the squad is forced to make scoping decisions and rally as a team to ensure there’s something to ship.

Most teams use 1, 2 or 3 week sprints to dictate how often they ship to production.

Tip #3: Follow the Ceremonies

There are only 5 meetings that should happen every sprint:

Backlog Grooming

A weekly hour-long session where the top of the backlog is discussed and estimated by the squad.  Prior to this meeting, the product owner should prioritize the backlog such that the top items represent the most meaningful ideas to deliver business value / move the product KPIs.

During the meeting, the product owner should present each ticket, starting with the top of the backlog and the Scrum Master should ask the squad to estimate (I prefer estimating via planning poker because it eliminates any bias in estimates but there are other techniques).  I’ve found asking for an estimate is the best way to force clarifying questions to come up from squad members – things like what users will this impact, what browsers are included, how will we know if we’ve succeeded (tie the story back to a KPI), etc.  Anything to clarify the scope of the story or what needs to be true in order to be successful.

Spring Planning

Assuming the top of your backlog is estimated, sprint planning is where the Scrum Master asks the squad to make a commitment to delivering specific stories/bugs in the next sprint.  Ideally the squad is just determining how much of the top of the backlog they think they can take on, but there might be scenarios

Key questions to ask during this meeting:

  • How much did we take on last sprint?  Did it feel like too much? Too little? Just right?
  • Are there any other factors in this sprint we should consider?  Considerable unknowns with some of the work?  Production support spikes we might anticipate?  Planned squad member vacations?

At the end of the meeting, the commitment should be clear to all, and the sprint should begin.  The product owner may want to communicate the commitment to stakeholders.

Daily Standup

This is a daily 10-15 minute meeting, typically in the morning but as long as it’s the same time every day, any time is fine.  Each squad member should answer the following questions:

  • What did you do yesterday?
  • What are you planning to do today?
  • What’s blocking you from doing what you planned to do today? (for example: I got pulled into a lot of meetings related to some other project at the company, I can’t test because the environment is down, I don’t understand what we’re trying to do, etc.)

Other squad members should listen for blockers in particular, but also items that might require some collaboration (for example: “oh, you’re working on that? I had some ideas, I’ll find you after standup”).  Scrum Masters should be making sure the squad is on track to deliver their sprint commitment (burn down charts are helpful for this) and that there is a game plan to remove any blockers that day.

Showcase

This is typically a 30-60 minute session open to the squad and all stakeholders to see what was shipped that sprint.  Squad members (including technology) should demo the changes, perhaps providing some color commentary or “behind the scenes” on what it took to implement.  This is a great place to test squad communication  – anyone should be able to get up at the showcase and explain the background/rationale for the change, along with demoing it.

After or during this meeting, stakeholders should provide feedback to the product owner. (ex. “looks great”, “wait, I didn’t realize that’s what we were gonna build – tell me more”, etc.).

Retrospective

This is typically a 30 minute meeting for the squad between the showcase and sprint planning.  The outcome of this session is to identify from a process perspective, what went well that sprint and what could be improved.  There are a few ways to do this:

  • Go around the room and ask each squad member what went well and what they would change
  • Ask folks to fill out a short form before this meeting, then identify themes and discuss
  • Ask the team to write down what went well on Post-Its and put them up on a wall.  Then do the same for what could be better and discuss each Post-It afterwards.

I’d recommend switching up the format of this ceremony periodically (not necessarily ever sprint) to keep this meeting fresh.

It’s really tempting to skip these ceremonies because “it feels like a lot of meetings” but if you’re doing them right, they’ll go by fast and you’ll understand the value of them.

Want to talk more about Agile?

There’s no way this one post can do Agile justice.  To talk more about how you can unlock the power of Agile for your business, please either email me at startupproductcoach@gmail.com or schedule a free 30-minute call.

Capturing Customer Insights for Your Squad

Not everyone from the squad (design, engineering, QA) may be there when you talk to customers. So how do capture insights to share what you learned with them?

My vote is the Job Story and I feel this article does a great job explaining it.  A Job Story summarizes key insights way better than an audio recording or written notes.

Let’s write an example job story from my experience in FinTech: planning for retirement.

When I start a new job, I want guidance in selecting my 401k contribution rate, so that I can feel confident about my decision, finish my HR paperwork and get to work! 

Let’s look at each clause in further detail.

The When Clause

This is perhaps the most critical part – it provides important information about the context in which the person is trying to do a job. Some key questions to ask when writing this:

  • What’s prompted the person to think about doing this job? (In the CREATE framework I wrote about previously, this is the cue). In this example the cue was probably filling out some HR paperwork.
  • Where is the person when the cue occurs? As you might imagine things like whether this person is at home, at work, or on the road would be important. Do they have a computer nearby? Or just a phone?

In the example above, if I had just talked to a new hire about their journey to make this decision, I might refine my when clause:

When I’m sitting in my first day HR orientation and am asked to fill out paperwork about how much I want to contribute to my 401k…

The I Want Clause

This is where you explain the job to be done. In this case, it’s deciding how much to save for retirement. Note that I intentionally didn’t specify a solution – no mention that we should show something like “you’d have $50,000 per year if you saved 5% and $70,000 per year if you saved 7%” or “to max out your employer match you should contribute 6%”.  I’m not saying you can’t suggest solutions when reviewing the story with your squad, but I’m a big believer that product managers should be presenting problems, not solutions.

In this example, as the squad starts brainstorming possible solutions, they may come up with either a tool to explore the implications of different contribution rates or an engine that just spits out a recommendation with an explanation. Both solutions might work for the person.

The So That Clause

This is where you explain why it’s important to the person to complete this job.

In this example, it’s important to note that for this person, they want to make a good decision but the real goal is to get through the paperwork and start the role she was hired to do. There might have been other reasons the person wanted to do that job:

…So that I can retire as soon as possible (he doesn’t like working!)

…So that I can minimize my annual tax bill (trying to lower her take home income)

Think about how different the squad might interpret the story based on the way it’s written.

Don’t forget the emotional and social aspects

Every job story has 3 components:

  1. Functional
  2. Emotional
  3. Social

I’ve only written about the functional here to start: what is the person trying to do? It’s important to consider the other two aspects though.

Emotional. How does the person feel before, during, and after they’re doing this job in their life? In this example, we know that doubt is a prominent feeling when retirement planning; no one is really sure when planning for something that might feel so far away.

Social. Will others know that the person has done this job? Will they discuss it openly? What will others think about the person based on how they did this job? In this example, we know it’s not common to discuss your 401k contribution rate with others. Maybe part of the solution is to make it easy to share your contribution decision anonymously within the company or to publish anonymous data from HR so new hires know what others have done in the past.

Want to talk more about how your team is collecting, documenting and using customer insights? Drop me a line. 

Why You Should Design Experiences Like Real World Conversations

Nc1U8Di
Does your digital experience feel like a real world conversation?

Yesterday Michaela Hackner, Head of Content Strategy at Capital One, came to talk to us at work about how they’ve used content strategy to improve their user experiences and business outcomes.  She also mentioned that her team is part of the Conversation Design and that they design interactions using “talk bubbles” to imagine what the back-and-forth between C1 and the user will be, as if the person had walked into a retail branch (or in this case, was talking to Alexa).

It got me thinking to how great digital experiences mimic great real-world interactions – they all start with a conversation.  A common place I see startups failing with this is asking users to register before demonstrating value or even understanding what the user is trying to accomplish.  Don’t assume this person read your entire marketing site or your app’s entire description in the App Store or Google Play.

I’ve written about this before (with the shame-on-you example being LetGo).  Asking me to register before even understanding what I’m registering for or the value to me of registering is like:

  • A car salesman asking you to fill out a customer lead form the minute you walk into a dealership, before he greets you or you even tell him why you’re there
  • Asking for my credit card as soon as I walk into a retail store, even if I’m just browsing
  • A banker shaking my hand and immediately asking me for my Social Security Number and address so he can open a new account for me, without even trying to understand why I might be interested in a new account

Instead, imagine if you had a retail store – how would you train your employees to greet a prospective customer when he/she walks in the door?  What types of questions or responses would you expect?  Start there and see how much better the user feedback and business outcomes can be.

Want to talk about making your user experience more conversational?

Should You Let a Customer Subsidize the Development of a New Product?

When you’ve identified a new need in the market, it’s tempting to let the first enterprise customer subsidize the development costs.  Here’s a look at the pros and cons of that scenario, along with a recommendation.

Pros

  • There’s no better way to validate the need for a new product than to have a customer willing to pay for it to be built.
  • There’s also no better way to ensure it does the job it’s supposed to than to have a real customer who is looking to use it to solve a real problem (likely urgently, so don’t forget to release often, even in beta format, to get feedback from that first customer).

Cons

  • The customer thinks the product will be “theirs” and that they own your roadmap. Make sure you set the expectation up front that the product is intended to be a SaaS solution that serves other customers as well.
  • You build the product using only one customer’s use case, making it unusable/irrelevant for other customers.

So Should You Do It?

Yes, but just make sure you:

a. Make it clear to your first customer the product is not “theirs” but that  you value their partnership/feedback in being the first customer. (This means you can’t sign exclusives!)

b. Get out with your sales team to pitch the new solution to more customers.  Ideally sales is landing a second/third customer while the product is being developed, so that you can get more input on how it will do the same job for many customers.

Want to talk more about launching a new product? Drop me a line.

How to CREATE Experiences that Spur Users to Action

help-your-users-take-action-funnel.png

I work with Dr. Steve Wendel, who is our Head of Behavioral Science.  He literally wrote the book on designing software for behavior change.  In that book he outlines his CREATE framework that suggests there are 5 hurdles you need to overcome to get a user or prospective user to take an action, and that at each hurdle, you’re going to lose a few people (hence the funnel shape).  Let’s take a look at the funnel in greater detail.

Cue

You’ve gotta prompt your user to take the action you want – be it an email, push notification, or ad.  This is the one I think most product development squads forget.  They focus so much on building a great user experience that they forget how and why the user would come to their product in the first place.  (if you build it, they will NOT come)

Reaction

As you may have read in books like Blink, our minds use shortcuts to make instantaneous judgements in response to a cue.  You should be aware of how your audience might react to your cue.  For example, if you’re telling them you have the greatest budgeting app in the world, be aware of what people think about the word “budget” (mostly negative reactions, as it implies work or an unpleasant realization that they’re spending too much).

Evaluation

If you can get past the knee-jerk emotional response, then you have to get past a more rational cost-benefit evaluation.  If you’re asking them to switch to your product, this is the point in time in the Jobs to be Done framework where they might decide whether your product is superior to what they’re using today, and whether the switching costs will be worth it.

Ability

Your audience has to have the ability to take the action you’re asking them to take.  For example, if you send an email asking them to log in to your app and check on some changes but they’re in a cellular dead zone on the subway, they’re not logging in then (and chances are, they’re not going to remember to log in later, unless they open that email again).

Timing

Which brings us to the last step – even if the person can take the action, you have to convince them why NOW is the time to do so.  For example, in my line of work, getting younger employees to save for retirement is hard – there are more immediate financial goals like buying a house or paying off student loans that feel more urgent that something that’s 30 years away.

Here are a couple behavioral techniques that can help create urgency:

  • Scarcity: you might have seen Amazon do this with a message like “only 3 left in stock” under a product’s description.  Or with a limited number of beta invites for an upcoming app launch.
  • Deadlines: you may have seen this on TicketMaster – a clock that counts down saying that the price for those Bieber tickets you’re thinking about buying will only stay locked in for 3 more minutes.  Same idea with limited time offers.

Execution

This one is not on the visual, but the last step is actually taking the action.  This is where you give your user a virtual high five for taking an action that you both wanted her to.

Want to discuss using behavioral science to spur action with your users?

Email me using the form below or schedule a call now.

3 Tips for Picking KPIs to Measure Your Product’s Success

new_kia_car_financing

Of all the items on a new car sticker, which ones should matter most to the car’s product manager?

Product managers, how do you know if you’re doing a good job?  Your manager tells you so? A customer leaves a glowing product review? Your coworker likes the new feature you launched?  Nope. You’ll know if you’re doing a good job if your product is successful.

But what does success means? It depends on your product, but no matter what, if you don’t have a way to measure success, you’ll never know, and neither will your stakeholders – internal or external.  Here are some tips on choosing Key Performance Indicators to measure the success of your product.

Tip 1: Start With the Money

I’ve written about this before: the ultimate measure of success should be a business outcome, so you should have at least one KPI that has a dollar sign in front of it.  Profitability is ideal, but revenue or cost savings for your company are also good.  If you need a shorter feedback loop on a KPI like revenue (because your sales cycle is really long), use a proxy KPI (for example, maybe you know that your sales team closes 80% of deals after 2 meetings with the decision maker – great, use that instead of revenue).

Tip 2: Measure Often

KPIs don’t matter if you can’t measure them easily, especially if you’re releasing frequently as a part of an Agile process.  Make sure you can measure your KPIs within a few minutes, and that the data needed to measure them is updated often – at least daily.  If you need to prioritize time to instrument your product to make measurement easier, do it.  Otherwise you’re either flying blind, or there’s too much of a delay in your feedback loop.  Also, don’t forget to publish your KPIs regularly for internal stakeholders, so that they can also see how your product is doing.

Tip 3: Don’t Pick Too Many KPIs

At most, I’d suggest 2 or 3.  Why? Because if you’re prioritizing changes (or tests) to improve your KPIs, it’s hard to juggle too many.  Don’t forget about the “K” in KPI.  In an ideal world,  you might even assign a KPI to each of your product owners and development squads, so that they know whether their work is resulting in meaningful business value.

Want to discuss your KPIs?