Why Mechanical Turk is the Best Market Research tool You're Not Using

[caption id="attachment_5362" align="alignright" width="361"] image source: Priceonomics.com[/caption]

A lot of people have no idea what Mechanical Turk is, which is a shame, because it’s quite possibly the best market research tool that you’ve never heard of.

I don't feel like I'm just part of the choir singing the praises of Mechanical Turk, but more like the preacher behind the pulpit. I tell other startup people and other writers about everything they can learn and achieve, automate and outsource, research and survey with this service. Jeff Bezos should send me a thank you card.

Mechanical Turk, for the uninitiated, is “artificial artificial intelligence.” It’s a huge workforce of people looking to do small tasks online, about half Americans and about half Indians, with some Canadians and other nationalities mixed in. They will perform small tasks for anywhere from five cents to a dollar. I love it. And they’re not a bunch of dumb bums, either. Mechanical Turkers in America are better educated than the general population, and I’m often amazed still by the care they put into their work, and the quality of feedback I get from Turkers. I’ve had someone email me to make sure his or her research was accurate on a five-cent task. They want to do a great job, and they guard their approval rates voraciously, as having a high rate allows them to get higher quality jobs.

Here are some specific research examples that you can do with Mechanical Turk:

• Contact information research – This is something I often do with Mechanical Turk, usually for customer prospecting. For my screen printing company, Vacord Screen Printing, I have posted jobs that were “go through this directory of student groups at this university and compile the contact names, email addresses, and group name.” I did it not too long ago, and it was a directory of 300+ groups, and I paid four bucks. Just four bucks. And that job was taken nearly immediately, which meant I could have probably paid less. You can also upload a CSV of contacts to find, so if you have for example a list of startups that you’d like to email, you can upload a list of their URLs and then pay people to find the contact information and names for the founders of those startups. I usually pay nine cents for a task like that, and they almost always get filled. When you upload a CSV, each line of your CSV becomes its own job (or HIT in Turk terminology, which is short for Human Intelligence Task) so you will have potentially hundreds of different turkers going through your HITs. This expedites the return of data like crazy. I’ve posted 200 HITs to find a contact name, then gone out to lunch, and come back and seen the HITs 80% filled. And maybe that cost me $20, maybe less, but it saved me huge amounts of time. (Be aware that Amazon itself adds a 10% surcharge to your HIT, so a ten cent task will cost you eleven cents)
• DIY press releases – I would really like to try press releases for a traction model for my startup that provides critiques of online dating profiles, but I don’t want to pay a company $1000 to do it for me. So what’s a boy to do? Use Google News API and Mechanical Turk and do it way way cheaper. Special thanks to my buddy Justin Wilcox who wrote this hack how to do this at customerdevlabs.com. You can use his tool and search the Google News API for articles in the last month about “online dating” or whatever keyword phrase, then download the results as CSV files. You can search for multiple keywords, like “onling dating” and “okcupid” and “eharmony” and get several CSV files, then combine them and cull the duplicates and delete the ones that aren’t really relevant (you could perhaps pay a Turker to do this too, but I want to do it myself). By searching for news articles about your industry and competitors, you are finding reporters who tend to write about your industry. Upload the CSV to Mechanical Turk and have the Turkers find out the reporter’s name and email address, then use a tool like quickmail.io to send them all an email about what you are doing. Blammo, there is your cheap press release, in which you can email hundreds of reporters for very cheap.
• Customer discovery surveys – I’ve written about this in more detail in other posts, but if you’re thinking about a startup or a new business, you really need to validate that idea, and customer discovery is key. Lucky for us, Mechanical Turk is a great place to post customer discovery surveys. You can use the title of the HIT to limit who takes the survey, and target a certain subset of the population. I have a pre-launch startup that will provide training to pilots for the written FAA exams and I needed to survey student pilots to make sure they would actually care about my service. One thing about being a student pilot is that they are broke because flight lessons are so expensive, so I hoped I could find student pilots on MTurk, and I had no issue filling 20 surveys in an evening, with pilots and student pilots who were on MTurk for a little extra cash. That survey showed me:
  • 75+% of the survey takers were interested in the service, so it’s worth building
  • Most people were willing to pay what I wanted to charge or more for the service
  • What they wanted most out of the service
• User testing surveys – My interest in MTurk was inspired by a site that basically just posted user testing surveys to the system. Instead of spending a ludicrous $50 a test with usertesting.com, I can post a written survey to MTurk for a dollar per user, or less, and get incredible feedback. By doing user testing surveys on MTurk, you can learn:
  • What the conversion issues are on your site
  • If people actually understand what your site provides
  • What keeps people from trusting your site
  • How to increase trust
  • How to improve your site
  • What people like and dislike
  • What they think of your logo, branding, etc
• Image sorting – This is a classic example of why MTurk makes sense for tiny jobs. Let’s say I have a thousand pictures of automobiles. A script can’t tell what’s a truck and what’s a car, but a human can tell instantly, and sort accordingly. Okay, maybe an El Camino will throw them for a loop, but this is an example of when you need a human, any human, to do a tiny task that a computer just can’t do.
• Transcription – There are lots of transcription jobs posted on MTurk, and some of them go for $20 or more. So if you have audio that you need turned into text accurately, you can hire a turker to type it up.
• Fear research – This relates to customer discovery and marketing. I wrote a longer article on it here, but the idea is that you would post surveys and use them to find out what concerned your potential customers about a problem or issue or business model, and then use it to your advantage in copywriting. For example, with this sort of research I found that most guys are afraid they are wasting their time with online dating, so buy playing that up in my copy, I saw an 800% increase in traffic to the order page for my dating profile critique service. That’s a huge improvement and all I did was run $12 worth of surveys to learn the most common fears.
• Content market research, what and where – By “what and where” I mean that with surveying you can learn what people want to learn about and where they would try to find it. For example, I found out what graphic designers wanted to learn about most in regards to screen printing, and also what blogs they would check for this information. That gives me an excellent guest blogging opportunity, by being able to say to an established blog “I have these topic ideas that your audience would really love, can I do a guest post please?”

image source: crowdsourcing.org

Here are some issues that you can run into with Mechanical Turk, and how to avoid them:

• Specificity – When I first posted surveys about online dating, I didn’t require people to actually have tried online dating, so I got a lot of people taking the survey honestly who couldn’t give me any usable feedback. You can set the title of your HIT to say something like “Men who are active with online dating wanted to take a survey” to narrow your range.
• Fraud – I use SurveyMonkey to create the surveys. I put up to 9 questions, then create a second page with just text that says something like “Thanks for taking the survey, please go put “banana” in the HIT to let me know you completed it.” That way I can review the submissions within Mechanical Turk and know that anyone who put banana in the HIT actually finished the survey.
• Junk entries – I very rarely see it, but sometimes people will put smart alecky answers into a HIT. Just reject their work, and it’ll hurt their approval rating. This is so uncommon in my experience. It’s very common for people to want their approval rating as high as possible.

General tips for best results:

• Use popular keywords – When I was first running tests putting online dating profiles on Mechanical Turk to be critiqued, I didn’t use any keywords, and it could take days to get five reviews completed. After adding proper keywords, I can now get ten reviews completed within an hour. Use keywords like “English, fun, writing, easy, survey” and so on and people who search these keywords will find your HITs easily. NYU published a wonderful paper on Mechanical Turk keyword usage that is incredibly valuable.
• Pay quickly – Pay your turkers for their work as soon as possible. It matters to them. And you don’t want to annoy them, because turkers have their own message boards and will review you as a HIT provider.
• Pay fairly – Yes, you can pay almost nothing, but the quality of work will be affected by it, and so will how quickly your work gets completed. And you could get reviewed as a cheapskate on the turker boards. Nobody wants that.
• Turn off “Masters” – There’s really no point to keeping the master designation on. Amazon charges you an extra 10% to use someone with a master rating. You can set minimums of how many HITs someone has had to complete and a minimum approval percentage in order to get higher quality workers without the surcharge.
• Limit by country – I have used foreign workers a lot in the past for development and virtual assistants, often with great results, but the language and cultural differences can hurt your research. If you are in the USA, you can limit your jobs to just Americans, but you should also include Canadians as they are looking for quality HITs too and I’ve had Canadians request to be allowed in on my HITs before.

Mechanical Turk is an underutilized and often unknown research tool that more people should use. The possibilities seem endless with what you can do with the digital workforce. I couldn’t see myself without it, at this point. Give it a shot the next time you need to research something.

Author Bio

Stuart Brent specializes in customer discovery, marketing automation, and startup strategy. He’s the founder of Vacord Screen Printing, Ignite Your Match - Online Dating Profile Critiques, among other projects. Follow him on Twitter at @Vacord.

Share

PREVIOUS
NEXT