Entrepreneurship ENT

 View Only

  • 1.  Qualtrics panel recruitment or Amazon Mechanical Turk?

    Posted 12-06-2016 00:44

    Dear all,

    I'm looking at administering a survey/experiment.  The (Qualtrics) survey is basically finished but I'm exploring different possibilities for my respondent pool.

    Has anybody here ever used Qualtrics for recruiting respondents?  Or Amazon Mechanical Turk?  And any ideas about comparing the costs / effectiveness of those two programs?

    And finally, any idea about the respect for those two sample recruitment strategies in our (entrepreneurship) academic community?  It seems that there is at least some advocacy for AMT in the broader academic community.

    Regards, -chihmao.

     

    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!


  • 2.  Qualtrics panel recruitment or Amazon Mechanical Turk?

    Posted 12-06-2016 06:41
    Hi Chihmao,

    I have used both MTurk and Qualtrics as sources for participant recruitment.

    There are complicated differences that include price and demographics, as well as attention span, expected rewards/compensation, and usefulness of respondent's experience.

    Publications from both sources are getting more common (see below for some relevant citations).

    I sought entrepreneurs from a Qualtrics Panel and found that the people who self-reported in response to my qualifying questions were not the "entrepreneurs" I anticipated (they were mostly folks with small side businesses with very modest goals-e.g., make $1,000 extra a year on Etsy or Uber). If you want just a general representative panel, then this is less important. But, if you want a specific demographic, Qualtrics can do it, it just costs more and you need to be very careful with the qualifying questions you ask. Prices for Qualtrics Panels very widely. I have paid anywhere from $5/participant (for general audiences) to $50/participant (for a matched sample of entrepreneurs and their spouses).

    For MTurk, it is important to consider the task you want them to do-some tasks are not suitable to the Human Intelligence Task (HIT) setting. And, it is harder to be selective about the characteristics you want in a sample-because anyone can self-report. Compensation expectations are much lower-typically based on an hourly rate. If your task takes 10 minutes, then compensation of between .$50 and $2.00 is common ($10/hour divided by the minutes it takes).

    Email me offline if you have questions.

    Best, Jeff


    Qualtrics Panel
    Hagtvedt, H. (2011). The impact of incomplete typeface logos on perceptions of the firm. Journal of Marketing, 75(4), 86-93.
    Brandon, D. M., Long, J. H., Loraas, T. M., Mueller-Phillips, J., & Vansant, B. (2013). Online instrument delivery and participant recruitment services: Emerging opportunities for behavioral accounting research. Behavioral Research in Accounting, 26(1), 1-23.
    Rosoff, H., Siko, R., John, R., & Burns, W. J. (2013). Should I stay or should I go? An experimental study of health and economic government policies following a severe biological agent release. Environment Systems & Decisions, 33(1), 121-13.

    MTurk
    Casler, K., Bickel, L., & Hackett, E. (2013). Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29(6), 2156-2160.
    Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk a new source of inexpensive, yet high-quality, data?. Perspectives on psychological science, 6(1), 3-5.
    Bohannon, J. (2011). Social science for pennies. Science, 334(6054), 307-307.



    Jeff Pollack
    Assistant Professor 
    Management, Innovation, and Entrepreneurship Department
    Poole College of Management
    NC State University
    2801 Founders Drive, Campus Box 7229
    Raleigh, NC 27695-7229
    804.397.0818 phone
    jmpolla3@ncsu.edu


    On Dec 6, 2016, at 12:43 AM, Chihmao HSIEH <c.hsieh@YONSEI.AC.KR> wrote:

    Dear all,

    I'm looking at administering a survey/experiment.  The (Qualtrics) survey is basically finished but I'm exploring different possibilities for my respondent pool.

    Has anybody here ever used Qualtrics for recruiting respondents?  Or Amazon Mechanical Turk?  And any ideas about comparing the costs / effectiveness of those two programs?

    And finally, any idea about the respect for those two sample recruitment strategies in our (entrepreneurship) academic community?  It seems that there is at least some advocacy for AMT in the broader academic community.

    Regards, -chihmao.

     
    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!

    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!


  • 3.  Qualtrics panel recruitment or Amazon Mechanical Turk?

    Posted 12-06-2016 08:19
    Another option is https://www.prolific.ac/

    You can reject participants who carelessly respond and only pay for those who complete the study. The service also provides a full demographic, downloadable data set, for the sample you collect. 

    On Tue, Dec 6, 2016 at 6:40 AM, Jeff Pollack <jmpolla3@ncsu.edu> wrote:
    Hi Chihmao,

    I have used both MTurk and Qualtrics as sources for participant recruitment.

    There are complicated differences that include price and demographics, as well as attention span, expected rewards/compensation, and usefulness of respondent's experience.

    Publications from both sources are getting more common (see below for some relevant citations).

    I sought entrepreneurs from a Qualtrics Panel and found that the people who self-reported in response to my qualifying questions were not the "entrepreneurs" I anticipated (they were mostly folks with small side businesses with very modest goals-e.g., make $1,000 extra a year on Etsy or Uber). If you want just a general representative panel, then this is less important. But, if you want a specific demographic, Qualtrics can do it, it just costs more and you need to be very careful with the qualifying questions you ask. Prices for Qualtrics Panels very widely. I have paid anywhere from $5/participant (for general audiences) to $50/participant (for a matched sample of entrepreneurs and their spouses).

    For MTurk, it is important to consider the task you want them to do-some tasks are not suitable to the Human Intelligence Task (HIT) setting. And, it is harder to be selective about the characteristics you want in a sample-because anyone can self-report. Compensation expectations are much lower-typically based on an hourly rate. If your task takes 10 minutes, then compensation of between .$50 and $2.00 is common ($10/hour divided by the minutes it takes).

    Email me offline if you have questions.

    Best, Jeff


    Qualtrics Panel
    Hagtvedt, H. (2011). The impact of incomplete typeface logos on perceptions of the firm. Journal of Marketing, 75(4), 86-93.
    Brandon, D. M., Long, J. H., Loraas, T. M., Mueller-Phillips, J., & Vansant, B. (2013). Online instrument delivery and participant recruitment services: Emerging opportunities for behavioral accounting research. Behavioral Research in Accounting, 26(1), 1-23.
    Rosoff, H., Siko, R., John, R., & Burns, W. J. (2013). Should I stay or should I go? An experimental study of health and economic government policies following a severe biological agent release. Environment Systems & Decisions, 33(1), 121-13.

    MTurk
    Casler, K., Bickel, L., & Hackett, E. (2013). Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29(6), 2156-2160.
    Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk a new source of inexpensive, yet high-quality, data?. Perspectives on psychological science, 6(1), 3-5.
    Bohannon, J. (2011). Social science for pennies. Science, 334(6054), 307-307.



    Jeff Pollack
    Assistant Professor 
    Management, Innovation, and Entrepreneurship Department
    Poole College of Management
    NC State University
    2801 Founders Drive, Campus Box 7229
    Raleigh, NC 27695-7229
    804.397.0818 phone
    jmpolla3@ncsu.edu


    On Dec 6, 2016, at 12:43 AM, Chihmao HSIEH <c.hsieh@YONSEI.AC.KR> wrote:

    Dear all,

    I'm looking at administering a survey/experiment.  The (Qualtrics) survey is basically finished but I'm exploring different possibilities for my respondent pool.

    Has anybody here ever used Qualtrics for recruiting respondents?  Or Amazon Mechanical Turk?  And any ideas about comparing the costs / effectiveness of those two programs?

    And finally, any idea about the respect for those two sample recruitment strategies in our (entrepreneurship) academic community?  It seems that there is at least some advocacy for AMT in the broader academic community.

    Regards, -chihmao.

     
    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!

    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!



    --
    "The price of anything is the amount of life you exchange for it." HDT

    Tim Michaelis
    PhD candidate, Innovation & Entrepreneurship
    Research Associate, Center for Innovation Management Studies
    North Carolina State University
    Phone: (336) 340 - 8791

    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!


  • 4.  Qualtrics panel recruitment or Amazon Mechanical Turk?

    Posted 12-06-2016 08:28
    I have used Amazon Mechanical Turk. Be careful because a lot of the participants are fake. In the trenches, folks set up fake accounts so that they can participate more than once to earn money. A lot of the turks are typically in very low income foreign countries and may not actually be entrepreneurs or whatever it is that you are looking for. So, I would not use them.  They are used quite a bit to boost social media responses and to help posts go viral. 

    Small businesses owners can pay them to post responses on Facebook, Twitter, Linkedin or to do small tasks for very little money. Not reliable for scholarly research.

    Dr. Clovia Hamilton

    On Tue, Dec 6, 2016 at 12:43 AM, "Chihmao HSIEH" <c.hsieh@yonsei.ac.kr> wrote:

    Dear all,

    I'm looking at administering a survey/experiment.  The (Qualtrics) survey is basically finished but I'm exploring different possibilities for my respondent pool.

    Has anybody here ever used Qualtrics for recruiting respondents?  Or Amazon Mechanical Turk?  And any ideas about comparing the costs / effectiveness of those two programs?

    And finally, any idea about the respect for those two sample recruitment strategies in our (entrepreneurship) academic community?  It seems that there is at least some advocacy for AMT in the broader academic community.

    Regards, -chihmao.

     

    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!

    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!


  • 5.  Qualtrics panel recruitment or Amazon Mechanical Turk?

    Posted 12-06-2016 10:32

    Chihmao.

     

    To Jeff's excellent suggestions, I add two more below as they pertain to management and entrepreneurship research using M-Turk:

     

    Leslie, Machester, & Dahm (2016, AMJ); Gupta, Goktan, & Gunay (2014 JBV)

     

    You may also want to look at the following:

     

    Aguinis, H., & Lawal, S. O. (2012). Conducting field experiments using eLancing's natural environment. Journal of Business Venturing, 27(4), 493-505.

     

    Hope these help!  

     

    Vishal

     

    On Tue, Dec 6, 2016 at 5:40 AM, Jeff Pollack <jmpolla3@ncsu.edu> wrote:

    Hi Chihmao,

     

    I have used both MTurk and Qualtrics as sources for participant recruitment.

     

    There are complicated differences that include price and demographics, as well as attention span, expected rewards/compensation, and usefulness of respondent's experience.

     

    Publications from both sources are getting more common (see below for some relevant citations).

     

    I sought entrepreneurs from a Qualtrics Panel and found that the people who self-reported in response to my qualifying questions were not the "entrepreneurs" I anticipated (they were mostly folks with small side businesses with very modest goals-e.g., make $1,000 extra a year on Etsy or Uber). If you want just a general representative panel, then this is less important. But, if you want a specific demographic, Qualtrics can do it, it just costs more and you need to be very careful with the qualifying questions you ask. Prices for Qualtrics Panels very widely. I have paid anywhere from $5/participant (for general audiences) to $50/participant (for a matched sample of entrepreneurs and their spouses).

     

    For MTurk, it is important to consider the task you want them to do-some tasks are not suitable to the Human Intelligence Task (HIT) setting. And, it is harder to be selective about the characteristics you want in a sample-because anyone can self-report. Compensation expectations are much lower-typically based on an hourly rate. If your task takes 10 minutes, then compensation of between .$50 and $2.00 is common ($10/hour divided by the minutes it takes).

     

    Email me offline if you have questions.

     

    Best, Jeff

     

     

    Qualtrics Panel

    Hagtvedt, H. (2011). The impact of incomplete typeface logos on perceptions of the firm. Journal of Marketing, 75(4), 86-93.

    Brandon, D. M., Long, J. H., Loraas, T. M., Mueller-Phillips, J., & Vansant, B. (2013). Online instrument delivery and participant recruitment services: Emerging opportunities for behavioral accounting research. Behavioral Research in Accounting, 26(1), 1-23.

    Rosoff, H., Siko, R., John, R., & Burns, W. J. (2013). Should I stay or should I go? An experimental study of health and economic government policies following a severe biological agent release. Environment Systems & Decisions, 33(1), 121-13.

     

    MTurk

    Casler, K., Bickel, L., & Hackett, E. (2013). Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29(6), 2156-2160.

    Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk a new source of inexpensive, yet high-quality, data?. Perspectives on psychological science, 6(1), 3-5.

    Bohannon, J. (2011). Social science for pennies. Science, 334(6054), 307-307.

     

     

     

    Jeff Pollack
    Assistant Professor 
    Management, Innovation, and Entrepreneurship Department
    Poole College of Management
    NC State University
    2801 Founders Drive, Campus Box 7229
    Raleigh, NC 27695-7229
    804.397.0818 phone
    jmpolla3@ncsu.edu

     

    On Dec 6, 2016, at 12:43 AM, Chihmao HSIEH <c.hsieh@YONSEI.AC.KR> wrote:

     

    Dear all,

    I'm looking at administering a survey/experiment.  The (Qualtrics) survey is basically finished but I'm exploring different possibilities for my respondent pool.

    Has anybody here ever used Qualtrics for recruiting respondents?  Or Amazon Mechanical Turk?  And any ideas about comparing the costs / effectiveness of those two programs?

    And finally, any idea about the respect for those two sample recruitment strategies in our (entrepreneurship) academic community?  It seems that there is at least some advocacy for AMT in the broader academic community.

    Regards, -chihmao.

     

    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!

     

    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!

     

    ************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!


Related Content