Hi Mallory,
Let me toss out some thoughts regarding your point b, and maybe c. I believe it is better to use frequency measures to assess behavior. I would further extend this to assessing exposures, e.g., workload, work-family conflict, bullying, etc. The practice of using agree/disagree anchors to assess behavior/exposures is not so common in other disciplines.
Items using agree/disagree response formats often fix the frequency of the behavior/exposure in the item and then ask people to agree/disagree with the item. Some items may not assume a level of frequency. In any case, these items provide no reasonable information on frequency of exposure, which I think is what researchers are really interested in. Our general use of vague qualifiers makes interpreting findings difficult enough. Using agree/disagree responses for behaviors/exposures only compounds problems of interpretation. I addressed this issue in a chapter where we discussed four major measurement issues with current measures of work-family conflict. See the section on appropriate response anchors in Bellavia and Frone (2005).
There are several other related issues:
1) The type of frequency anchors. Often there is no reason to use relative frequency measures (never, seldom, sometimes, often, very often). These are problematic because there is no reason to assume that "often" means the same thing to every respondent. When possible, use anchors that assess absolute frequency (never, less than monthly, 1-3 days per month, 1-2 times per week, 3-5 times per week, etc.). The coarseness of such a scale can be adjusted, but one needs to be careful how that is done (Schwarz, 1999). Or use an open-ended response format that asks for the number of times/days. There are trade-offs between these two types of absolute frequency response formats. For instance, if a behavior/exposure generally occurs frequently (high base rate), an opened format may be difficult for respondents. This general issue is discussed in Belavia and Frone (2005).
2) Fix the reporting period. When asking people to report their frequency of behaving some way or frequency of exposure to some event/condition, fix the reporting period in the instructional stem (or items)--past week, past month, past 6 months, past year, lifetime, etc. Surprisingly, this is not always done. Frequency responses that are not tied to a specific reporting period are difficult or impossible to interpret. And there is no reason to believe that in the absence of fixing the reporting period, all individuals would use the same period. Even if they did, you wouldn't know what it was. Also, a fixed reporting period is crucial to interpret prevalence rates. The need for a fixed reporting period is mentioned in Bellavia & Frone (2003).
3) How does one combine frequency of behavior/exposure items into an overall score. The standard approach is to compute the mean frequency across item. Another approach that can be useful is to create an overall score by taking the maximum score across items for each individual (for examples, see Frone, 1994, 2008; Frone & Brown, 2010, Frone & Trinidad, 2014; Grzywacz, Frone, Brewer, & Kovner, 2006). Mean scores can underestimate or overestimate the frequency of engaging in some behavior or frequency of exposure. If one's interest is exclusively in terms of associations, it may not matter much if one uses mean or maximum values. Nonetheless, it is something to consider. The difference between mean and maximum scores can have a larger impact if one is interested in prevalence rates and the actual levels of frequency.
I have had reviewers question the use of maximum values. Below is a response to one such reviewer, which lays out the logic in more detail than any of the papers cited above (though Frone 1994 has a footnote addressing the issue):
With items that assess the frequency of engaging in behaviors, such as substance use, deviant acts of various types, etc., we think computing an average score or the sum of the scores is not desirable. Logically, the maximum value will perform better. For example, consider three items that assess the frequency of using marijuana, cocaine, and heroin. The response options are:
0=never
1=less than monthly
2=1 to 3 days per month,
3=1 to 3 days per week
4=4 to 5 days per week
5=6 to 7 days per week
We want a score that represent the overall frequency of using illicit drugs. Consider a person with scores of 5 for marijuana, 0 for cocaine, and 0 for heroin. If we wanted to model the overall frequency of using illicit drugs, a mean score would not be useful. This person would have a mean score of 1.67, which is between less than monthly and monthly use. However, we know the person uses an illicit drug almost every day -- marijuana 6 to 7 days per week. So it seems more accurate to use the maximum value across the three items. A sum also has its problems. Consider a person who has scores of 5 on all three items. In this case the mean and maximum value are both 5, correctly indicting that the person used an illicit drug almost every day. However the sum would be 15. This score confounds the overall frequency of using an illicit drug with concurrent use of multiple drugs (i.e., number of illicit drugs being used). If one was interested in both overall frequency of using an illicit drug and concurrent use of multiple illicit drugs, it would probably be better to create separate measures of these two dimensions than use the sum, which confounds both dimensions. There are all sorts of variations in terms of means and sums, but in general if one wants a measure of overall frequency of illicit drug use, the maximum value is what a researcher probably wants.
In terms of the present paper, consider a measure of the frequency of workplace alcohol use (use during the workday). Using the same anchors, we ask how often the respondents drink while working, during lunch, and during other breaks. Exactly when the drinking occurs during the workday is not the issue. So if a person answers, 0, 5, and 0. The maximum value correctly classifies the person as drinking during the workday almost every day (obviously a score of 4 would do the same for a person who works 5 days per week). The mean would suggest infrequent drinking during the workday, which is incorrect. As with the illicit drug use example, the sum confounds frequency of drinking during the workday with the number of occasions during the workday when drinking occurs. Again, if one is interested in both of these dimensions, it may make more sense to create separate variables rather than use a sum that confounds both dimensions.
4) Level or severity of behavior/exposure. Not directly related to your question, but I think assessing frequency naturally leads to the issue of severity. For example, if one wants to understand predictors or outcomes of a behavior like alcohol use, frequency of use is not enough. One also would want to know among users, how much they consume. The level of drinking is more important for many outcomes than the frequency of use, or their may be interactive (synergistic). Assessing frequency of being injured at work would seem insufficient without knowing the severity of the injuries. How one assesses level/severity may be more dependent on the actual behaviors/exposures under investigation than assessing frequency of behaviors/exposures.
Bellavia, G., & Frone, M. R. (2005). Work-family conflict. In J. Barling, E. K. Kelloway, & M. R. Frone (Eds.), Handbook of work stress (pp. 113-147). Thousand Oaks, CA: Sage.
Frone, M. R., Cooper, M. L., & Russell, M. (1994). Stressful life events, gender, and substance use: An application of tobit regression. Psychology of Addictive Behaviors, 8, 59-69.
Frone, M. R. (2008). Are work stressors related to employee substance use? The importance of temporal context in assessments of alcohol and illicit drug use. Journal of Applied Psychology, 93, 199-206.
Frone, M. R., & Brown, A. L. (2010). Workplace substance use norms as predictors of employee substance use and impairment: A survey of U.S. workers. Journal of Studies on Alcohol and Drugs, 71, 526-534.
Frone, M. R., & Trinidad, J. R. (2014). Perceived physical availability of alcohol at work and workplace alcohol use and Impairment: Testing a structural model.
Psychology of Addictive Behaviors, 28, 1271-1277.
Grzywacz, J. G., Frone, M. R., Brewer, C. S., & Kovner, C. T. (2006). Quantifying work-family conflict among registered nurses. Research in Nursing & Health, 29, 414-426.
Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93-105.
**************************************************************************
Michael R. Frone, Ph.D.
Senior Research Scientist
Research Institute on Addictions
State University of New York at Buffalo
1021 Main Street
Buffalo, New York 14203
Office: 716-887-2519
Fax: 716-887-2477
E-mail: frone@ria.buffalo.edu
Alcohol and Illicit Drug Use in the Workforce and Workplace
RIA Website Google Scholar ResearchGate LinkedIn
***************************************************************************
-----Organizational Behavior Division Listserv <OB@AOMLISTS.AOM.ORG> wrote: -----To:
OB@AOMLISTS.AOM.ORGFrom: Mallory McCord
Sent by: Organizational Behavior Division Listserv
Date: 10/15/2017 03:18PM
Subject: [OB-LIST] OB Listserv Question
Greetings,
Can anyone recommend references which discuss optimal use of response scales? Some questions I am looking to answer include: a) Is it best to use the response scale with which a scale was originally validated?, b) It is best practice to use a frequency scale for behaviors and agree/disagree for attitudes?, c) Does base rate of assessed behaviors matter for a response scale?, d) Should all scales in a survey use the same number of scale points (i.e., all 5-point, all 7-point)? and e), Is it better to use a 7-point over a 5-point scale?
Thank you very much,
Mallory A McCord, Ph.D., M.B.A. Assistant Professor of I/O Psychology
Department of Psychology
University of Minnesota Duluth
326 Bohannon Hall
1207 Ordean Court
Duluth, MN 55812-3010