Key Mistakes to Avoid When Constructing Survey Questions
Mistakes to Avoid
These are questions that ask about 2 areas, issues, or groups but only allow for one general answer.
How often do you talk to your nurses and administrative staff when you have a problem?
Choose one: How often do you talk to your nurses when you have a problem?
Negatively worded items
These are questions that include a negative or even a double negative. Some people confirm negation with a “yes” and some do so with a “no.”
In an average week, have you ever not been able to start rounds on time?
Reframe as a positive question: In an average week, how many times do you start rounds on time?
Using statements instead of questions
Asking a statement requires the respondents to do multiple cognitive tasks instead of just one. The respondent must understand (1) what the statement means, (2) if it applies to them, and (3) which response captures how much the statement applies to them.
I am confident I can do well in this course: not at all, a little bit true, somewhat true, mostly true, completely true.
Reframe as a question rather than a statement: How confident are you in succeeding in this course? Not at all confident, a little confident, somewhat confident, mostly confident, completely confident.
Response scales do not match the question
This may take many forms in regard to discordance, such as number of response options, polarity of the scale, mutual exclusiveness of response options, or missing options.
Would you recommend this hospital to someone? Not likely at all, a little likely, somewhat likely, very likely.
Match the descriptors in your scale with the stem of your item: How likely are you to recommend this hospital to someone? Not likely at all, a little likely, somewhat likely, very likely.
Asking socially desirable or anxiety-provoking questions
Respondents have difficulty responding truthfully if they are afraid of being judged.
Do you think women and children should be given the first available flu shots?
Consider removing or rephrasing so that the stem sets up a nonbiased approach: Who should be given the first available flu shots (pick one)? (1) Infants (<1 y of age), (2) children (1–17 y), (3) adults (18–64 y), or (4) elderly (65 y and older).
Not using mutually exclusive answer categories
These are questions with overlapping response options, so that 2 categories may be simultaneously applicable.
How often have you worked >40 h in the past 30 d? (1) Never, (2) 1–5 times, (3) 5–10 times, (4) 10–15 times, (5) >15 times.
How often have you worked >40 h in the past 30 d? (1) Never, (2) 1–5 times, (3) 6–10 times, (4) 11–15 times, (5) >15 times.
Adapted with permission from Wolters Kluwer Health, Inc: Artino A, Gelbach H, and Durning S. AM last page: avoiding 5 common pitfalls of survey design. Acad Med. 2011;86(10):1327. http://www.academicmedicine.org.
Additional Resources for Different Aspects of Survey Design
If You Want to Learn More About. . .
A wide range of topics related to survey design and administration