Instant PDF download, name-and-email checkout. Complete Book Bundle: every Economics, Statistics, Mathematics, Econometrics and Research Methods book
Research Methods

Questionnaire Design: How to Write Good Survey Questions (with Rewrites)

Questionnaire design made simple: fix leading, double-barrelled, loaded and ambiguous survey questions, build Likert scales, order questions and pilot.

In 30 seconds
  • Every survey question should ask one thing, in plain words, without pushing the respondent towards an answer.
  • The four classic faults are leading, double-barrelled, loaded and ambiguous questions. Each has a simple fix.
  • Response options must be mutually exclusive (no overlaps) and exhaustive (everyone has an answer).
  • For rating scales, label every point, keep the direction consistent and think twice before using agree/disagree items.
  • Order questions from easy and general to specific and sensitive, then pilot the whole questionnaire before you launch.

The problem: a survey that cannot be analysed

A student studying part-time work sends out this question: “Don’t you agree that working long hours and studying is stressful and bad for grades?” Ninety people reply. Most tick “Agree”. What has the student learned? Almost nothing. The question pushes people to agree, asks about two things at once (stress and grades), and never defines “long hours”.

Bad questions cannot be fixed at the analysis stage. No statistical test can rescue data from a question that measured the wrong thing. This guide shows you how to write questions that work, with worked rewrites, Likert scales, response options, question order and piloting. It suits undergraduate and master’s dissertations, the IB Extended Essay and A-level NEA projects.

What a good survey question does

Key termQuestionnaire: a fixed set of written questions, answered by respondents themselves (online or on paper) or read out by an interviewer. A good questionnaire question is clear (everyone understands it the same way), neutral (it does not suggest an answer), single (it asks about one thing) and answerable (respondents know the answer and the options fit it).
Research Methods Made Simple textbook coverResearch Methods Made Simple takes you from questionnaire design to a finished dissertation, with 43 templates and worked examples.Get the textbook · $15

Krosnick and Presser’s review of the research on question design summarises the long-standing advice: use simple, familiar words, make questions specific, avoid leading or loaded wording, and ask about one thing at a time[1]. Dillman, Smyth and Christian give detailed guidance on wording and layout for online, phone and mail surveys[2]. The four faults below break those rules most often.

The four classic faults

FaultWhat it isBad exampleWhy it fails
LeadingWording that suggests the “right” answerDon’t you agree that the library should open longer?Invites agreement; inflates “yes”
Double-barrelledTwo questions in oneIs your job flexible and well paid?A flexible, badly paid job has no correct answer
LoadedEmotive words or a hidden assumptionHow often do you skip lectures because of your job?Assumes the respondent has a job and skips lectures; “skip” sounds like blame
AmbiguousVague words people read differentlyDo you work regularly?“Regularly” could mean daily, weekly or monthly
Worked example

Rewrite the four bad questions in the table so that each asks one neutral, clear thing.

Step 1Leading: “Don’t you agree that the library should open longer?” Remove the push and offer balanced options. Rewrite: “Would you prefer the library’s weekday opening hours to be longer, shorter or kept as they are?” (Longer / Kept as they are / Shorter / No preference.)
Step 2Double-barrelled: “Is your job flexible and well paid?” Split it. Rewrite: “How flexible are your working hours?” and, separately, “How satisfied are you with your hourly pay?” each with its own scale.
Step 3Loaded: “How often do you skip lectures because of your job?” Add a filter and neutral wording. Rewrite: Q1 “Have you had paid work during this term?” (Yes / No). If yes, Q2 “In the last four weeks, how many timetabled classes, if any, did you miss because of paid work?” (0 / 1–2 / 3–5 / 6 or more.)
Step 4Ambiguous: “Do you work regularly?” Replace the vague word with a time frame and a number. Rewrite: “In the last seven days, how many hours of paid work did you do?” (enter a whole number; 0 if none.)

Each rewrite gives data you can analyse: a count, a category or a rating that means the same thing to every respondent.

Response options: exclusive and exhaustive

Closed questions are only as good as their options. Two rules matter most. Options must be mutually exclusive, so no answer fits two boxes. And they must be exhaustive, so every respondent has a box to tick.

Worked example

Fix these options for “How many hours of paid work did you do last week?”: 0–5, 5–10, 10–20, 20+.

Step 1Check exclusivity. A student who worked 5 hours fits both “0–5” and “5–10”. So does 10 hours (“5–10” and “10–20”) and 20 hours (“10–20” and “20+”). The categories overlap at every boundary.
Step 2Check exhaustiveness. Students with no job are lumped with those who worked up to 5 hours. “None” should be its own category, because not working is different from working a little.
Step 3Check usefulness. If your hypothesis concerns 15 or more hours, you need a boundary at 15. “10–20” hides it.
Step 4Rewrite: None (0) / 1–5 / 6–10 / 11–15 / 16–20 / More than 20. Every whole number of hours fits exactly one box.
Step 5Better still, ask for the number itself (“Enter the number of hours, 0 if none”). You can always group numbers later, but you cannot ungroup categories.

The fixed version gives six non-overlapping groups with a boundary at 15 hours, and the open number keeps full detail for analysis.

Other option rules: include “Other (please specify)” when you cannot list every answer; use “Prefer not to say” for sensitive items; and keep options in a logical order (low to high, or alphabetical for unordered lists). Krosnick and Presser note that the order of options can itself shift answers: when options are shown on screen or paper, people tend to pick ones near the top (a primacy effect), while options read aloud favour those heard last[1].

Likert scales and rating questions

Key termLikert scale: a set of statements on one topic, each answered on an ordered agreement scale (for example Strongly disagree to Strongly agree), with the item scores added or averaged into one score. The method of summated ratings was introduced by Rensis Likert in 1932 to measure attitudes[3]. Strictly, the scale is the set of items; each single item is a Likert-type item.

Good practice for rating questions:

  • Label every point. Krosnick and Presser report that reliability is higher when all points carry words rather than only the ends[1].
  • Choose the length on purpose. In their review, reliability and validity tend to improve up to about seven points, and they conclude that 7-point scales are probably optimal in many instances[1]. Five points is common and easier on a phone screen. Whatever you choose, use it consistently.
  • Offer a midpoint for genuinely neutral people. Without one, neutral respondents are pushed to a side.
  • Be wary of agree/disagree. Some people tend to agree with any statement (acquiescence). Krosnick and Presser recommend asking about the underlying dimension directly where you can[1]: “How stressful do you find balancing work and study?” (Not at all / Slightly / Moderately / Very / Extremely) rather than “Balancing work and study is stressful. Agree or disagree?”
  • Mix item direction carefully. If you include some negatively worded items in a Likert scale, reverse-score them before adding up, and check they are not confusing.
Worked example

A five-item “work–study balance” scale uses 1 = Strongly disagree to 5 = Strongly agree. Item 3 is negatively worded (“I often feel I cannot keep up with my coursework because of work”). A respondent answers 4, 5, 2, 4, 3. Find the scale score.

Step 1Reverse-score item 3 so that high always means better balance. On a 1–5 scale, reversed score = 6 − original. So 6 − 2 = 4.
Step 2Item scores after reversing: 4, 5, 4, 4, 3.
Step 3Total: 4 + 5 + 4 + 4 + 3 = 20, out of a possible 25 (minimum 5).
Step 4Mean item score: 20 / 5 = 4.0, which sits between Agree and Strongly agree on the original labels.

Without reversing, the total would be 4 + 5 + 2 + 4 + 3 = 18, understating this respondent’s balance. (The figures are illustrative.) Before you combine items, check that they hang together, for example with Cronbach’s alpha; our post on reliability and validity explains how.

Research Methods Practice Questions coverPractise spotting flawed survey questions and rewriting them with the Practice Questions book: 1,004 questions, all with worked answers.Get 1,004 questions · $9

Question order

The order of questions changes answers and changes who finishes. Krosnick and Presser advise opening with easy, engaging items that clearly relate to the stated topic, grouping related items, and keeping in mind that fatigue lowers the quality of answers near the end[1].

A safe order for questionnaire sections student survey on part-time work
SectionSensitive items
1Introduction and consentPurpose, time needed, anonymity, right to withdraw2Easy opening and screeningSimple factual items linked to the topic; route people who do not qualify3Core questions: general to specificDo you work? Then hours, job type, shifts4Rating scalesGrouped by topic, every point labelled, same direction5Sensitive questionsIncome, health, behaviour: later, once trust is built6Demographics and thank youAge, year, subject; a closing open question; thanksFlow runs top to bottom. Questions move from easy and general to specific and sensitive.
Figure 1. Start easy and on topic, move from general to specific, and leave sensitive and demographic questions until later.

Two more order effects to know. First, an earlier question can colour a later one: asking about stress at work before asking “How satisfied are you with university life?” may pull satisfaction down. Ask the general question first. Second, filter questions save time: “Have you had paid work this term?” lets people without jobs skip the work section, so they do not have to answer questions that do not apply.

Common mistakePutting demographics and income at the very start. It feels logical, but it opens with personal questions before trust is built, and some people quit. Unless you need them to route respondents, move them to the end.

Piloting: test before you launch

A pilot is a small trial run with people like your real respondents. It is the cheapest insurance in research. Ask pilot respondents to complete the questionnaire, then talk to them: which questions were unclear, which options were missing, how long did it take? Clark and colleagues recommend piloting a questionnaire before using it, to check that the questions work as intended[4].

Pilot finding (15 students)NumberAction
Chose “Other” for job type6 of 15 (40%)Most wrote “delivery rider” or “tutoring”: add both as options
Asked what “regularly” meant4 of 15 (27%)Replace with “in the last seven days”
Completion timeMean 11.5 minutes (range 8 to 15)Cut the least useful questions to aim for under 10 minutes

(The figures are illustrative: 173 total minutes ÷ 15 = 11.5 minutes on average.) If 40% of people need “Other”, your options are missing something. If a quarter ask what a word means, the word is ambiguous. Fix, then pilot again if the changes are large. Do not include pilot answers in your main results if you changed the questions.

Common mistakePiloting with friends from your course and only asking “Was it OK?” Friends are polite and share your vocabulary. Pilot with people like your real sample and ask specific questions: “What did you think Q7 was asking?”
Dissertation tipPut the final questionnaire in an appendix and, in your methodology chapter, include a short table of what the pilot found and what you changed. It shows examiners that your instrument was tested, and it answers the question “how do you know your questions measured what you intended?” before they ask it.

A quick questionnaire checklist

  • Every question links to a research question or objective. If it does not, delete it.
  • Each question asks one thing, in plain words, with a clear time frame.
  • No leading, loaded or emotive wording; no hidden assumptions.
  • Options are mutually exclusive and exhaustive, with “Other” or “Prefer not to say” where needed.
  • Rating scales have every point labelled and run in the same direction.
  • Order goes easy to hard, general to specific, with sensitive items late.
  • Consent information is clear, and the questionnaire has ethics approval.
  • It has been piloted with people like the real sample, and changes are recorded.

For how many people to survey, see margin of error and sample size; for how to choose them, see sampling methods.

Research Methods Made Simple bundle coverGet through your survey design and the rest of your dissertation with both Research Methods Made Simple books for $19.Both books · $19

The Research Methods Made Simple textbook covers questionnaire and interview design with ready-to-use templates and a companion student survey dataset, and the Research Methods Made Simple Practice Questions book includes scenario questions where you spot the flaws in real-looking questionnaires, each with a worked answer.

Practice questions

EasyQ1. Name the fault and rewrite: “How satisfied are you with the cost and quality of campus food?” Show worked answer
Fault: double-barrelled. It asks about cost and quality at once, and someone may like the quality but not the cost.
Rewrite as two questions: “How satisfied are you with the price of campus food?” and “How satisfied are you with the quality of campus food?”, each on a fully labelled scale from Very dissatisfied to Very satisfied.
EasyQ2. What is wrong with these age options: Under 18 / 18–21 / 21–25 / Over 25? Show worked answer
They are not mutually exclusive: a 21-year-old fits both “18–21” and “21–25”.
Exhaustiveness is fine for whole years, since “Over 25” covers everyone older.
Fix: Under 18 / 18–20 / 21–24 / 25 or over.
MediumQ3. A 4-item scale is scored 1–7. Items 2 and 4 are negatively worded. A respondent answers 6, 3, 5, 1. Find the total and mean after reverse-scoring. Show worked answer
On a 1–7 scale, reversed score = 8 − original.
Item 2: 8 − 3 = 5. Item 4: 8 − 1 = 7.
Scores: 6, 5, 5, 7.
Total = 6 + 5 + 5 + 7 = 23 (out of 28). Mean = 23 / 4 = 5.75.
MediumQ4. Rewrite this loaded, leading question as a neutral question with balanced options: “Given the cost-of-living crisis, how worried are you about money?” Show worked answer
Problems: the opening clause primes worry and assumes the respondent sees a crisis; the question assumes the respondent is worried (“how worried”).
Rewrite: “Thinking about the next three months, how worried, if at all, are you about meeting your living costs?” Options: Not at all worried / Slightly worried / Moderately worried / Very worried / Extremely worried.
“If at all” and “Not at all” make no worry an equally acceptable answer.
ExamQ5. A student’s questionnaire on part-time work begins: “Q1 What is your household income? Q2 Don’t you think students who work long hours get worse grades? Q3 How often do you work? (Often / Sometimes / Rarely)”. Evaluate the design and propose improvements. [10 marks] Show worked answer
Q1 (up to 3 marks): sensitive item placed first, before trust is built, which risks drop-out (1); it should move to the end with “Prefer not to say” (1) and income bands that are mutually exclusive and exhaustive (1).
Q2 (up to 3 marks): leading (“Don’t you think”) (1); “long hours” and “worse grades” are undefined (1); it asks for an opinion about a relationship the study should measure, so replace it with separate factual questions on hours and marks (1).
Q3 (up to 2 marks): ambiguous vague quantifiers with no time frame (1); replace with “In the last seven days, how many hours of paid work did you do? (0 if none)” (1).
Overall (up to 2 marks): open with an easy filter question such as “Have you had paid work this term?” (1); pilot with students like the target sample and record changes (1).

Frequently asked questions

Should I use a 5-point or a 7-point scale?

Either can work. Research summarised by Krosnick and Presser suggests reliability and validity improve up to about seven points, so 7 is a good choice for attitude items. Five points is shorter and easier on phones. The more important rules are to label every point and to use the same format throughout.

Should I include a “Don’t know” option?

Include it for factual questions that some people genuinely cannot answer. For opinion questions it is less clear: Krosnick and Presser report that “Don’t know” options do not improve measurement and that some people who choose them do hold views. If you omit it, make sure every question is one your respondents can answer.

How long should a questionnaire be?

As short as your research questions allow. Every extra question raises the chance of drop-out and careless answers. Time your pilot and cut anything that does not link directly to an objective. For a student survey, aim for something most people can finish in about 10 minutes.

Can I use questions from published studies?

Yes, and it is often better than writing your own. Validated scales have been tested for reliability and validity. Cite the source, check whether permission is needed, and do not change the wording without saying so, because changes can affect how the items work.

References

  1. Krosnick, J.A. and Presser, S. (2010) ‘Question and questionnaire design’, in Marsden, P.V. and Wright, J.D. (eds) Handbook of Survey Research. 2nd edn. Bingley: Emerald, ch. 9. https://web.stanford.edu/dept/communication/faculty/krosnick/docs/2010/2010%20Handbook%20of%20Survey%20Research.pdf
  2. Dillman, D.A., Smyth, J.D. and Christian, L.M. (2014) Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 4th edn. Hoboken, NJ: Wiley.
  3. Likert, R. (1932) ‘A technique for the measurement of attitudes’, Archives of Psychology, 22(140), pp. 1–55.
  4. Clark, T., Foster, L., Sloan, L. and Bryman, A. (2021) Bryman’s Social Research Methods. 6th edn. Oxford: Oxford University Press.
Research Methods Made Simple Complete Bundle cover
Research Methods Made Simple takes you through every step of a dissertation, with 43 templates, and the bundle adds 1,004 worked practice questions.Research Methods Made Simple Complete Bundle — Textbook + 1,004 Practice Questions
Get it for $19
E

The EconTutorials team writes clear, exam-focused lessons in economics and statistics for A-Level, IB, AP and university students, with more than 13 years of classroom teaching behind every guide and book in the Made Simple series.