- Every survey question should ask one thing, in plain words, without pushing the respondent towards an answer.
- The four classic faults are leading, double-barrelled, loaded and ambiguous questions. Each has a simple fix.
- Response options must be mutually exclusive (no overlaps) and exhaustive (everyone has an answer).
- For rating scales, label every point, keep the direction consistent and think twice before using agree/disagree items.
- Order questions from easy and general to specific and sensitive, then pilot the whole questionnaire before you launch.
The problem: a survey that cannot be analysed
A student studying part-time work sends out this question: “Don’t you agree that working long hours and studying is stressful and bad for grades?” Ninety people reply. Most tick “Agree”. What has the student learned? Almost nothing. The question pushes people to agree, asks about two things at once (stress and grades), and never defines “long hours”.
Bad questions cannot be fixed at the analysis stage. No statistical test can rescue data from a question that measured the wrong thing. This guide shows you how to write questions that work, with worked rewrites, Likert scales, response options, question order and piloting. It suits undergraduate and master’s dissertations, the IB Extended Essay and A-level NEA projects.
What a good survey question does
Krosnick and Presser’s review of the research on question design summarises the long-standing advice: use simple, familiar words, make questions specific, avoid leading or loaded wording, and ask about one thing at a time[1]. Dillman, Smyth and Christian give detailed guidance on wording and layout for online, phone and mail surveys[2]. The four faults below break those rules most often.
The four classic faults
| Fault | What it is | Bad example | Why it fails |
|---|---|---|---|
| Leading | Wording that suggests the “right” answer | Don’t you agree that the library should open longer? | Invites agreement; inflates “yes” |
| Double-barrelled | Two questions in one | Is your job flexible and well paid? | A flexible, badly paid job has no correct answer |
| Loaded | Emotive words or a hidden assumption | How often do you skip lectures because of your job? | Assumes the respondent has a job and skips lectures; “skip” sounds like blame |
| Ambiguous | Vague words people read differently | Do you work regularly? | “Regularly” could mean daily, weekly or monthly |
Rewrite the four bad questions in the table so that each asks one neutral, clear thing.
Each rewrite gives data you can analyse: a count, a category or a rating that means the same thing to every respondent.
Response options: exclusive and exhaustive
Closed questions are only as good as their options. Two rules matter most. Options must be mutually exclusive, so no answer fits two boxes. And they must be exhaustive, so every respondent has a box to tick.
Fix these options for “How many hours of paid work did you do last week?”: 0–5, 5–10, 10–20, 20+.
The fixed version gives six non-overlapping groups with a boundary at 15 hours, and the open number keeps full detail for analysis.
Other option rules: include “Other (please specify)” when you cannot list every answer; use “Prefer not to say” for sensitive items; and keep options in a logical order (low to high, or alphabetical for unordered lists). Krosnick and Presser note that the order of options can itself shift answers: when options are shown on screen or paper, people tend to pick ones near the top (a primacy effect), while options read aloud favour those heard last[1].
Likert scales and rating questions
Good practice for rating questions:
- Label every point. Krosnick and Presser report that reliability is higher when all points carry words rather than only the ends[1].
- Choose the length on purpose. In their review, reliability and validity tend to improve up to about seven points, and they conclude that 7-point scales are probably optimal in many instances[1]. Five points is common and easier on a phone screen. Whatever you choose, use it consistently.
- Offer a midpoint for genuinely neutral people. Without one, neutral respondents are pushed to a side.
- Be wary of agree/disagree. Some people tend to agree with any statement (acquiescence). Krosnick and Presser recommend asking about the underlying dimension directly where you can[1]: “How stressful do you find balancing work and study?” (Not at all / Slightly / Moderately / Very / Extremely) rather than “Balancing work and study is stressful. Agree or disagree?”
- Mix item direction carefully. If you include some negatively worded items in a Likert scale, reverse-score them before adding up, and check they are not confusing.
A five-item “work–study balance” scale uses 1 = Strongly disagree to 5 = Strongly agree. Item 3 is negatively worded (“I often feel I cannot keep up with my coursework because of work”). A respondent answers 4, 5, 2, 4, 3. Find the scale score.
Without reversing, the total would be 4 + 5 + 2 + 4 + 3 = 18, understating this respondent’s balance. (The figures are illustrative.) Before you combine items, check that they hang together, for example with Cronbach’s alpha; our post on reliability and validity explains how.
Question order
The order of questions changes answers and changes who finishes. Krosnick and Presser advise opening with easy, engaging items that clearly relate to the stated topic, grouping related items, and keeping in mind that fatigue lowers the quality of answers near the end[1].
Two more order effects to know. First, an earlier question can colour a later one: asking about stress at work before asking “How satisfied are you with university life?” may pull satisfaction down. Ask the general question first. Second, filter questions save time: “Have you had paid work this term?” lets people without jobs skip the work section, so they do not have to answer questions that do not apply.
Piloting: test before you launch
A pilot is a small trial run with people like your real respondents. It is the cheapest insurance in research. Ask pilot respondents to complete the questionnaire, then talk to them: which questions were unclear, which options were missing, how long did it take? Clark and colleagues recommend piloting a questionnaire before using it, to check that the questions work as intended[4].
| Pilot finding (15 students) | Number | Action |
|---|---|---|
| Chose “Other” for job type | 6 of 15 (40%) | Most wrote “delivery rider” or “tutoring”: add both as options |
| Asked what “regularly” meant | 4 of 15 (27%) | Replace with “in the last seven days” |
| Completion time | Mean 11.5 minutes (range 8 to 15) | Cut the least useful questions to aim for under 10 minutes |
(The figures are illustrative: 173 total minutes ÷ 15 = 11.5 minutes on average.) If 40% of people need “Other”, your options are missing something. If a quarter ask what a word means, the word is ambiguous. Fix, then pilot again if the changes are large. Do not include pilot answers in your main results if you changed the questions.
A quick questionnaire checklist
- Every question links to a research question or objective. If it does not, delete it.
- Each question asks one thing, in plain words, with a clear time frame.
- No leading, loaded or emotive wording; no hidden assumptions.
- Options are mutually exclusive and exhaustive, with “Other” or “Prefer not to say” where needed.
- Rating scales have every point labelled and run in the same direction.
- Order goes easy to hard, general to specific, with sensitive items late.
- Consent information is clear, and the questionnaire has ethics approval.
- It has been piloted with people like the real sample, and changes are recorded.
For how many people to survey, see margin of error and sample size; for how to choose them, see sampling methods.
The Research Methods Made Simple textbook covers questionnaire and interview design with ready-to-use templates and a companion student survey dataset, and the Research Methods Made Simple Practice Questions book includes scenario questions where you spot the flaws in real-looking questionnaires, each with a worked answer.
Practice questions
EasyQ1. Name the fault and rewrite: “How satisfied are you with the cost and quality of campus food?” Show worked answer
Rewrite as two questions: “How satisfied are you with the price of campus food?” and “How satisfied are you with the quality of campus food?”, each on a fully labelled scale from Very dissatisfied to Very satisfied.
EasyQ2. What is wrong with these age options: Under 18 / 18–21 / 21–25 / Over 25? Show worked answer
Exhaustiveness is fine for whole years, since “Over 25” covers everyone older.
Fix: Under 18 / 18–20 / 21–24 / 25 or over.
MediumQ3. A 4-item scale is scored 1–7. Items 2 and 4 are negatively worded. A respondent answers 6, 3, 5, 1. Find the total and mean after reverse-scoring. Show worked answer
Item 2: 8 − 3 = 5. Item 4: 8 − 1 = 7.
Scores: 6, 5, 5, 7.
Total = 6 + 5 + 5 + 7 = 23 (out of 28). Mean = 23 / 4 = 5.75.
MediumQ4. Rewrite this loaded, leading question as a neutral question with balanced options: “Given the cost-of-living crisis, how worried are you about money?” Show worked answer
Rewrite: “Thinking about the next three months, how worried, if at all, are you about meeting your living costs?” Options: Not at all worried / Slightly worried / Moderately worried / Very worried / Extremely worried.
“If at all” and “Not at all” make no worry an equally acceptable answer.
ExamQ5. A student’s questionnaire on part-time work begins: “Q1 What is your household income? Q2 Don’t you think students who work long hours get worse grades? Q3 How often do you work? (Often / Sometimes / Rarely)”. Evaluate the design and propose improvements. [10 marks] Show worked answer
Q2 (up to 3 marks): leading (“Don’t you think”) (1); “long hours” and “worse grades” are undefined (1); it asks for an opinion about a relationship the study should measure, so replace it with separate factual questions on hours and marks (1).
Q3 (up to 2 marks): ambiguous vague quantifiers with no time frame (1); replace with “In the last seven days, how many hours of paid work did you do? (0 if none)” (1).
Overall (up to 2 marks): open with an easy filter question such as “Have you had paid work this term?” (1); pilot with students like the target sample and record changes (1).
Frequently asked questions
Should I use a 5-point or a 7-point scale?
Either can work. Research summarised by Krosnick and Presser suggests reliability and validity improve up to about seven points, so 7 is a good choice for attitude items. Five points is shorter and easier on phones. The more important rules are to label every point and to use the same format throughout.
Should I include a “Don’t know” option?
Include it for factual questions that some people genuinely cannot answer. For opinion questions it is less clear: Krosnick and Presser report that “Don’t know” options do not improve measurement and that some people who choose them do hold views. If you omit it, make sure every question is one your respondents can answer.
How long should a questionnaire be?
As short as your research questions allow. Every extra question raises the chance of drop-out and careless answers. Time your pilot and cut anything that does not link directly to an objective. For a student survey, aim for something most people can finish in about 10 minutes.
Can I use questions from published studies?
Yes, and it is often better than writing your own. Validated scales have been tested for reliability and validity. Cite the source, check whether permission is needed, and do not change the wording without saying so, because changes can affect how the items work.
References
- Krosnick, J.A. and Presser, S. (2010) ‘Question and questionnaire design’, in Marsden, P.V. and Wright, J.D. (eds) Handbook of Survey Research. 2nd edn. Bingley: Emerald, ch. 9. https://web.stanford.edu/dept/communication/faculty/krosnick/docs/2010/2010%20Handbook%20of%20Survey%20Research.pdf
- Dillman, D.A., Smyth, J.D. and Christian, L.M. (2014) Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 4th edn. Hoboken, NJ: Wiley.
- Likert, R. (1932) ‘A technique for the measurement of attitudes’, Archives of Psychology, 22(140), pp. 1–55.
- Clark, T., Foster, L., Sloan, L. and Bryman, A. (2021) Bryman’s Social Research Methods. 6th edn. Oxford: Oxford University Press.

