Your organising group understands what “regular participation” means because you have discussed it for weeks. A new reader may count one annual festival, while another thinks only weekly meetings qualify. A questionnaire can collect tidy-looking answers to different questions without anyone noticing.
Before launching it, invite a few people who were not involved in writing it to try the draft. The aim of this pilot is to discover how the questions work, not to estimate what the whole community thinks.
The Office for National Statistics describes questionnaire development that includes testing how people understand and answer questions. A small community group can apply the same basic concern for interpretation without pretending its informal pilot is a representative survey.
Observe before explaining
Ask the tester to complete the draft in a realistic format. If the final questionnaire will be on a phone, do not test only a large printed copy. Note where they pause, skip a question or search for an answer that is not offered.
Afterwards, use neutral prompts such as asking what a particular phrase meant to them or how they chose between two options. Do not immediately explain what they were supposed to think. Their initial interpretation is the information you need.
- Can the reader identify the period the question covers?
- Does it ask one thing, or combine two experiences?
- Is there an appropriate answer for someone who does not use the service?
- Can the person complete it using their usual access tools or assistance?
Include people with different relevant experiences, while being honest that a few testers cannot cover every need. Offer an accessible way to participate and explain that criticism of the draft is welcome.
Keep test responses separate
Label the pilot clearly. If wording or answer options change, do not quietly merge its responses into the main dataset as though everyone received the same question. Record the version and the reason for each substantial revision.
For example, replacing “Do you attend regularly?” with a specific, relevant time period makes the question easier to interpret. The exact period should fit the project, not an arbitrary template.
Run a short second check after important changes. A pilot has done its job when it helps you ask a clearer question, not when testers reassure the committee that its first draft was already excellent.