develop a  research proposal

Develop a Research Proposal

 

home
Writing the Proposal - Data Collection Questionnaires

Questionnaires

Questionnaires are a popular means of collecting data, but are difficult to design and often require many rewrites before an acceptable questionnaire is produced.

Advantages:

  • Can be used as a method in its own right or as a basis for interviewing or a telephone survey.
  • Can be posted, e-mailed or faxed.
  • Can cover a large number of people or organizations.
  • Wide geographic coverage.
  • Relatively cheap.
  • No prior arrangements are needed.
  • Avoids embarrassment on the part of the respondent.
  • Respondent can consider responses.
  • Possible anonymity of respondent.
  • No interviewer bias.

Disadvantages:

  • Design problems.
  • Questions have to be relatively simple.
  • Historically low response rate (although inducements may help).
  • Time delay whilst waiting for responses to be returned.
  • Require a return deadline.
  • Several reminders may be required.
  • Assumes no literacy problems.
  • No control over who completes it.
  • Not possible to give assistance if required.
  • Problems with incomplete questionnaires.
  • Replies not spontaneous and independent of each other.
  • Respondent can read all questions beforehand and then decide whether to complete or not. For example, perhaps because it is too long, too complex, uninteresting, or too personal.

Design of questionnaires

Theme and covering letter

The general theme of the questionnaire should be made explicit in a covering letter. You should state who you are; why the data is required; give, if necessary, an assurance of confidentiality and/or anonymity; and contact number and address or telephone number. This ensures that the respondents know what they are committing themselves to, and also that they understand the context of their replies. If possible, you should offer an estimate of the completion time. Instructions for return should be included with the return date made obvious. For example: ‘It would be appreciated if you could return the completed questionnaire by... if at all possible’.

Instructions for completion

You need to provide clear and unambiguous instructions for completion. Within most questionnaires these are general instructions and specific instructions for particular question structures. It is usually best to separate these, supplying the general instructions as a preamble to the questionnaire, but leaving the specific instructions until the questions to which they apply. The response method should be indicated (circle, tick, cross, etc.). Wherever possible, and certainly if a slightly unfamiliar response system is employed, you should give an example.

Appearance

Appearance is usually the first feature of the questionnaire to which the recipient reacts. A neat and professional look will encourage further consideration of your request, increasing your response rate. In addition, careful thought to layout should help your analysis. There are a number of simple rules to help improve questionnaire appearance:

  • Liberal spacing makes the reading easier.
  • Photo-reduction can produce more space without reducing content.
  • Consistent positioning of response boxes, usually to the right, speeds up completion and also avoids inadvertent omission of responses.
  • Choose the font style to maximize legibility.
  • Differentiate between instructions and questions. Either lower case and capitals can be used, or responses can be boxed.

Length

There may be a strong temptation to include any vaguely interesting questions, but you should resist this at all costs. Excessive size can only reduce response rates. If a long questionnaire is necessary, then you must give even more thought to appearance. It is best to leave pages unnumbered; for respondents to flick to the end and see ‘page 27’ can be very disconcerting.

Order

Probably the most crucial stage in questionnaire response is the beginning. Once the respondents have started to complete the questions they will normally finish the task, unless it is very long or difficult. Consequently, you need to select the opening questions with care. Usually the best approach is to ask for biographical details first, as the respondents should know all the answers without much thought. Another benefit is that an easy start provides practice in answering questions.

Once the introduction has been achieved the subsequent order will depend on many considerations. You should be aware of the varying importance of different questions. Essential information should appear early, just in case the questionnaire is not completed. For the same reasons, relatively unimportant questions can be placed towards the end. If questions are likely to provoke the respondent and remain unanswered, these too are best left until the end, in the hope of obtaining answers to everything else.

Coding

If analysis of the results is to be carried out using a statistical package or spreadsheet it is advisable to code non-numerical responses when designing the questionnaire, rather than trying to code the responses when they are returned.

An example of coding is:

Male [   ]

Female [   ]

1

2

The coded responses (1 or 2) are then used for the analysis.

Thank you

Respondents to questionnaires rarely benefit personally from their efforts and the least the researcher can do is to thank them. Even though the covering letter will express appreciation for the help given, it is also a nice gesture to finish the questionnaire with a further thank you.

Questions

  • Keep the questions short, simple and to the point; avoid all unnecessary words.
  • Use words and phrases that are unambiguous and familiar to the respondent. For example, ‘dinner’ has a number of different interpretations; use an alternative expression such as ‘evening meal’.
  • Only ask questions that the respondent can answer. Hypothetical questions should be avoided. Avoid calculations and questions that require a lot of memory work, for example, ‘How many people stayed in your hotel last year?’
  • Avoid loaded or leading questions that imply a certain answer. For example, by mentioning one particular item in the question, ‘Do you agree that Colgate toothpaste is the best toothpaste?’
  • Vacuous words or phrases should be avoided. ‘Generally’, ‘usually’, or ‘normally’ are imprecise terms with various meanings. They should be replaced with quantitative statements, for example, ‘at least once a week’.
  • Questions should only address a single issue. For example, questions like: ‘Do you take annual holidays to Spain?’ should be broken down into two discreet stages, firstly find out if the respondent takes an annual holiday, and then secondly find out if they go to Spain.
  • Do not ask two questions in one by using ‘and’. For example, ‘Did you watch television last night and read a newspaper?’
  • Avoid double negatives. For example, ‘Is it not true that you did not read a newspaper yesterday?’ Respondents may tackle a double negative by switching both negatives and then assuming that the same answer applies. This is not necessarily valid.
  • State units required but do not aim for too high a degree of accuracy. For instance, use an interval rather than an exact figure:

Ex.

How much did you earn last year?

Less than $10,000 [   ]

Between $10,000 and  $20,000 [   ]

Avoid emotive or embarrassing words – usually connected with race, religion, politics, sex, money.

Types of questions

Closed questions

A question is asked and then a number of possible answers are provided for the respondent. The respondent selects the answer which is appropriate. Closed questions are particularly useful in obtaining factual information:

Ex.

Sex:    Male [   ] Female [   ]

Did you watch television last night?     Yes [   ] No [   ]

Some ‘Yes/No’ questions have a third category ‘Do not know’. Experience shows that as long as this alternative is not mentioned people will make a choice. Also the phrase ‘Do not know’ is ambiguous:

Ex.

Do you agree with the introduction of the school policy?

Yes [   ] No [   ] Do not know [   ]

What was your main way of traveling to the hotel? Tick one box only.

Car

[   ]

Coach

[   ]

Motor bike

[   ]

Train

[   ]

Other means, please specify

Top of Form

 

Bottom of Form

With such lists you should always include an ‘other’ category, because not all possible responses might have been included in the list of answers.

Sometimes the respondent can select more than one from the list. However, this makes analysis difficult:

Why have you visited the historic house? Tick the relevant answer(s). You may tick as many as you like.

I enjoy visiting historic houses

[   ]

The weather was bad and I could not enjoy outdoor activities

[   ]

I have visited the house before and wished to return

[   ]

Other reason, please specify

Top of Form

Bottom of Form

Attitude questions

Frequently questions are asked to find out the respondents’ opinions or attitudes to a given situation. A Likert scale provides a battery of attitude statements. The respondent then says how much they agree or disagree with each one:

Read the following statements and then indicate by a tick whether you strongly agree, agree, disagree or strongly disagree with the statement.

 

Strongly agree

Agree

Disagree

Strongly disagree

My visit has been good value for money

 

 

 

 

There are many variations on this type of question. One variation is to have a ‘middle statement’, for example, ‘Neither agree nor disagree’. However, many respondents take this as the easy option. Only having four statements, as above, forces the respondent into making a positive or negative choice. Another variation is to rank the various attitude statements; however, this can cause analysis problems:

Which of these characteristics do you like about your job? Indicate the best three in order, with the best being number 1.

Varied work

[   ]

Good salary

[   ]

Opportunities for promotion

[   ]

Good working conditions

[   ]

High amount of responsibility

[   ]

Friendly colleagues

[   ]

A semantic differential scale attempts to see how strongly an attitude is held by the respondent. With these scales double-ended terms are given to the respondents who are asked to indicate where their attitude lies on the scale between the terms. The response can be indicated by putting a cross in a particular position or circling a number:

School is: (circle the appropriate number)

Difficult 

1   2   3   4   5   6   7

Easy

Useless

1   2   3   4   5   6   7

Useful

Interesting

1   2   3   4   5   6   7

Boring

For summary and analysis purposes, a ‘score’ of 1 to 7 may be allocated to the seven points of the scale, thus quantifying the various degrees of opinion expressed. This procedure has some disadvantages. It is implicitly assumed that two people with the same strength of feeling will mark the same point on the scale. This almost certainly will not be the case. When faced with a semantic differential scale, some people will never, as a matter of principle, use the two end indicators of 1 and 7. Effectively, therefore, they are using a five-point scale. Also scoring the scale 1 to 7 assumes that they represent equidistant points on the continuous spectrum of opinion. This again is probably not true. Nevertheless, within its limitations, the semantic differential can provide a useful way of measuring and summarizing subjective opinions.

Other types of questions to determine peoples’ opinions or attitudes are:

Which one/two words best describes...?

Which of the following statements best describes...?

How much do you agree with the following statement...?

Open questions

An open question such as ‘What are the essential skills a manager should possess?’ should be used as an adjunct to the main theme of the questionnaire and could allow the respondent to elaborate upon an earlier more specific question. Open questions inserted at the end of major sections, or at the end of the questionnaire, can act as safety valves, and possibly offer additional information. However, they should not be used to introduce a section since there is a high risk of influencing later responses. The main problem of open questions is that many different answers have to be summarized and possibly coded.

Testing – pilot survey

Questionnaire design is fraught with difficulties and problems. A number of rewrites will be necessary, together with refinement and rethinks on a regular basis. Do not assume that you will write the questionnaire accurately and perfectly at the first attempt. If poorly designed, you will collect inappropriate or inaccurate data and good analysis cannot then rectify the situation.

To refine the questionnaire, you need to conduct a pilot survey. This is a small-scale trial prior to the main survey that tests all your question planning. Amendments to questions can be made. After making some amendments, the new version would be re-tested. If this re-test produces more changes, another pilot would be undertaken and so on. For example, perhaps responses to open-ended questions become closed; questions which are all answered the same way can be omitted; difficult words replaced, etc.

It is usual to pilot the questionnaires personally so that the respondent can be observed and questioned if necessary. By timing each question, you can identify any questions that appear too difficult, and you can also obtain a reliable estimate of the anticipated completion time for inclusion in the covering letter. The result can also be to use coding and analytical procedures to be performed later.

Here's an excellent resource for creating questions for data collection.

Back to Data Collection Tools