Graziano & Raulin
Research Methods (9th edition)
Composing Survey Items
Composing Different Types of Items
There are several types of items used in surveys.
Open-Ended, Low-Structure Items
An open-ended question allows respondents to answer the question in their own words. These items are best used in early phases of research when ideas are being sought for hypothesis development. They are less suitable for survey research, in which hypothesis testing is carried out and therefore greater item structure is needed. For example, a campus survey on students' evaluations of the campus security might contain:
How much confidence do you have in the campus police?
Structured Items
Structured items restrict the responses to a defined set of alternatives. Structured items can be presented in several ways. Here are two examples.
I have confidence in the campus police.
( ) Strongly agree
( ) Agree
( ) Uncertain
( ) Disagree
( ) Strongly Disagree
Compared with other departments at the university, would you say
the campus security department is:
( ) Excellent
( ) Good
( ) Fair
( ) Poor
A numerical scaling for items can also be used. This is illustrated in the following item. It is common in surveys to construct the kind of visual scale shown in this item.
On the following scale, where do you place yourself with regard to the campus police?
Complete No Confidence Confidence At All _______________________________________________________________ 1 2 3 4 5 6 7
Another approach is based on the semantic differential developed by Osgood, Suci, and Tannenbaum (1957). Seven-point scales are anchored in polar-opposite adjectives that describe the concept that is being evaluated. The respondents indicate the place on each scale where their evaluation falls by circling one of the vertical lines on the implied scale. The example below indicts how efficient this format is in obtaining ratings on a variety of dimensions quickly and easily.
The Campus Security Department is:
Competent | ----- | ----- | ----- | ----- | ----- | ----- |
Incompetent
Effective | ----- | ----- | ----- | ----- | ----- | ----- |
Ineffective
Helpful | ----- | ----- | ----- | ----- | ----- | ----- | Unhelpful
Necessary | ----- | ----- | ----- | ----- | ----- | ----- |
Unnecessary
All of the structured items illustrated so far have been implicitly or explicitly scaled. That is, there is an obvious ordering of the responses. However, structured items do not need to be scaled. People can be categorized on dimensions that are not based on a particular ordering, as shown in the following example.
My political affiliation is:
( ) Democrat
( ) Republican
( ) Independent
( ) Other
( ) I have no political affiliation
The examples illustrated above were all answered by giving a single choice. However, there are times when you might want your participants to make several selections from a single item. The following item illustrates this situation.
We want to know more about our customers and what they enjoy
doing. Please check each of the activities below that you engage in
on a regular basis:
( ) breathing
( ) eating
( ) thinking
( ) chilling out
( ) driving like a maniac
( ) living like a maniac
( ) studying research methods diligently
( ) lying about studying research methods diligently
General Rules for Writing Good Items
We have just described the various types of items typically used in a survey instrument. Now we want to focus on how to make each item that you write as effective as possible. Posavac and Carey (1980) suggest several good rules to keep in mind when writing items. We will illustrate each of these rules with good and bad examples of items.
* Always keep in mind for whom you are writing these items.
If you remembering who will be taking your survey, it will help you to keep the items appropriate for your intended audience. For example, if you are planning to survey fourth graders on homework, the language of the items must be appropriate for the average fourth grader. The following items represent appropriate and inappropriate language for a sample of fourth graders.
Bad Item
Do you find the volume and complexity of the homework assigned by your teacher excessive?
Good Item
Does your teacher give
___ too much homework
___ too little homework
___ the right amount of homework
* Avoid negatives and, especially, double negatives.
Negatives and double negatives are difficult to read and to understand. You especially want to avoid negatives when the response format is true/false. Rewrite any such items in order to eliminate the negatives.
Bad Item
( true / false ) I never felt like I would not be willing to support the proposed project.
Good Item
( true / false ) I support the proposed project.
* Do not mix issues in a single item.
The best items include only one issue or concept.
Bad Item
New faculty members are more demanding and less understanding than those with many years of experience. (Agree/Disagree).
[The two concepts, "more demanding" and "less understanding" are very different and make this item impossible for some to answer--i.e., they may believe young faculty are too demanding but nevertheless are more understanding. If both concepts are important, then two items are needed as shown in the good item example.]
Good Item
New faculty members are more demanding than those with many years of experience. (Agree/Disagree).
New faculty members are less understanding than those with many years of experience. (Agree/Disagree).
* Use good grammar.
Bad grammar makes items confusing, and it reduces the survey's face validity in the eyes of the respondent.
Bad Item
Check the statement that most describes your position.
___ Those guys are wrong headed.
___ I think the mayor is right on.
___ They was wrong to not back the governor.
___ I ain't got no opinion.
Good Item
Check the statement that best describes your opinion.
___ I support the proposal made by the governor.
___ I support the proposal made by the mayor.
___ I do not support either proposal.
___ I have no opinion on the matter.
* Make sure each item is logical.
Be sure that there are no logical errors in your items. Logical errors will confuse the respondent.
Bad Item
Which statement best describes your feelings about your job?
___ I am generally satisfied with my job.
___ I am generally unsatisfied with my job.
[Forgetting to make provisions for the fact that not everyone has a job in this item makes it impossible to interpret the answers obtained. Would someone without a job just leave it blank? Perhaps people who have jobs but do not have strong feelings about the jobs would also leave it blank? If one of the choices was checked, does that mean that the person has a job? Could those without jobs be dissatisfied that they are out of work and so they checked the dissatisfied option? What if someone has two jobs and is satisfied with one and dissatisfied with the other?]
Good Item
Which statement best describes your feelings about your job?
___ I am generally satisfied with my job.
___ I am generally unsatisfied with my job.
___ I currently do not have a job.
___ I currently have more than one job.
* Make sure each item is as concise as possible.
Bad Item
( true / false ) On balance, and given all of the factors that impinge on the proposed project and the possible consequences for future development, I think that this project is important and should go forward as proposed by the management consultants hired by the state.
Good Item
( true / false ) I am supportive of the proposed plan.