Field research includes low-constraint naturalistic research, which was discussed early in this text, and experiments and other high-constraint research conducted in natural settings, which are discussed here.
There are three major reasons for conducting experiments in field settings: (1) to test the external validity of causal conclusions developed in the laboratory, (2) to determine the effects of events that occur in the field, and (3) to improve generalization across settings.
We need to conduct experiments in field settings so as to be able to draw causal inferences, but doing so can be difficult. The essential question is: "How can we answer causal questions in natural settings when we cannot utilize many of the usual manipulations and controls of the laboratory?"
Flexibility and creativity are important in research. Researchers listen to their hunches, flashes of insight, flights of creativity, and alertness to interesting, unanticipated events. All can open new directions to be explored, especially in field research. However, for the student it is better to learn the fundamentals before taking such free-flight leaps.
In some situations, we cannot meet all of the demands of an experiment, but still want to draw causal inferences. We can use quasi-experimental designs in those situations, which allow us to draw causal inferences, although not with high confidence. However, they still give us more useful information than not experimenting at all. In quasi-experiments:
Often in field research we must compare already existing groups. But this does not allow random assignment or matching, and the groups may not be equivalent at the start of the study. However, the groups might be similar on most characteristics, thus allowing us to draw causal conclusions, albeit with less confidence.
There are two major problems in using nonequivalent groups: (1) The groups may differ on dependent measures at the start of the study, and (2) there may be other differences between the groups that have not been controlled by random assignment. To address the first issue, a basic strategy is to measure the dependent variable both before and after the manipulation. Then, following the manipulation, the difference scores between pre-manipulation and post-manipulation scores are taken as an indication of the effects of the independent variable. To address the second issue, we carefully examine the procedure and try to rule out confounding factors.
In interrupted time-series designs, a single group of participants is measured several times both before and after some event or manipulation. Time-series designs have two potential confounding factors: history and instrumentation. The interrupted time-series design can be improved by adding one or more comparison groups.
The task in program evaluation is to evaluate how successfully a program is meeting its goals. This has obvious practical value in social services and health matters.
There are many serious practical constraints on program evaluation. These include ethical as well as technical issues.
Surveys gather information by asking participants about their experiences, attitudes, or knowledge. Survey research is not a single research design but, rather, uses several basic procedures to obtain information from people in their natural environments.
Status surveys describe the current status of some population characteristics. An example is a status survey to determine what proportion of current voters are Republicans, Democrats, or Independents.
Survey research is more complex, seeking not only the status of characteristics but also attempting to discover relationships among variables.
Listed below are several steps in the survey research process:
Determine what area of information is to be sought.
Define the population to be studied.
Decide how the survey is to be administered.
Construct the first draft of the survey instrument; edit and refine the draft.
Pretest survey with a subsample; refine it further.
Develop a sampling frame and draw a representative sample.
Administer the final form of the instrument to the sample.
Analyze, interpret, and communicate the results.
The instrument can be a questionnaire or an interview schedule, a mailed survey, or a group-administered questionnaire. Surveys begin by explaining their purpose and the questions then fall into two main categories: demographic (e.g., respondent’s age, gender, occupation, education, etc.) and content (respondent’s opinions, attitudes, knowledge, behavior, etc.).
The survey researcher should identify the general area to be surveyed, determine what questions are to be answered, construct the survey instrument, and determine the procedures for its administration. The items must cover the area of information being studied and be written in language appropriate for the intended respondents. Items can be open ended, multiple choice, or Likert scales.
The population is the larger group about which we want to gain information. Obtaining an adequate sample from that population is one of the most important factors in conducting surveys.
The researcher must also determine the research plan or design. Two basic designs are used in survey research.
Most of the research designs discussed in this text have been cross-sectional designs. Some of the designs discussed in Chapter 11 and many of the designs discussed in this chapter represent longitudinal research. In cross-sectional research, each participant is measured at only one point in time, while in longitudinal research, each participant is measured at more than one point in time.
One must be especially sensitive to ethical issues in program evaluation research. Programs are designed to meet people's needs, and it is critical that people in the programs not feel obligated to be involved in the evaluation. If they do, their participation is coerced.