Hypothesis testing is crucial in experimental research, and developing the research hypothesis is a major task. The researcher begins with initial ideas (theoretical concepts) and refines them into a statement of the problem, which is converted into the research hypothesis when operational definitions of variables are added to the problem statement. By combining the statement of the problem and operational definitions of variables, the researcher makes a prediction about the effects of the specific, operationally defined, independent variable on the specific, operationally defined, dependent variable. That prediction is the research hypothesis.
The research process begins with one or more initial ideas, which are phrased as questions. The researcher then searches the literature and refines, modifies, retains, or discards the initial ideas.
Problem statements for experimental research are in the form of questions that concern causality. They contain a good deal of information and we find in each: (1) a statement about an expected causal effect, (2) the identification of at least two major variables, and (3) an indication of the direction of the expected causal effects.
There are several criteria for a good problem statement: (1) It should state clearly the expected relationships between variables; (2) it must be stated in the form of a question; and (3) it must at least imply the possibility of an empirical test of the problem.
Constructs in research must be developed to the level of an operational definition in order to measure and manipulate a variable and to communicate to other researchers exactly what was done in the research. The operational definition specifies exactly how variables are to be measured or manipulated. Operational definitions make it possible to precede a step closer to formulating the research hypothesis.
A good research hypothesis: (1) makes a statement about the relationship among variables and (2) implies clearly that the relationship can be empirically tested. The research hypothesis is a declarative statement.
Theory is critical in developing the research hypothesis. Detailed theories guide the researcher's thinking, suggesting which research studies will uncover the most interesting and useful information.
The experimental research hypothesis is not a single statement, but actually encompasses three hypotheses, each of which must be carefully checked: the null or statistical hypothesis, the confounding variable hypothesis, and the causal hypothesis.
Most problem statements can be developed into several different research hypotheses, each of which can be tested. Thus, several different studies can be generated from the same problem statement. This can be done because the variables indicated in the problem statement can be operationally defined in several different ways. By doing that, we can develop more than one research hypothesis and, essentially, study many facets of the original problem statement. This gives us a degree of replication and produces a more thorough study of the problem.
A major concern in scientific research is the validity of the procedures and conclusions. Validity refers to the methodological soundness of the research, which is a concern at all levels of constraint. Many factors may affect the outcome of an experiment; some of these factors are likely to be threats to the validity of the experiment. Therefore, two of the major tasks of the experimenter are to anticipate potential threats to validity and create procedures to eliminate or reduce them. Threats to validity and their control are two sides of the same coin, and control over threats to validity is a major concern for the researcher.
Two threats to statistical validity are (1) unreliable dependent variable and (2) violations of statistical assumptions.
Construct validity refers to how well the study's results support the theory or constructs behind the research and asks if the theory that is supported by the findings provides the best available explanation of the results. Threats to construct validity can be reduced by using clearly stated operational definitions and by building the hypotheses on solid, well-validated constructs.
External validity is the degree to which we are able to generalize the results of an experiment to other participants, conditions, times, and places. Threats to external validity are best controlled by having an adequate, representative sample of participants.
Internal validity is of great concern for the experimenter because it involves the very heart of the experimental goals: the demonstration of causality. In an experiment, internal validity concerns the question, "Was the independent variable, and not an extraneous variable, responsible for the observed changes in the dependent variable?" An experiment is internally valid when we can conclude with confidence that it was the independent variable that brought about the observed changes in the dependent variable. Any factor that reduces that confidence is a threat to the internal validity of the study.
There are a number of confounding variables that researchers must avoid.
Many threats to the validity of a study are due to the expectations and biases of the participants and the experimenters.
Participants enter experiments with a variety of motivations, biases, and expectations, all of which may affect the outcome. Demand characteristics are characteristics of the setting and procedures that unintentionally suggest the behavior expected of the participants. A related phenomenon, the placebo effect, can occur when participants expect some fairly specific effect of an experimental manipulation.
Experimenters may carry their own biases and expectations into a study, which may affect the outcome. Experimenter expectancies are potential threats to validity and must be controlled.
Many of the most important threats to validity will be controlled if the researcher designs into each experiment the random selection of participants, the random assignment of participants to conditions, and a proper control group.
The researcher has an ethical obligation to conduct carefully planned research so that the research will be useful. Failing to control confounding makes the research of little value.
Some confounding variables can only be controlled by violating ethical principles. For example, to avoid selection problems, we would have to force all selected individuals to participate, which violates the principle of informed consent. We never violate ethics to enhance validity.