CP 6691 - Week 4

Group Comparison or Causal-Comparative Research Designs

Interactive Table of Contents (Click on any Block)

Purpose of Causal-Comparative Research
How to Identify This Type of Design
A Limitation of Causal-Comparative Research
Evaluating Causal-Comparative Research Studies
Evaluating Sample Study #14 (Summer Birthdate Children ...)

Assignment for Week 5

Purpose of Causal-Comparative Research

The purpose of Causal-Comparative research (also called Group Comparison research in your text) is to identify possible cause-and-effect relationships. Now why do you imagine someone would be content to discover possible cause-and-effect relationships? If you figure out the answer let me know, because I don't know. Researchers looking for cause-and-effect relationships are searching for actual ones. In other words, they want to be sure that the effect they find is really caused by the variable (treatment) being studied. To determine actual cause-and-effect relationships, researchers resort to doing experiments (something we'll talk more about in a later lesson.

So where does causal-comparative research come in? Well, it turns out that it's simply not possible to conduct an experiment every time a cause-and-effect relationship is being sought. Some situations are unethical, illegal, or unsafe to explore with an experimental study. Here's an example of what I mean.

Anyone who smokes (or used to smoke) is familiar with the warning labels on cigarette packages. A typical warning label says: "Caution, cigarette smoking causes lung cancer." But, do you remember what the first warning labels put on cigarette packages said? They said "Warning, cigarette smoking may be hazardous to your health." That's a far cry from the current warning labels. Why did it take 15 years or more before we could say with certainty that cigarette smoking causes lung cancer (cause-and-effect)? Let's imagine an experiment that would determine the actual cause-and-effect link between cigarette smoking and lung cancer. The experiment could be constructed like this:

We begin with 10,000 randomly selected newborn babies. We could stratify the selection to aid in generalization, but let's not get too complicated. Once we have the sample, we randomly assign them to one of two groups: the smoking group and the control (non-smoking) group. Parents of babies assigned to the smoking group are instructed to begin their children smoking at the age of 10 years, and to ensure they keep smoking a pack of cigarettes a day until they are 18 years of age (by then they'll be addicted and will keep smoking for the rest of their lives). Meanwhile, parents of the control group babies are instructed to do everything in their power as parents to prevent their children from smoking until their 18th birthday. Now all we have to do is to wait till the children have reached 18 years of age and give each of them a physical to determine which ones have contracted some form of lung cancer. If we find that the percentage of lung cancers in the smoking group is significantly higher than in the control group, we can be quite sure that it the lung cancers were due to smoking since the groups were equal in all other respects (because they were initially randomly assigned to the groups).

Obviously, this would be a most unethical (not to mention illegal) experiment. It could never be done. So, how did the National Institutes of Health determine that smoking causes lung cancer? Yep - you guessed it - causal-comparative research.

In causal-comparative research, the way researchers get around the ethical, legal, and safety problems is to use subjects who have already engaged in or encountered the treatment (independent variable) of interest on their own. In other words, the smoking study would use two groups of subjects, those who smoke and those who don't. So, the smokers would have already been smokers before the study began (we say they self-selected the independent variable). So, causal-comparative research always occurs after the independent variable has occurred. That's why this type of research is sometimes called ex post facto research (Latin for "after-the-fact").

How to Identify This Type of Design

As was the case with Descriptive Research designs, it's highly unlikely you'll see a purpose statement in a research report that says "The purpose of this study was to determine the possible cause-and-effect relationship between .... " Fortunately, however, there's an alternative method for identifying a causal-comparative design. There are really only two "markers" you need to look for:

  1. The researcher creates two or more distinct groups on the independent variable (usually one group gets all of the variable and the other(s) get(s) none, or one group gets more of the variable than the other(s).
     

  2. The independent variable occurred before the researcher did the study.
In other words, if you recognize that the researcher has created two or more groups, and if you can determine that the independent variable already occurred before the researcher did his study, then it's a causal-comparative research study. You should be able to see both of these "markers" in the smoking study described above. (Smoking would be the independent variable and development of lung cancer would be the dependent variable. These are the two groups formed in the study. Also, notice that the subjects' choice to smoke or not to smoke was made long before we, the researchers, did this study.)

Now you can easily distinguish between descriptive and causal-comparative research. Since descriptive research does not create groups, if you determine that groups are being formed by the researcher, it cannot be descriptive research (it must be causal-comparative); contrary-wise, if you determine that no groups are formed, then it cannot be causal-comparative research (it must be descriptive). Keep in mind, however, this only works because you only know two research designs. As we learn more designs, we will become more sophisticated in determining the design type. And always remember, if you can clearly identify the purpose statement, then you can clearly identify the research design type without worrying about these "markers."

A Limitation of Causal-Comparative Research

The biggest problem with causal-comparative research is that it is impossible to randomly assign subjects to the various groups. This means that there are several extraneous variables not controlled (eliminated) in the study. Any variables not controlled remain possible explanations for why the dependent variable came out the way it did. That is, uncontrolled extraneous variables are alternate explanations or possible causes for the dependent variable. That's why causal-comparative can only determine possible cause-and effect relationships and not actual ones. In the smoking study, for instance, we could not possibly control for other "possible causes" for developing lung cancer, such as heredity, work environment, home environment, level of personal stress, etc. These would have all been controlled if we could have randomly assigned subjects to an experimental (smoking) or control (non-smoking) group. Therein lies the power of the experimental research design, which we'll discuss more in a later lesson.

Evaluating Causal-Comparative Research Studies

When evaluating causal-comparative research studies, you should evaluate whether the two (or more) groups are similar except for the independent variable. Pay special attention to what the reseacher does to control for (reduce or eliminate) extraneous variables. Ask yourself what other variables could also possibly explain the effect being studied. Then check to see if the researcher addresses these other variables. If you find that important extraneous variables are not controlled by the researcher, then it weakens the study appropriately (because it leaves these extraneous variables as possible causes). Remember, the goal of the researcher is to control as many important extraneous variables as possible.

Typically, inferential statistics are used to analyze causal-comparative research studies (the same kind of statistics used in experimental research studies). See Table 11.4 on page 249 of the course text. Anytime inferential statistics are used, it is your responsibility to determine whether the researcher has used an appropriate (parametric or non-parametric) statistic. This information was presented in the Week 3 lesson on "Statistical Tools."  If you wish, you may review the lesson on determining appropriate inferential statistics by selecting the link. Either press your browser's BACK button or select the link (at the top of the page) that says Return to your previous location to return to this page.

As usual, you also need to evaluate the sampling method, the instrumentation used to collect data, and the validity of the research hypotheses, objectives, or questions.

Evaluating Sample Study #14
(Summer Birthdate Children:
Kinergarten Entrance Age and Academic Achievement)

1. What kind of research design is this?

This is a causal-comparative design because both markers are satisfied:

  1. Two groups of subjects are formed: Students who entered kindergarten at age 5 and students who entered at age 6.
  2. The independent variable (entry into kindergarten) had occurred long before the researcher did this study. This is obvious since the students are now in seventh-ninth grades.

2. What is the research hypothesis, objective, or question(s), or if none, so state.

A research hypothesis is stated on page 142, right-hand column, end of the first full paragraph. This hypothesis is restated on page 143, left-hand column, in the section labeled Design (second paragraph). Notice also that the researcher establishes an alpha level of .05.

3. To what population would you feel comfortable generalizing results of this study?

A relatively large sampling (253) of subjects from the seventh-ninth grades from seven public school districts in northwestern Ohio was used. We're not told whether the subjects were drawn randomly or not (so we assume not). But, of this potential sample of 253 subjects, only 45 pairs of subjects survived the matching process used to control for gender and IQ differences.  It's unlikely that 90 subjects would be representative of the 253 potential subjects.  Therefore, I would not feel very comfortable generalizing these results to any population.
 

4. Identify the strengths and threats to validity in this study.

Let's consider the strengths and threats by External and Internal threat categories. 

External Threat Categories:

1.  Population Validity - It appears at first sight that a large sample was selected ("253 potential subjects").  However, Due to the causal-comparative nature of this study, several extraneous variables were controlled which led to losing some of these 253 potential subjects.  As you read through the "Subjects" portion of the Method section, you find that the researcher matched subjects in each group (the five-year-old kindergarten entry group and the six-year-old kindergarten entry group) based on gender and IQ.  As a result of this, no all subjects could be matched and, thus, had to be dropped from the study.  The final number of subjects.  By virtue of the required matching that was done, the selection process for these subjects used in this study is given in the first paragraph on pg 143:  45 pairs of 90 subjects was non-random.  So, with a relatively small sample size (90 subjects out of a possible 253) and a non-random sampling method, population validity should be considered a threat in this study.

2.  Demographics - I can't think of any additional demographic data that should have been included in this study.  So, I'll conclude that demographics are a strength.  However, if you believe there is one or more variables not included in the study that would be important to understanding the results, then you should bring them up and indicate that demographics is a threat in your mind.

Internal Threat Categories:

1.  Instrumentation - Researcher used standardized tests so she only had to assess reliability on them in this study.  She did not assess the reliability herself in this study.  Rather, she reported reliability results from previous studies.  I find that to be a threat to instrumentation.

2.  Appropriate Use of Inferential Statistics - Researcher used a normalized standard score as a measure of achievement (rather than percentiles) which represent continuous data. Thus the parametric t-test used by the researcher to analyze data was appropriate. If you did not realize that the NCE (normal curve equivalent) score represents continuous data, that's ok.  In that case,  if you aren't sure what form the data being analyzed are in, you must use both sides of the rule.  You should say something like this:
 

I'm not sure what form the data are in.  If the data are continuous,
then the most appropriate inferential statistic to use would be
parametric.  If, however, the data are discreet, then the most
appropriate inferential statistic to use would be non-parametric. 
 

5. Are there any ethical problems in this study?

There are no ethical problems in this study because nothing is done to the subjects. Only their academic scores are examimed. The only ethical problems likely with causal-comparative studies are likely to relate to data collection.

If you have any questions concerning this evaluation (if you found things I didn't discuss here, or if you don't understand something I've discussed here), talk with other members of the class to see if you can resolve the issues with them. If not, discuss your questions with the instructor in class or via E-mail.

End of Week 4 Lesson
Assignment For Next Week
Gall: Chapter 12 (Correlational Designs)
SB: Study 15
Extra Evaluation Practice:  Try your hand at evaluating these scenario-based research studies (I call them mini-studies.)  For each problem, read the scenario (first page) and try to evaluate it using the 5 evaluation questions in the Typical Evaluation Quiz format we've been using above.  Don't look at the answers on the second page until you have answered all the questions yourself.  Then compare your answers with those provided in the problem.  If you have questions or don't agree with or understand the answers provided, E-mail me and let's discuss it.

Scenario Pack 1 (MS Word documents):   Scenario A, Scenario B,    Scenario C, Scenario D, Scenario E, Scenario F, Scenario G.

Due Next Week
Prepare for Evaluation Quiz 1