Lab: Understanding Experiments

Does random assignment work?

A true experiment is a research design in which subjects are divided into treatment and control groups randomly (as we will do in this excercise). The treatment group is given an intervention of some type while the control group does not get the intervention. An example of a treatment might be a math tutoring program for high school students. The treatment group would get tutoring. The control group would not get tutoring. The outcome measure might be scores on the Regents math test.

Experimental design is considered the strongest design for testing cause and effect relationships because (it is claimed) random assignment insures that any differences in the outcome variable are not due to differences between the treatment and control groups. That is, any relationship between the intervention and the outcome variable is not spurious.

In this lab we will see whether randomly assigning people to two groups really creates two groups which are almost identical on all of the background variables. 

Hypothesis: Groups that are created by random assignment will be virtually identical on all characteristics.

Go into SPSS and open GSS96. We will repeat the following steps 5 times. Fill in the table below as you go along. Please do not print.

1. Randomly divide the sample into two groups:

• Click on Transform and then compute.
• Create a new variable called group by typing GROUP in the box underneath Target Variable.  
• For Numeric Expression you wnat to either type RV.BINOM(1,.5) or go down the list of functions and select RV.BINOM(n,p) and type in the 1 and the .5.
Click OK. This will divide the sample into two groups: 1s and 0s. If you wish, you may go to "variable view" and create value labels for the variable group. For example 1=treatment 0=control.

2. Get the number of 1s and 0s by running a frequency on the variable GROUP.

3. Find the percentage male and the percentage Black in the two groups by running a crosstab with GROUP as the independent (column) variable and SEX and RACE and another variable of your choice (not age, educ or any variable with a lot of attributes) as the dependent variables (make sure to ask for the correct percentages). For your variable, report the percent with one of the attributes (responses).

4. Find the mean ages and years of education of the two groups by using Analyze, Compare Means, Means. Use GROUP as the independent variable and AGE and EDUC as the dependent variables.

Overall, how well do you think that assigning people to  groups using random numbers works at creating groups that are similar to each other? Why (give evidence from your table)?

Was the hypothesis supported?

In your own words, why is experimental design is the strongest design for testing cause and effect relationships?