Okay, welcome to our next lecture. This lecture marks our final test of significance, so we will have completed an introduction to some of the basic ones. Pat yourself on the back, we are almost done! In this lecture, we are going to cover our final test of significance called chi-square.
Up until now, our dependent variable has always been quantitative. It's been some quantifiable variable that you could directly measure. Now we're going to move to a categorical dependent variable, so both the independent and dependent variable will be categorical, so it will be items that you can count rather than items that you can directly measure.
If you remember in ANOVA study, we looked at ACT scores from 3 types of high schools: rural, urban, and suburban. In ANOVA study, the dependent variable was quantitative. We took the mean ACT score and compared them for those 3 types of high schools. What if we took the ACT scores and said, "If you got a 20 or lower, we're going to call it a low ACT score and maybe code that with number 1. If you got a 20 or higher, we're going to call that a high ACT score, and we're going to give that a code of 2. Then we can just simply count how many low and high ACTs for rural high schools, suburban high schools, and urban high schools. Chi-square can tell us whether those counts significantly differ from one another.
The chi-square, that test significance, is often called a ‘goodness of fit’ test because we're comparing counts. What are we comparing? We're comparing the actual number to what the expected number would be, so we're taking those two counts and comparing them to one another.
In chi-square, like I said, we're comparing actual to expected, and establishing expected can be a little tricky. We need to talk about that a little bit. First thing, the easy way you can do it is just to consider all categories equal. In the rural, suburban, and urban high school ACT example, we can just make the assumption that the number would be the same. We can also compare the actual number to an expected that we get from some type of outside source, so one example might be some type of national norm. If we know nationally how many type O blood types there are, we could compare our number to that if our variable was blood type; or, we can do some type of past experiences. I think it's done a lot in smoking studies, where you know it's a categorical dependent variable. You're either a smoker, or you're a non-smoker, and you can compare that number, let's say entering college freshman, the number that smoke versus the number that don't smoke, compare the number in your current entering freshman class to the number ten years ago or five years ago. Or, you can compare it to a state school versus a private school or students in Illinois versus students in Iowa. Establishing expected, we've got to spend some time thinking about how we're going to define "expected."
There's some important assumptions with chi-square we need to review (three important assumptions.) The first is (I've already really talked about this) you've got to deal with frequency data. If there's a way that we can convert if we get mean scores, or maybe some way that we can convert that mean score, the ACT I gave you as an example, creating a cutoff and saying this person has a high ACT, this person has a low ACT. You have to create cutoffs or hopefully there's some type of logical cutoff that makes sense. I think in the book they use the example a dependent variable was age and the logical cutoffs they used were grade school, junior high, high school. They made three categories and just put them whether they were in the age that would be in a grade school student, a junior high student, or a high school student. You have to sometimes convert frequency data to categorical data.
The second assumption is that you have an adequate sample size. The picture here is a picture of a chi distribution, or a chi-squared distribution. It is a skewed distribution. If you think about it, that kind of makes sense. If we have observations you can't have negative observations, so it does not run into—the left side of the normal curve really doesn't exist, because you can't have negative observations. The book talks about establishing a set sample size. Some books will say you have to have a minimum of 10. I'm not too worried about having you guys memorize a number. All the cells need to be filled, so if you've created three categories, you have to have members in each, and small cell representation is a problem. I think 10 is probably a good rule of thumb, but individual situations you might have different. I think one text book suggests that if you have a sample of over 100, that at least 20% fall into each category if you have 3 categories. Just remember that small sample sizes can be problematic because we can't—you've to get a large enough observation that you can't fall into a potential negative observations because they're just nonsensical. Just remember that adequate sample sizes are an important assumption.
The final assumption is that observations are made independent of one another. That can mean really in chi-square one of two things. I'll start with the second one. Subjects cannot influence one another, so if they're participating in a study, we can’t have information that they're talking to one another. Hopefully, ideally, these subjects would not know one another or if they did, we would just ask them during the study not to talk to one another about what their experiences have been that possibly could contaminate your study. The other assumption related to independent observations is that each subject must only be counted as a single observation. I think the book used the example. If you're counting admissions to a hospital, and let's say you're collecting data over a 6-month time, if a person is admitted twice to the hospital, you can't count that person twice. You've got to make sure you are not counting one person twice. The example I can tell you about, we did a study once of counting people who are walking and talking on a cell phone at the same time to see if there are gender difference, so we counted men and women. I sent a class of students out on our college campus to count men and women who are on the phone to see if there were more men walking and talking or more women. It's possible that a person—let’s say they're walking from one part of campus to the other. That's possible that two of my students could have counted that same subject twice. What we did to deal with that is to count them. They had to take their picture, use their cell phone to snap a picture of them, then we looked at all of the photos and made sure that the same person and we didn't have two identical photos or the same person was not counted twice. Independent observations can talk about subjects influencing each other, but really more common in chi-square is to make sure that the same subject is not counted twice.
Let's look at an example. This is an example of a study where two PT clinics want to compare to see whether they have similar age groups visiting their facilities during a typical week, so let's take a look at our data.
Now for our data, we have 2 different types of clinics, we'll call them Clinic A and Clinic B, and then we have divided our dependent variable. It's a quantifiable variable, age, so we cannot compare the mean age. We put those ages into 3 categories: under 35 (so we'll call them young); middle age is 35 to 54, and then over 55 we'll call geriatric. Then you'll see how many of those visited those 2 clinics during a particular week. We can observe differences in the numbers but what we want to know is if it's statistically significant.
Here's an example of our SPSS printout. We have our observed number in our three categories. Then have our expected (and I'll show you in our screencast how you can enter the expected number), but it generates an expected number and then we get a residual. Below that is our actual chi-square test, and you see our significance is 0.001 so we can say there is a significant difference in that breakdown. Now, you might be thinking to yourself, "Can we do a post-hoc test, like with ANOVA to see where the difference is? Are there certain differences that aren't significant and other differences that are significant?" You can't do that with chisquare. You see we have a residual there in the third column. We can talk about the highest residual. The highest residual was in the middle age group, closely followed by the young group and then very little difference in the 55 or over group. We can talk about which residual was highest but we cannot do a post-hoc test to say certain differences were or were not significant. We can just say the overall difference in counts in this case was statistically significant at the 0.05 level.
If you would like to go through this presentation again, simply click the Replay button, and you will be returned to the beginning.