This is the screen cast tutorial for how to use the program G power to determine two things. First of all is a priori power, and that is computing the required sample size that you need to achieve a desired level of power for a study given levels of alpha and effect size. The other thing is to compute post-hoc power; that is, exactly what is the obtained level of power you have in a study given your sample statistics to estimate things like effect size as well as the alpha level that you use in your study. The nice thing about G power is, as you can see here, there are a variety of statistical tests that you can select from. We are not going to cover all of these this semester or last semester or even over the course of this curriculum, but there really is a lot of flexibility in the program G power for any designer you might come up with. There are different test families as well. Right now we are going to stick with just the t-tests, but as you can see F tests, which may be required for your upcoming project, are available in here as well in addition to things like chi-squared and D tests that we have done in the past.
Again, what we are going to do is stick with t-tests for now. Let's look at the first example. Consider what we want to do is to determine what sample size would be required to achieve the desired level of power for a paired sample's t-test. We know we are in the t-test family, and then we go over to statistical tests and find the test in which we are interested. In this case a paired sample, or what they are calling a matched pairs t-test, is indeed what we are looking at; the difference between two dependent means, so we select that test. As I mentioned, in G power all you have to do is fill in the different values you know and it will calculate the unknown value for you. In this case the first thing we want to do is to tell it what type of test we're looking at—a priori or post-hoc. Now you can see there are a lot of other options in here as well, but these are going to be the only two in which we are interested in.
For knowledge to the a priori test, which is going to tell us what is our required sample size to achieve a desired level of power. Then what we need to do is to let it know whether it is a directional or a nondirectional hypothesis by telling at the number of tails. We know that a nondirectional test is a two-tailed test; a directional test is a one-tailed test. Let's assume for now that we are going to be going with a two-tailed test. As mentioned in lecture all we then need to know are what our effect size, our alpha level, and our desired power and then what we can do is compute the sample size that would be necessary in order to achieve that level of power. The effect size here at 0.5 is a moderate effect size. We can simply change that value to anything we want. If we think it would be a little bit larger—0.06—if we think it would be a small effect size—0.02–0.08 for a large effect size. Those are conventional levels.
Let's just stick with a moderate effect size of 0.05. Our alpha level is indeed going to be a value of 0.05 as it is always going to be in this course and in behavioral science in general, and than what we can do is to say, "What is the level of power that we would desire?" Now 0.95 is a great level of power. Recall what this means is that if there really is an effect we want there to be a 95% chance of detecting this effect in our study. We might not want such a stringent requirement. Let's lower this down to something like 90%, and that is all it takes is inputting your effect size estimate D, your alpha level, and your desired power and then clicking on calculate. What it is going to produces a graph similar to the one that we saw in lecture as well where it is showing you the red distribution represents the null destination, the dashed blue distribution represents the potential alternative or research hypothesis, and you can see the shaded areas for alpha and beta similar to the way they were discussed in lecture.
What is going to be important for us than is looking at what is going to be the sample size that is required to achieve this level of power. The place you can find that is right here under total sample size, and we can see that we need a total of 44 participants in the study; that is, in a within-subject situation where we are comparing two dependent means we would need 44 people. Then, of course, we know that this would be a within-subject study where 44 people are participating in both conditions.
There are a couple of other things we want to look at here. In particular one thing we can do is to look at the X-Y plot for a range of possible values. If we want to do that then all you need to do is to click on the button at the bottom that brings up this plotting window, and you can change everything you want in here as well. You can change the effect size. You can change your alpha level, and you can set some other specific details as well. For example, say that what you want to see is a sample size required to achieve different levels of power ranging from 0.5 all of the way up to 0.95. Once you set the details and the values how you like them then all you have to do is click "draw plot" and then it will show you the values. Specifically, to achieve a certain level of power which is shown in the X axis say that we want to achieve power of 0.8 we can use this plot than to see how many participants are going to be required. If we read over on the Y axis it is going to show us about 33 participants are required total sample size given our effect size and alpha level in order to achieve that power of 0.8.
It is really that simple. There is a second example we can look at as well. Say that you don't have your effect size in particular. For example, what you may be doing is basing your effect size off of previous research. Say you have run one study and things didn't go so well, yet you have the data—the sample data—from that previous study that you can use to estimate effect size for an upcoming study, for example. Any situation like this where you want to use the sample statistics to determine your effect size can be done by clicking on this determined but here on the left. What that is going to do is open up this little side window or drawer. In here what you can do is calculate the sample statistics, and to them in directly, and then it will determine the effect size for you.
For example, we know that when we are calculating a paired sample's t-test we end up calculating a means different score and a standard deviation of that different score. If you come and select this button "from differences" and say that we have done a study where we find a mean difference of 2 and a standard deviation among different scores of say 5.5, then we can click this calculate button here at the bottom and it will do the effect size calculation for us. This is a pretty simple calculation, which I am sure you can verify using the calculator as well, but if you have these sample statistics handy this is a convenient way for it to calculate the effect size, and then by clicking the other button "transfer the main window" then you can see what it does is it copies the effect size into the main window of G power directly for you. From there everything else perceives as normal. You can simply click calculate to determine your total sample size necessary. In this case it would be easy to. You can see as the effect size went down from 0.5 to 0.36 the sample size required went up to maintain the same level of power. Once again, you can click for the X-Y plot to show the entire range of values.
Let's look at one other type of test, the independent sample's t-test, because all of the logic is the same, and am sure you can figure it out for yourself but it is handled a little bit differently, but specifically this is going to be the differences between means of two independent groups. This then is the independent sample's t-test in G power. So clicking on here, again what we may still want to know is what is the sample size that is going to be required to achieve a specific desired level of power? Again, everything is the same. Let's look at the same effect size of 0.5, alpha 0.05, power again of let's say 0.90—just changing the value in the boxes here. The only other box that is on here is the allocation ratio N2 to N1. Specifically, what we are going to strive for in an independent sample's test is to have equal sample sizes in the two groups. If that is the case, then the ratio of sample size between the groups is going to be equal to 1, but it may not always be the case. For now let's assume it is, and if it is you can simply calculate and then it is going to show you the sample size in each group—70 and 70—for a total sample size of 140.
Now if you have a different allocation ratio—let's say that you have different numbers of people within two different groups. Say that it is an allocation ratio of 0.75. This would be an example if you have 30 people and one group and 40 people in the other group. Then, the number of people and one group—30—over the number of people in the other room—40—would equal ¾ or 0.75. In this case you still just click calculate, and again it is going to show you the sample size that you need which you can see is 142 here and then how they are going to be allocated across the two different groups. You may notice that the total sample size necessary in this case has gone up a little bit. This is an important point and an independent sample's t-test that you actually achieve the highest level of power if your sample, your total sample, is allocated evenly across two groups.
Another thing that we can do here is to think about using the determine window in order to enter the sample's statistics just like we did for the paired sample's test. Now in this case, again, what we may want to do is just enter the sample data. Say we've collected some data and we find the mean of the first group is 55, the mean of the second group is 45, standard deviation from one group is 6, say the other groups 7—we can use this to calculate our effect size here as well, transfer it to the main window which it has done and then complete the calculation for us. In this case, of course there was such a large effect size we see that we need relatively fewer people in the two groups.
If we do not have equal effect sizes, which is the top button here in this calculation or determination drawer, than what we may do is to enter in the mean of the two groups. Again, let's say its 55 and 45. Them what it asks for here is the standard deviation within each group. It is important to know what this is going to be is essentially the pooled standard deviation; that is, the square root of the pooled variance estimate that we know how to calculate. Let's say that is something like 6.2, just to select a number. Again, we can calculate the effect size, transfer it to the main window, and then calculate the sample size necessary to produce the desired level of power. Once again, with such a large effect size, the sample size that we need in each groups relatively pretty small.
Finally, let's look an example of how we can then calculate post-hoc power. Remember, this is going to change a little bit because what we are looking at finding now is not for a given level of power what is our sample size but, given our sample size, what is our achieve level of power? For this we are just going to stick with independent sample's t-test and doing it for the dependent test is very similar, so I am not going to go back through the motions there. Again, what you can see is that it has happened to retain the values that we calculated in our last analysis. This may or may not be the case, but what you can do in this case especially, because you are doing this post-hoc, you are typically going to have exactly the values that would go in these boxes over here. Just to change the numbers to produce a different example, say we have one class that gets a 75 on average on an exam and another class gets a 72 average. We have the standard deviation for the exam scores in each group. If that is the case, again we can just calculate the effect size. These are going to be values that come directly out of your analysis either through SPSS or Excel or whatever you are going to calculate them on. Once again, all we have to do is transferred to the main window and now then we are also going to know the sample we had in each group. Let's say that we had 18 people in the first group and 20 people in the second group. In this case when we calculate what it is going to show us is the calculated—the achieved—level of power, in this case rather low. It is 0.27. You can see that in terms of how we talked about it in terms of lecture especially with the overlap here of the alternative distribution, specifically how much of it is to the left of our critical value which is shown in green.
That's the last example that I wanted to go through. What we have seen now is how to do two things; how to calculate a priori power, that is for a given desired level of power what is the sample size necessary to achieve it, and post-hoc power—given all of our sample statistics, that is the values that are associated with our exact experimental design or R study that is our sample means and standard deviations as well as sample size what is the obtained level of power that we have achieved within our study? This should be enough to prepare you to not only complete the homework given the effect sizes and other calculations as well as the values that are provided in the homework problems themselves but it is also going to start familiarizing you with G power so that you can do a power analysis for your own upcoming project. Again, the basic logic is the same even though exactly the boxes you click on her going to be slightly different for your study depending on the exact nature of the design, but that is something your lab instructors can walk you through as well depending on whether or not, for example, you have a factorial design or a situation where you have a single independent variable with multiple levels so forth and so on.
Thanks for tuning in today, and good luck with the homework.