Overview
In a pretest-postest design, a sample is randomly assigned to two or more groups (usually one or more treatment groups and one control group); Subjects in each group are measured at two time periods: pretest (before treatment) and posttest (after treatment). Subjects in the same group receive the same treatment. The treatment, especially for a control group, may be a placebo or no treatment at all.
We consider multiple designs for the simple case of a single treatment group and a single control group, as shown in Figure 1.
Figure 1 – Pretest-Posttest Data
T-test of the Gain
One approach is to simply compare the gain after treatment for the two groups. Here the gain is the difference between the posttest and pretest scores. This can be done by using a two-sample t-test based on the data in range E4:E8 vs. E9:E13, as shown in Figure 2.
Figure 2 – t-test on the gain
With p-value = .003, we see there is a significant difference between the Treatment and Control groups.
Note that the study as a whole depends on the fact that there isn’t a significant difference between the Treatment and Control groups in the pretest phase. This can be checked by running a two-sample t-test based on the data in range C4:C8 vs. C9:C13, as shown in Figure 3.
Figure 3 – t-test on pretest data
The p-value = .58 confirms that there is no significant difference between the Treatment and Control groups, although there is a larger medium-sized effect (d = .36) with a non-negligible difference between the group means (19 vs. 35).
Repeated Measures ANOVA
Another approach is to perform a repeated-measures ANOVA with one between-subjects factor (treatment and control) and one within-subjects factor (pre- and post), as shown in Figure 4.
Figure 4 – Repeated measures analysis
Since there are only two between-subjects levels, we don’t need to worry about sphericity. For this analysis, p-value = .002875, which again shows that there is a significant difference between the Treatment and Control groups. Note that it is the p-value of the Interaction effect that is most relevant. Also, note that this value is identical to the p-value shown in cell K13 of Figure 2. This will be so for any similar analysis.
ANCOVA approach
Click here to learn about the ANCOVA approach to pretest-posttest design.
More than two between-subjects groups
Click here to learn more about pretest-posttest design with more than two between-subjects groups.
Reference
Dimitrov, D. M. and Rumrill Jr., P. D. (2003) Pretest-posttest designs and measurement of change
https://content.iospress.com/download/work/wor00285?id=work%2Fwor00285
Hi Charles my pre test post test research involves a health education intervention for about 500 individuals. The outcome is binary uptake of vaccine (y/n) in the same group of participants. Should I use a paired t test, do I need a regression analysis
Edith,
Does this mean that your data looks like the following (for 10 rows instead of 500) where 1 = yes and 0 = no?
Pre Post
1 0
1 1
0 0
0 1
0 0
1 1
1 0
0 1
0 0
0 0
Dear Charles,
Thank you for the informative descriptions. For my research I have a pretest-posttest control group experimental design. The two groups (experimental and control) were either exposed to material that was expected to change their attitude or keep the attitude neutral. Both their attitude was measured before the material and after the material. The change in attitude is expected for only the group that saw the material expected to change their attitude (H1). Furthermore, I measured two demographic features which I expected to have a strengthening/positive impact on the relationship between the independent on the dependent variables (moderation effect). Which test would be best to use for analyzing?
Kind regards,
Vivian
Hello Vivian,
If you are asking about the moderating effect then you probably want to use regression. Also see
https://real-statistics.com/multiple-regression/moderation-analysis/
Charles
Hi Charles,
I am doing a control group and intervention group quasi-experimental design, each group will have a pre-test and a post-test. I did not pair the pre-tests and post-tests. What would be my best way of analyzing this data? The pre-test and post-test are the same test for both the control group and the intervention group, an 8-question, 5-point Likert scale. I appreciate your assistance.
Hi Stacy,
1. Please explain what you mean by “I did not pair the pre-tests and post-tests.”
2. What hypotheses are you trying to test?
Charles
Hi Charles,
What could I do if I have two groups (control and intervention) and I am doing pre-test/post-test and wanted to look at within group and between groups differences?
Would I still do paired t-test? My dependent variable is PSS (perceived stress scale) scores and independent variable is enrolment in a meditation-based stress reduction course.
Hi Gagan,
You can perform a paired t-test for the intervention group (or two t-test for the intervention and control groups). This would test whether there is a significant difference pre and post. It would not test the intervention vs control group.
Charles
I have a question.
I am conducting a study involving 2 pretest and post test measured on the likert scale with 30 couples. I will be evaluating the effectiveness of changing three non verbal behaviors to
see if doing so will improve feelings of trust (test a) and security (test b). I am thinking ANCOVA is the most appropriate measure. What are your thoughts?
Hi Mel,
Can you you explain in more detail what you mean by “2 pretest and post test”.
Charles
Hello Charles,
I’m writing in hope that you could provide some guidance please. My current study is to evaluate the effectiveness of a new training programme for staff, which aims to reduce physical restraint used on patients. This is a pretest-posttest without a control group, whereby all staff had completed the new training programme. There are no participants in my study, as I am using secondary data that is available. The data that was collated: 3 months prior the training (Feb 2022 to Apr 2022) and 3 months after the training (Mar 2023 to May 2023).
The dependent variables are: the frequency of physical restraint (per month), the average duration of restraint (in minutes, per month) and the number of injuries reported by patients (per month). Based on previous studies, my hypotheses are that there will be a reduction in frequency, duration and injuries reported, after all staff have completed the training.
I am unsure whether independent t-test is the appropriate analysis to use (as I think the data is continuous), however I have read other research papers which have used chi squared instead.
Looking forward to hearing from you.
Best regards,
Kathy
Hello Kathy,
Generally, if you had one dependent variable, you would use a paired t-test provided the assumptions were met. Since you have multiple dependent variables, you could perform multiple paired t-tests (reducing the significant level to take experimentwise error into account). If you think there is some correlation between the dependent variables, it is probably better to use a paired Hotelling’s T-square test.
Charles
Hello Charles,
That’s been really helpful, thank you.
Best regards,
Kathy
Hello Charles,
I am doing research. I have two questions:
1) I have taken 200 participants, in which I am looking pre-effect of medicine on different health factors (10 continuous variables related with health) and post-effect of medicine on same health factors (10 continuous variables). to check the mean difference of all the health factors, May I use multiple measure ANOVA in this situation and evaluate each health factor separately (pre- and post-effect of medicine?
2) I would also want to check the association of this pre- and post test results with number of other independent variables (categorical variables). DO I need the mean of each health factor reading and then run multivariate regression? or tell me any other method.
Hello Seen,
1) If you have multiple dependent variables pre- vs post- you could use multiple paired t-tests (pre/post), but then you need to take the familywise error into account (since you are doing multiple tests). You could start with a paired Hotelling’s T-square test which seems to be a good fit for your situation. If you get a significant result, you can then use a follow-up test. Depending on what hypotheses you are trying to test and the nature of your data, this may in the end result in essentially multiple t-tests.
2) This sounds like multivariate regression, but I would need more information to say for sure.
Charles
Many thanks Charles!
I would use multiple paired t test as it seems appropriate for this data.
If I want to use this design, can I use a paired t-test to compare the pre-test and post-test results of the experimental group that received both a control and independent variable to see if both variables together improved their performance while also conducting an independent t-test on the post-test results of the control group (who will receive the control variable alone) and the experimental group (who will receive the control and independent variable) to isolate the independent variable to determine if that alone had a significantly contribute to the enhancement of the results. Is this a valid approach?
Hi Jasmin,
Yes, this is a valid option. It is the approach that I used in a research paper I wrote a couple of years ago.
Charles
Thank you.
Another option I have is to do a post-test only true experimental design in which the same concept is used, but only using the independent t-test to measure the results of the post-tests across the 2 groups. In this situation, I would still have the same control variable only control group and control variable plus independent variable combination experimental group. In this design I would still isolate the independent variable, but I would do it to test 2 differing forms of the control variable (one which is regular and the other which is an experimental version adding the independent variable) on the dependent variable (the post-test). Would this have any advantage over the previous design I mentioned, and which would you recommend? Note that I will be testing human behavior and people’s ability to develop a certain skill.
Jasmin,
Sorry, but I don’t understand the scenario that you are describing.
Charles
Hi
I have baseline and endline results from different study groups on knowledge about RH.
Baseline Endline
In-school intervention group 43% 94%
Out-of-school intervention group 50% 96%
Comparison/Non-intervention/ group 48% 85%
Based on the above information, how do we test whether the significant or not? NB. the subject at baseline and endline are different
If the subjects in the baseline and endline groups are different, then you don’t have the usual pretest/posttest situation.
If I understand correctly, you have 4 groups: In-school & no knowledge about RH, In-school & knowledge about RH, Out-of–school & no knowledge about RH, and Out-of-school & knowledge about RH. You likely need to use some version of ANOVA, but the design depends on the specific hypothesis or hypoetheses you want to test. E.g. you can use one-way ANOVA to test whether there is a difference between the 4 groups and then Tukey HSD to get more details. Another approach is to use two-way ANOVA with In-school vs Out-of-school as one factor and no knowledge of RH vs knowledge of RH as the other factor.
Charles
hello professor
I am preparing a proposal for my doctoral thesis and I don’t know how to analyze my results. The topic is as folllow
I am using a mixed-method quasi-experimental design with pre-test post-test control group and I want to see the effect of the Flipped classroom on students’ metacognitive development and writing performance.I am going to deliver a writing test twice as well as the Metacognitive scale to see whether there is a significant difference before and after the treatment.
please would you suggest any statistical measurements for my case
Any of the approaches described on this webpage might be appropriate.
Do you specific doubts or questions about these approaches_
Charles
Thank you
but which one is more appropriate for my research objective?I am really confused
Hello Mery,
If you only want to see whether there is a significant difference before and after treatment, then the paired t test would be sufficient (provided the assumptions of the test are met). I would need more information to say more.
Charles
hello professor
In fact, I HAve two groups, a control, and an experimental group. I want to see whether the experimental will score better than the control group in the writing test as well as in metacognitive strategy use in other words whether the flipped model will consolidate students’ usage of metacognitive strategies, whether they will jump for example from the tactic level to strategic level. for the instrument, I will be having pre and post-writing tests, pre and post-metacognition questionnaires, and lastly an interview
Any of the approaches described on this webpage might be appropriate.
Do you specific doubts or questions about these approaches?
Charles
Hi , I am doing Pre and post study of the training in the community level .i.e with the same population group. I have the same multiple choice question for the pre and post test. what statistical analysis should i use to compare these same groups (before and after the training). My sample size is 100.
Hello Royan,
Are you trying to compare the pre with the post for one group or do you have multiple groups (e.g. compare men vs women)?
You can compare pre vs post in one group by using a paired t-test provided the normality assumption is not violated.
If you have multiple groups, the situation becomes more complicated.
Charles
Mr. Charles,
Which design do you recommend for the following, Experimental Pretest/Post test three treatment groups and 3 control groups?
RQ1. How does Test A and Test b scores compare across Subject 1, Subject 2, Subject 3 teachers that implemented x within instruction and those that did not?
RQ2.How do Subject 1, Subject, and Subject 3 teachers rate on their overall knowledge of X?
RQ3.How does Subject 1, Subject 2, Subject 3 teachers rate on their implementation of x compared to their knowledge of x?
Hello Arie,
Please clarify the following:
What is the relationship between the 5 groups and tests A and B?
What is the relationship between the 5 groups and Subjects 1, 2, and 3?
RQ1, RQ2, and RQ3 don’t seem to refer to pre- and post-tests. Where is this information used?
Charles
the focus is teachers implementing a set of strategies within instruction to see the average difference in scores between students that receive the treatment and students that do not.
A benchmark assessment will be administered to all students across groups. One set of students will receive instruction with the inclusion of the strategies and one set of students will not. Students will then take their end of unit assessments. This same process will be done for Unit 2 assessment (Benchmark, Instruction, and end of Unit assessment)
The problem is poor teacher implementation of these set strategies due to a lack of in-service training which is negatively impacting underperforming students in a high school setting
My position is if teachers receive intensive support and coaching on the implementation of set strategies student achievement will improve.
Sorry, but I don’t know how to answer your questions.
Charles
Hello Charles,
I am doing a study where I have 3 different conditions but no control condition that receives no treatment at all. In this study, the opinion on a Concept is tested before the treatment and then additional information is presented (1=Goals, 2=Progammes, 3=Goals+Programmes). After that, the same questionnaire/test is administered. I have 138 participants (46 each). Which statistical analysis approach would you recommend?
Best Regards,
Lia
Hello Lia,
I understand that you have three groups, each with 46 subjects distributed randomly. All the subjects are tested on their opinion about a Concept, and then subjects in each of the three groups are presented with the additional information (1, 2, or 3) specific to their group. What test to use depends on the null hypothesis that you are trying to test. E.g. if you want to see whether there is a significant difference (post-score minus pre-score) between the three groups you can use one-way ANOVA.
Charles
Hi Charles,
I am writing a research proposal and am not sure what to use to analyze results. I am using a pretest posttest control group design, with one group receiving EMDR therapy for 12 weeks, the other group receiving traditional therapy for 12 weeks. I will be testing prior to treatment, during treatment, and after treatment. Thanks for you help,
Talia
Hi Talia,
Any of the three approaches described on this webpage can be used. If all you want to do is compare the pretest with the posttest then the first approach (essentially a paired t-test) is the simplest. If you want to make sure that the pretest and posttest samples are not biased as well then the repeated measures ANOVA and ANCOVA approaches have merit.
Charles
Hi,
I had a set of 40 students and made a control group and experimental group of 20 each. Conducted pretest of both group
Then used traditional teaching on control group and technology-based teaching on experimental group. Then I conducted a posttest and collected the 40 results.
Please explain which statistical analysis needs to be done
Hi Kavitha,
What tests to perform depends on what hypotheses you want to test. Some possibilities include:
1. Use a t-test to compare the pretest results of the Treatment vs Control group to make sure that you randomly split the two groups. You expect no significant difference between the groups.
2. Use a t-test to compare the posttest results of the Treatment vs Control group to see whether traditional teaching is significantly different from technology-based teaching.
Charles
Thank you so much Mr Charles for this very useful guide.
From what I understood in your explanation, I will be using the “Repeated Measures Anova” to analyse the result of a set of farmers I trained. I conducted a pre and post training test which will be used as the variables while the treatment will be their level of education (Graduate and Non Graduates). I hope I am on the right lane.
What if the treatments are more than two, and the training was held in up to 4 locations. Can I still use the same “Repeated Measures Anova” ?
Best Regards
Adebisi
Yes, you can use Repeated Measures Anova in this case.
Charles
Dear Mr.Zaiontz,
I’m writing to ask a question.
I’m studying teaching English in university. I’m a MA student. I have to present proposal.
My subject is : The effect of flipped classroom on EFL learners’ self-regulating.
I’m wondering if you help me , which design is suitable for this subject. I chose quasi experimental, but I think isn’t good because I don’t need post-test .
I’m looking forward to hearing from you as soon as possible.
Respectfully yours,
Azadeh. Harandi
Hello Azadeh,
What design to use depends especially on the hypothesis or hypotheses that you are trying to test and the nature of your data.
Let’s start there. (1) What hypothesis are you trying to test? (2) How are you measuring the effect that you are trying to study?
Charles
Thank you for providing an explanation of how to run and how to interpret a pre-test and post-test design. I will be using this basically in biological research for growth measures in determining the effectiveness of treatments on plant growth.
Glad I could help.
Charles