MANOVA Follow up using ANOVA

Basic Concepts

Since there is a significant difference in mean vectors between the groups for Example 1 of Manova Basic Concepts, we would like to better understand where this difference lies. The natural first step is to see whether there is a difference between the groups for any of the dependent variables using ANOVA.

While this is the natural first step, it isn’t usually the best option since the whole idea of using MANOVA is to capture the correlations between the dependent variables, something which is lost when performing ANOVA on each of the dependent variables.

Example

For Example 1 of Manova Basic Concepts, we can perform the ANOVA tests using the standard Excel ANOVA: Single Factor data analysis, but it is much easier to use the Multiple Anova option in the Real Statistics MANOVA data analysis tool (see Figure 2 of Manova Real Statistics Support). You should normally keep the default alpha value of .05. A Bonferroni correction will automatically be applied, and so the alpha value for each ANOVA test will be α/k = .05/3 = 0.016667. You can always override this value by changing the value of Alpha in the output.

The output is shown in Figure 1 (slightly reformatted to fit in the figure).

Multiple ANOVA follow up

Figure 1 – Multiple Single Factor Anovas

Note that despite the fact that a significant difference was detected by the MANOVA test, none of the ANOVA tests shows a significant difference (p-value > .05/3 = 0.16667). In fact, none shows a significant difference even if α = .05.

Examples Workbook

Click here to download the Excel workbook with the examples described on this webpage.

Reference

Howell, D. C. (2010) Statistical methods for psychology (7th ed.). Wadsworth, Cengage Learning.
https://labs.la.utexas.edu/gilden/files/2016/05/Statistics-Text.pdf

25 thoughts on “MANOVA Follow up using ANOVA”

  1. Hi Charles,

    For some reason RealStats is returning me an Eta-sq value intstead of an F-Crit value as shown in your figures here when performing multiple ANOVAs. Any idea on why this is happening?

    Reply

  2. Hi Charles,
    Many thanks for such insightful website!
    I’ve remarked that the Std. Error is calculated as sqrt(MSw/n) ans used in the CI calculation. If I am not mistaken, it should be equal to sqrt(2*MSw/n) (if equal sample sizes) based on the std. err definition (sqrt(S1^2/n1+S2^2/n2) or Spooled*sqrt(1/n1+1/n2)).
    The same way of calculation is used in the Tukey HSD test for CI (lower/upper bounds). Any explanation is highly appreciated.
    Regards,
    Hamadi

    Reply

    • Sorry Charles,
      I’ve compared t-test to Tukey HSD and found the same results.
      There is no error in the SE calculation.
      Regards,
      Hamadi

      Reply

  3. Hello again Charles,

    Sorry for the big amount of questions. I think I am near the test I need and I want to “have all the ends tied” (is a literal traduction of an spanish expression). It is correct to perform group by groups comparisons after doing the Multiples ANOVAS? If in the example above you want to compare the variable water between clay and loam for example you can perform pairwaise comparison between both?

    As I understand a MANOVA is accompanied by those results:
    -A global p value that indicates you that there exist or not statistical differences between the groups having into account all the dependent variables.
    – The planned contrast, were you can see if there exist differences between two groups or 2 vs 2 groups for example having into account all the dependent variables.
    – The multiple ANOVA that tell you If there exist differences one by one dependent variable between all the groups.
    So the last comparison I need is to see the direction of the differences in the multiple ANOVA as mentioned…

    I am a little scared about the Box test as mentioned in other section of the webpage, because I have obtained a p value really significant… I do not know if in this case I can use the One Way MANOVA.

    Thanks a lot Charles,

    Gabriel

    Reply

  4. Hi there 🙂

    Can I please have some help!?

    I am reading a research paper that is looking at the effect of Task specific training on performances in 3 motor tasks. Measures are taken before and after training.
    Training is on a ‘feeding’ tasks, and improvements in this task are measured after 5 days as well as performances in two untrained tasks: dressing and feeding.
    So it’s repeated measures. IV: before and after training.
    DV: performance (in 3 tasks)
    What statistical test is better? I thought maybe MANOVA BUT these tasks are very similar (all upper limb motor tasks, presumably highly correlated) AND they are measured on different/unequal scales

    Reply

      • Hi Charles

        Thanks for this – although I am slightly confused so bear with me.. this is not my statistical analyses, but a research paper I’m critiquing

        The paper I read did multiple repeated measure ANOVA’s without correcting for type 1. They said they did this as the 3 different tasks used different measures

        However, I am unable to work out whether MANOVA was more appropriate: they are measuring performance in 3 different (but very similar) motor tasks? Overall Dependent variable is performance but does this mean there was 3 DV’s?

        But, as they are highly correlated (presumably) does this mean that MANOVA isn’t appropriate?

        Thanks

        Megan

        What do you think is most appopriate?

        Reply

        • Hi again Charles

          IF the research only had 11 participants, would MANOVA be inappropriate (it says this on the link that sample size should be 10 + k)

          Reply

  5. Hello Charles,
    Is Real Statistics able to perform a one-way MANOVA on an unbalanced data set? I have a data set with three levels of the independent variable and three dependent variables. The 3 levels of the independent variable have 6, 8, and 21 experimental subjects, respectively. The P values for the Pillai, Wilks, Hotelling and Roy tests are all highly significant (0.0001 or less). However, when I do a Box’s M test, I get the following: M=32.46, F=2.23, df1=12, df2=1546.575 (could this be correct?), p=0.0088.
    Two of the 3 follow up one-way ANOVAs showed highly significant main effects (p=0.0009 and p= 5.12E-09, respectively). Levene tests for each of the 3 follow up one-way ANOVAs were in the range of 0.9, 0.25, and 0.09 for the three dependent variables (about same for mean, median and trimmed for each variable).
    Am I OK with the Real Statistics program for this data?
    Is it permissible to do Tukey HSD paired comparisons for the 2 follow up one-way ANOVAs that showed significant main effects (?
    Thanks very much for your help. I very much enjoy instruction and your commentary on the website.
    Joel

    Reply

    • Hello Joel and Happy New Year,
      The Real Statistics MANOVA data analysis tool handles unbalanced models (at least in the sense that you have described).
      The MANOVA data analysis tool will generate results, including p-values, whether or not the assumptions are met. If the assumptions are not met (including homogeneity of covariance matrices), then the conclusions may be inaccurate. This is a problem for many tests including MANOVA, where someof the assumptions may not be met.
      I have run test cases in the past and the results that the Real Statistics MANOVA data analysis tool generates are consistent with the results from other tools. Without looking in more details, however, I can’t say whether the results you got are OK.
      Yes, you can do Tukey HSD follow-up tests after significant ANOVA results. Two cautions are needed: (1) indiviual ANOVA tests may not be the best way to proceed since they don’t take the correlations between the dependent variables into account and (2) whereas for each ANOVA the Tukey HSD test takes experimentwise error into account (and so you don’t need to use a Bonferroni correction), the fact that you are performing multiple such tests (for the two ANOVA’s) is not corrected This may mean that you need to use an alpha value of .025 (instead of .05) to take into account that you are performing two Tukey HSDs, one for each ANOVA.
      Charles

      Reply

  6. Joel
    Hello Charles,
    1) I am unable to reproduce the spreadsheet you display in Figure 1 – Multiple Single Factor Anova, for multiple ANOVA as a follow-up to MANOVA . I copied Figure 1 – Data for Example 1 in standard form from your example spreadsheet (Multivariate Examples, Sheet MANOVA 1 a) to a blank spreadsheet, chose Multiple Anova option in the Real Statistics MANOVA data analysis tool (Figure 2 of Manova Real Statistics Support), and highlighted A3:D35 as input range. My program output the MANOVA table as well as the three single factor ANOVAs displayed in your Figure 1 – Multiple Single Factor Anova, but the Alpha factor on my printout was 0.05 instead of 0.0167 (no Bonferroni correction, and the lower and upper limits were wider apart for each soil condition (presumably corresponding to the lower degree of significance requested). The corresponding between groups F-critical was smaller, again presumably related to the different P value. What did I do wrong?
    2) If your one-way follow-up ANOVA for one of the dependent variables (e.g. yield) had revealed a significant main effect, would it be justifiable to perform Tukey HSD paired comparisons to determine which soil condition(s) was (were) responsible for the difference? If so, what level of significance would you require for the Tukey HSD? Does the Tukey HSD have a Bonferroni correction “built-in”, or would I need to change my alpha to 0.05/6, since I would be comparing different pairs of soil conditions?
    Thanks very much for your help and best wishes for a happy holiday season.
    Joel

    Reply

    • Hello Joel,
      1) The program neglected to apply the Bonferroni correction. This will be corrected in the next release of the Real Statistics software. I believe the Real Statistics software calculates the confidence intervals correctly, but this is not correctly reflected on the website and in the Examples workbook. I will correct these when I issue the next software release.
      2) In general, to isolate the paired comparison after ANOVA, you can certainly perform a Tukey HSD test. This test will automatically correct for familywise error and you don’t need to change the alpha value using a Bonferroni correction.
      Charles

      Reply

  8. Attractive seϲtion off ϲontent. I just stumbled ᥙpon your bblog and in accession capital tο assert that Ӏ get actually enjoiyed account yoսr blog
    posts. Anywaqy ӏ’ll be subscribing to your augment aand even I achievement
    you access consistently rapidly.

    Reply

  9. Hi Charles,

    Thank you for this webpage. The MANOVA Real Statistics Support page is not available. Is it still in construction or is there a problem with it?

    Thanks!

    Reply

Leave a Comment