Consultation 1: Repeated-Measures ANOVA for Office-Image Effects on Stress

I. Client’s Inquiry

This study aims to examine the effects of indoor office environment images on stress levels. To achieve this, three spatial variables were defined: color (cool/warm), ceiling height (low/high), and window view (urban/natural).

The experiment consists of two stages (Test 1 and Test 2) conducted on the same group of 30 participants.

Test 1:

To investigate the individual effects of each spatial variable, images representing each condition (a total of 6 images) will be presented separately.
Paired-sample t-tests will be used to compare stress indices across conditions in order to determine whether there are significant differences between them.

Test 2:

In the second stage, combinations of the three variables will be used to create 2 × 2 × 2 conditions (color × ceiling height × view), resulting in a total of 8 image conditions.
Stress levels will be measured under each condition, with stress index as the dependent variable and environmental condition (8 combinations) as the independent variable.
A One-way Repeated Measures ANOVA will be conducted to examine whether there are significant differences in mean stress levels among the combined environmental conditions.

We would like to ask for your review regarding whether this statistical design and analysis approach is appropriate, and whether there might be a simpler or clearer method for analyzing the data.
In particular, we would like to confirm whether the use of One-way ANOVA or One-way Repeated Measures ANOVA is suitable in this case, and whether the sample size of 30 participants is sufficient or should be increased.

II. Answer

1. Test 1

The model is given as \[Y_{ij} = \mu + \tau_j + s_i + \epsilon_{ij}\] where

  • \(Y_{ij}\) : ith subject’s stress figure when exposed to jth condition (color, …)
  • \(\mu\) : mean
  • \(\tau_j\) : the effect of color condition ( Cool colors vs. warm colors)
  • \(s_i\)​: the effect of subjects (random effect)
  • \(\epsilon_{ij}\)​: error

the way to adjust the significance level (Bonferroni) : \(\alpha_B = \alpha/n\) That is: each test is considered significant only if its p-value is less than \(\alpha/n\). This ensures that the overall significance level remains at \(\alpha\).

If there are more than 3 levels? => One-way Repeated Measures ANOVA. ex) 3 colors.

Idendical model, but the number of \(\tau_j\) is 3: \(j=1,2,3\). Then, the null hypothesis is

\[H_0 : \tau_1 = \tau_2 = \tau_3\]

If the null hypothesis is rejected, it means that at least one of the levels has a different effect. The specific levels that differ can be identified through pairwise t-tests. For instance,

  • cool color vs neutral color
  • cool color vs warm color
  • neutral color vs warm color

At this point, a Bonferroni correction (or similar adjustment) should be applied.

We may have the following question : > If we conduct a Three-way Repeated Measures ANOVA from the start, can we still check the significance of each factor in the same way?

The Three-way RM ANOVA can encompass the t-test results from Test 1, but it does not guarantee completely identical outcomes. Test 1 is designed to evaluate each factor in isolation, whereas the Three-way RM ANOVA analyzes the effect of each factor while all factors act simultaneously.

If all interaction effects are not significant, the main effects of each factor can be interpreted independently. In this case, the main effect results from the Three-way RM ANOVA and the results from analyzing each factor separately using t-tests can provide practically similar or even identical conclusions.

However, the main effects in a Three-way RM ANOVA represent the average effects while controlling for the mean influence of other factors, whereas t-tests examine the effects with other factors held constant. Therefore, due to the differences in design and interpretation, the numerical results may differ.

2. Test 2

(1) one-way RM anova

model : \[Y_{ij} = \mu + \tau_j + s_i + \epsilon_{ij}\]

  • \(\tau_j\) : jth effect (j = 1,…,8)
  • \(s_i\) : ith subject effect, random effect

Then,

null hypothesis : tau_1 = … = tau_8, alternative hypothesis : At least one of them has a different stress index.

If the null hypothesis is rejected, post-hoc tests can be used to determine which conditions differ.

In conclusion, this approach is suitable if the goal is simply to test whether the three factors have no overall effect. However, it is not appropriate if you want to understand the individual effects of each factor or their interaction effects.

(2) three-way rm anova

\[Y_{ijkl} = \mu + color_j + level_k + view_l + ... + s_i + \epsilon_{ijkl}\] - \(s_i\) : subject effect (random effect)

It also allows you to identify both the main effects of each factor and the presence of interaction effects. Through this, you can determine: - whether each factor (e.g., color) has a significant effect, and - whether there are interaction effects between two or among all three factors.
(This is done by checking whether p < 0.05 ⇒ significant.)

3. Better model

(1) Client’s Analysis Objective:

  • To determine whether each design element (color, ceiling height, and window view) has an individual effect.

  • To see whether stress levels differ depending on the combination of conditions.

  • If possible, to further identify which specific combinations are particularly effective.

(2) Comparison:

  • Interpretability: One-way RM ANOVA shows only the mean differences among the 8 conditions,
    whereas Three-way RM ANOVA reveals the main effects of color, ceiling height, and window view, as well as their interaction effects.

  • Clarity of interpretation: One-way RM ANOVA allows statements like “Which combination resulted in lower stress,”
    while Three-way RM ANOVA allows statements such as “What is the average effect of color on stress.”

(3) Conclusion: Three-way RM ANOVA

The client’s main interest is not just in comparing combinations,
→ but in understanding how much each design factor (color, ceiling height, view) influences stress.

A One-way RM ANOVA only shows whether there are differences among the 8 combinations,
→ but it can never tell whether those differences are due to color, ceiling height, or their interaction.

In contrast, a Three-way RM ANOVA can analyze
→ the main effects of color, ceiling height, and window view,
→ as well as all possible interaction effects among them.

Given that the core purpose of the experiment is to understand how these design factors affect stress,
→ the Three-way RM ANOVA is a more suitable approach than the One-way RM ANOVA.

4. [30] participants

Appropriate Minimum Benchmark

When the sample size is around 30 or more, the sampling distribution of the mean approximately follows a normal distribution.
→ Therefore, under the assumption of normality, the use of t-tests and ANOVA becomes valid.
→ For this reason, “30 participants” is widely used as a practical benchmark in research.

cf) Advantage of Repeated Measures Design:
It provides higher statistical power with a smaller sample size compared to an independent-sample design.