Htykuut

Given a dataset that has a numerical variable called "number of children" and a categorical variable "Standard of living" with $4$ levels, I used anova to see if there is a relationship between the number of children and the different standards of living.

But first, I evaluated the mean of the number of children of each standard of living ($1$=low$,2,3,$$4$=high):

begin{align}
&text{standard of living} hspace{20pt} 1 hspace{30pt} 2 hspace{27pt} 3 hspace{30pt} 4 \
&text{number of children} hspace{14pt} 3.25 hspace{15pt} 3.30 hspace{15pt} 3.28 hspace{17pt} 3.42 \
end{align}

After running the ANOVA command in R, the result was F value $= 0.05$
and Pr(>F)$= 0.985$.

Since the F value is low and the means are very close, does that mean that there is not a
significant relationship between the number of children in a couple
and their standard of living?

I did the same thing but now for the categorical variable wife's education which has four levels $(1$=low$, 2,3, 4=$high$)$ and the result was

begin{align}
&text{Wife's education} hspace{20pt} 1 hspace{30pt} 2 hspace{27pt} 3 hspace{30pt} 4 \
&text{number of children} hspace{11pt} 4.42 hspace{15pt} 3.51 hspace{15pt} 3.23 hspace{17pt} 2.83 \
end{align}

With F value $=20.67$ and Pr(>F)$=4.06e-13$. There is a strong
relationship between the number o children in a couple and the wife's
education?

In this case what is "strong relationship"? And what values of $F$ are "high" or "low"?

Running a Tukey post hoc test:

          diff        lwr         upr     p adj



2-1 -0.9120706 -1.4940399 -0.33010131 0.0003402



3-1 -1.1869063 -1.7517540 -0.62205855 0.0000005



4-1 -1.5891636 -2.1314528 -1.04687427 0.0000000



3-2 -0.2748357 -0.7132637  0.16359229 0.3719251



4-2 -0.6770930 -1.0860475 -0.26813844 0.0001286



4-3 -0.4022573 -0.7864558 -0.01805873 0.0360224

There is not a significant difference in the number of children
between the wife's education level $3$ and level $2$. And there is a
significant difference between the wife's education level $4$ and
level $1$ etc.

But what is a "significant difference" in this context?

(1) Standard of living and number of children: Small F-statistic and consequent large P-value mean that no significant differences have been found. Thus there is no reason
to do ad hoc Tukey tests.

(2) Wife's education and number of children: Large F-statistic and consequent small P-value mean that there are some significant differences. You do an ad hoc Tukey procedure to
see what can be determined about the pattern of differences.

Roughly speaking, the number of children tend to decrease as wife's education level increases. If we used ${4 choose 2} = 6$ different t-tests to check for differences, results might be confusing. We could
do each t-test at the 5% level of significance, but we would have no idea what
risk we run of falsely declaring differences in some of the six comparisons. We say
that the 'family error' rate for the pattern of differences among the four educational levels is indeterminate, at worst is might almost as high as $6(.05) = 0.3)$ or 30%.

The Tukey test is somewhat more 'reluctant' to declare differences. The criterion for
declaring an 'Honest Significant Difference' (HSD) is chosen in such a way as to
keep the family error rate below 5%. Thus the difference $3.51 - 3.23 = 0.28$ between education levels 2 and 3 is not sufficiently large to be declared 'significant'.

By contrast, for example, the difference $3.23 - 2.83 = 0.40$ between education levels 3 and 4 is (barely) large enough to be declared significant. The much larger difference between education levels 1 and 2 is (more easily) declared significant. (If the sample sizes differ from level to level of the categorical variable the value of HSD may differ from one comparison to another.)