Test statistic is a quantity calculated based on the sample data. Based on the test-statistic value we make the decision on whether we reject the null hypothesis. With a large value of test-statistic (for every test there is a cut-off value) the null hypothesis is rejected. There are many test-statistic around. For example, Z-test, t-test, Chi Square-test, F-test etc. We need to select the right test statistic based on the hypothesis to be tested.
Confidence Interval (CI) is a range of values with a lower limit and an upper limit calculated from the sample data. If the null hypothesised value falls inside the CI, we do not reject the null hypothesis. A wider CI means that there is more uncertainty in the observed results which also means that the observed results are less reliable. The meaning of a 95% CI is that “we are 95% confident that this interval contains the true value”.
P-value is a useful measure however people at this age misuse it more than use it in the right way. P-value is the probability that a larger effect or strong association is obtained by random chance only. A lower p-value (usually <0.05) means that the results are statistically significant.
All three measures should give us the same decision because the p-value and CI are calculated based on the test statistic. Please keep in mind that the test statistic, CI, and p-value provide the decision about the statistical hypothesis only not the clinical hypothesis. In the table below some test statistics are listed with their use.
Table 2 Test-statistic and their use.
| Test statistic | When to use? |
| t-test (independent) | To compare the means of two independent samples. |
| t-test (paired) | To compare the means of two paired samples. |
| t-test | To test the significance of regression coefficients and correlation coefficients. |
| Z-test | To compare the proportions of two independent samples. |
| Chi-Square – test | To test if two categorical variables are correlated. |
| F-test (Also known as ANOVA) | To compare means of 3 or more samples. Also, to assess the overall significance of a regression model. |
| Repeated measure ANOVA | Compare means when measurements are taken from the same person two or more times. |
Al Emran
‘A wider CI means that there is more uncertainty in the observed results’ at the same time ‘a 95% CI is that “we are 95% confident that this interval contains the true value”.’ It seems self contradiction generating.
We hope more lucid here
.
Moin Bahezi
Thank you for your comment. Wider CI meaning here the magnitude of a 95% interval is wider. For example, there are two odds ratio (OR), 1.3 (OR1) and 1.7 (OR2) and their 95% CIs are 1.2-1.5 (for OR1) and 0.67-3.7 (for OR2) respectively. So the width of the intervals are 0.3 (for OR1) and 3.03 (for OR2). Though the OR2 is larger meaning stronger association but because of it’s wider CI, the association is more uncertain than the OR1.