Skip to content

How significant is the difference between the means?

One-sample, Welch two-sample and paired t tests from data or summary statistics, with the t distribution drawn live.

Test
Enter

Test settings
Alternative
%

p-value

0.0379

t = 2.269 (df = 15.53) gives p = 0.0379: significant at the 5% level. The difference of means A − B is 3.869; 95% confidence interval 0.2459 to 7.491.

The t distribution with the test statistic

Key values as a table
StatisticValue
t2.269
Critical t−2.125
Critical t2.125
p-value0.0379

t-test results

t
2.269
Degrees of freedom
15.53
p-value
0.0379
Difference of means A − B
3.869
95% confidence interval
0.2459 to 7.491
Standard error
1.705
Critical t
−2.125 / 2.125
Sample A
mean 36.09, SD 4.908, n = 11
Sample B
mean 32.22, SD 2.539, n = 9

One-sample, Welch two-sample and paired t tests from data or summary statistics, with the t distribution drawn live.

How to use

  1. Choose the test: one sample against a known mean, two independent samples, or paired measurements of the same subjects.
  2. Paste the values, or switch to Summary and enter each group's mean, standard deviation and size.
  3. Read the p-value and the sentence; the curve shades the rejection region and the tail beyond your t. Drag the critical line, or open Test settings, to change the confidence level or test one side only.

How it works

A t test asks whether a difference between means is larger than chance alone would easily produce. It divides the difference by its standard error: t = (x̄ − μ₀) ÷ (s ÷ √n) for one sample, where x̄ is the sample mean, μ₀ the mean tested against, s the standard deviation and n the size. The p-value is the chance of a t at least this far from zero if the true difference were zero; a small p (below α, often 5 %) counts as significant.

Two independent samples use Welch's version, which does not assume equal variances: t = (x̄₁ − x̄₂) ÷ √(s₁²/n₁ + s₂²/n₂), with the Welch–Satterthwaite degrees of freedom. In the NIST example, assembly times of an old process (mean 36.09, SD 4.91, n = 11) against a new one (mean 32.22, SD 2.54, n = 9) give t = 2.27 with about 15.5 degrees of freedom, a two-sided p of about 0.038. A paired test works on the differences of each pair, so it removes the variation between subjects. The formulas follow the NIST/SEMATECH e-Handbook of Statistical Methods.

The test assumes independent observations and roughly normal data or, for larger samples (about 30 per group), roughly normal means. A significant result says the difference is unlikely to be chance, not that it is large or important; the confidence interval shows how big it plausibly is.

Worked example

Do two samples with means 36.09 (n = 11) and 32.22 (n = 9) really differ?

  1. The difference of the means is 3.87, with a standard error of 1.7.
  2. Welch's t = difference ÷ standard error = 2.27, with 15.5 degrees of freedom.
  3. The two-sided p-value for that t is 0.038.

t = 2.27 and p = 0.038: a p-value below 0.05 means the difference is significant at the 5 % level.

Frequently asked questions

Which t test should I use?

One sample: one group against a fixed value. Two samples: two separate groups, such as a treatment and a control group. Paired: the same subjects measured twice, such as before and after, or matched pairs.

Why Welch's test and not Student's equal-variance test?

Welch's test stays correct when the two groups have different spreads and loses almost nothing when they are equal, so it is the safer default; many statistics packages now use it by default.

What does p = 0.03 mean?

If there were truly no difference, a t at least this extreme would turn up in about 3 % of samples. It is not the probability that the hypothesis is true.

When should I use a one-sided test?

Only when the direction was decided before seeing the data and a difference the other way would not matter. A one-sided p is half the two-sided p when the difference lies in the chosen direction.

How does the test relate to the confidence interval?

A two-sided test is significant at 5 % exactly when the 95 % confidence interval leaves out the hypothesised value.

Should I enter the sample or population standard deviation?

Enter the sample standard deviation, calculated with n − 1 in the variance denominator. Do not enter the standard error of the mean in the SD field.

Send feedback