Updated for the 2026-2027 CFA® Level I curriculum.
A hypothesis test brings several ideas together. The test statistic measures how far the sample result lies from the value stated in the null hypothesis. The p-value describes how unusual that result would be under the null, while the significance level provides the threshold for making a decision.
For CFA Level I, focus on how these parts connect. Once you identify the hypotheses, test direction, and significance level, the decision process becomes much easier to follow.
Quick Answer
A test statistic expresses the difference between a sample result and the hypothesized value in units of standard error. The p-value is the probability, assuming the null hypothesis is true, of obtaining a result at least as extreme as the observed one. Reject when the p-value is less than or equal to , or when the test statistic falls within the rejection region.
Key Takeaways About P-Values, Test Statistics, and Decision Rules
A test statistic standardizes the difference between the sample statistic and the hypothesized value.
A p-value measures how unusual the observed result would be under a true null hypothesis.
The sample data determine the p-value.
The analyst selects the significance level before evaluating the evidence.
Reject when .
The p-value method and critical-value method should produce the same decision when they use the same distribution, tail, and significance level.
The correct decisions are “reject the null hypothesis” and “fail to reject the null hypothesis.”
Statistical significance addresses sampling uncertainty. Economic significance addresses whether the result is large enough to affect an investment decision.
What You Need to Know for CFA Level I
For CFA Level I, you should be able to:
Calculate or interpret a standardized test statistic.
Explain what a p-value represents.
Compare a p-value with the significance level.
Compare a test statistic with the correct critical value.
Identify the rejection region for a left-tailed, right-tailed, or two-tailed test.
State the statistical decision using the correct language.
Explain how statistical significance differs from economic significance.
Recognize when an incorrect tail or critical value has been used.
What Is a Test Statistic?
A test statistic measures how far the sample result lies from the value specified under the null hypothesis. The difference is divided by the statistic’s standard error, which adjusts for the expected amount of sampling variation.
The general form is:
A larger absolute test statistic indicates that the sample result lies farther from the hypothesized value relative to its sampling variability.
For example, a sample mean that exceeds the hypothesized mean by 2% may provide strong evidence when the standard error is 0.5%. The same 2% difference provides much weaker evidence when the standard error is 5%.
Test Statistic for a Population Mean
When the population variance is unknown and estimated from the sample, the test statistic for a population mean is commonly written as:
Where:
= calculated t-statistic
= sample mean
= hypothesized population mean
= sample standard deviation
= number of observations
= estimated standard error of the sample mean
The appropriate test statistic depends on the hypothesis being tested. Tests involving means commonly use a t-statistic or z-statistic. Tests involving variances may use an F-statistic, while tests based on contingency tables use a chi-square statistic.
The interpretation remains consistent: the test statistic measures distance from the null-hypothesis value in standardized terms.
What Is a P-Value in Hypothesis Testing?
The p-value is the probability of obtaining a test statistic at least as extreme as the observed statistic, assuming the null hypothesis is true.
Suppose a test produces a p-value of 0.03. You can read this as follows:
If the null hypothesis were true, a test statistic this extreme or more extreme would occur approximately 3% of the time under the assumed model.
A smaller p-value indicates that the observed result would be less common under the null hypothesis. This provides stronger evidence against .
The meaning of “extreme” depends on the alternative hypothesis:
In a right-tailed test, larger positive statistics are more extreme.
In a left-tailed test, larger negative statistics are more extreme.
In a two-tailed test, statistics far from zero in either direction are more extreme.
The p-value therefore depends on both the calculated test statistic and the direction of the test.
P-Value vs Significance Level
The p-value and significance level play different roles in a hypothesis test.
Feature | P-Value | Significance Level |
|---|---|---|
Symbol | ||
Meaning | Probability of a result at least as extreme as the observed result under | Maximum probability of a Type I error the analyst is prepared to accept |
Determined by | Sample data and test setup | Analyst or researcher |
Timing | Calculated after observing the sample | Selected before evaluating the evidence |
Main use | Measures the strength of evidence against | Establishes the decision threshold |
Decision comparison | Compared with | Serves as the cutoff for the p-value |
The decision rule is:
For example, when :
A p-value of 0.02 leads to rejection of .
A p-value of 0.08 leads to failure to reject .
A p-value of 0.05 falls on the rejection threshold.
A significance level of 5% corresponds to a 95% confidence level. The p-value should still be compared with 0.05, rather than with 0.95.
What Is a Decision Rule in Hypothesis Testing?
A decision rule specifies the condition under which you reject the null hypothesis. You can apply the rule using either the p-value or the critical value.
P-Value Method
Critical-Value Method
Reject when the calculated test statistic falls within the rejection region established by the critical value.
Test Direction | Rejection Rule |
|---|---|
Right-tailed | Reject when the statistic is at or above the upper critical value |
Left-tailed | Reject when the statistic is at or below the lower critical value |
Two-tailed | Reject when the statistic lies beyond either critical value |
The critical value is the boundary between the rejection region and the non-rejection region. It is determined by:
The significance level
The test direction
The selected probability distribution
The degrees of freedom, when applicable
The p-value and critical-value methods express the same statistical decision in different ways. One compares probabilities, while the other compares standardized statistics.
How Do Test Statistics, Critical Values, and P-Values Work Together?
You can organize most hypothesis-testing questions into six steps.
Step 1: State the Hypotheses
Write the null hypothesis and alternative hypothesis . The alternative hypothesis determines whether the test is left-tailed, right-tailed, or two-tailed.
Step 2: Select the Significance Level
Choose before evaluating the sample result. Common significance levels include 0.01, 0.05, and 0.10.
Step 3: Select the Appropriate Test Statistic
Choose the statistic and sampling distribution that match the parameter, available information, and assumptions of the test.
Step 4: Calculate the Test Statistic
Use the sample data to calculate the standardized distance from the hypothesized value.
Step 5: Apply the Decision Rule
Use either of the following approaches:
Compare the p-value with .
Compare the test statistic with the appropriate critical value.
Step 6: State the Conclusion
Explain what the decision means in the context of the research question.
How Should You State the Hypothesis-Test Conclusion?
A hypothesis test produces one of two decisions.
Reject the Null Hypothesis
Rejecting means the evidence is strong enough, at the selected significance level, to support the alternative hypothesis.
A contextual conclusion might read:
At the 5% significance level, the evidence supports the conclusion that the fund’s mean active return differs from zero.
Fail to Reject the Null Hypothesis
Failing to reject means the available evidence is insufficient to support the alternative hypothesis at the selected significance level.
A contextual conclusion might read:
At the 5% significance level, there is insufficient evidence to conclude that the fund’s mean active return differs from zero.
Use “fail to reject” rather than “accept.” The test evaluates whether the evidence against crosses a specified threshold. A result outside the rejection region leaves the evidence below that threshold.
Sample size and variability also affect this outcome. A small sample or high standard error can produce low statistical power, allowing an economically meaningful effect to remain undetected.
Statistical Significance vs Economic Significance
Statistical significance shows that the observed effect is unlikely to result solely from sampling variation under the null hypothesis.
Economic significance considers whether the size of the effect is meaningful enough to influence an investment decision.
For example, a strategy may produce a statistically significant excess return of three basis points per month. Transaction costs, taxes, liquidity constraints, and management fees could still make the strategy unattractive.
Large samples can make very small effects statistically significant because increasing n reduces the standard error. You should therefore evaluate both:
Whether the result is statistically distinguishable from the null-hypothesis value.
Whether the estimated effect is large enough to matter in practice.
Worked Example: Applying Both Decision Rules
An analyst wants to determine whether a long-short equity fund generates a mean monthly active return different from zero.
The analyst has the following information:
Number of observations:
Sample mean active return:
Sample standard deviation:
Hypothesized mean:
Significance level:
Step 1: State the Hypotheses
The analyst is interested in a difference in either direction, so the test is two-tailed.
Step 2: Calculate the Standard Error
Step 3: Calculate the Test Statistic
The sample has 59 degrees of freedom:
Step 4: Apply the Critical-Value Rule
For a two-tailed t-test with 59 degrees of freedom and , the approximate critical values are:
The calculated statistic of 2.30 exceeds the upper critical value of 2.001, so it lies within the rejection region.
Decision: Reject .
Step 5: Apply the P-Value Rule
The two-tailed p-value associated with a t-statistic of approximately 2.30 and 59 degrees of freedom is about:
Because:
the p-value method also leads to rejection of .
Step 6: State the Conclusion
At the 5% significance level, the evidence supports the conclusion that the fund’s mean monthly active return differs from zero.
The analyst should then evaluate the size of the return and determine whether it remains economically meaningful after fees, trading costs, and other practical constraints.
Common Exam Traps
Interpreting the p-value as the probability that is true. Read the p-value as the probability of an equally or more extreme statistic under an assumed true null hypothesis.
Comparing the p-value with the confidence level. Compare the p-value with . A 95% confidence level corresponds to .
Reversing the p-value rule. Smaller p-values provide stronger evidence against .
Using the wrong test direction. A two-tailed p-value includes probability from both tails.
Comparing unlike quantities. Compare the p-value with , and compare the test statistic with the critical value.
Using a standard normal critical value for every test. A t-distribution critical value depends on the degrees of freedom.
Writing “accept .” Use “fail to reject .”
Treating statistical significance as an investment recommendation. Practical decisions also depend on effect size, costs, risk, and implementation constraints.
Rounding too early. A heavily rounded p-value or test statistic can change a decision when the result lies close to the threshold.
Practice Question
An analyst conducts a right-tailed standard normal hypothesis test at the 5% significance level. The calculated test statistic is 1.78, the critical value is 1.645, and the p-value is 0.038.
Which decision is most appropriate?
Fail to reject because the p-value of 0.038 is greater than the critical value of 1.645
Reject because the p-value is below 0.05 and the test statistic lies in the rejection region
Fail to reject because the test statistic does not exceed 1.96
Correct Answer: B
The p-value of 0.038 is below the significance level of 0.05, so the p-value method leads to rejection of .
The test statistic of 1.78 also exceeds the right-tailed critical value of 1.645, placing it inside the rejection region. Both decision rules therefore support the alternative hypothesis at the 5% significance level.
Option A compares a probability with a standardized test statistic. The p-value should be compared with , while the test statistic should be compared with the critical value.
Option C uses 1.96, which is the standard normal critical value for a two-tailed test at the 5% significance level. The question specifies a right-tailed test.
Continue Your CFA Level I Prep With KeyPoint
Use structured lessons, practice questions, mock exams, and progress tracking to focus on the time you have left
FAQs About P-Values, Test Statistics, and Decision Rules
What Does a P-Value Mean in Hypothesis Testing?
A p-value is the probability, assuming the null hypothesis is true, of obtaining a test statistic at least as extreme as the observed statistic.
A small p-value means the sample result would be relatively unusual under , providing stronger evidence in favor of the alternative hypothesis.
What Is the Difference Between a P-Value and a Significance Level?
The p-value is calculated from the sample result. The significance level is selected before the evidence is evaluated and represents the analyst’s chosen Type I error threshold.
The hypothesis-test decision comes from comparing the two values.
When Should You Reject the Null Hypothesis?
Reject the null hypothesis when the p-value is less than or equal to .
You can reach the same decision by determining whether the test statistic falls inside the rejection region established by the critical value.
Does a Smaller P-Value Mean a Larger Investment Effect?
A smaller p-value indicates stronger statistical evidence against the null hypothesis. The size of the investment effect is measured separately through the estimated return, coefficient, spread, or other relevant metric.
A large sample can produce a small p-value for an effect that remains economically minor.
What Is the Difference Between Statistical and Economic Significance?
Statistical significance indicates that an observed effect is unlikely to result solely from sampling variation under the null hypothesis.
Economic significance considers whether the size of that effect is meaningful enough to affect an investment decision after costs, risks, taxes, and implementation limits are considered.