what does it mean to reject your hypothesis

Hypothesis Testing (cont...)

Hypothesis testing, the null and alternative hypothesis.

In order to undertake hypothesis testing you need to express your research hypothesis as a null and alternative hypothesis. The null hypothesis and alternative hypothesis are statements regarding the differences or effects that occur in the population. You will use your sample to test which statement (i.e., the null hypothesis or alternative hypothesis) is most likely (although technically, you test the evidence against the null hypothesis). So, with respect to our teaching example, the null and alternative hypothesis will reflect statements about all statistics students on graduate management courses.

The null hypothesis is essentially the "devil's advocate" position. That is, it assumes that whatever you are trying to prove did not happen ( hint: it usually states that something equals zero). For example, the two different teaching methods did not result in different exam performances (i.e., zero difference). Another example might be that there is no relationship between anxiety and athletic performance (i.e., the slope is zero). The alternative hypothesis states the opposite and is usually the hypothesis you are trying to prove (e.g., the two different teaching methods did result in different exam performances). Initially, you can state these hypotheses in more general terms (e.g., using terms like "effect", "relationship", etc.), as shown below for the teaching methods example:

Depending on how you want to "summarize" the exam performances will determine how you might want to write a more specific null and alternative hypothesis. For example, you could compare the mean exam performance of each group (i.e., the "seminar" group and the "lectures-only" group). This is what we will demonstrate here, but other options include comparing the distributions , medians , amongst other things. As such, we can state:

Now that you have identified the null and alternative hypotheses, you need to find evidence and develop a strategy for declaring your "support" for either the null or alternative hypothesis. We can do this using some statistical theory and some arbitrary cut-off points. Both these issues are dealt with next.

Significance levels

The level of statistical significance is often expressed as the so-called p -value . Depending on the statistical test you have chosen, you will calculate a probability (i.e., the p -value) of observing your sample results (or more extreme) given that the null hypothesis is true . Another way of phrasing this is to consider the probability that a difference in a mean score (or other statistic) could have arisen based on the assumption that there really is no difference. Let us consider this statement with respect to our example where we are interested in the difference in mean exam performance between two different teaching methods. If there really is no difference between the two teaching methods in the population (i.e., given that the null hypothesis is true), how likely would it be to see a difference in the mean exam performance between the two teaching methods as large as (or larger than) that which has been observed in your sample?

So, you might get a p -value such as 0.03 (i.e., p = .03). This means that there is a 3% chance of finding a difference as large as (or larger than) the one in your study given that the null hypothesis is true. However, you want to know whether this is "statistically significant". Typically, if there was a 5% or less chance (5 times in 100 or less) that the difference in the mean exam performance between the two teaching methods (or whatever statistic you are using) is as different as observed given the null hypothesis is true, you would reject the null hypothesis and accept the alternative hypothesis. Alternately, if the chance was greater than 5% (5 times in 100 or more), you would fail to reject the null hypothesis and would not accept the alternative hypothesis. As such, in this example where p = .03, we would reject the null hypothesis and accept the alternative hypothesis. We reject it because at a significance level of 0.03 (i.e., less than a 5% chance), the result we obtained could happen too frequently for us to be confident that it was the two teaching methods that had an effect on exam performance.

Whilst there is relatively little justification why a significance level of 0.05 is used rather than 0.01 or 0.10, for example, it is widely used in academic research. However, if you want to be particularly confident in your results, you can set a more stringent level of 0.01 (a 1% chance or less; 1 in 100 chance or less).

One- and two-tailed predictions

When considering whether we reject the null hypothesis and accept the alternative hypothesis, we need to consider the direction of the alternative hypothesis statement. For example, the alternative hypothesis that was stated earlier is:

The alternative hypothesis tells us two things. First, what predictions did we make about the effect of the independent variable(s) on the dependent variable(s)? Second, what was the predicted direction of this effect? Let's use our example to highlight these two points.

Sarah predicted that her teaching method (independent variable: teaching method), whereby she not only required her students to attend lectures, but also seminars, would have a positive effect (that is, increased) students' performance (dependent variable: exam marks). If an alternative hypothesis has a direction (and this is how you want to test it), the hypothesis is one-tailed. That is, it predicts direction of the effect. If the alternative hypothesis has stated that the effect was expected to be negative, this is also a one-tailed hypothesis.

Alternatively, a two-tailed prediction means that we do not make a choice over the direction that the effect of the experiment takes. Rather, it simply implies that the effect could be negative or positive. If Sarah had made a two-tailed prediction, the alternative hypothesis might have been:

In other words, we simply take out the word "positive", which implies the direction of our effect. In our example, making a two-tailed prediction may seem strange. After all, it would be logical to expect that "extra" tuition (going to seminar classes as well as lectures) would either have a positive effect on students' performance or no effect at all, but certainly not a negative effect. However, this is just our opinion (and hope) and certainly does not mean that we will get the effect we expect. Generally speaking, making a one-tail prediction (i.e., and testing for it this way) is frowned upon as it usually reflects the hope of a researcher rather than any certainty that it will happen. Notable exceptions to this rule are when there is only one possible way in which a change could occur. This can happen, for example, when biological activity/presence in measured. That is, a protein might be "dormant" and the stimulus you are using can only possibly "wake it up" (i.e., it cannot possibly reduce the activity of a "dormant" protein). In addition, for some statistical tests, one-tailed tests are not possible.

Rejecting or failing to reject the null hypothesis

Let's return finally to the question of whether we reject or fail to reject the null hypothesis.

If our statistical analysis shows that the significance level is below the cut-off value we have set (e.g., either 0.05 or 0.01), we reject the null hypothesis and accept the alternative hypothesis. Alternatively, if the significance level is above the cut-off value, we fail to reject the null hypothesis and cannot accept the alternative hypothesis. You should note that you cannot accept the null hypothesis, but only find evidence against it.

Support or Reject Null Hypothesis in Easy Steps

What does it mean to reject the null hypothesis.

• General Situations: P Value
• P Value Guidelines
• A Proportion
• A Proportion (second example)

In many statistical tests, you’ll want to either reject or support the null hypothesis . For elementary statistics students, the term can be a tricky term to grasp, partly because the name “null hypothesis” doesn’t make it clear about what the null hypothesis actually is!

The null hypothesis can be thought of as a nullifiable hypothesis. That means you can nullify it, or reject it. What happens if you reject the null hypothesis? It gets replaced with the alternate hypothesis, which is what you think might actually be true about a situation. For example, let’s say you think that a certain drug might be responsible for a spate of recent heart attacks. The drug company thinks the drug is safe. The null hypothesis is always the accepted hypothesis; in this example, the drug is on the market, people are using it, and it’s generally accepted to be safe. Therefore, the null hypothesis is that the drug is safe. The alternate hypothesis — the one you want to replace the null hypothesis, is that the drug isn’t safe. Rejecting the null hypothesis in this case means that you will have to prove that the drug is not safe.

To reject the null hypothesis, perform the following steps:

Step 1: State the null hypothesis. When you state the null hypothesis, you also have to state the alternate hypothesis. Sometimes it is easier to state the alternate hypothesis first, because that’s the researcher’s thoughts about the experiment. How to state the null hypothesis (opens in a new window).

Step 2: Support or reject the null hypothesis . Several methods exist, depending on what kind of sample data you have. For example, you can use the P-value method. For a rundown on all methods, see: Support or reject the null hypothesis.

If you are able to reject the null hypothesis in Step 2, you can replace it with the alternate hypothesis.

That’s it!

When to Reject the Null hypothesis

Basically, you reject the null hypothesis when your test value falls into the rejection region . There are four main ways you’ll compute test values and either support or reject your null hypothesis. Which method you choose depends mainly on if you have a proportion or a p-value .

Support or Reject the Null Hypothesis: Steps

Click the link the skip to the situation you need to support or reject null hypothesis for: General Situations: P Value P Value Guidelines A Proportion A Proportion (second example)

Support or Reject Null Hypothesis with a P Value

If you have a P-value , or are asked to find a p-value, follow these instructions to support or reject the null hypothesis. This method works if you are given an alpha level and if you are not given an alpha level. If you are given a confidence level , just subtract from 1 to get the alpha level. See: How to calculate an alpha level .

Step 1: State the null hypothesis and the alternate hypothesis (“the claim”). If you aren’t sure how to do this, follow this link for How To State the Null and Alternate Hypothesis .

Step 2: Find the critical value . We’re dealing with a normally distributed population, so the critical value is a z-score . Use the following formula to find the z-score .

Click here if you want easy, step-by-step instructions for solving this formula.

Step 4: Find the P-Value by looking up your answer from step 3 in the z-table . To get the p-value, subtract the area from 1. For example, if your area is .990 then your p-value is 1-.9950 = 0.005. Note: for a two-tailed test , you’ll need to halve this amount to get the p-value in one tail.

Step 5: Compare your answer from step 4 with the α value given in the question. Should you support or reject the null hypothesis? If step 7 is less than or equal to α, reject the null hypothesis, otherwise do not reject it.

P-Value Guidelines

Use these general guidelines to decide if you should reject or keep the null:

If p value > .10 → “not significant ” If p value ≤ .10 → “marginally significant” If p value ≤ .05 → “significant” If p value ≤ .01 → “highly significant.”

Back to Top

Support or Reject Null Hypothesis for a Proportion

Sometimes, you’ll be given a proportion of the population or a percentage and asked to support or reject null hypothesis. In this case you can’t compute a test value by calculating a z-score (you need actual numbers for that), so we use a slightly different technique.

Example question: A researcher claims that Democrats will win the next election. 4300 voters were polled; 2200 said they would vote Democrat. Decide if you should support or reject null hypothesis. Is there enough evidence at α=0.05 to support this claim?

Step 1: State the null hypothesis and the alternate hypothesis (“the claim”) . H o :p ≤ 0.5 H 1 :p > .5

Step 3: Use the following formula to calculate your test value.

Where: Phat is calculated in Step 2 P the null hypothesis p value (.05) Q is 1 – p

The z-score is: .512 – .5 / √(.5(.5) / 4300)) = 1.57

Step 4: Look up Step 3 in the z-table to get .9418.

Step 5: Calculate your p-value by subtracting Step 4 from 1. 1-.9418 = .0582

Step 6: Compare your answer from step 5 with the α value given in the question . Support or reject the null hypothesis? If step 5 is less than α, reject the null hypothesis, otherwise do not reject it. In this case, .582 (5.82%) is not less than our α, so we do not reject the null hypothesis.

Support or Reject Null Hypothesis for a Proportion: Second example

Example question: A researcher claims that more than 23% of community members go to church regularly. In a recent survey, 126 out of 420 people stated they went to church regularly. Is there enough evidence at α = 0.05 to support this claim? Use the P-Value method to support or reject null hypothesis.

Step 1: State the null hypothesis and the alternate hypothesis (“the claim”) . H o :p ≤ 0.23; H 1 :p > 0.23 (claim)

Step 3: Find ‘p’ by converting the stated claim to a decimal: 23% = 0.23. Also, find ‘q’ by subtracting ‘p’ from 1: 1 – 0.23 = 0.77.

Step 4: Use the following formula to calculate your test value.

If formulas confuse you, this is asking you to:

• Multiply p and q together, then divide by the number in the random sample. (0.23 x 0.77) / 420 = 0.00042
• Take the square root of your answer to 2 . √( 0.1771) = 0. 0205
• Divide your answer to 1. by your answer in 3. 0.07 / 0. 0205 = 3.41

Step 5: Find the P-Value by looking up your answer from step 5 in the z-table . The z-score for 3.41 is .4997. Subtract from 0.500: 0.500-.4997 = 0.003.

Step 6: Compare your P-value to α . Support or reject null hypothesis? If the P-value is less, reject the null hypothesis. If the P-value is more, keep the null hypothesis. 0.003 < 0.05, so we have enough evidence to reject the null hypothesis and accept the claim.

Note: In Step 5, I’m using the z-table on this site to solve this problem. Most textbooks have the right of z-table . If you’re seeing .9997 as an answer in your textbook table, then your textbook has a “whole z” table, in which case don’t subtract from .5, subtract from 1. 1-.9997 = 0.003.

Check out our Youtube channel for video tips!

Everitt, B. S.; Skrondal, A. (2010), The Cambridge Dictionary of Statistics , Cambridge University Press. Gonick, L. (1993). The Cartoon Guide to Statistics . HarperPerennial.

What 'Fail to Reject' Means in a Hypothesis Test

Casarsa Guru/Getty Images

• Inferential Statistics
• Statistics Tutorials
• Probability & Games
• Descriptive Statistics
• Applications Of Statistics
• Math Tutorials
• Pre Algebra & Algebra
• Exponential Decay
• Worksheets By Grade
• Ph.D., Mathematics, Purdue University
• M.S., Mathematics, Purdue University
• B.A., Mathematics, Physics, and Chemistry, Anderson University

In statistics , scientists can perform a number of different significance tests to determine if there is a relationship between two phenomena. One of the first they usually perform is a null hypothesis test. In short, the null hypothesis states that there is no meaningful relationship between two measured phenomena. After a performing a test, scientists can:

• Reject the null hypothesis (meaning there is a definite, consequential relationship between the two phenomena), or
• Fail to reject the null hypothesis (meaning the test has not identified a consequential relationship between the two phenomena)

Key Takeaways: The Null Hypothesis

• In a test of significance, the null hypothesis states that there is no meaningful relationship between two measured phenomena.

• By comparing the null hypothesis to an alternative hypothesis, scientists can either reject or fail to reject the null hypothesis.

• The null hypothesis cannot be positively proven. Rather, all that scientists can determine from a test of significance is that the evidence collected does or does not disprove the null hypothesis.

It is important to note that a failure to reject does not mean that the null hypothesis is true—only that the test did not prove it to be false. In some cases, depending on the experiment, a relationship may exist between two phenomena that is not identified by the experiment. In such cases, new experiments must be designed to rule out alternative hypotheses.

Null vs. Alternative Hypothesis

The null hypothesis is considered the default in a scientific experiment . In contrast, an alternative hypothesis is one that claims that there is a meaningful relationship between two phenomena. These two competing hypotheses can be compared by performing a statistical hypothesis test, which determines whether there is a statistically significant relationship between the data.

For example, scientists studying the water quality of a stream may wish to determine whether a certain chemical affects the acidity of the water. The null hypothesis—that the chemical has no effect on the water quality—can be tested by measuring the pH level of two water samples, one of which contains some of the chemical and one of which has been left untouched. If the sample with the added chemical is measurably more or less acidic—as determined through statistical analysis—it is a reason to reject the null hypothesis. If the sample's acidity is unchanged, it is a reason to not reject the null hypothesis.

When scientists design experiments, they attempt to find evidence for the alternative hypothesis. They do not try to prove that the null hypothesis is true. The null hypothesis is assumed to be an accurate statement until contrary evidence proves otherwise. As a result, a test of significance does not produce any evidence pertaining to the truth of the null hypothesis.

Failing to Reject vs. Accept

In an experiment, the null hypothesis and the alternative hypothesis should be carefully formulated such that one and only one of these statements is true. If the collected data supports the alternative hypothesis, then the null hypothesis can be rejected as false. However, if the data does not support the alternative hypothesis, this does not mean that the null hypothesis is true. All it means is that the null hypothesis has not been disproven—hence the term "failure to reject." A "failure to reject" a hypothesis should not be confused with acceptance.

In mathematics, negations are typically formed by simply placing the word “not” in the correct place. Using this convention, tests of significance allow scientists to either reject or not reject the null hypothesis. It sometimes takes a moment to realize that “not rejecting” is not the same as "accepting."

Null Hypothesis Example

In many ways, the philosophy behind a test of significance is similar to that of a trial. At the beginning of the proceedings, when the defendant enters a plea of “not guilty,” it is analogous to the statement of the null hypothesis. While the defendant may indeed be innocent, there is no plea of “innocent” to be formally made in court. The alternative hypothesis of “guilty” is what the prosecutor attempts to demonstrate.

The presumption at the outset of the trial is that the defendant is innocent. In theory, there is no need for the defendant to prove that he or she is innocent. The burden of proof is on the prosecuting attorney, who must marshal enough evidence to convince the jury that the defendant is guilty beyond a reasonable doubt. Likewise, in a test of significance, a scientist can only reject the null hypothesis by providing evidence for the alternative hypothesis.

If there is not enough evidence in a trial to demonstrate guilt, then the defendant is declared “not guilty.” This claim has nothing to do with innocence; it merely reflects the fact that the prosecution failed to provide enough evidence of guilt. In a similar way, a failure to reject the null hypothesis in a significance test does not mean that the null hypothesis is true. It only means that the scientist was unable to provide enough evidence for the alternative hypothesis.

For example, scientists testing the effects of a certain pesticide on crop yields might design an experiment in which some crops are left untreated and others are treated with varying amounts of pesticide. Any result in which the crop yields varied based on pesticide exposure—assuming all other variables are equal—would provide strong evidence for the alternative hypothesis (that the pesticide does affect crop yields). As a result, the scientists would have reason to reject the null hypothesis.

• Null Hypothesis Examples
• Hypothesis Test for the Difference of Two Population Proportions
• Type I and Type II Errors in Statistics
• Null Hypothesis and Alternative Hypothesis
• How to Conduct a Hypothesis Test
• An Example of a Hypothesis Test
• What Is a P-Value?
• The Difference Between Type I and Type II Errors in Hypothesis Testing
• What Is a Hypothesis? (Science)
• Null Hypothesis Definition and Examples
• Hypothesis Test Example
• What Level of Alpha Determines Statistical Significance?
• How to Do Hypothesis Tests With the Z.TEST Function in Excel
• The Runs Test for Random Sequences
• What Is the Difference Between Alpha and P-Values?
• Scientific Method Vocabulary Terms

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Hypothesis Testing and Confidence Intervals

By Jim Frost 20 Comments

Confidence intervals and hypothesis testing are closely related because both methods use the same underlying methodology. Additionally, there is a close connection between significance levels and confidence levels. Indeed, there is such a strong link between them that hypothesis tests and the corresponding confidence intervals always agree about statistical significance.

A confidence interval is calculated from a sample and provides a range of values that likely contains the unknown value of a population parameter . To learn more about confidence intervals in general, how to interpret them, and how to calculate them, read my post about Understanding Confidence Intervals .

In this post, I demonstrate how confidence intervals work using graphs and concepts instead of formulas. In the process, I compare and contrast significance and confidence levels. You’ll learn how confidence intervals are similar to significance levels in hypothesis testing. You can even use confidence intervals to determine statistical significance.

Read the companion post for this one: How Hypothesis Tests Work: Significance Levels (Alpha) and P-values . In that post, I use the same graphical approach to illustrate why we need hypothesis tests, how significance levels and P-values can determine whether a result is statistically significant, and what that actually means.

Significance Level vs. Confidence Level

Let’s delve into how confidence intervals incorporate the margin of error. Like the previous post, I’ll use the same type of sampling distribution that showed us how hypothesis tests work. This sampling distribution is based on the t-distribution , our sample size , and the variability in our sample. Download the CSV data file: FuelsCosts .

There are two critical differences between the sampling distribution graphs for significance levels and confidence intervals–the value that the distribution centers on and the portion we shade.

The significance level chart centers on the null value, and we shade the outside 5% of the distribution.

Conversely, the confidence interval graph centers on the sample mean, and we shade the center 95% of the distribution.

The shaded range of sample means [267 394] covers 95% of this sampling distribution. This range is the 95% confidence interval for our sample data. We can be 95% confident that the population mean for fuel costs fall between 267 and 394.

Confidence Intervals and the Inherent Uncertainty of Using Sample Data

The graph emphasizes the role of uncertainty around the point estimate . This graph centers on our sample mean. If the population mean equals our sample mean, random samples from this population (N=25) will fall within this range 95% of the time.

We don’t know whether our sample mean is near the population mean. However, we know that the sample mean is an unbiased estimate of the population mean. An unbiased estimate does not tend to be too high or too low. It’s correct on average. Confidence intervals are correct on average because they use sample estimates that are correct on average. Given what we know, the sample mean is the most likely value for the population mean.

Given the sampling distribution, it would not be unusual for other random samples drawn from the same population to have means that fall within the shaded area. In other words, given that we did, in fact, obtain the sample mean of 330.6, it would not be surprising to get other sample means within the shaded range.

If these other sample means would not be unusual, we must conclude that these other values are also plausible candidates for the population mean. There is inherent uncertainty when using sample data to make inferences about the entire population. Confidence intervals help gauge the degree of uncertainty, also known as the margin of error.

Related post : Sampling Distributions

Confidence Intervals and Statistical Significance

If you want to determine whether your hypothesis test results are statistically significant, you can use either P-values with significance levels or confidence intervals. These two approaches always agree.

The relationship between the confidence level and the significance level for a hypothesis test is as follows:

Confidence level = 1 – Significance level (alpha)

For example, if your significance level is 0.05, the equivalent confidence level is 95%.

Both of the following conditions represent statistically significant results:

• The P-value in a hypothesis test is smaller than the significance level.
• The confidence interval excludes the null hypothesis value.

Further, it is always true that when the P-value is less than your significance level, the interval excludes the value of the null hypothesis.

In the fuel cost example, our hypothesis test results are statistically significant because the P-value (0.03112) is less than the significance level (0.05). Likewise, the 95% confidence interval [267 394] excludes the null hypotheses value (260). Using either method, we draw the same conclusion.

Hypothesis Testing and Confidence Intervals Always Agree

The hypothesis testing and confidence interval results always agree. To understand the basis of this agreement, remember how confidence levels and significance levels function:

• A confidence level determines the distance between the sample mean and the confidence limits.
• A significance level determines the distance between the null hypothesis value and the critical regions.

Both of these concepts specify a distance from the mean to a limit. Surprise! These distances are precisely the same length.

A 1-sample t-test calculates this distance as follows:

The critical t-value * standard error of the mean

Interpreting these statistics goes beyond the scope of this article. But, using this equation, the distance for our fuel cost example is $63.57.

P-value and significance level approach : If the sample mean is more than $63.57 from the null hypothesis mean, the sample mean falls within the critical region, and the difference is statistically significant.

Confidence interval approach : If the null hypothesis mean is more than $63.57 from the sample mean, the interval does not contain this value, and the difference is statistically significant.

Of course, they always agree!

The two approaches always agree as long as the same hypothesis test generates the P-values and confidence intervals and uses equivalent confidence levels and significance levels.

Related posts : Standard Error of the Mean and Critical Values

I Really Like Confidence Intervals!

In statistics, analysts often emphasize using hypothesis tests to determine statistical significance. Unfortunately, a statistically significant effect might not always be practically meaningful. For example, a significant effect can be too small to be important in the real world. Confidence intervals help you navigate this issue!

Similarly, the margin of error in a survey tells you how near you can expect the survey results to be to the correct population value.

Learn more about this distinction in my post about Practical vs. Statistical Significance .

Learn how to use confidence intervals to compare group means !

Finally, learn about bootstrapping in statistics to see an alternative to traditional confidence intervals that do not use probability distributions and test statistics. In that post, I create bootstrapped confidence intervals.

Neyman, J. (1937).  Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability .  Philosophical Transactions of the Royal Society A .  236  (767): 333–380.

Share this:

Reader Interactions

December 7, 2021 at 3:14 pm

I am helping my Physics students use their data to determine whether they can say momentum is conserved. One of the columns in their data chart was change in momentum and ultimately we want this to be 0. They are obviously not getting zero from their data because of outside factors. How can I explain to them that their data supports or does not support conservation of momentum using statistics? They are using a 95% confidence level. Again, we want the change in momentum to be 0. Thank you.

December 9, 2021 at 6:54 pm

I can see several complications with that approach and also my lack of familiarity with the subject area limits what I can say. But here are some considerations.

For starters, I’m unsure whether the outside factors you mention bias the results systematically from zero or just add noise (variability) to the data (but not systematically bias).

If the outside factors bias the results to a non-zero value, then you’d expect the case where larger samples will be more likely to produce confidence intervals that exclude zero. Indeed, only smaller samples sizes might produce CIs that include zero, but that would only be due to the relative lack of precision associated with small sample sizes. In other words, limited data won’t be able to distinguish the sample value from zero even though, given the bias of the outside factors, you’d expect a non-zero value. In other words, if the bias exists, the larger samples will detect the non-zero values correctly while smaller samples might miss it.

If the outside factors don’t bias the results but just add noise, then you’d expect that both small and larger samples will include zero. However, you still have the issue of precision. Smaller samples will include zero because they’re relatively wider intervals. Larger samples should include zero but have narrower intervals. Obviously, you can trust the larger samples more.

In hypothesis testing, when you fail to reject the null, as occurs in the unbiased discussion above, you’re not accepting the null . Click the link to read about that. Failing to reject the null does not mean that the population value equals the hypothesized value (zero in your case). That’s because you can fail to reject the null due to poor quality data (high noise and/or small sample sizes). And you don’t want to draw conclusions based on poor data.

There’s a class of hypothesis testing called equivalence testing that you should use in this case. It flips the null and alternative hypotheses so that the test requires you to collect strong evidence to show that the sample value equals the null value (again, zero in your case). I don’t have a post on that topic (yet), but you can read the Wikipedia article about Equivalence Testing .

I hope that helps!

September 19, 2021 at 5:16 am

Thank you very much. When training a machine learning model using bootstrap, in the end we will have the confidence interval of accuracy. How can I say that this result is statistically significant? Do I have to convert the confidence interval to p-values first and if p-value is less than 0.05, then it is statistically significant?

September 19, 2021 at 3:16 pm

As I mention in this article, you determine significance using a confidence interval by assessing whether it excludes the null hypothesis value. When it excludes the null value, your results are statistically significant.

September 18, 2021 at 12:47 pm

Dear Jim, Thanks for this post. I am new to hypothesis testing and would like to ask you how we know that the null hypotheses value is equal to 260.

Thank you. Kind regards, Loukas

September 19, 2021 at 12:35 am

For this example, the null hypothesis is 260 because that is the value from the previous year and they wanted to compare the current year to the previous year. It’s defined as the previous year value because the goal of the study was to determine whether it has changed since last year.

In general, the null hypothesis will often be a meaningful target value for the study based on their knowledge, such as this case. In other cases, they’ll use a value that represents no effect, such as zero.

I hope that helps clarify it!

February 22, 2021 at 3:49 pm

Hello, Mr. Jim Frost.

Thank you for publishing precise information about statistics, I always read your posts and bought your excellent e-book about regression! I really learn from you.

I got a couple of questions about the confidence level of the confidence intervals. Jacob Cohen, in his article “things I’ve learned (so far)” said that, in his experience, the most useful and informative confidence level is 80%; other authors state that if that level is below 90% it would be very hard to compare across results, as it is uncommon.

My first question is: in exploratory studies, with small samples (for example, N=85), if one wishes to generate correlational hypothesis for future research, would it be better to use a lower confidence level? What is the lowest level you would consider to be acceptable? I ask that because of my own research now, and with a sample size 85 (non-probabilistic sampling) I know all I can do is generate some hypothesis to be explored in the future, so I would like my confidence intervals to be more informative, because I am not looking forward to generalize to the population.

My second question is: could you please provide an example of an appropriate way to describe the information about the confidence interval values/limits, beyond the classic “it contains a difference of 0; it contains a ratio of 1”.

I would really appreciate your answers.

Greetings from Peru!

February 23, 2021 at 4:51 pm

Thanks so much for your kind words and for supporting my regression ebook! I’m glad it’s been helpful! 🙂

On to your questions!

I haven’t read Cohen’s article, so I don’t understand his rationale. However, I’m extremely dubious of using a confidence level as low as 80%. Lowering the confidence level will create a narrower CI, which looks good. However, it comes at the expense of dramatically increasing the likelihood that the CI won’t contain the correct population value! My position is to leave the confidence level at 95%. Or, possibly lower it to 90%. But, I wouldn’t go further. Your CI will be wider, but that’s OK. It’s reflecting the uncertainty that truly exists in your data. That’s important. The problem with lowering the CIs is that it makes your results appear more precise than they actually are.

When I think of exploratory research, I think of studies that are looking at tendencies or trends. Is the overall pattern of results consistent with theoretical expectations and justify further research? At that stage, it shouldn’t be about obtaining statistically significant results–at least not as the primary objective. Additionally, exploratory research can help you derive estimated effect sizes, variability, etc. that you can use for power calculations . A smaller, exploratory study can also help you refine your methodology and not waste your resources by going straight to a larger study that, as a result, might not be as refined as it would without a test run in the smaller study. Consequently, obtaining significant results, or results that look precise when they aren’t, aren’t the top priorities.

I know that lowering the confidence level makes your CI look more information but that is deceptive! I’d resist that temptation. Maybe go down to 90%. Personally, I would not go lower.

As for the interpretation, CIs indicate the likely range that a population parameter is likely to fall within. The parameter can be a mean, effect size, ratio, etc. Often times, you as the researcher are hoping the CI excludes an important value. For example, if the CI is of the effect size, you want the CI to exclude zero (no effect). In that case, you can say that there is unlikely to be no effect in the population (i.e., there probably is a non-zero effect in the population). Additionally, the effect size is likely to be within this range. Other times, you might just want to know the range of values itself. For example, if you have a CI for the mean height of a population, it might be valuable on its own knowing that the population mean height is likely to fall between X and Y. If you have specific example of what the CI assesses, I can give you a more specific interpretation.

Additionally, I cover confidence intervals associated with many different types of hypothesis tests in my Hypothesis Testing ebook . You might consider looking in to that!

July 26, 2020 at 5:45 am

I got a very wide 95% CI of the HR of height in the cox PH model from a very large sample. I already deleted the outliers defined as 1.5 IQR, but it doesn’t work. Do you know how to resolve it?

July 5, 2020 at 6:13 pm

Hello, Jim!

I appreciate the thoughtful and thorough answer you provided. It really helped in crystallizing the topic for me.

If I may ask for a bit more of your time, as long as we are talking about CIs I have another question:

How would you go about constructing a CI for the difference of variances?

I am asking because while creating CIs for the difference of means or proportions is relatively straightforward, I couldn’t find any references for the difference of variances in any of my textbooks (or on the Web for that matter); I did find information regarding CIs for the ratio of variances, but it’s not the same thing.

Could you help me with that?

Thanks a lot!

July 2, 2020 at 6:01 pm

I want to start by thanking you for a great post and an overall great blog! Top notch material.

I have a doubt regarding the difference between confidence intervals for a point estimate and confidence intervals for a hypothesis test.

As I understand, if we are using CIs to test a hypothesis, then our point estimate would be whatever the null hypothesis is; conversely, if we are simply constructing a CI to go along with our point estimate, we’d use the point estimate derived from our sample. Am I correct so far?

The reason I am asking is that because while reading from various sources, I’ve never found a distinction between the two cases, and they seem very different to me.

Bottom line, what I am trying to ask is: assuming the null hypothesis is true, shouldn’t the CI be changed?

Thank you very much for your attention!

July 3, 2020 at 4:02 pm

There’s no difference in the math behind the scenes. The real difference is that when you create a confidence interval in conjunction with a hypothesis test, the software ensures that they’re using consistent methodology. For example, the significance level and confidence level will correspond correctly (i.e., alpha = 0.05 and confidence level = 0.95). Additionally, if you perform a two-tailed test, you will obtain a two-sided CI. On the other hand, if you perform a one-tailed test, you will obtain the appropriate upper or lower bound (i.e., one-sided CIs). The software also ensures any other methodological choices you make will match between the hypothesis test and CI, which ensures the results always agree.

You can perform them separately. However, if you don’t match all the methodology options, the results can differ.

As for your question about assuming the null is true. Keep in mind that hypothesis tests create sampling distributions that center on the null hypothesis value. That’s the assumption that the null is true. However, the sampling distributions for CIs center on the sample estimate. So, yes, CIs change that detail because they don’t assume the null is correct. But that’s always true whether you perform the hypothesis test or not.

Thanks for the great questions!

December 21, 2019 at 6:31 am

Confidence interval has sample static as the most likely value ( value in the center) – and sample distribution assumes the null value to be the most likely value( value in the center). I am a little confused about this. Would be really kind of you if you could show both in the same graph and explain how both are related. How the the distance from the mean to a limit in case of Significance level and CI same?

December 23, 2019 at 3:46 am

That’s a great question. I think part of your confusion is due to terminology.

The sampling distribution of the means centers on the sample mean. This sampling distribution uses your sample mean as its mean and the standard error of the mean as its standard deviation.

The sampling distribution of the test statistic (t) centers on the null hypothesis value (0). This distribution uses zero as its mean and also uses the SEM for its standard deviation.

They’re two different things and center on different points. But, they both incorporate the SEM, which is why they always agree! I do have section in this post about why that distance is always the same. Look for the section titled, “Why They Always Agree.”

November 23, 2019 at 11:31 pm

Hi Jim, I’m the proud owner of 2 of your ebooks. There’s one topic though that keeps puzzling me: If I would take 9 samples of size 15 in order to estimate the population mean, the se of the mean would be substantial larger than if I would take 1 sample of size 135 (divide pop sd by sqrt(15) or sqrt(135) ) whereas the E(x) (or mean of means) would be the same.

Can you please shine a little light on that.

Tx in advance

November 24, 2019 at 3:17 am

Thanks so much for supporting my ebooks. I really appreciate that!! 🙂

So, let’s flip that scenario around. If you know that a single large sample of 135 will produce more precise estimates of the population, why would you collect nine smaller samples? Knowing how statistics works, that’s not a good decision. If you did that in the real world, it would be because there was some practical reason that you could not collect one big example. Further, it would suggest that you had some reason for not being able to combine them later. For example, if you follow the same random sampling procedure on the same population and used all the same methodology and at the same general time, you might feel comfortable combining them together into one larger sample. So, if you couldn’t collect one larger example and you didn’t feel comfortable combining them together, it suggests that you have some reason for doubting that they all measure the same thing for the same population. Maybe you had differences in methodology? Or subjective measurements across different personnel? Or, maybe you collected the samples at different times and you’re worried that the population changed over time?

So, that’s the real world reason for why a researcher would not combine smaller samples into a larger one.

As you can see, the expected value for the population standard deviation is in the numerator (sigma). As the sample size increases, the numerator remains constant (plus or minus random error) because the expected value for the population parameter does not change. Conversely, the square root of the sample size is in the denominator. As the sample size increases, it produces a larger values in the denominator. So, if the expected value of the numerator is constant but the value of the denominator increases with a larger sample size, you expect the SEM to decrease. Smaller SEM’s indicate more precise estimates of the population parameter. For instance, the equations for confidence intervals use the SEM. Hence, for the same population, larger samples tend to produce smaller SEMS, and more precise estimates of the population parameter.

I hope that answers your question!

November 6, 2018 at 10:26 am

first of all: Thanks for your effort and your effective way of explaining!

You say that p-values and C.I.s always agree. I agree.

Why does Tim van der Zee claim the opposite? I’m not enough into statistcs to figure this out.

http://www.timvanderzee.com/not-interpret-confidence-intervals/

Best regards Georg

November 7, 2018 at 9:31 am

I think he is saying that they do agree–just that people often compare the wrong pair of CIs and p-values. I assume you’re referring to the section “What do overlapping intervals (not) mean?” And, he’s correct in what he says. In a 2-sample t-test, it’s not valid to compare the CI for each of the two group means to the test’s p-values because they have different purposes. Consequently, they won’t necessarily agree. However, that’s because you’re comparing results from two different tests/intervals.

On the one hand, you have the CIs for each group. On the other hand, you have the p-value for the difference between the two groups. Those are not the same thing and so it’s not surprising that they won’t agree necessarily.

However, if you compare the p-value of the difference between means to a CI of the difference between means, they will always agree. You have to compare apples to apples!

April 14, 2018 at 8:54 pm

First of all, I love all your posts and you really do make people appreciate statistics by explaining it intuitively compared to theoretical approaches I’ve come across in university courses and other online resources. Please continue the fantastic work!!!

At the end, you mentioned how you prefer confidence intervals as they consider both “size and precision of the estimated effect”. I’m confused as to what exactly size and precision mean in this context. I’d appreciate an explanation with reference to specific numbers from the example above.

Second, do p-values lack both size and precision in determination of statistical significance?

Thanks, Devansh

April 17, 2018 at 11:41 am

Hi Devansh,

Thanks for the nice comments. I really appreciate them!

I really need to write a post specifically about this issue.

Let’s first assume that we conduct our study and find that the mean cost is 330.6 and that we are testing whether that is different than 260. Further suppose that we perform the the hypothesis test and obtain a p-value that is statistically significant. We can reject the null and conclude that population mean does not equal 260. And we can see our sample estimate is 330.6. So, that’s what we learn using p-values and the sample estimate.

Confidence intervals add to that information. We know that if we were to perform the experiment again, we’d get different results. How different? Is the true population mean likely to be close to 330.6 or further away? CIs help us answer these questions. The 95% CI is [267 394]. The true population value is likely to be within this range. That range spans 127 dollars.

However, let’s suppose we perform the experiment again but this time use a much larger sample size and obtain a mean of 351 and again a significant p-value. However, thanks to the large sample size, we obtain a 95 CI of [340 362]. Now we know that the population value is likely to fall within this much tighter interval of only 22 dollars. This estimate is much more precise.

Sometimes you can obtain a significant p-value for a result that is too imprecise to be useful. For example, with first CI, it might be too wide to be useful for what we need to do with our results. Maybe we’re helping people make budgets and that is too wide to allow for practical planning. However, the more precise estimate of the second study allows for better budgetary planning! That determination how much precision is required must be made using subject-area knowledge and focusing on the practical usage of the results. P-values don’t indicate the precision of the estimates in this manner!

I hope this helps clarify this precision issue!

Comments and Questions Cancel reply

User Preferences

Content preview.

Arcu felis bibendum ut tristique et egestas quis:

• Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
• Duis aute irure dolor in reprehenderit in voluptate
• Excepteur sint occaecat cupidatat non proident

Keyboard Shortcuts

S.3.1 hypothesis testing (critical value approach).

The critical value approach involves determining "likely" or "unlikely" by determining whether or not the observed test statistic is more extreme than would be expected if the null hypothesis were true. That is, it entails comparing the observed test statistic to some cutoff value, called the " critical value ." If the test statistic is more extreme than the critical value, then the null hypothesis is rejected in favor of the alternative hypothesis. If the test statistic is not as extreme as the critical value, then the null hypothesis is not rejected.

Specifically, the four steps involved in using the critical value approach to conducting any hypothesis test are:

• Specify the null and alternative hypotheses.
• Using the sample data and assuming the null hypothesis is true, calculate the value of the test statistic. To conduct the hypothesis test for the population mean μ , we use the t -statistic \(t^*=\frac{\bar{x}-\mu}{s/\sqrt{n}}\) which follows a t -distribution with n - 1 degrees of freedom.
• Determine the critical value by finding the value of the known distribution of the test statistic such that the probability of making a Type I error — which is denoted \(\alpha\) (greek letter "alpha") and is called the " significance level of the test " — is small (typically 0.01, 0.05, or 0.10).
• Compare the test statistic to the critical value. If the test statistic is more extreme in the direction of the alternative than the critical value, reject the null hypothesis in favor of the alternative hypothesis. If the test statistic is less extreme than the critical value, do not reject the null hypothesis.

Example S.3.1.1

Mean gpa section  .

In our example concerning the mean grade point average, suppose we take a random sample of n = 15 students majoring in mathematics. Since n = 15, our test statistic t * has n - 1 = 14 degrees of freedom. Also, suppose we set our significance level α at 0.05 so that we have only a 5% chance of making a Type I error.

Right-Tailed

The critical value for conducting the right-tailed test H 0 : μ = 3 versus H A : μ > 3 is the t -value, denoted t \(\alpha\) , n - 1 , such that the probability to the right of it is \(\alpha\). It can be shown using either statistical software or a t -table that the critical value t 0.05,14 is 1.7613. That is, we would reject the null hypothesis H 0 : μ = 3 in favor of the alternative hypothesis H A : μ > 3 if the test statistic t * is greater than 1.7613. Visually, the rejection region is shaded red in the graph.

Left-Tailed

The critical value for conducting the left-tailed test H 0 : μ = 3 versus H A : μ < 3 is the t -value, denoted -t ( \(\alpha\) , n - 1) , such that the probability to the left of it is \(\alpha\). It can be shown using either statistical software or a t -table that the critical value -t 0.05,14 is -1.7613. That is, we would reject the null hypothesis H 0 : μ = 3 in favor of the alternative hypothesis H A : μ < 3 if the test statistic t * is less than -1.7613. Visually, the rejection region is shaded red in the graph.

There are two critical values for the two-tailed test H 0 : μ = 3 versus H A : μ ≠ 3 — one for the left-tail denoted -t ( \(\alpha\) / 2, n - 1) and one for the right-tail denoted t ( \(\alpha\) / 2, n - 1) . The value - t ( \(\alpha\) /2, n - 1) is the t -value such that the probability to the left of it is \(\alpha\)/2, and the value t ( \(\alpha\) /2, n - 1) is the t -value such that the probability to the right of it is \(\alpha\)/2. It can be shown using either statistical software or a t -table that the critical value -t 0.025,14 is -2.1448 and the critical value t 0.025,14 is 2.1448. That is, we would reject the null hypothesis H 0 : μ = 3 in favor of the alternative hypothesis H A : μ ≠ 3 if the test statistic t * is less than -2.1448 or greater than 2.1448. Visually, the rejection region is shaded red in the graph.

Statistics Made Easy

Understanding the Null Hypothesis for ANOVA Models

A one-way ANOVA is used to determine if there is a statistically significant difference between the mean of three or more independent groups.

A one-way ANOVA uses the following null and alternative hypotheses:

• H 0 :  μ 1  = μ 2  = μ 3  = … = μ k  (all of the group means are equal)
• H A : At least one group mean is different   from the rest

To decide if we should reject or fail to reject the null hypothesis, we must refer to the p-value in the output of the ANOVA table.

If the p-value is less than some significance level (e.g. 0.05) then we can reject the null hypothesis and conclude that not all group means are equal.

A two-way ANOVA is used to determine whether or not there is a statistically significant difference between the means of three or more independent groups that have been split on two variables (sometimes called “factors”).

A two-way ANOVA tests three null hypotheses at the same time:

• All group means are equal at each level of the first variable
• All group means are equal at each level of the second variable
• There is no interaction effect between the two variables

To decide if we should reject or fail to reject each null hypothesis, we must refer to the p-values in the output of the two-way ANOVA table.

The following examples show how to decide to reject or fail to reject the null hypothesis in both a one-way ANOVA and two-way ANOVA.

Example 1: One-Way ANOVA

Suppose we want to know whether or not three different exam prep programs lead to different mean scores on a certain exam. To test this, we recruit 30 students to participate in a study and split them into three groups.

The students in each group are randomly assigned to use one of the three exam prep programs for the next three weeks to prepare for an exam. At the end of the three weeks, all of the students take the same exam. 

The exam scores for each group are shown below:

When we enter these values into the One-Way ANOVA Calculator , we receive the following ANOVA table as the output:

Notice that the p-value is 0.11385 .

For this particular example, we would use the following null and alternative hypotheses:

• H 0 :  μ 1  = μ 2  = μ 3 (the mean exam score for each group is equal)

Since the p-value from the ANOVA table is not less than 0.05, we fail to reject the null hypothesis.

This means we don’t have sufficient evidence to say that there is a statistically significant difference between the mean exam scores of the three groups.

Example 2: Two-Way ANOVA

Suppose a botanist wants to know whether or not plant growth is influenced by sunlight exposure and watering frequency.

She plants 40 seeds and lets them grow for two months under different conditions for sunlight exposure and watering frequency. After two months, she records the height of each plant. The results are shown below:

In the table above, we see that there were five plants grown under each combination of conditions.

For example, there were five plants grown with daily watering and no sunlight and their heights after two months were 4.8 inches, 4.4 inches, 3.2 inches, 3.9 inches, and 4.4 inches:

She performs a two-way ANOVA in Excel and ends up with the following output:

We can see the following p-values in the output of the two-way ANOVA table:

• The p-value for watering frequency is 0.975975 . This is not statistically significant at a significance level of 0.05.
• The p-value for sunlight exposure is 3.9E-8 (0.000000039) . This is statistically significant at a significance level of 0.05.
• The p-value for the interaction between watering  frequency and sunlight exposure is 0.310898 . This is not statistically significant at a significance level of 0.05.

These results indicate that sunlight exposure is the only factor that has a statistically significant effect on plant height.

And because there is no interaction effect, the effect of sunlight exposure is consistent across each level of watering frequency.

That is, whether a plant is watered daily or weekly has no impact on how sunlight exposure affects a plant.

Additional Resources

The following tutorials provide additional information about ANOVA models:

How to Interpret the F-Value and P-Value in ANOVA How to Calculate Sum of Squares in ANOVA What Does a High F Value Mean in ANOVA?

Featured Posts

Hey there. My name is Zach Bobbitt. I have a Masters of Science degree in Applied Statistics and I’ve worked on machine learning algorithms for professional businesses in both healthcare and retail. I’m passionate about statistics, machine learning, and data visualization and I created Statology to be a resource for both students and teachers alike.  My goal with this site is to help you learn statistics through using simple terms, plenty of real-world examples, and helpful illustrations.

2 Replies to “Understanding the Null Hypothesis for ANOVA Models”

Hi, I’m a student at Stellenbosch University majoring in Conservation Ecology and Entomology and we are currently busy doing stats. I am still at a very entry level of stats understanding, so pages like these are of huge help. I wanted to ask, why is the sum of squares (treatment) for the one way ANOVA so high? I calculated it by hand and got a much lower number, could you please help point out if and where I went wrong?

As I understand it, SSB (treatment) is calculated by finding the mean of each group and the grand mean, and then calculating the sum of squares like this: GM = 85.5 x1 = 83.4 x2 = 89.3 x3 = 84.7

SSB = (85.5 – 83.4)^2 + (85.5 – 89.3)^2 + (85.5 – 84.7)^2 = 18.65 DF = 2

I would appreciate any help, thank you so much!

Hi Theo…Certainly! Here are the equations rewritten as they would be typed in Python:

### Sum of Squares Between Groups (SSB)

In a one-way ANOVA, the sum of squares between groups (SSB) measures the variation due to the interaction between the groups. It is calculated as follows:

1. **Calculate the group means**: “`python mean_group1 = 83.4 mean_group2 = 89.3 mean_group3 = 84.7 “`

2. **Calculate the grand mean**: “`python grand_mean = 85.5 “`

3. **Calculate the sum of squares between groups (SSB)**: Assuming each group has `n` observations: “`python n = 10 # Number of observations in each group

ssb = n * ((mean_group1 – grand_mean)**2 + (mean_group2 – grand_mean)**2 + (mean_group3 – grand_mean)**2) “`

### Example Calculation

For simplicity, let’s assume each group has 10 observations: “`python n = 10

ssb = n * ((83.4 – 85.5)**2 + (89.3 – 85.5)**2 + (84.7 – 85.5)**2) “`

Now calculate each term: “`python term1 = (83.4 – 85.5)**2 # term1 = (-2.1)**2 = 4.41 term2 = (89.3 – 85.5)**2 # term2 = (3.8)**2 = 14.44 term3 = (84.7 – 85.5)**2 # term3 = (-0.8)**2 = 0.64 “`

Sum these squared differences: “`python sum_of_squared_diffs = term1 + term2 + term3 # sum_of_squared_diffs = 4.41 + 14.44 + 0.64 = 19.49 ssb = n * sum_of_squared_diffs # ssb = 10 * 19.49 = 194.9 “`

So, the sum of squares between groups (SSB) is 194.9, assuming each group has 10 observations.

### Degrees of Freedom (DF)

The degrees of freedom for SSB is calculated as: “`python df_between = k – 1 “` where `k` is the number of groups.

For three groups: “`python k = 3 df_between = k – 1 # df_between = 3 – 1 = 2 “`

### Summary

– **SSB** should consider the number of observations in each group. – **DF** is the number of groups minus one.

By ensuring you include the number of observations per group in your SSB calculation, you can get the correct SSB value.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Join the Statology Community

Sign up to receive Statology's exclusive study resource: 100 practice problems with step-by-step solutions. Plus, get our latest insights, tutorials, and data analysis tips straight to your inbox!

By subscribing you accept Statology's Privacy Policy.

P-Value And Statistical Significance: What It Is & Why It Matters

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

The p-value in statistics quantifies the evidence against a null hypothesis. A low p-value suggests data is inconsistent with the null, potentially favoring an alternative hypothesis. Common significance thresholds are 0.05 or 0.01.

Hypothesis testing

When you perform a statistical test, a p-value helps you determine the significance of your results in relation to the null hypothesis.

The null hypothesis (H0) states no relationship exists between the two variables being studied (one variable does not affect the other). It states the results are due to chance and are not significant in supporting the idea being investigated. Thus, the null hypothesis assumes that whatever you try to prove did not happen.

The alternative hypothesis (Ha or H1) is the one you would believe if the null hypothesis is concluded to be untrue.

The alternative hypothesis states that the independent variable affected the dependent variable, and the results are significant in supporting the theory being investigated (i.e., the results are not due to random chance).

What a p-value tells you

A p-value, or probability value, is a number describing how likely it is that your data would have occurred by random chance (i.e., that the null hypothesis is true).

The level of statistical significance is often expressed as a p-value between 0 and 1.

The smaller the p -value, the less likely the results occurred by random chance, and the stronger the evidence that you should reject the null hypothesis.

Remember, a p-value doesn’t tell you if the null hypothesis is true or false. It just tells you how likely you’d see the data you observed (or more extreme data) if the null hypothesis was true. It’s a piece of evidence, not a definitive proof.

Example: Test Statistic and p-Value

Suppose you’re conducting a study to determine whether a new drug has an effect on pain relief compared to a placebo. If the new drug has no impact, your test statistic will be close to the one predicted by the null hypothesis (no difference between the drug and placebo groups), and the resulting p-value will be close to 1. It may not be precisely 1 because real-world variations may exist. Conversely, if the new drug indeed reduces pain significantly, your test statistic will diverge further from what’s expected under the null hypothesis, and the p-value will decrease. The p-value will never reach zero because there’s always a slim possibility, though highly improbable, that the observed results occurred by random chance.

P-value interpretation

The significance level (alpha) is a set probability threshold (often 0.05), while the p-value is the probability you calculate based on your study or analysis.

A p-value less than or equal to your significance level (typically ≤ 0.05) is statistically significant.

A p-value less than or equal to a predetermined significance level (often 0.05 or 0.01) indicates a statistically significant result, meaning the observed data provide strong evidence against the null hypothesis.

This suggests the effect under study likely represents a real relationship rather than just random chance.

For instance, if you set α = 0.05, you would reject the null hypothesis if your p -value ≤ 0.05. 

It indicates strong evidence against the null hypothesis, as there is less than a 5% probability the null is correct (and the results are random).

Therefore, we reject the null hypothesis and accept the alternative hypothesis.

Example: Statistical Significance

Upon analyzing the pain relief effects of the new drug compared to the placebo, the computed p-value is less than 0.01, which falls well below the predetermined alpha value of 0.05. Consequently, you conclude that there is a statistically significant difference in pain relief between the new drug and the placebo.

What does a p-value of 0.001 mean?

A p-value of 0.001 is highly statistically significant beyond the commonly used 0.05 threshold. It indicates strong evidence of a real effect or difference, rather than just random variation.

Specifically, a p-value of 0.001 means there is only a 0.1% chance of obtaining a result at least as extreme as the one observed, assuming the null hypothesis is correct.

Such a small p-value provides strong evidence against the null hypothesis, leading to rejecting the null in favor of the alternative hypothesis.

A p-value more than the significance level (typically p > 0.05) is not statistically significant and indicates strong evidence for the null hypothesis.

This means we retain the null hypothesis and reject the alternative hypothesis. You should note that you cannot accept the null hypothesis; we can only reject it or fail to reject it.

Note : when the p-value is above your threshold of significance,  it does not mean that there is a 95% probability that the alternative hypothesis is true.

One-Tailed Test

Two-Tailed Test

How do you calculate the p-value ?

Most statistical software packages like R, SPSS, and others automatically calculate your p-value. This is the easiest and most common way.

Online resources and tables are available to estimate the p-value based on your test statistic and degrees of freedom.

These tables help you understand how often you would expect to see your test statistic under the null hypothesis.

Understanding the Statistical Test:

Different statistical tests are designed to answer specific research questions or hypotheses. Each test has its own underlying assumptions and characteristics.

For example, you might use a t-test to compare means, a chi-squared test for categorical data, or a correlation test to measure the strength of a relationship between variables.

Be aware that the number of independent variables you include in your analysis can influence the magnitude of the test statistic needed to produce the same p-value.

This factor is particularly important to consider when comparing results across different analyses.

Example: Choosing a Statistical Test

If you’re comparing the effectiveness of just two different drugs in pain relief, a two-sample t-test is a suitable choice for comparing these two groups. However, when you’re examining the impact of three or more drugs, it’s more appropriate to employ an Analysis of Variance ( ANOVA) . Utilizing multiple pairwise comparisons in such cases can lead to artificially low p-values and an overestimation of the significance of differences between the drug groups.

How to report

A statistically significant result cannot prove that a research hypothesis is correct (which implies 100% certainty).

Instead, we may state our results “provide support for” or “give evidence for” our research hypothesis (as there is still a slight probability that the results occurred by chance and the null hypothesis was correct – e.g., less than 5%).

Example: Reporting the results

In our comparison of the pain relief effects of the new drug and the placebo, we observed that participants in the drug group experienced a significant reduction in pain ( M = 3.5; SD = 0.8) compared to those in the placebo group ( M = 5.2; SD  = 0.7), resulting in an average difference of 1.7 points on the pain scale (t(98) = -9.36; p < 0.001).

The 6th edition of the APA style manual (American Psychological Association, 2010) states the following on the topic of reporting p-values:

“When reporting p values, report exact p values (e.g., p = .031) to two or three decimal places. However, report p values less than .001 as p < .001.

The tradition of reporting p values in the form p < .10, p < .05, p < .01, and so forth, was appropriate in a time when only limited tables of critical values were available.” (p. 114)

• Do not use 0 before the decimal point for the statistical value p as it cannot equal 1. In other words, write p = .001 instead of p = 0.001.
• Please pay attention to issues of italics ( p is always italicized) and spacing (either side of the = sign).
• p = .000 (as outputted by some statistical packages such as SPSS) is impossible and should be written as p < .001.
• The opposite of significant is “nonsignificant,” not “insignificant.”

Why is the p -value not enough?

A lower p-value  is sometimes interpreted as meaning there is a stronger relationship between two variables.

However, statistical significance means that it is unlikely that the null hypothesis is true (less than 5%).

To understand the strength of the difference between the two groups (control vs. experimental) a researcher needs to calculate the effect size .

When do you reject the null hypothesis?

In statistical hypothesis testing, you reject the null hypothesis when the p-value is less than or equal to the significance level (α) you set before conducting your test. The significance level is the probability of rejecting the null hypothesis when it is true. Commonly used significance levels are 0.01, 0.05, and 0.10.

Remember, rejecting the null hypothesis doesn’t prove the alternative hypothesis; it just suggests that the alternative hypothesis may be plausible given the observed data.

The p -value is conditional upon the null hypothesis being true but is unrelated to the truth or falsity of the alternative hypothesis.

What does p-value of 0.05 mean?

If your p-value is less than or equal to 0.05 (the significance level), you would conclude that your result is statistically significant. This means the evidence is strong enough to reject the null hypothesis in favor of the alternative hypothesis.

Are all p-values below 0.05 considered statistically significant?

No, not all p-values below 0.05 are considered statistically significant. The threshold of 0.05 is commonly used, but it’s just a convention. Statistical significance depends on factors like the study design, sample size, and the magnitude of the observed effect.

A p-value below 0.05 means there is evidence against the null hypothesis, suggesting a real effect. However, it’s essential to consider the context and other factors when interpreting results.

Researchers also look at effect size and confidence intervals to determine the practical significance and reliability of findings.

How does sample size affect the interpretation of p-values?

Sample size can impact the interpretation of p-values. A larger sample size provides more reliable and precise estimates of the population, leading to narrower confidence intervals.

With a larger sample, even small differences between groups or effects can become statistically significant, yielding lower p-values. In contrast, smaller sample sizes may not have enough statistical power to detect smaller effects, resulting in higher p-values.

Therefore, a larger sample size increases the chances of finding statistically significant results when there is a genuine effect, making the findings more trustworthy and robust.

Can a non-significant p-value indicate that there is no effect or difference in the data?

No, a non-significant p-value does not necessarily indicate that there is no effect or difference in the data. It means that the observed data do not provide strong enough evidence to reject the null hypothesis.

There could still be a real effect or difference, but it might be smaller or more variable than the study was able to detect.

Other factors like sample size, study design, and measurement precision can influence the p-value. It’s important to consider the entire body of evidence and not rely solely on p-values when interpreting research findings.

Can P values be exactly zero?

While a p-value can be extremely small, it cannot technically be absolute zero. When a p-value is reported as p = 0.000, the actual p-value is too small for the software to display. This is often interpreted as strong evidence against the null hypothesis. For p values less than 0.001, report as p < .001

Further Information

• P-values and significance tests (Kahn Academy)
• Hypothesis testing and p-values (Kahn Academy)
• Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a world beyond “ p “< 0.05”.
• Criticism of using the “ p “< 0.05”.
• Publication manual of the American Psychological Association
• Statistics for Psychology Book Download

Bland, J. M., & Altman, D. G. (1994). One and two sided tests of significance: Authors’ reply.  BMJ: British Medical Journal ,  309 (6958), 874.

Goodman, S. N., & Royall, R. (1988). Evidence and scientific research.  American Journal of Public Health ,  78 (12), 1568-1574.

Goodman, S. (2008, July). A dirty dozen: twelve p-value misconceptions . In  Seminars in hematology  (Vol. 45, No. 3, pp. 135-140). WB Saunders.

Lang, J. M., Rothman, K. J., & Cann, C. I. (1998). That confounded P-value.  Epidemiology (Cambridge, Mass.) ,  9 (1), 7-8.

Related Articles

Exploratory Data Analysis

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

Convergent Validity: Definition and Examples

Content Validity in Research: Definition & Examples

Construct Validity In Psychology Research

Using a confidence interval to decide whether to reject the null hypothesis

Suppose that you do a hypothesis test. Remember that the decision to reject the null hypothesis (H 0 ) or fail to reject it can be based on the p-value and your chosen significance level (also called α). If the p-value is less than or equal to α, you reject H 0 ; if it is greater than α, you fail to reject H 0 .

• If the reference value specified in H 0 lies outside the interval (that is, is less than the lower bound or greater than the upper bound), you can reject H 0 .
• If the reference value specified in H 0 lies within the interval (that is, is not less than the lower bound or greater than the upper bound), you fail to reject H 0 .
• Minitab.com
• License Portal
• Cookie Settings

You are now leaving support.minitab.com.

Click Continue to proceed to:

Stack Exchange Network

Stack Exchange network consists of 183 Q&A communities including Stack Overflow , the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Does failing to reject the null hypothesis mean rejecting the alternative? [duplicate]

Does "failing to reject" the null hypothesis entails rejecting the alternative one?

I think rigorously thinking, we can't, unless alpha is very high.

• hypothesis-testing

• $\begingroup$ This is also related stats.stackexchange.com/questions/125541/… $\endgroup$ –  Sextus Empiricus Commented Dec 19, 2020 at 19:22
• $\begingroup$ also stats.stackexchange.com/questions/60670 and stats.stackexchange.com/questions/85903 $\endgroup$ –  Sextus Empiricus Commented Dec 19, 2020 at 19:24

In statistics there are two types of errors:

• Type I : when the null hypothesis is correct. If in this case we reject null, we make this error.
• Type II : when the alternative is correct. If in this case we fail to reject null, we make this error.

A type I error is connected to statistical significance. a type II error is connected to statistical power.

Many frequentists remember about significance, and forget about power. This leads to the situation, that they state, that failing to reject null means accepting null - IT IS WRONG. The true statement is failing to reject null means that we do not know anything . Unless of course we have knowledge about the power.

Let’s imagine an example, that we have a test with 5% significance, but also very low power - let’s say 10%. We failed to reject null. So now, a false positive (making an error of type I) is not our concern. Now we wish to think if we should accept the null (reject alternative), and without knowledge about the power of the test we can do nothing. But if we know the power of this test, which is 10%, we know that when the alternative is true, the test will correctly reject null only in 10% of cases - in 90% of cases where the alternative is correct, we will fail to reject null!

The problem with power is that in most cases it is a function of many aspects connected to the test itself, the sample size, unknown parameters, satisfaction of test assumptions, and probably more. In most cases it can not be calculated directly, and is approximated by Monte Carlo simulations. But every time those conditions change, the power is completely different.

For some more information about this problem, read Valentin et al. (2019) - a short, popular science, article in Nature, which describes the issue in a more elaborate way. For those more curious I'd suggest taking a look at Wasserstein and Lazar (2016) - the original ASA statement.

Amrhein, Valentin, Sander Greenland, and Blake McShane. " Scientists rise up against statistical significance. " (2019): 305-307.

Wasserstein, Ronald L., and Nicole A. Lazar. " The ASA statement on p-values: context, process, and purpose. " (2016): 129-133.

• 2 $\begingroup$ I spent too long trying to write a good answer to the question with same arguments, this explains it very clearly $\endgroup$ –  RaphaelS Commented Dec 18, 2020 at 14:28
• $\begingroup$ "Many frequentists remember about significance, and forget about power. This leads to the situation, that they state, that failing to reject null means accepting null" Power has nothing to do with misinterpreting a failure to reject the null hypothesis ( all a large p-value means is that the data are reasonably consistent with the null hypothesis) $\endgroup$ –  Graham Bornholt Commented Feb 25 at 22:15

Not the answer you're looking for? Browse other questions tagged hypothesis-testing or ask your own question .

• Featured on Meta
• Upcoming sign-up experiments related to tags

Hot Network Questions

• Colored underline and overline in math mode
• Is the OP_PUSHBYTES_X opcode always required after OP_RETURN?
• Cut and replace every Nth character on every row
• How to find your contract and employee handbook in the UK?
• Exception handling: 'catch' without explicit 'try'
• Would Eldritch Blast using a Dexterity Save instead of Spell Attack appropriate for a low-confidence player?
• How to join two PCBs with a very small separation?
• Should mail addresses for logins be stored hashed to minimize impact of data loss?
• What is the best way to set a class value to a variable in Python if it exists in a dictionary?
• Diagnosing tripped breaker on the dishwasher circuit?
• Is this professor being unnecessarily harsh or did I actually make a mistake?
• What is a quarter in 19th-century England converted to contemporary pints?
• How to merge two videos with openshot?
• Why is polling data for Northern Ireland so differently displayed on national polling sites?
• Writing a generic makefile for C projects
• Can a planet have a warm, tropical climate both at the poles and at the equator?
• Sets of algebraic integers whose differences are units
• Why do many philosophers consider a past-eternal universe to be self-explanatory but not a universe that began with no cause?
• SEPIC DC-DC converter
• Can I replace a GFCI outlet in a bathroom with an outlet having USB ports?
• Fairy Tale where parents have dozens of kids and give them all the same name
• Styling histograms
• New faculty position – expectation to change research direction
• Is it legal to discriminate on marital status for car insurance/pensions etc.?

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

An Easy Introduction to Statistical Significance (With Examples)

Published on January 7, 2021 by Pritha Bhandari . Revised on June 22, 2023.

If a result is statistically significant , that means it’s unlikely to be explained solely by chance or random factors. In other words, a statistically significant result has a very low chance of occurring if there were no true effect in a research study.

The p value , or probability value, tells you the statistical significance of a finding. In most studies, a p value of 0.05 or less is considered statistically significant, but this threshold can also be set higher or lower.

Table of contents

How do you test for statistical significance, what is a significance level, problems with relying on statistical significance, other types of significance in research, other interesting articles, frequently asked questions about statistical significance.

In quantitative research , data are analyzed through null hypothesis significance testing, or hypothesis testing. This is a formal procedure for assessing whether a relationship between variables or a difference between groups is statistically significant.

Null and alternative hypotheses

To begin, research predictions are rephrased into two main hypotheses: the null and alternative hypothesis .

• A null hypothesis ( H 0 ) always predicts no true effect, no relationship between variables , or no difference between groups.
• An alternative hypothesis ( H a or H 1 ) states your main prediction of a true effect, a relationship between variables, or a difference between groups.

Hypothesis testin g always starts with the assumption that the null hypothesis is true. Using this procedure, you can assess the likelihood (probability) of obtaining your results under this assumption. Based on the outcome of the test, you can reject or retain the null hypothesis.

• H 0 : There is no difference in happiness between actively smiling and not smiling.
• H a : Actively smiling leads to more happiness than not smiling.

Test statistics and p values

Every statistical test produces:

• A test statistic that indicates how closely your data match the null hypothesis.
• A corresponding p value that tells you the probability of obtaining this result if the null hypothesis is true.

The p value determines statistical significance. An extremely low p value indicates high statistical significance, while a high p value means low or no statistical significance.

Next, you perform a t test to see whether actively smiling leads to more happiness. Using the difference in average happiness between the two groups, you calculate:

• a t value (the test statistic) that tells you how much the sample data differs from the null hypothesis,
• a p value showing the likelihood of finding this result if the null hypothesis is true.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

The significance level , or alpha (α), is a value that the researcher sets in advance as the threshold for statistical significance. It is the maximum risk of making a false positive conclusion ( Type I error ) that you are willing to accept .

In a hypothesis test, the  p value is compared to the significance level to decide whether to reject the null hypothesis.

• If the p value is  higher than the significance level, the null hypothesis is not refuted, and the results are not statistically significant .
• If the p value is lower than the significance level, the results are interpreted as refuting the null hypothesis and reported as statistically significant .

Usually, the significance level is set to 0.05 or 5%. That means your results must have a 5% or lower chance of occurring under the null hypothesis to be considered statistically significant.

The significance level can be lowered for a more conservative test. That means an effect has to be larger to be considered statistically significant.

The significance level may also be set higher for significance testing in non-academic marketing or business contexts. This makes the study less rigorous and increases the probability of finding a statistically significant result.

As best practice, you should set a significance level before you begin your study. Otherwise, you can easily manipulate your results to match your research predictions.

It’s important to note that hypothesis testing can only show you whether or not to reject the null hypothesis in favor of the alternative hypothesis. It can never “prove” the null hypothesis, because the lack of a statistically significant effect doesn’t mean that absolutely no effect exists.

When reporting statistical significance, include relevant descriptive statistics about your data (e.g., means and standard deviations ) as well as the test statistic and p value.

There are various critiques of the concept of statistical significance and how it is used in research.

Researchers classify results as statistically significant or non-significant using a conventional threshold that lacks any theoretical or practical basis. This means that even a tiny 0.001 decrease in a p value can convert a research finding from statistically non-significant to significant with almost no real change in the effect.

On its own, statistical significance may also be misleading because it’s affected by sample size. In extremely large samples , you’re more likely to obtain statistically significant results, even if the effect is actually small or negligible in the real world. This means that small effects are often exaggerated if they meet the significance threshold, while interesting results are ignored when they fall short of meeting the threshold.

The strong emphasis on statistical significance has led to a serious publication bias and replication crisis in the social sciences and medicine over the last few decades. Results are usually only published in academic journals if they show statistically significant results—but statistically significant results often can’t be reproduced in high quality replication studies.

As a result, many scientists call for retiring statistical significance as a decision-making tool in favor of more nuanced approaches to interpreting results.

That’s why APA guidelines advise reporting not only p values but also  effect sizes and confidence intervals wherever possible to show the real world implications of a research outcome.

Aside from statistical significance, clinical significance and practical significance are also important research outcomes.

Practical significance shows you whether the research outcome is important enough to be meaningful in the real world. It’s indicated by the effect size of the study.

Clinical significance is relevant for intervention and treatment studies. A treatment is considered clinically significant when it tangibly or substantially improves the lives of patients.

Prevent plagiarism. Run a free check.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Descriptive statistics
• Measures of central tendency
• Correlation coefficient

Methodology

• Cluster sampling
• Stratified sampling
• Types of interviews
• Cohort study
• Thematic analysis

Research bias

• Implicit bias
• Cognitive bias
• Survivorship bias
• Availability heuristic
• Nonresponse bias
• Regression to the mean

Statistical significance is a term used by researchers to state that it is unlikely their observations could have occurred under the null hypothesis of a statistical test . Significance is usually denoted by a p -value , or probability value.

Statistical significance is arbitrary – it depends on the threshold, or alpha value, chosen by the researcher. The most common threshold is p < 0.05, which means that the data is likely to occur less than 5% of the time under the null hypothesis .

When the p -value falls below the chosen alpha value, then we say the result of the test is statistically significant.

A p -value , or probability value, is a number describing how likely it is that your data would have occurred under the null hypothesis of your statistical test .

P -values are usually automatically calculated by the program you use to perform your statistical test. They can also be estimated using p -value tables for the relevant test statistic .

P -values are calculated from the null distribution of the test statistic. They tell you how often a test statistic is expected to occur under the null hypothesis of the statistical test, based on where it falls in the null distribution.

If the test statistic is far from the mean of the null distribution, then the p -value will be small, showing that the test statistic is not likely to have occurred under the null hypothesis.

No. The p -value only tells you how likely the data you have observed is to have occurred under the null hypothesis .

If the p -value is below your threshold of significance (typically p < 0.05), then you can reject the null hypothesis, but this does not necessarily mean that your alternative hypothesis is true.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). An Easy Introduction to Statistical Significance (With Examples). Scribbr. Retrieved June 24, 2024, from https://www.scribbr.com/statistics/statistical-significance/

Is this article helpful?

Pritha Bhandari

Other students also liked, understanding p values | definition and examples, what is effect size and why does it matter (examples), hypothesis testing | a step-by-step guide with easy examples, what is your plagiarism score.

Stack Exchange Network

Stack Exchange network consists of 183 Q&A communities including Stack Overflow , the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

what does it mean to reject a hypothesis?

This is a simple problem I cannot understand. The task is to solve:

$$\frac{2}{3} \times |5-2x| - \frac{1}{2} = 5$$

• As a first step, I isolate the absolute value, like this:

\begin{align} 2/3 \times |5-2x| &= 5 + 1/2\\ 2/3 \times |5-2x| &= 11/2\\ |5-2x| &= 11/2 ÷ 2/3 \\ |5-2x| &= 33/4\\ |5-2x| &= 8.25 \end{align} 2. Now, $|5-2x|$ can be two things: $$ |5-2x| =\begin{cases} 5-2x & x \geq 2.5\\ 5+2x & x < 2.5. \end{cases} $$

• Let's solve the equation $|5-2x| = 8.25$

if $x \geq 2.5$, then: \begin{align} 5-2x &= 8.25\\ -2x &= 3.25\\ x &= -1.625 \end{align} But we said that $x \geq 2.5$!

I get a similar contradiction if I calculte the other conditional: $x < 2.5$ \begin{align} -5+2x &= 8.25\\ 2x &= 13.25\\ x &= 6.625 \end{align} which is not smaller than $2.5$.

As far as I know, in such situations, we tend "reject the solution." But what does it mean for a solution not to confirm to a hypothesis?

(The problem is from Stitz & Zeager (2013) Precalculus, exercise: 2.2.1/8. I apologise for not being able to use nice formatting.)

• self-learning
• absolute-value

• 1 $\begingroup$ Why do you say $|5-2x|=5-2x$ when $x\ge 2.5$? $\endgroup$ –  Angina Seng Commented Jul 26, 2018 at 15:43

3 Answers 3

$$5-2x\ge 0\iff |5-2x|=5-2x$$ but $$5-2x\ge0\iff 2x\le 5,$$

unlike what you wrote.

In this particular exercise, no hypothesis is rejected, so it doesn't illustrate the concept.

• $\begingroup$ I didn't know that multiplying an inequality by -1 changes the direction of the inequality sign. Thank you so much for pointing that out, Yves! Also, please, let me know if I should edit or do something with the post, because (I realise) it's really not a question about hypothesis rejection. Thank you. $\endgroup$ –  malasi Commented Jul 26, 2018 at 17:13
• $\begingroup$ @malasi: I did'nt work this out with a multiplication but by changing members. $\endgroup$ –  user65203 Commented Jul 26, 2018 at 17:44
• $\begingroup$ what do you mean by "changing members?" I'm confused now. $\endgroup$ –  malasi Commented Jul 26, 2018 at 17:52
• $\begingroup$ I got it, you simply added +2x to the inequality 5-2x ≥ 0. I had somehow gotten confused by the order of members. Thanks for the notice. $\endgroup$ –  malasi Commented Jul 26, 2018 at 20:13

In addition to what has been said, note that we can solve the equation by noting that $$ |x|=a\iff x=\pm a $$ In particular $$ |5-2x|=8.25\iff 5-2x=8.25 \quad \text{or} \quad 5-2x=-8.25 $$ which is perhaps easier to solve.

A good way to solve this kind of equation is consider two cases

1) For $5-2x\ge 0 \iff x\le \frac25$ we have

$$\frac23 |5-2x| - 1/2 = 5 \iff \frac23 (5-2x) - 1/2 = 5 \iff 20-8x-3=30 \\\iff 8x=-13 \iff x=-\frac{13}8$$

that solution is acceptable since it is consistent to the assumption $x\le \frac25$.

2) For $5-2x< 0 \iff x> \frac25$ we have

$$\frac23 |5-2x| - 1/2 = 5 \iff \frac23 (2x-5) - 1/2 = 5 \iff 8x-20-3=30 \\\iff 8x=53 \iff x=\frac{53}8$$

which is also acceptable.

As an example with some solution to reject refer to Why am I getting a wrong answer on solving $|x-1|+|x-2|=1$ .

You must log in to answer this question.

Not the answer you're looking for browse other questions tagged self-learning absolute-value ..

• Featured on Meta
• Upcoming sign-up experiments related to tags

Hot Network Questions

• In By His Bootstraps (Heinlein) why is Hitler's name Schickelgruber?
• Would Eldritch Blast using a Dexterity Save instead of Spell Attack appropriate for a low-confidence player?
• Sorting and Filtering using Dynamic Parameters
• Less ridiculous way to prove that an Ascii character compares equal with itself in Coq
• Can God transcend human logic and reasoning?
• Is this professor being unnecessarily harsh or did I actually make a mistake?
• How does the router know to send packets to a VM on bridge mode?
• Does anyone know what the heck this tool is? I got it in a box lot or tools and dont have a clue
• Determine the continuity of a function
• What exactly is beef bone extract, beef extract, beef fat (all powdered form) and where can I find it?
• Synthesis of racemic nicotine
• Is the OP_PUSHBYTES_X opcode always required after OP_RETURN?
• How to merge two videos with openshot?
• Do magic states only work for CSS codes?
• Writing a generic makefile for C projects
• What is the translation of lawfare in French?
• What rights does an employee retain, if any, who does not consent to being monitored on a work IT system?
• How to refer to the locations in lower depths of a waterbody (such as a lake)?
• How will the ISS be decommissioned?
• Is it legal to discriminate on marital status for car insurance/pensions etc.?
• In an interview how to ask about access to internal job postings?
• SEPIC DC-DC converter
• Does "my grades suffered" mean "my grades became worse" or "my grades were bad"?
• Co-authors with little contribution

How can hypothesis testing be performed in Python, and what are some examples of its application?

Table of Contents

Hypothesis testing is a statistical method used to determine if a certain hypothesis or claim about a population is true or not. In Python, hypothesis testing can be performed using various statistical packages such as SciPy and Statsmodels. These packages provide functions and methods for conducting different types of hypothesis tests, such as t-tests, ANOVA, and chi-square tests.

To perform hypothesis testing in Python, one must first define the null and alternative hypotheses and select an appropriate test based on the type of data and research question. The necessary data must then be imported into the Python environment and the chosen test function can be applied to the data. The test results will provide a p-value, which is used to determine the statistical significance of the hypothesis.

Some examples of applications of hypothesis testing in Python include determining if there is a significant difference in sales between two versions of a product, testing the effectiveness of a new marketing strategy, and analyzing the impact of a certain variable on a population. Hypothesis testing in Python allows for efficient and accurate analysis of data, making it a valuable tool in various fields such as business, healthcare, and social sciences.

Perform Hypothesis Testing in Python (With Examples)

A is a formal statistical test we use to reject or fail to reject some statistical hypothesis.

This tutorial explains how to perform the following hypothesis tests in Python:

• One sample t-test
• Two sample t-test
• Paired samples t-test

Let’s jump in!

Example 1: One Sample t-test in Python

A is used to test whether or not the mean of a population is equal to some value.

For example, suppose we want to know whether or not the mean weight of a certain species of some turtle is equal to 310 pounds.

To test this, we go out and collect a simple random sample of turtles with the following weights:

Weights : 300, 315, 320, 311, 314, 309, 300, 308, 305, 303, 305, 301, 303

The following code shows how to use the ttest_1samp() function from the scipy.stats library to perform a one sample t-test:

The t test statistic is  -1.5848 and the corresponding two-sided p-value is  0.1389 .

The two hypotheses for this particular one sample t-test are as follows:

• H 0 :  µ = 310 (the mean weight for this species of turtle is 310 pounds)
• H A :  µ ≠310 (the mean weight is not  310 pounds)

Because the p-value of our test (0.1389) is greater than alpha = 0.05, we fail to reject the null hypothesis of the test.

We do not have sufficient evidence to say that the mean weight for this particular species of turtle is different from 310 pounds.

Example 2: Two Sample t-test in Python

For example, suppose we want to know whether or not the mean weight between two different species of turtles is equal.

To test this, we collect a simple random sample of turtles from each species with the following weights:

Sample 1 : 300, 315, 320, 311, 314, 309, 300, 308, 305, 303, 305, 301, 303

Sample 2 : 335, 329, 322, 321, 324, 319, 304, 308, 305, 311, 307, 300, 305

The following code shows how to use the function from the scipy.stats library to perform this two sample t-test:

The t test statistic is – 2.1009 and the corresponding two-sided p-value is 0.0463 .

The two hypotheses for this particular two sample t-test are as follows:

• H 0 :  µ 1 = µ 2 (the mean weight between the two species is equal)
• H A :  µ 1 ≠ µ 2 (the mean weight between the two species is not equal)

Since the p-value of the test (0.0463) is less than .05, we reject the null hypothesis.

This means we have sufficient evidence to say that the mean weight between the two species is not equal.

Example 3: Paired Samples t-test in Python

A is used to compare the means of two samples when each observation in one sample can be paired with an observation in the other sample.

For example, suppose we want to know whether or not a certain training program is able to increase the max vertical jump (in inches) of basketball players.

To test this, we may recruit a simple random sample of 12 college basketball players and measure each of their max vertical jumps. Then, we may have each player use the training program for one month and then measure their max vertical jump again at the end of the month.

The following data shows the max jump height (in inches) before and after using the training program for each player:

Before : 22, 24, 20, 19, 19, 20, 22, 25, 24, 23, 22, 21

After : 23, 25, 20, 24, 18, 22, 23, 28, 24, 25, 24, 20

The following code shows how to use the from the scipy.stats library to perform this paired samples t-test:

The t test statistic is – 2.5289  and the corresponding two-sided p-value is 0.0280 .

The two hypotheses for this particular paired samples t-test are as follows:

• H 0 :  µ 1 = µ 2 (the mean jump height before and after using the program is equal)
• H A :  µ 1 ≠ µ 2 (the mean jump height before and after using the program is not equal)

Since the p-value of the test (0.0280) is less than .05, we reject the null hypothesis.

This means we have sufficient evidence to say that the mean jump height before and after using the training program is not equal.

Additional Resources

You can use the following online calculators to automatically perform various t-tests:

Related terms:

• How can the Chi-Square Test of Independence be performed in R, and what are some examples of its application?
• How to Perform Hypothesis Testing in Python (With Examples)
• 4 Examples of Hypothesis Testing in Real Life?
• How can the duplicated function be used in R, and what are some examples of its application?
• How can I use the DGET function in Excel, and what are some examples of its application?
• How can I utilize the DCOUNT function in Excel and what are some examples of its application?
• How can I calculate the Levenshtein Distance in R, and what are some examples of its application?
• How can the droplevels function be used in R, and what are some examples of its application?
• What is Cronbach’s Alpha and how is it calculated to measure the internal consistency of a scale or test? Can you provide examples of its application in research studies?
• How can the lines() function be used in R and what are some examples of its application?

• Trending Now
• Foundational Courses
• Data Science
• Practice Problem
• Machine Learning
• System Design
• DevOps Tutorial

Significance Tests (Hypothesis Testing)

Significance tests provide a formal method for using sample data to determine the likelihood of a claim regarding a population value. Learn how to use significance tests and p-values to determine the likelihood of a sample result occurring by chance. You'll also learn how we use p-values to draw conclusions regarding hypotheses.

Which of the following best describes a null hypothesis (H₀)?

The null hypothesis states there is an effect or a difference.

The null hypothesis states there is no effect or no difference.

The null hypothesis is what the researcher aims to prove.

The null hypothesis is always true.

What does an alternative hypothesis (H₁) suggest?

There is no change or effect.

There is a change, effect, or difference.

The results are inconclusive.

The null hypothesis is correct.

If the significance level (alpha) is set at 0.05, what does this mean?

There is a 5% chance of rejecting a true null hypothesis.

There is a 5% chance of accepting a false null hypothesis.

There is a 95% chance of rejecting a false null hypothesis.

There is a 95% chance of accepting a true null hypothesis.

What does a p-value of 0.03 indicate in a significance test?

The null hypothesis is definitely true.

The null hypothesis is definitely false.

There is a 3% probability that the observed data occurred by chance under the null hypothesis.

There is a 97% probability that the observed data occurred by chance under the null hypothesis.

What is a Type I error in hypothesis testing?

Failing to reject a true null hypothesis.

Rejecting a false null hypothesis.

Rejecting a true null hypothesis.

Failing to reject a false null hypothesis.

There are 5 questions to complete.

How to avoid a desk reject: do’s and don’ts

• Published: 17 June 2024

Cite this article

• Sjoerd Beugelsdijk 1 &
• Allan Bird 2  

2779 Accesses

1 Altmetric

Explore all metrics

Avoid common mistakes on your manuscript.

Introduction

The number of manuscripts submitted to academic journals has increased significantly, and along with that the desk-reject rate also, that is, the rate at which manuscripts are rejected at the very first stage of the review process (Ansell & Samuels, 2021 ). At the Journal of International Business Studies ( JIBS ), roughly 65% of submissions are desk-rejected. In other words, the authors of nearly two-thirds of the manuscripts sent to this journal will not see their submissions reach an area editor or reviewers. Obviously, no one wants to receive a rejection letter; when it is a desk reject, authors may well feel they never even got a fair hearing through a peer-review process. Not only has the number of submissions risen but so has their overall quality. It was inevitable that the desk-reject bar would be raised. Not doing so would have risked overburdening editorial teams and the pool of qualified and dedicated reviewers on which they rely. Already, across all fields of science, potential reviewers are more frequently declining invitations to review, and this further increases the pressure on reviewing editors to desk-reject manuscripts (Dance, 2023 ). Footnote 1

Getting past the desk-reject stage is critical because even if a manuscript is not eventually accepted for publication, the suggestions and comments of an area editor and reviewers— invariably acknowledged experts in the field—can be immeasurably valuable in making improvements for submission to another journal. A desk reject differs from rejection later in the review process because the objective and the process of the two are quite different. Reviewing editors decide by themselves the fate of a manuscript, while peer review is a shared responsibility. The workload of reviewing editors is such that they need to rely on heuristics to make their decisions. Hence, they hone in on specific elements which, if not present, will result in a desk reject. In this editorial, we describe these elements under two headings: (1) effective communication, and (2) theory- and method-related rigor. Our goal is to relay what reviewing editors look for when deciding whether to forward a manuscript to the next level. Our tips and actionable suggestions are summarized in the Appendix, where we also provide a list of 80 questions that serve as a ‘checklist’ of do’s and don’ts. These suggestions are based on more than 4000 Journal of International Business Studies desk-reject decisions between 2016 and 2024. Many of the suggestions and recommendations we provide apply equally to overall guidance about research in international business.

The role of reviewing editors

Reviewing editors are tasked with (1) conserving the time, attention, and energies of area editors, reviewers, and submitting authors, and (2) maintaining the focus and integrity of the journal mission as embodied in its statement of aims and scope. The mechanics of a desk review are straightforward: their purpose is to assess whether a submission meets the journal’s fit, quality, and contribution thresholds. In addition to determining if a manuscript meets those three content criteria, reviewing editors are charged with ensuring that the review process is fair, specifically that it is free of bias, ethical lapses and errors, and that, to a reasonable extent, author concerns or requests are accommodated. More details can be found in the Journal of International Business Studies guidelines for reviewers. Footnote 2

The first step in the desk-review process entails reading the cover letter. Although not required, many authors submit them. They can be used to explain distinctive aspects of the manuscript, for example, a unique approach taken in framing the research question. A cover letter might also be used to make a request, such as for a reviewer knowledgeable about a new analytical approach. We recommend that authors provide pertinent information, such as the names of persons who have previously read and commented on the manuscript, thereby avoiding the possibility of compromising the double-blind peer review should the manuscript move forward in the review process. Footnote 3 Authors should also alert the reviewing editor if other manuscripts or published articles by the author and co-authors address the same topic or draw upon the same dataset as the submitted manuscript. Footnote 4 We recommend submitting a detailed overview of any overlap and differences between the submitted manuscript and the authors’ existing work in the same vein (sometimes referred as an originality matrix), so as to help the reviewing editor assess the contribution of the manuscript. There is an expectation that authors will be transparent with editors.

Reviewing editors are anxious to avoid making type I or type II errors. They do not want to desk reject a manuscript that might end up being a high-quality, impactful article. Making the wrong decision can mean a loss for the journal as well as deny the authors timely publication. On the other hand, forwarding for full review a manuscript that does not meet fit, quality, and contribution thresholds and has little chance of reaching publication is an inefficient use of the time, attention, and effort of editors and reviewers. It also bogs down authors who end up having devoted time pursuing an ultimately fruitless review process rather than improving the manuscript and submitting it elsewhere.

Because there are recurring patterns in the types of issues that lead to a desk reject, reviewing editors use heuristics in making their assessments. In general, a manuscript is desk-rejected if there is not a good fit with the aims and scope of the journal. For the Journal of International Business Studies , this implies the topic has to address an international business topic as explained in the editorial guidelines. Footnote 5 Manuscripts should address topics from an international comparative and/or cross-border angle. This means that ‘just’ analyzing a cross section of countries is not sufficient to be considered for this journal. Similarly, ‘just’ adding some country-specific variables as control variables is not sufficient to qualify as making a contribution to international business. Single-country studies without an IB dimension are a substantial portion of all desk-rejected articles. The heuristics that reviewing editors use can be categorized into two main domains: (1) effective communication and (2) theory- and method-related rigor. Each domain consists of a series of do’s and don’ts. These do’s and don’ts are summarized in the Appendix.

Effective communication

Writing a good manuscript involves reading prior research, data analysis, sense-making, writing, re-analyzing, presenting to colleagues, re-writing, and eventually accepting a certain degree of imperfection. A positive correlation exists between manuscript quality and the time spent on it, but that correlation is far from 1. Certainly, as some seem to believe, a manuscript does not merit review simply because the author claims a lot of time has been spent on it. Underestimation of the importance of effectively communicating with readers is at the root of many desk rejects. We discuss five of them here.

Develop a story

Human beings are pattern-seeking, sense-making, story-telling animals (Leamer, 2009 ). A good manuscript tells a story, one that is believable and memorable. The story may be based on a phenomenological observation or be a theory-based narrative, but a good story is critical to scholarly understanding because storytelling is a cognitive process with sense-making at its core.

It is a mistake to think that storytelling in science is limited to manuscripts using interviews, for which it is a recommended theory-development strategy. It is also a critical part of effective communication for manuscripts based on secondary data where there is often a focus on statistical relationships without a clear understanding of underlying processes. To minimize the probability that a regression result becomes merely a statistical artefact, authors should understand what is driving the statistically significant relationships between the variables. If they do, their story is much better than that of authors who rely on statistical software packages to tell the story for them. In other words, a coefficient that differs significantly from zero is never the essence of the story, but only a part of it. This is one important reason why authors who first analyze the data and then develop hypotheses on that basis (i.e., who practice what is called harking –  h ypothesizing a fter r esults a re k nown) are generally not good storytellers. Harking is not only unscientific, but it also results in unpersuasive stories.

What does make for a good story? In a word, focus. We do not mean honing in on detail to such an extent that the result is a marginal contribution. Far from it. Still, the most valuable contributions are typically very focused. By focus we mean that the core concept is succinctly stated and concisely explained in just a few sentences. The key takeaway should be delivered in plain English understandable to a non-academic audience. Preparing 15-min mock presentations and rehearsing—out loud—the opening and concluding sections can be especially useful in developing a focused story.

Focus alone will not suffice. Delivery is extremely important. Good writing enhances storytelling. We hasten to add that reviewing editors do not reject manuscripts out of hand because of low readability—although obviously the manuscript must be intelligible. Nonetheless, there can be a horn’s effect. A carelessly put together manuscript with typos, misspellings, and grammatical errors that could have easily been caught by running a spelling and grammar check, or table and figure headings that do not match content, or referencing that is incomplete, inconsistent or not applicable, raise doubts about the rigor and precision with which theory is developed and data analyzed. Authors need to take the time to polish their manuscripts; even established researchers spend a considerable amount of time doing that. Poor writing can be fixed by careful language editing down the line.

It is also a mistake to overcomplicate the story by trying to do too much. This typically happens when an author tries to eclectically mix different theories. Reviewing editors are not likely to forward manuscripts in which authors use multiple theories, e.g., the resource-based view of the firm, transaction costs theory, population ecology, and institutional theory. First, each theory comes with its own set of assumptions, causal mechanisms, and boundary conditions, and these can be hard—if not impossible—to integrate into one overarching framework. Second, combining multiple theories tends to result in convoluted arguments with no real punchline.

Another mistake is to center the story around the use of a different method or a distinctive sample to empirically examine relationships that have already been studied extensively. While that strategy might work when submitting to a second-tier journal, top journals expect there to be a clear theoretical contribution and novelty beyond a new method or distinctive sample. Showing that a relationship already examined in other studies holds when expanding the sample, e.g., to different countries or perhaps by using an alternative method, will trigger interest only if there is an unusual theoretical rationale for using the new method or sample. For example, suppose a specific theory has been tested primarily in economically developed countries and that good theoretical arguments exist for why the theory may not apply outside that context; then expanding the sample to less economically developed countries makes sense. The same holds true for a manuscript that an author attempts to ‘sell’ based on the use of a new method. With the exception of method-focused journals such as Organizational Research Methods , most reviewing editors will only forward a method-focused manuscript if the method element has interesting theoretical implications.

What makes for a good story is to some extent time-specific. Management trends come and go, and so does what is seen as a legitimate story. For a long time, authors specified what was called a ‘gap’ in the literature. They would claim to have uncovered a theoretical hole and then outline it in the introduction of their manuscript. Their story was essentially based on their observation that aspect A of theory X had not yet been addressed. As time has passed, phenomenological research has become popular and it is now increasingly legitimate for authors to start their story with a new, interesting, even odd empirical observation. With that, a good story has become one that piques the interest of readers and makes them curious about what comes out. It leaves them thinking to themselves, “Good point. Why didn’t I think of that?” An effective way of gauging what is trending in a particular community of scholars is to read the introductions of conference papers and recently published articles to see what kind of ‘hook’ is used.

Write a clear introduction that explains the what, so what, and now what

The introduction can be a make-or-break point. A desk reject is likely if the introduction is not clear. The reviewing editor will look for focus, a good story, convincing theorizing, and tight empirical tests. There is no universal template for a high-quality introduction, but that does not mean that crafting one is a random process. The best introductions include several recurring elements (Grant & Pollock, 2011 ). The introduction of articles published in top journals may differ from the pattern explained below because of differences in topic, method, data, field, research tradition, and findings. Still, we can discuss several elements that all reviewing editors look for when reading an introduction. Often those elements correspond to the four paragraphs that we propose should form the introduction.

The first paragraph should set the scene. It should include (1) the topic, (2) why it matters, and (3) what is already known about it, including theories used. Writing the opening paragraph is quite challenging because the author must summarize in just a few sentences the state-of-affairs in a field. The second paragraph discusses what we do not yet know about the topic. This can be theory or phenomenon-driven. For example, there is well-established and vast literature on why people resign and change jobs, but we do not yet really understand the recently identified phenomenon of quiet quitting. Describing why quiet quitting could be important is key because it provides the motivation behind the manuscript. The third paragraph describes what the author does to address the question, specifically the theory used, key characteristics of the data (e.g., sample size and country context), as well as the method used. In this paragraph the author should also summarize the findings. In the fourth and final paragraph, authors should circle back to the broader topic – in our example, quiet quitting. They should show why their findings matter as well as the implications. The contribution should be as explicit as possible, not just repeat the empirical findings, but discuss their broader meaning. The last paragraph often ends with a road map indicating how the manuscript is structured.

The typical introduction in management journal articles is around 600 words, divided more or less equally between the paragraphs described above. This means that in each case the material to be covered is handled in just six to eight sentences. The first and last sentences of each paragraph are critical. If those two alone convey the message, the manuscript is probably properly focused. In fact, one way of checking whether a paragraph makes sense is to read those sentences, ignoring the ones in between, to see if the core message is still conveyed. If so, the manuscript is focused and the storyline clear. Another test is to string together the opening sentence of each paragraph. There should be a coherent story supported by a clear line of reasoning. Obviously there are many variations in the way successful authors craft introductions. We describe here what we, as reviewing editors, have found effective introductions have in common.

Know your audience and the language they speak

Imagine entering a room in which the ten most-cited scholars in your area are debating the very topic on which you are writing. They turn to look at you. You have their attention. What can you say about your manuscript that would interest them? Would it impress them if you were to say that you show that the relation between X, Y, and Z—something which they have already analyzed – holds true using your data? What if instead you were able to tell them a powerful story in field-specific language, words that carry a particular connotation and labels with well-known associations? The point is, in a twist to the normal advice to use your own words, you need to tell the story in their kinds of words.

Authors need to immerse themselves in the language used in their area. They need to read the classic articles and books as well as the latest ones on the topic, bearing in mind that there is a significant time lag between manuscript submission and final publication. They need to stay on top of what is happening in the scholarly community in other ways as well. Taking part in academic conferences is one of them—attending panels, observing debates, engaging in discussions, especially delivering papers—all help in understanding where a field is heading. Topics, methods, approaches, and terminology are ‘in the air’ at workshops and during webinars. All of this is part of knowing the audience. Despite all recent advancements in artificial intelligence (AI), this aspect of targeting your audience has so far not been successfully integrated in existing AI tools.

One less tacit, more formal aspect of audience expectations is understanding the style and format in which core ideas are communicated. Journals have set limits on the number of words used. It is important for authors to stick to them. Reviewing editors do sometimes wade through manuscripts that are considerably longer than the norm, but they are ever mindful of the contribution-to-length ratio. Authors should not try to be exhaustive in providing references. Peppering a text with references, especially when placed mid-sentence, reduces readability. Reviewing editors are familiar with a wide range of research areas. They will catch careless referencing, such as backing up a statement with a reference to an article or book in which no such support can be found, or misattributing a contribution. Inaccurate or excessive referencing reflects badly on the scholarship of a submitting author and may lead to a desk reject. It is important to use current references, as submissions with references ending 15 or 20 years ago signal the manuscript is outdated. Finally, there is no formal rule regarding what particular works authors should reference, but if none of the references have been published in the journal to which they are submitting, it is likely to be taken as evidence of not being in touch with ongoing discussions in the journal, thereby raising the question of fit.

Avoid vague wording

Words matter. Scientific research requires precision. Formal modeling provides it in economics, finance, operations research, and some subfields in sociology and political science. Social sciences, including business and management, rely on precise, unambiguous language. Unfortunately, many authors are not so meticulous. Reviewing editors are not taken in by meaningless jargon or pretentious verbiage. Rather, such language might be taken as an indication that an author has not totally grasped the topic or is attempting to oversell the contribution.

Consider the following seven examples taken from actual manuscripts—followed by our critical comments. (1) We show that an integrated approach is required. This kind of generic statement holds for virtually all topics . (2) We provide a nuanced picture of the complex relationship between X and Y. Attempting to add nuance to a complex concept is an endless exercise—not a goal in itself. The goal should be to make the complex simple without making it simplistic. We mean E = mc 2 simple. (3) Managers should take care of their international HR function. No study is needed to reach this obvious conclusion. (4) We discuss some implications of… Some? Are there others? Vague statements like these make us wonder what is left unsaid, or unresearched, or if the author is unsure of what the implications might be or how to explain them. (5) We uncover heterogeneity that has not been addressed before. To our knowledge, we are the first to analyze... An author may have found something of importance that escaped all others, but maybe it is not sufficiently interesting or indeed even relevant enough to merit publication. (6) We draw upon… What exactly does this mean? Does the author intend to take – in whole or in part – elements from a theory and eclectically combine them? (7) The relation between subsidiary and headquarters: some insights from country X . This last example has to do with crafting meaningful titles. The manuscript title, as well as those of the figures and tables, should be precise and specific and convey meaningful information.

Finally, a word of caution about acknowledging limitations. It is not a recommended strategy to discuss all possible limitations, especially when done at the end of a manuscript, as this may leave readers wondering why they have taken the time to read something the authors themselves think is significantly flawed. Two types of limitations should be identified, but not necessarily addressed in a specific section labeled as such and found at the end of the manuscript. Methodological limitations are ideally addressed in the Method section along with steps taken to mitigate or overcome them. Theoretical limitations relate specifically to what interpretations or conclusions can be drawn from the empirical findings. Rather than listing them as limitations, they can be framed as future lines of inquiry opened up as a result of what was learned from the study.

Write a clear self-standing abstract

Many authors underestimate the importance of the abstract. This is hard to understand because a good abstract gets the attention of potential readers and can entice them to continue reading. An article read is possibly one cited. The abstract is also important in the review process. It is the first thing that a reviewing editor reads. The abstract should give the topic and research question (the motivation), the theoretical angle taken, what the author does (the empirical setting if relevant), the findings, and why the study matters (the contribution). In short, it must convey a considerable amount of information. Writing one takes time and attention, and the abstract should not be the last quick thing authors attend to before submission. All too often abstracts are overly technical and hard to understand without having read the full manuscript.

What can authors do to be sure that what they write in the abstract is meaningful? One way of testing is to draw a line through the key construct named in the abstract and put in its place some other construct in the field. If the abstract makes just as much sense after plugging in that randomly chosen construct, the original abstract is probably uninformative and unconvincing. Let us illustrate the point with a concrete example. Do the following test on this hypothetical abstract: “Institutions have been recognized as a crucial topic in international business research with wide-ranging implications for internationalizing firms. As a result, there are a wide variety of studies in different contexts, using different methods, a diverse set of theories, and a variety of empirical measures. In this article, we review the existing literature, evaluate current approaches critically, and highlight directions for future research.” Now, suppose ‘institutions’ were to be substituted by ‘headquarter–subsidiary relationships’. There is nothing jarring about the resulting version, a sign of an abstract that is too generic.

Theory- and method-related rigor

Distinguish between theory and literature review.

Authors sometimes confuse the literature review with the theory section. Whereas a literature review provides an overview of established findings thereby providing the frame into which a manuscript fits, a theory section provides a set of arguments (embedded in underlying assumptions) that logically lead to a proposition or testable hypothesis. A theory is about the arrows linking construct A to construct B (Thomas et al., 2011 ). In short, theories explain relationships. But rather than providing an integrated framework based on causal theoretical arguments, the theory section in many manuscripts is just a literature review that provides an overview of what other authors have argued or found in their empirical studies. The lack of a strong theory section is an important reason for a desk reject.

Theoretical arguments are often not precise because authors work with overly broad concepts. The result is loosely linked arguments. Another common mistake is to mix arguments from different schools of thought, leading to theoretical imprecision. This, as noted before, leads to poor stories. Reviewing editors are senior scholars and thus aware of the most important differences between the core theories used in a field. This does not mean that manuscripts need only develop narrow arguments derived from a single theoretical framework, but it is generally recognized that combining lenses is challenging (Okhuysen & Bionardi, 2011 ).

Finally, reviewing editors are likely to desk-reject a manuscript when the author excessively uses quotations. Instead of relying on others to say what you want to argue, it is far better to explain the mechanisms directly and explicitly in your own words. There is a risk of misstating what the cited author means to say because quotations are snapshots of broader arguments, and often individual sentences are taken from longer paragraphs.

Spell out the theoretical mechanisms

Ultimately, the theoretical contribution lies in highlighting the set of mechanisms that logically explain the relationship between A and B. Hypotheses are testable predictions derived from a set of arguments that causally and logically relate to one another. Often authors present hypotheses as the result of a set of empirical findings. This leads to truisms—claims that are so self-evident that they are too obvious to mention. In these cases, reviewing editors are inclined to reject manuscripts. Examples can be an effective way to present arguments, but they are no substitute for clear theoretical argumentation. In other words, the plural of anecdote may be data, but data cannot by themselves be the basis for hypotheses.

Hypotheses make testable statements on the relationship between abstract constructs. A good hypothesis is the logical outcome of proper theorizing (Santangelo & Verbeke, 2022 ). Because they are unable to examine theoretical relationships directly, researchers rely on empirical proxies, e.g., patent filings as a proxy for firm innovation, and return on investment for firm performance. It is not uncommon for authors to shift focus from constructs to proxies, and to make statements on relationships between empirical proxies while overlooking the theoretical constructs the proxies are purported to represent. As a general rule of thumb, one should not discuss measurement-related issues (e.g., the variables used as proxies) in the theory section. This makes it possible to keep it as clean as possible and reduces the risk of conflating the theoretical argument supporting hypotheses with the empirical tools used to test them.

Many phenomena in international business are multi-level by nature. For example, country-level variables, such as national cultural differences, may moderate lower-level relationships, such as the dynamic between team leaders and team members. When data are nested in countries, firms, teams, and individuals, one needs to use multi-level methods to disentangle the impact of variations at each level. The real challenge is often not in using multi-level methods, but in developing multi-level theories. Reviewing editors look for a description of the mechanisms linking the micro and the macro levels. If they are not made explicit, a desk reject is likely. To avoid that, authors should make sure they discuss the causal relationships between the different levels.

As a rule, authors should also avoid hypotheses that involve more than one relationship. For example, a model where an increase in A is theorized to cause a decrease in B and the A–B relationship is moderated by C should have two hypotheses, not one. Compound hypotheses are inherently complex and consequently often poorly worded, and this may lead to a desk reject.

Isolate the theoretical channels empirically

In addition to clearly specifying the nature of the theoretical argument, empirical tests of hypothesized relationships need to get as close as possible to a direct test of the proposed mechanisms. This is done by providing convincing theoretical arguments and a series of empirical tests that serve two goals. First, to show that the mechanism that is theorized exists empirically. Second, to rule out alternative plausible explanations. Ruling out alternative explanations is at least as important as providing evidence for the theoretical mechanisms. This should be taken into account when designing the study and prior to data collection. A number of methods are available to identify mechanisms, including—but not limited to—instrumental variables, natural or quasi experiments, regression discontinuity design, difference-in-difference analysis, randomized control trials, propensity score matching, and longitudinal studies.

Increasingly, authors combine multiple methods to corroborate the main effects found, combining quantitative and qualitative methods, including AI. In all cases, it is critical to explain why a specific method was used, the problem it addresses, and how it helps us better understand the theoretical mechanisms. Reviewing editors will evaluate whether the methods used are adequate to test the proposed theoretical relationship between constructs. If the answer is no, a desk reject is likely. Using multiple inadequate methods does not substitute for using a (single) adequate one.

Although theorizing is all about developing causal arguments, establishing causation is often empirically difficult. Authors should therefore avoid mentioning causation unless they can empirically test for it. Language should be precise and distinguish between association, e.g., an increase in political risk is associated with a decrease in foreign direct investment, and causation, e.g., an increase in an MNE’s foreign investments reduces its organizational slack. Note that all journals prefer to see evidence of causality, but will often accept association.

Match construct and empirical measure

Empirical research relies on proxy measures for theoretical constructs. More often than not, proxy measures are imperfect. The alignment between construct and measurement is critical in empirical research, and ideally already addressed at the design stage of a research project. Researchers doing survey-based studies typically develop custom-made measurement instruments, other researchers using those instruments in later studies need to make sure that the instruments align definitionally with their own theoretical constructs. Similarly, secondary data-based research often relies on data collected for other purposes, and hence the variables used to measure the theoretical constructs are often imperfect proxies.

One way to check if proxies are distal is to write the definition of a construct and the way the construct is measured on separate pieces of paper, and to then, without looking at the rest of the text, ask whether the two are aligned. With survey instruments, it can be useful to examine the individual items used to measure the construct. For example, research using Hofstede’s power distance dimension might compare Hofstede’s definition of the power distance construct with the original items used to measure it. Distal proxies are relatively easy for reviewing editors to detect, and are a common reason for desk rejects. Harking not only leads to poor stories, as explained earlier, but also to the use of distal proxies as authors try to retrofit an already-existing measure to a theoretical construct.

Link research question, theory, hypotheses, and implications

By the time the reviewing editor reaches the Discussion section, the primary focus is on the third criterion—contribution. It is not enough to provide a convincing answer to the research question. Authors must demonstrate that the answer contributes to a broader or deeper understanding of theoretical concerns or practical phenomena. Often described in terms of ‘implications’, what the Discussion section ideally accomplishes is an explanation of how the findings of the study should be understood, i.e., what the findings mean. A failure to position a manuscript’s contribution into a broader theoretical context may lead the reviewing editor to conclude that the manuscript’s contribution is narrow or trivial.

Theoretical implications are difficult to describe, yet doing so well is essential. One way to elicit them is by asking what changes should be made to extant theory to account for the empirical results found. When stating theoretical and empirical implications, it is best not to overreach and claim overly bold implications that do not logically follow from the findings. To sum up: reviewing editors look for a logical fit between the research question, the hypotheses, and the overall theoretical implications; and they expect the implications to be substantive.

Authors as prosecuting attorneys

The metaphor of trying a case in a court of law is useful when conceptualizing the challenges facing authors in getting their manuscripts published. Authors are like prosecuting attorneys in that they must have a convincing story supported by reliable witnesses and credible evidence. Prosecuting attorneys need to relate the various elements of a crime—motive, means, and opportunity—in a compellingly persuasive way. Likewise, authors must craft a story that explains a phenomenon, gather evidence—primary and secondary data, elicit reliable testimony from unimpeachable witnesses—authors of other relevant research, and finally, validate their closing arguments using quantitative and qualitative analytical tools. In essence, both prosecuting attorneys and authors are saying, “This is my story and I can back it up, so believe me.” In this metaphor, reviewing editors act like judges overseeing preliminary hearings in that they weigh the validity of the case before them. Is it strong enough, i.e., sufficiently credible, to warrant proceeding further? If a manuscript does not communicate persuasively that it is sufficiently compelling in terms of theory, method, analysis, and conclusion, the answer will be no, a desk reject.

We have attempted to demystify the desk-review stage of the review process by sharing our insights and the heuristics we use as reviewing editors. We trust that authors will find our suggestions helpful and look forward to reviewing their manuscripts. Our suggestions are subject to some limitations. Most of the articles published in the Journal of International Business Studies , and business and management journals more broadly, are hypothesis-testing. Thus, our recommendations are predominantly derived from reviewing such manuscripts. Relatedly, most manuscripts submitted to social science journals, including the Journal of International Business Studies , fall within the domain of the logical positivist tradition. Despite these limitations, we believe that following our suggestions can increase the probability a manuscript will pass the desk-review stage, which is a critical step towards publication.

Appendix: How to minimize the probability of a desk rejection

Can I explain the story of my paper in 2 min in non-academic language?

Suppose I take out the first and final sentence of each paragraph, do the two sentences make sense?

Is my story focused, straightforward, and not complicated?

Is my story about a theory or practice, not about a sample or method?

If I have a story on method or sample, do I explain why this matters theoretically?

Did I present the paper before submitting it?

Did I rehearse a 15-min presentation out loud?

Do figures and diagrams add substantively to descriptions and explanations in the text?

Write a clear introduction

Is my introduction in the range of 500 to 750 words?

Can I explain in one sentence why the topic matters to non-academics? (Don’t answer “yes,” write out the sentence).

Does my first paragraph clearly: (1) identify the topic, (2) explain why it matters, (3) describe what is already known?

Select the first and final sentence of each paragraph, do those two sentences make sense? And do those eight to ten sentences from the paragraphs in the Introduction pull the reader in?

Know your audience

Can I write down three names of scholars that I would like to read the article?

Can I explain why I selected these three names?

Did I check if members of the editorial team have recently published on the topic of my paper?

Do I stay within the recommended word length of the journal?

If I exceed the word length, do I provide an explanation for why in the accompanying cover letter?

Did I check the latest editorials in the journal?

Did I check if there are relevant forthcoming articles published on the website already?

Do I refer to articles published in the journal to which I am submitting?

Did I read the journal’s style guide and prepare my manuscript accordingly?

Am I explicit about what is novel in my paper?

Did I perform a search in the journal to which I am submitting using the key terms in my manuscript?

Is each sentence in the entire manuscript no longer than two lines?

Do I limit the number of abbreviations and acronyms in my article?

If I use an abbreviation, do I explain it the first time I introduce it?

Are figures and diagrams comprehensible without reference to the written text?

Are my tables and figures logically numbered and put at the end of the manuscript, not embedded in the main text?

Write a clear abstract

Does the abstract tell the story in the manuscript?

Does the abstract give the topic, research question (motivation), theoretical approach, empirical setting (if relevant), findings, and why the study matters (contribution)?

In the abstract, if I replace the key construct of the manuscript with some other key construct, does the abstract no longer make sense?

Did I ask colleagues to read my abstract without them knowing the entire paper?

Distinguish between literature review and theory

Does the literature review clearly frame my research question in terms of prior research?

Is my literature review focused on work relevant to my specific research question, the key constructs, and chosen theoretical lens?

Do I identify a specific theory, define key constructs, and delineate relevant premises/assumptions?

Do all references used in the text refer to the statement made in that particular sentence? (In other words, do I make sure there are no ‘casual’ references?)

Spell out theoretical mechanisms

Do I rely on a well-defined theoretical model?

Do I present a compelling logic, e.g., line of reasoning, rather than rely on references to prior empirical works to support my hypotheses?

If I combine multiple theories, do I explain how the assumptions of these theories are compatible?

Do I rule out alternative explanations for the findings I report?

Do my hypotheses have a counterfactual? Put differently, can my hypotheses also not be true?

Do I avoid hypotheses that include more than one relationship?

Do I minimize the use of quotations to make my argument?

Isolate theoretical channels empirically

Are my hypotheses predicated on a theoretical argument? Alternatively: Do I make sure my hypotheses are not predicated on empirical findings (i.e., merely a retest with a different data set of prior empirical findings?)

Do my hypotheses constitute tests of theoretical (as opposed to empirical) relationships?

If I test for moderating/interaction effects, do I discuss the economic effect size of the total effect (e.g., plot the marginal effects in a graph)?

Do I address endogeneity?

Do I discuss how my methods and measures are suitable to test for the mechanisms I theorize?

Do I describe how I arrive at my sample?

Do I explain why my sample is appropriate for answering my research question and testing my hypotheses?

Do I provide a table with the characteristics of the observations and possible subsamples (e.g., countries, firms per country, number of teams, etc.)?

If my data are nested, do I control for the nested structured of my data, for example using multi-level methods?

If I use multi-level methods, do I provide the intra-class correlations?

Do I include a correlation table?

Is each empirical proxy I use in my analysis closely aligned with its respective abstract construct in my theoretical model?

Do I explain how a measure that was developed and used in other studies is appropriate for use in my study?

If I adapt existing measures to my study context, do I explicitly explain why and how?

If my dependent and independent variables are from the same survey instrument, do I address and mitigate common method variance?

Do I provide a list of variables I use in my analysis (e.g., in the appendix)?

Do I write down the names of the variables in full in the tables and figures?

Do I provide data sources for all variables (in the text and in the appendix)?

If I use AI tools to collect my data, am I transparent on the process and coding?

Do I include references to the data sources in the paper (main text, footnote, reference)?

Do I provide references of scholars who have used the same measures?

Do I provide a discussion of the economic effect size?

Do I explain novelty in a consistent manner in the abstract, introduction, and discussion sections?

Do I identify theoretical implications of my findings (being careful not to extrapolate beyond what the method and data allow)?

Do I identify practical implications of my findings, i.e., specific, actionable options?

If I read the practical implications independent of the rest of the manuscript, are they meaningful? (In other words, do I make sure my implications are not obvious/generic?)

Do I clearly describe what I can explain and what I cannot explain (sometimes referred to as ‘limitations’) of my study?

Miscellaneous

If I submitted this manuscript before to another journal and it was rejected after review, did I incorporate the comments provided?

Did I prepare a cover letter?

Do I have a possible conflict of interest (e.g., colleagues who have reviewed the manuscript before, or an editor with whom I am close friends, or an editor who has been my co-author)? If yes, am I transparent about that in my cover letter?

If my manuscript is based on data I used in other manuscripts (published or not), do I explain this in my cover letter?

If my manuscript is based on data I used in other manuscripts (published or not), can I explain the difference in theory and/or variables used?

If this paper is part of a series of studies on a related topic, do I make sure there is no textual overlap between this new manuscript and other ones?

Did I check if all in-text references are listed?

Are all references in the same style and format and does that format comply with journal’s requirements?

Do I acknowledge the limits of using AI tools in my efforts to speak to the audience I have in mind?

Am I transparent about how, when, and where I have used AI in my study (e.g., literature review or analytical tools)?

Journals differ in who they nominate to handle the desk-reject stage. Sometimes it is the Editor-in-Chief, sometimes the Managing Editor, and sometimes, like at this journal, desk rejects are handled by dedicated reviewing editors.

See https://www.palgrave.com/gp/journal/41267/authors/review-process

Section 3.3.5 of the Journals Code of Ethics of the Academy of International Business provides helpful examples of potential conflicts of interest between authors and an editor or reviewer: “(1) one of the Authors is at the same institution as the nominated Editor or Reviewer; (2) one of the Authors was a member of the Editor or Reviewer’s dissertation committee, or vice versa; or (3) one of the Authors, and the Editor or Reviewer, are currently Co‐Authors on another manuscript or have been Co‐Authors on a manuscript within the past three years.”

See https://www.palgrave.com/gp/journal/41267/authors/frequently-asked-questions for a sample originality matrix.

https://www.palgrave.com/gp/journal/41267/authors/editorial-policy

Ansell, B. W., & Samuels, D. J. (2021). Desk rejecting: a better use of your time. PS: Political Science & Politics, 54 , 686–689.

Google Scholar  

Dance, A. (2023). Peer review needs a radical rethink. Nature, 614 , 581–583.

Article   Google Scholar  

Grant, A. M., & Pollock, T. G. (2011). Publishing in AMJ part 3: Setting the hook. Academy of Management Journal, 54 (5), 873–879.

Leamer, E. (2009). Macroeconomic patterns and stories . Springer.

Book   Google Scholar  

Okhuysen, G., & Bonardi, J. P. (2011). The challenges of building theory by combining lenses. Academy of Management Review, 36 (1), 6–11.

Santangelo, G., & Verbeke, A. (2022). Actionable guidelines to improve ‘theory related’ contributions to international business research. Journal of International Business Studies, 53 (9), 1843–1855.

Thomas, D. C., Cuervo-Cazurra, A., & Brannen, M. Y. (2011). Explaining theoretical relationships in international business research: Focusing on the arrows, NOT the boxes. Journal of International Business Studies, 42 (9), 1073–1078.

Download references

Author information

Authors and affiliations.

Darla Moore School of Business, University of South Carolina, 1014 Greene Street, Columbia, SC, 29208, USA

Sjoerd Beugelsdijk

Goa Institute of Management, Sanquelim Campus, Poriem, Sattari, Goa, 403505, India

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sjoerd Beugelsdijk .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Beugelsdijk, S., Bird, A. How to avoid a desk reject: do’s and don’ts. J Int Bus Stud (2024). https://doi.org/10.1057/s41267-024-00712-8

Download citation

Published : 17 June 2024

DOI : https://doi.org/10.1057/s41267-024-00712-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research