Nobody is above mistakes but there are situations where a single mistake can cause physical, psychological and economical problems. In these types of situations, one has to be very careful. Imagine going to a hospital and getting misdiagnosed, being convicted for a crime you did not commit or giving the public some false information that causes them to panic for no reason. These are all serious problems that can be a result of some error somewhere.
In this article, you will learn about errors in hypothesis testing, their causes and how to balance them.
Hypothesis Testing
Before we can talk about hypothesis testing and the errors that can occur, you should first know what a hypothesis is.
A hypothesis is a proposed claim or idea about the characteristics of a population.
A hypothesis can be challenged based on some newfound knowledge about a population. It can be tested and compared to other claims to confirm or to draw a new conclusion.
Let's see the definition of hypothesis testing.
Hypothesis testing is a procedure that uses sample data to confirm a hypothesis or claim about a population by comparing it with another claim.
See the example below.
A battery company claims that their batteries last 18 hours but some new information comes up to say that the battery lasts less than 18 hours. Testing these claims by comparing them will help decide which is true and which is false.
To find out more about hypothesis testing, see our article on Hypothesis Testing
Hypothesis testing is used to find out if a claim is true or not but during this process of testing, some errors can occur. These errors can affect the conclusion of the test. It can lead to wrong results and decisions. Before we dive deep into errors in hypothesis testing, there are some terms you should know.
From the example above, the original claim by the battery company that the batteries last for 18 hours is the null hypothesis while the claim that it lasts for less is the alternative hypothesis.
There are two types of errors in hypothesis testing, Type I and Type II errors. Read on to find out more about them.
Type I Error in Hypothesis Testing
What is a Type I error?
Type I error is the error that occurs when the null hypothesis (\(H_0\)) is concluded to be false or is rejected when it is actually true.
Let's take a look at an example.
A man is being accused of murder and the judge is trying to decide if he is guilty or not. The possibility that he is not guilty is the null hypothesis and the possibility that he is guilty is the alternative hypothesis.
The hypotheses will be written as:
\[ \begin {align} &H_0: y = \text{man is not guilty}\\ &H_a: y = \text{man is guilty} \end {align} \]
where \(y\) represents the man's verdict.
If at the end of the trial, the judge concludes that he is guilty when he actually isn't, that will be a Type I error and an innocent man will go to jail.
The probability of a Type I error is called the level of significance of the test and it is denoted by \(\alpha\). So, if you have \(\alpha = 0.05\), it means that the level of significance of the test is \(0.05\).
You can also define \(\alpha\) as the probability of rejecting a null hypothesis when it is true.
\[ P(rejecting \ H_0 / H_0 \ is \ true) = \alpha \]
Type II Error in Hypothesis Testing
What is a Type II error?
Type II error is the error that occurs when the null hypothesis (\(H_0\)) is accepted when it is false.
Let's use the previous example about a murder case to illustrate a Type II error.
A man is being accused of murder and the judge is trying to decide if he is guilty or not. The possibility that the man is not guilty is the null hypothesis and the possibility that he is guilty is the alternative hypothesis.
The hypothesis will be written as:
\[ \begin {align} &H_0: y = \text{man is not guilty}\\ &H_a: y = \text{man is guilty} \end {align} \]
where \(y\) represents the man's verdict.
If at the end of the trial, the judge concludes that he is not guilty when he actually is, that will be a Type II error and a murderer will be left unpunished.
The probability of a Type II error is denoted by \(\beta\)
\[ P(\text{accepting }H_0:\; H_0 \text{ is false}) = \beta . \]
Either Type I or Type II error can occur in any test but the error that is more serious or significant depends on the situation.
Let's take a look at an example.
A doctor's diagnosis of a patient is to be confirmed with a test. The null hypothesis is that the patient has the disease and the alternative hypothesis is that he doesn't have the disease.
If the test concludes that the patient has the disease when he doesn't then that's a Type II error. In this case, a Type II error is much more serious than a Type I because assuring someone that they are healthy when they are not can cause serious problems.
Balancing Errors in Hypothesis
Ideally, the probability of having a Type I and Type II error should be zero i.e \(\alpha = \beta = 0 \), but the only way this can be possible is if the information you use from a census is taken from the population instead of a sample. Since that is highly unlikely, you have to prepare for errors. If you decrease the value for \(\alpha\) or choose a small value for \(\alpha\), this would mean that you are reducing the chances of a Type I error happening. This might sound good but doing this will increase the chances of a Type II error happening. So, as you decrease \(\alpha\), \(\beta\) increases.
Both errors have their consequences but you have to try to balance this. To balance this, you have to use an appropriate value for \(\alpha\) and \(beta\). Trying to figure out the right values to use will depend on the situation. To create a balance try not to make the value of \(\alpha\) too small. You should assess the consequences of both errors and use the largest value of \(\alpha\) and/or \(\beta\) that can be accepted for the situation.
Relationship between Type I and Type II Errors in Hypothesis Testing
When you are dealing with a sample from a population, you are bound to encounter a Type I or Type II error. In trying to balance and minimize the errors, you will notice their relationship. The relationship is seen when varying the values of \(\alpha\) and \(\beta\). The smaller the value of \(\alpha\), the bigger the value of \(\beta\) gets. In other words, as \(\alpha\) increases, \(\beta\) decreases. The errors are not independent of each other but they are inversely proportional.
As stated in the section above, making the value of \(\alpha\) as small as possible in hopes to reduce Type I error results in a higher possibility of Type II error occurring. Solving this will be to find a balance by choosing tolerable values for both \(\alpha\) and \(\beta\).
Causes of Type I and Type II Errors in Hypothesis Testing
Some of the causes of Type I error are listed below.
- Type I errors can occur as a result of a small sample size. A study or research is usually done on a sample of a population and sometimes, the sample may not contain enough information to come to an accurate conclusion. Increasing the sample size will help reduce the chances of a Type I error happening.
- Using poor research methods can lead to Type I error. If you don't use good methods and don't gather sufficient and correct data, a Type I error can occur.
- An external factor can affect the variables involved and can cause the result of the test to be against the null hypothesis.
- Setting the level of significance (\(\alpha\)) before the test without taking into account the sample size and how long the test will take can cause a Type I error.
Some of the causes of Type II errors are listed below.
- Type II error will occur when the power of the test is low.
- Using a small sample size can cause a Type II error.
The power of a test is the ability of a test to reject a null hypothesis when it is false. For more information see the article on Hypothesis Testing.
Examples of Errors in Hypothesis Testing
Let's see some examples of errors in hypothesis testing.
According to a poll, \(15\%\) of the students in a school do not like eating food from the cafeteria. The principal of the school decides to take a sample of her school's population to test and see if this claim is true. Let \(y\) be the percentage of the students who do not like eating food from the cafeteria. The hypothesis she used is below. \[ \begin {align} &H_0:y = 0.15 \\&H_a:y \neq 0.15. \end {align} \]
In which of the following conditions did the principal commit a Type I error?
- She concludes the percentage of students is not \(15\%\) when it isn't.
- She concludes the percentage of students is not \(15\%\) when it is.
- She concludes the percentage of students is \(15\%\) when it is.
- She concludes the percentage of students is \(15\%\) when it isn't.
Solution:
The correct option is option B. Concluding that it is not \(15\%\) when it actually is, is rejecting the null hypothesis which is a Type I error.
Option A is not an error. Concluding that it is not \(15\%\) when it is not, is not an error at all.
Option C is also not an error. Concluding that it is \(15\%\) when it is actually \(15\%\) is no error at all.
Option D is a Type II error. Concluding that it is \(15\%\) when it is not, is a Type II error.
Let's take a look at another example.
After research, it was concluded that then men in Town A are \(5\) times more likely to have lung cancer than the men in Town B. In an attempt to verify this result, it was found that the men in Town A are not \(5\) times more likely to have lung cancer than the men in Town B. What type of error occurred here?
Solution:
The null hypothesis \(H_0\) is that men in Town A are \(5\) times more likely to have lung cancer than the men in Town B. The alternative hypothesis \(H_a\) is that the men in Town A are not \(5\) times more likely to have lung cancer than the men in Town B.
After verification, it was found that \(H_0\) is false. This means that it was accepted to be true when it is indeed false. This is a Type II error.
Classifying which kind of error is happening and the cause of it can help improve testing and quality.
All the bottles manufactured by a certain company have an average diameter of \(4\, \mathrm{cm}\). The company suspects that the measurement has changed meaning they will need to re-calibrate their machine. Before they do that, they decide to take a sample of their product for testing. The hypotheses are:
\[ \begin {align} &H_0: x = 4\, \mathrm{cm} \\ &H_a: x \neq 4\, \mathrm{cm} \end {align} \]
where \(x\) is the average diameter of the bottle. After testing, \(H_0\) was rejected in favor of \(H_a\) when \(H_0\) was true.
What type of error is this and what may have caused it?
Solution:
The error here is a Type I error. A Type I error occurs when the null hypothesis is rejected in favor of the alternative hypothesis even though the null hypothesis is true.
This may have been caused by using a very small sample size for testing, an external factor may have contributed to altering the result, or the testing method may have been faulty.
Let's see one more example.
The following is are the null and alternative hypothesis for a new medical treatment:
\[ \begin {align} &H_0: p = 0.75 \\ &H_a: p &< 0.75 \end {align} \] where \(p\) represents the success of the treatment. In which of the following conditions is there a Type II error? Explain the effect or consequences of each of the options on a patient.
- Accepting \(H_0\) in favor of \(H_a\) when \(H_0\) is true.
- Rejecting \(H_0\) in favor of \(H_a\) when \(H_0\) is true.
- Accepting \(H_0\) in favor of \(H_a\) when \(H_0\) is false.
- Accepting \(H_a\) in favor of \(H_0\) when \(H_a\) is true.
Solution:
The correct option is option C. Accepting the null hypothesis when it is false is a Type II error. The consequence of this error is that you will give a patient a treatment that is not as effective as it is believed to be. There will be little or no improvement in the patient's condition which can lead to worsened health or even death.
Option A is not an error at all because \(H_0\) is accepted when it is indeed true. If you give the treatment to a patient, you will see the expected change in the patient's condition.
Option B is a Type I error because \(H_0\) is rejected when it is true. This error will mean that you will not give the patient the treatment because it is now thought of to be less effective than expected. The patient will not have the opportunity to improve their health with this treatment.
Option D is not an error at all because \(H_a\) is accepted when it is indeed true. Time and effort will not be wasted in giving a patient a treatment that is not effective.
Errors in Hypothesis Testing - Key takeaways
- Type I error is the error that occurs when the null hypothesis (\(H_0\)) is concluded to be false or is rejected when it is actually true.
- Type II error is the error that occurs when the null hypothesis (\(H_0\) is accepted when it is false.
- Type I error and Type II error is inversely proportional to each other.
Subjects Free & expert-verified explanations.
Exams Exam preparation made easy.
Textbooks Free and expert-verified textbook solutions.
Magazine Best study tips and tricks for your exams.
