{"id":14236,"date":"2023-01-25T11:29:45","date_gmt":"2023-01-25T10:29:45","guid":{"rendered":"https:\/\/www.physiotutors.com\/?post_type=wiki&p=14236"},"modified":"2023-01-25T11:29:45","modified_gmt":"2023-01-25T10:29:45","slug":"type-1-error-rate-control","status":"publish","type":"wiki","link":"https:\/\/www.physiotutors.com\/wiki\/type-1-error-rate-control\/","title":{"rendered":"Type 1 Error Rate Control"},"featured_media":15379,"template":"","class_list":["post-14236","wiki","type-wiki","status-publish","has-post-thumbnail","hentry"],"acf":{"sections":[{"acf_fc_layout":"page-header-detail-page","background":{"background_image":15379,"background_image_alt_text":"Type 1 Error Rate | Statistics"},"heading":"Type 1 Error Rate | Statistics","subheading":"Stats & Methodology","content_editor":"","button_type":"internal_link","button_label":"Check our shop","button_style":"button-five","button_internal_link":4646,"button_anchor":"","button_url":"","button_file":"","label":""},{"acf_fc_layout":"submenu","links":[{"button_type":"anchor","button_label":"Learn","button_internal_link":null,"button_anchor":"section-number-3","button_url":"","button_file":null},{"button_type":"anchor","button_label":"Assessment E-Book","button_internal_link":null,"button_anchor":"section-number-4","button_url":"","button_file":null},{"button_type":"anchor","button_label":"Reviews","button_internal_link":null,"button_anchor":"section-number-5","button_url":"","button_file":null}],"dropdown_label":"Go to","button_right":{"button_type":"internal_link","button_label":"Become a member","button_style":"button-three","button_internal_link":16662,"button_anchor":"","button_url":"","button_file":""},"button_description":"Find this wiki on the Physiotutors platform"},{"acf_fc_layout":"content-content","section_options":{"vertical_alignment":"top","column_right_box":false,"column_left_width":"col-12 col-lg-4 offset"},"column_left":{"content_editor":"

Learn<\/h4>"},"column_right":{"content_editor":"

Type 1 Error Rate | Statistics<\/h2>\nTesting multiple variables inflates the type 1 error rate or the false positive rate. This is called the multiple comparison problem<\/a>. Correcting for this alpha-inflation is not hard. There are two main ways, namely the Bonferroni correction and the Holm correction.\n
<\/h3>\n
Bonferroni correction<\/h3>\nThe Bonferroni correction is simple but quite conservative. You divide your alpha level<\/a> by the number of tests you are about to perform. This will be the new significance level. So in this case:\n\n\u0251 \/ n\n\n\u0251: alpha or significance level\n\nn: number of tests\n\n0.05 \/ 10 = 0.005\n\nYou can thus do this quite easily yourself when reading a paper. If five variables are tested, you know the alpha level<\/a> should be about 0.01 instead of 0.05 (0.05 \/ 5). This is under the assumption that the researchers did not perform a boatload of tests \u201cbehind the scenes\u201d while not reporting them. This is called data-dredging or p-hacking<\/a>.\n\nAnother way is by simply multiplying the p-value<\/a> in the paper by the number of tests.\n\nEg.<\/em>\n\nP-value = 0.03<\/em>\n\n0.03 * 10 = 0.3<\/em>\n\nThis means that the previously significant p-value<\/a> now became insignificant if 10 variables were tested.\n
Bonferroni correction limitations<\/h3>\nThe Bonferroni correction is a widely used method to adjust the significance level for multiple comparisons<\/a> in order to control the overall Type I error rate. However, it has several limitations.\n\nOne of the main issues is that it can be overly stringent, which may lead to a loss of statistical power. Additionally, it assumes that all comparisons are independent, which may not be the case in real-world data, potentially leading to higher Type II error<\/a> rates.\n\nAnother limitation of the Bonferroni correction is that it increases the chance of false negatives or Type II errors<\/a>, meaning that there is a higher chance of missing a true effect.\n\nFinally, the Bonferroni correction is most appropriate for situations where the number of comparisons is relatively small, as it may not be as effective when the number of comparisons is very large. Therefore, researchers should carefully consider the appropriateness of the Bonferroni correction for their research question and data set, and be aware of its limitations.\n
Holm correction<\/h3>\nA second way to correct the alpha inflation is the Holm correction. Let\u2019s say the researchers did five tests and thus became five p-values<\/a>. For the Holm correction to work, they should be ranked from lowest to highest.\n\nEg.<\/em>\n