Statistical significance under low power: A Gettier case?
Epistemologists and researchers alike are concerned with the conditions under which one can be said to possess knowledge. In philosophy there is a classic account of knowledge as being "Justified True Belief". Under this account, a person can only truly know something if they have a belief that is both justified and true (i.e. knowledge as JTB).
For example: An individual S knows a proposition P, iff (if and only if):
i) S believes that P,
ii) P is true, and
iii) S is justified in believing that P.
So-called "Gettier cases" arise when an individual is justified in believing something to be true and yet would probably not be said to have knowledge because they only got the right answer as a result of luck.
I posit that some instances in classical (frequentist) statistics are analogous to Gettier cases: A researcher who performs a study with low power, in which the alternative hypothesis is true, AND who observes a significant p-value, could be considered to be "lucky", given the likelihood that they would've observed non-significant findings. Still, they have inferred from that the alternative hypothesis is true; despite the improbability of doing so. It is unclear whether or not this should be considered "knowledge", under JTB.
·
Gettier criteria are useful but dangerously simplistic. Problems include:"P-true" might be wrong or unmeaningful&S-justified may be over-vague. Your p-value example for P-true is good, especially as there could be justification for believing either P OR for not-P where neither P nor NOT-P is true (both meaningless, because totally unrelated, unsuspected Q is true.) I give an example at:
https://fullduplexjonrichfield.blogspot.com/2014/01/normal-0-false-false-false_2.html in the section:"Well asked is half answered"
Jon Richfield · 8 Dec, 2020
Gettier criteria are useful but dangerously simplistic. Problems include:"P-true" might be wrong or unmeaningful&S-justified may be over-vague. Your p-value example for P-true is good, especially as there could be justification for believing either P OR for not-P where neither P nor NOT-P is determinable, or neither is true (both meaningless because of underdetermination of our accessible hypotheses, or because totally unrelated, unsuspected Q is true.) I give an example at:
https://fullduplexjonrichfield.blogspot.com/2014/01/normal-0-false-false-false_2.html in the section:"Well asked is half answered"
Jon Richfield · 8 Dec, 2020
Gettier criteria are useful but dangerously simplistic. Problems include:"P-true" might be wrong or unmeaningful&S-justified may be over-vague. Your p-value example for P-true is good, especially as there could be justification for believing either P OR for not-P where neither P nor NOT-P is determinable, or neither is true (both meaningless because of underdetermination of our accessible hypotheses, or because totally unrelated, unsuspected Q is true.) I give an example at:
https://fullduplexjonrichfield.blogspot.com/2014/01/normal-0-false-false-false_2.html in the section:"Well asked is half answered"
Jon Richfield · 8 Dec, 2020
Sorry, new here and I don't yet know how to delete comments. Of my preceding two, only the second is intended.
Jon Richfield · 8 Dec, 2020
very good ！
Xiaohong LI · 22 Mar, 2021