Juanmanner
New member
- Joined
- Dec 4, 2019
- Messages
- 1
Recent reviews (e.g., Button et al. 2013) suggests that neuroscience research papers have a median power (1? )=0.18, given ? =0.05. Being a skeptic, your prior belief is that only 1 0% of research studies examine “real” effects; you consider the rest to have true effect sizes of effectively zero. Given this information, answer the following questions:
a. Before examining a neuroscience paper, what probability describes your prior belief about whether it will report a real effect (denoted by ? ( ? ? ))? What probability describes your prior belief that no effect exists (denoted by ? ( ? ∅))?
b. Let “sig.” denote a significant result within a given study. Given the information above, report the values of the two probabilities: ? (sig.| ? ∅) and ? (sig.| ? ?). Additionally, identify each of these probabilities by name.
c. Given the four probabilities reported above, what is the value of ? (sig.)? Also, describe in English what this probability tells us.
d. Using Bayes’ Rule, calculate your posterior belief that the published significant result describes a true effect. Interpret the resulting value in English, and compare it to your prior belief.
e. Suppose that you re-read the same study and realize that it is guilty of p-hacking, to the extent that its effective rate of false positives is actually closer to ?=0.40. Assuming the power of the study is still the same (0.18), re-calculate your posterior belief that this published significant result describes a true effect. What does this result tell you about the dangers of p-hacking (or running multiple comparisons, or using any other method that inflates your Type I Error Rate)?
a. Before examining a neuroscience paper, what probability describes your prior belief about whether it will report a real effect (denoted by ? ( ? ? ))? What probability describes your prior belief that no effect exists (denoted by ? ( ? ∅))?
b. Let “sig.” denote a significant result within a given study. Given the information above, report the values of the two probabilities: ? (sig.| ? ∅) and ? (sig.| ? ?). Additionally, identify each of these probabilities by name.
c. Given the four probabilities reported above, what is the value of ? (sig.)? Also, describe in English what this probability tells us.
d. Using Bayes’ Rule, calculate your posterior belief that the published significant result describes a true effect. Interpret the resulting value in English, and compare it to your prior belief.
e. Suppose that you re-read the same study and realize that it is guilty of p-hacking, to the extent that its effective rate of false positives is actually closer to ?=0.40. Assuming the power of the study is still the same (0.18), re-calculate your posterior belief that this published significant result describes a true effect. What does this result tell you about the dangers of p-hacking (or running multiple comparisons, or using any other method that inflates your Type I Error Rate)?