Short form of the question:
What is the (best way to speculate about the) probability of H0 being true, if an observation was just made that only happens 1 in 1000 times if H0 is true ?
Elaborations about the question (optional):
Say I have 3 hypothesis, H0 = drug has 0 effect. H1 = drug has positive effect > 0, H2= drug has positive effect < 0)
From conducting tests, one can get observations, that may be unlikely given certain assumptions, often one use null hypothesis testing and get results like this :
If H0 was true, the chance of getting these good results are say 1 in 1000. p 0.001. And I get that one can subjectively set thresholds one are pleased with and say reject that hypothesis based on that subjective thresholdbeing crossed or not, but that seems very subjective to me, there must be some more scientific way than that surely ?
How can one best move from here, without fallacies, to find (or most mathematically correct guess/speculate) about the probabilities of for example H0 actually being true or false based on whatever observation is made? Is it impossible ?
What is the (best way to speculate about the) probability of H0 being true, if an observation was just made that only happens 1 in 1000 times if H0 is true ?
Elaborations about the question (optional):
Say I have 3 hypothesis, H0 = drug has 0 effect. H1 = drug has positive effect > 0, H2= drug has positive effect < 0)
From conducting tests, one can get observations, that may be unlikely given certain assumptions, often one use null hypothesis testing and get results like this :
If H0 was true, the chance of getting these good results are say 1 in 1000. p 0.001. And I get that one can subjectively set thresholds one are pleased with and say reject that hypothesis based on that subjective thresholdbeing crossed or not, but that seems very subjective to me, there must be some more scientific way than that surely ?
How can one best move from here, without fallacies, to find (or most mathematically correct guess/speculate) about the probabilities of for example H0 actually being true or false based on whatever observation is made? Is it impossible ?