Suppose a doctor wants to diagnose whether a particular patient B has a certain disease D. The patient belongs to the reference class of people having both symptoms S1 and S2. The doctor knows that the probability of having the disease given having a symptom S1 is 0.9 (P(D|S1)=0.9 ) and she knows that the probability of having the disease having a different symptom S2 is 0.4 (P(D|S2)=0.4). Unfortunately, that is all she knows. What is the probability that the patient has the disease? 0.9? 0.4? 0.65? Or some different value?
This problem of conflicting reference classes is of main importance, because often there is no data available for the reference class of having both symptoms S1 and S2 (P(D|S1,S2) is unknown). Indeed, many people have claimed that in this case there is no valid probability assessment of the person B having the disease (see for instance, Venn, J.: The logic of chance, 3rd edition, 1988, p.213, Reichenbach, H.:The theory of probability. University of Berkley Press, 1949).
Personally, I recommend to use the value 0.83 in this case. This line of thought gives more weight to the fact that the patient B belongs to a reference class with 0.9 probability. Finding individuals in the reference class S1 that do not have the disease D is very hard. Finding individuals in the reference class S2 that do have the disease D is not very hard. Hence, the reference class S1 should be given more weight to.
While this argument is relying on intuition, it is backed up by more elaborated approaches to reference class reasoning. There are two approaches that recommend the value 0.83: John Pollock’s approach to reference class reasoning (Pollock, J. Probable Probabilities. Unpublished) and the maximum entropy approach (Williamson, J:. In Defense of objective Bayesianism. OUP, 2010) applied to reference class reasoning. This comment is too short to go into more detail, but both approaches are derived from very reasonable principles.
I warmly invite you to join the contest and to suppose a different probability value for the patient B having the disease D.