## Uncertainty in individual factuality scores

These methods apply to analyses of any one of our malarkey scores.

## Estimating the probability distribution

Step 1: Sample the percentages of statements in each category of a each available report card from a Dirichlet distribution with parameters equal to the total number of events each category plus one.

Step 2: Caclulate malarkey from the sample report card percentages.

Step 3: Repeat steps 1 and 2 many times. The more the better. When Brash started doing this, he did it 10,000 times. Now he does it at least 100,000 times.

Step 4: The set of proportions of factuality scores at a each given value estimates the probability distribution of the factuality score in question.

Step 2: Caclulate malarkey from the sample report card percentages.

Step 3: Repeat steps 1 and 2 many times. The more the better. When Brash started doing this, he did it 10,000 times. Now he does it at least 100,000 times.

Step 4: The set of proportions of factuality scores at a each given value estimates the probability distribution of the factuality score in question.

## Estimating the confidence intervals

From the estimated probability distribution, calculate values at the 2.5th and 97.5th percentile.

Voila. 95% confidence interval.

Voila. 95% confidence interval.

## Extensions to aggregate malarkey and comparisons

The method is essentially the same when estimating the uncertainty in aggregate malarkey, and in comparisons between two individuals or groups.