What’s a UCL?
The upper confidence limit (UCL) of the arithmetic mean is often used as a metric of compliance against an exposure standard. In this post we will look at what a UCL is, how they are calculated, and how to interpret the values calculated. For the rest of this post the UCL will always be in reference to the arithmetic mean. You can, of course, have UCLs for other statistics like the exceedance fraction.
Definition
The 95% UCL is often defined as “the range within which we are X% confident that the true mean lies”. This definition is pervasive even in academic literature. While it’s not a terrible simplification, it’s not technically true. This is closer to the definition of a credible interval which is a bayesian idea.
A more apt definition of a 95% UCL is “a range generated by a sampling program below which the true mean lies 95% of the time for repeated sampling programs”. In other words, if you re-performed the same sampling program 100 times, 95 of the UCLs would contain the true arithmetic mean.
Many confidence intervals calculated from repeated sampling. This graphic has the full confidence interval, but the same logic applies.
You could argue that the difference between what people usually think and the true definition doesn’t matter. And you’re probably right. But the clarification helped me to realise two things.
That what matters from a health risk perspective is the arithmetic mean (and its relation to accumulative exposure) and not the UCL, despite the fact it is used as a measure of compliance.
That the UCL is just as much of a commentary of the quality of your sampling program than it is about the value of the AM.
“A 95% confidence interval does not mean that there is a 95% probability that the interval contains the true mean. [It] either contains the true mean or it does not. Instead, the level of confidence is associated with the method of calculating the interval.”
Say you were monitoring an airborne chemical. You find that the exposure mean is 10% of the OEL, but the 95% UCL is above the OEL. Under some legislation this is a non-compliance. In reality, it’s more likely the sample plan was too small for the variability, rather than showing workers are at unacceptable risk. A high UCL does not necessarily mean a high AM. It could be that not enough samples were collected given the amount of data variability. This should become more apparent when we look at how the CI is calculated.
Calculation
The interpretation of the confidence interval can be reinforced by understanding how it’s calculated:
From the equation above you’ll notice that:
The UCL gets bigger as the mean and standard deviation increases
The UCL gets smaller as the sample size increases.
This means when you have more variability, and the mean is closer to the OEL, you’ll need more samples to prove compliance. Also notice that the sample size is square rooted in the equation - which means each additional sample will have less impact on the UCL.
The “Land’s Exact” calculation used in IHStats is a slightly modified version of the above calculation to suit a lognormal distribution. I like to think of it as the UCL version of the MVUE of the AM. The modified equation is scary:
Its not important to understand Land’s Exact equation but if you look closely you see that sample size (“n”) and variance (“Sy”) appear much more often.
Interpretation
A statistician would interpret confidence intervals like this:
I think the two non-compliance scenarios would not be common for occupational hygienists to come across in their statistical analysis. Most would have already stopped monitoring and begun implementing controls.
Many hygienists would take “unproven compliance” to mean “non-compliance” and recommend additional control. While I would never discourage control exposures as low as possible, hopefully you can now interpret this situation with nuance.
I hope, too, that this has given you more appreciation for the importance of the design of monitoring programs.
Further Reading / References
NIST Confidence limits for the Mean (Website)
Tan et al (2010) The correct interpretation of confidence intervals. Proceedings of Singapore Healthcare Vol 19, No 3. Link
The Original Paper on CI Neyman (1937) Outline of a theory of statistical estimation based on the classical theory of probability. Philosophical transactions of the Royal Society of London. Vol 236 Link
Morey et al. (2016) The fallacy of placing confidence in confidence intervals. Psychonomic Bulletin and Review. 23. Link