Introduction
The Poisson Regression model assumes that the observed count data follows a Poisson distribution. One problem with this is that seldom does the assumption that the mean = variance hold. Often for this is because the data may contain a lot more zeros than is consistent with a Poisson distribution. Before we address this issue, however, we address the opposite issue, namely the case where the data contains no zero values.
The typical example here is length of hospital stay, where zero values don’t occur.
If the mean of a Poisson distribution is 10, then the probability of a zero value is POISSON.DIST(0,10,FALSE) = .00454%, and so we aren’t expecting many zeros anyway. Even when the mean is 5, we expect that less than 1% of the data is a zero. In these cases, we don’t really need to worry about this issue. Of course, if the mean is lower, say 2, then 13.5% of the data values are zero, and so it may be better to use a zero-truncated Poisson regression model. These models are based on a Zero-Truncated Poisson distribution.
Key Properties
As described in Zero-Truncated Poisson Distribution, the likelihood function in this case is
with a corresponding log-likelihood of
The maximum LL is therefore
It now follows that the regression model deviation is
with individual deviations of
where ri = yi – μi.
Since the mean and variance of a ZTP distribution are
we calculate the Pearson deviations as
and the Pearson deviation for the regression is equal to the sum of the squares of the pi.
We will use Solver to find the coefficients that maximize the LLfit value and estimate the covariance matrix by using the inverse of the k × k matrix C whose elements are
where k = the number of independent variables including the intercept.
Topics
Using these properties, we can now construct a ZTP regression model, assess its fit, and use it to make count predictions. Click on each of the following to see how this is done.
References
Hilbe, J. M. (2014) Modeling count data. Cambridge University Press
https://assets.cambridge.org/97811070/28333/frontmatter/9781107028333_frontmatter.pdf
UCLA (2024) Zero-Truncated Poisson Regression
https://stats.oarc.ucla.edu/stata/dae/zero-truncated-poisson-regression/