Why am I getting "algorithm did not converge" and "fitted prob numerically 0 or 1" warnings with glm?

Solution 1:

If you look at ?glm (or even do a Google search for your second warning message) you may stumble across this from the documentation:

For the background to warning messages about ‘fitted probabilities numerically 0 or 1 occurred’ for binomial GLMs, see Venables & Ripley (2002, pp. 197–8).

Now, not everyone has that book. But assuming it's kosher for me to do this, here's the relevant passage:

There is one fairly common circumstance in which both convergence problems and the Hauck-Donner phenomenon can occur. This is when the fitted probabilities are extremely close to zero or one. Consider a medical diagnosis problem with thousands of cases and around 50 binary explanatory variable (which may arise from coding fewer categorical variables); one of these indicators is rarely true but always indicates that the disease is present. Then the fitted probabilities of cases with that indicator should be one, which can only be achieved by taking β_i = ∞. The result from glm will be warnings and an estimated coefficient of around +/- 10. There has been fairly extensive discussion of this in the statistical literature, usually claiming non-existence of maximum likelihood estimates; see Sautner and Duffy (1989, p. 234).

One of the authors of this book commented in somewhat more detail here. So the lesson here is to look carefully at one of the levels of your predictor. (And Google the warning message!)

Solution 2:

This is probably due to complete separation, i.e. one group being entirely composed of 0s or 1s.

There are several options to deal with this:

(a) Use Firth's penalized likelihood method, as implemented in the packages logistf or brglm in R. This uses the method proposed in Firth (1993), "Bias reduction of maximum likelihood estimates", Biometrika, 80,1.; which removes the first-order bias from maximum likelihood estimates.

(b) By using median-unbiased estimates in exact conditional logistic regression. Package elrm or logistiX in R can do this.

(c) Use LASSO or elastic net regularized logistic regression, e.g. using the glmnet package in R.

(d) Go Bayesian, cf. the paper Gelman et al (2008), "A weakly informative default prior distribution for logistic & other regression models", Ann. Appl. Stat., 2, 4 and function bayesglm in the arm package.

(e) Use a hidden logistic regression model, as described in Rousseeuw & Christmann (2003),"Robustness against separation and outliers in logistic regression", Computational Statistics & Data Analysis, 43, 3, and implemented in the R package hlr.

You need to recode your factor as a factor first though using dat$bid1 = as.factor(dat$bid1))

Solutions to this problem are also discussed here:

https://stats.stackexchange.com/questions/11109/how-to-deal-with-perfect-separation-in-logistic-regression

https://stats.stackexchange.com/questions/45803/logistic-regression-in-r-resulted-in-perfect-separation-hauck-donner-phenomenon

https://stats.stackexchange.com/questions/239928/is-there-any-intuitive-explanation-of-why-logistic-regression-will-not-work-for

https://stats.stackexchange.com/questions/5354/logistic-regression-model-does-not-converge?rq=1

Solution 3:

If you have correctly specified the GLM formula and the corresponding inputs (i.e., design matrix, link function etc...). The glm algorithm may not converge due to not enough iterations used in the iteratively re-weighted least squares (IRLS) algorithm. Change maxit=25 (Default) to maxit=100 in R.