Finding multinomial logistic regression coefficients using Newton’s method

Basic Concepts

Instead of using Solver, we can use Property 3 of Basic Concepts of Multinomial Logistic Regression to calculate the multinomial logistic regression coefficients. In order to demonstrate how to use Newton’s method, we initialize the coefficients with the result from Solver shown in Figure 2 of Finding Multinomial Logistic Regression Coefficients using Solver and perform one iteration using the equation in Property 3 of Basic Concepts of Multinomial Logistic Regression,

Figure 1 – One iteration of Newton’s Method

The key formulas used in Figure 1 are shown in Figure 2.

Figure 2 – Key formulas from Figure 1

If we set the initial values of coefficients to zeros then we can use Newton’s Method to find the values of the multinomial logistic regression coefficients (e.g. using the MLogitCoeff worksheet function described in Finding Multinomial Logistic Regression Coefficients) as shown in the Figures 3, 4, and 5.

Results

Figure 3 – Multinomial logistic reg. via Newton’s Method (part 1)

Figure 4 – Multinomial logistic reg. via Newton’s Method (part 2)

Logistic Regression Results

Figure 5 – Multinomial logistic reg. via Newton’s Method (part 3)

Examples Workbook

Click here to download the Excel workbook with the examples described on this webpage.

References

Wikipedia (2014) Multinomial logistic regression
https://en.wikipedia.org/wiki/Multinomial_logistic_regression

Field, A. (2005) Discovering Statistics Using SPSS. 3rd ed. Sage
https://profandyfield.com/discoverse/dsus/

Cheng, H., (2021) Multinomial logistic regression
https://bookdown.org/chua/ber642_advanced_regression/multinomial-logistic-regression.html

Agresti, A. (2002) Categorical data analysis, Wiley & Sons