Exponential Regression Newton’s Method – Advanced

Property 1: Given samples {x1, …, xn} and {y1, …, yn} and let ŷ = αeβx, then the values of α and β that minimize \sum{}(yi − ŷi)2 satisfy the following equations:

image9215image9216

Proof: The minimum is obtained when the first partial derivatives are 0. Let

image9217

Thus we seek values for α and β such that \frac{\partial h}{\partial \alpha} = 0 and \frac{\partial h}{\partial \beta} = 0; i.e.

image9218image9219

Property 2: Under the same assumptions as Property 1, given initial guesses α0 and β0 forα  and β, let F = [f  g]T where f and g are as in Property 1 and

image9220

Now define the 2 × 1 column vectors Bn and the 2 × 2 matrices Jn  recursively, as follows

image9221

image9222

image9223

Provided α0 and β0 are sufficiently close to the coefficient values that minimize the sum of the deviations squared, then Bn converges to such coefficient values.

Proof: Now

image9224

image9225

image9226

image9227

Thus

image9228

The proof now follows by Property 2 of Newton’s Method.

Links

↑ Exponential regression

↑ Nonlinear model using Newton’s method

References

Foth, R. (2024) Modeling exponential relationships with regression
https://pimaopen.pressbooks.pub/topicsinmathematics/chapter/5-4-modeling-exponential-relationships-with-regression

Abramson, J. (2024) Exponential and logarithmic regressions
https://math.libretexts.org/Workbench/1250_Draft_3/06%3A_Exponential_and_Logarithmic_Functions/6.09%3A_Exponential_and_Logarithmic_Regressions

Libre Texts (2024) Nonlinear regression
https://math.libretexts.org/Workbench/Numerical_Methods_with_Applications_(Kaw)/6%3A_Regression/6.04%3A_Nonlinear_Regression

Geeks for Geeks (2024) Non-linear regression with examples
https://www.geeksforgeeks.org/non-linear-regression-examples-ml/

3 thoughts on “Exponential Regression Newton’s Method – Advanced”

  1. Hi Charles:
    Regarding Question 1, it is standing no more. I thought it was the Fcrit, but instead it is like the p-value, then it does have the value you presented. Sorry for my confusion.

    Question 2 still stands.

    All the best.

    Reply

  2. Hi Charles:
    I find your site very interesting and useful. Thanks for the great effort !!!

    In your “Exponential Regression using Newton’s Method” page, it does not have a “Leave a Reply” at the end, so I am posting here. Sorry for any inconvenience.

    Question 1. In Figure 3. Which is the formula for “Significance F”…is it “=F.INV(0.05, 1, 9)”? If that is the case, Excel gives the result of 0.00416, but you have 1.175E-06.

    Question 2. Which is the rationale to have the formulas in cells Q25 and Q26, for the standard error of a and b (alfa and beta)? I mean, why you take values from the JTJ-1 matrix? Please clarify the reason.

    Thanks in advance for the clarifications.

    Reply

Leave a Comment