Newton's method replaces the curve by its tangent at xn and takes the point where the tangent meets the axis: xn+1=xn−f′(xn)f(xn)
Lesson 3 of 3Show all lessonsHide
Try the code and the simulations beside the lesson.
What you will be able to do
derive Newton's formula from the tangent line and use it;
explain why it converges so fast near a simple root;
recognise the starting values and the functions for which it fails.
Follow the tangent
Stand on the curve y=f(x) at the point (xn,f(xn)). Near that point the curve looks like its tangent line. Where does the tangent meet the x-axis? The tangent is y−f(xn)=f′(xn)(x−xn). Put y=0:
xn+1=xn−f′(xn)f(xn).
This is the Newton-Raphson formula. It needs f′(xn)=0. Isaac Newton used the idea in the 1660s, and Joseph Raphson published the general rule in 1690.
A worked example
Solve x3+2x−5=0 with x0=1.5. Here f′(x)=3x2+2.
First step.x1=1.5−8.751.375≈1.342857.
Second step.x2≈1.328384.
Third step.x3≈1.3282689.
The errors are about 1.5×10−2, 1.2×10−4 and 7×10−9. Bisection would need about 27 steps for the accuracy of x3.
Two panels. On the left, a rising curve with three tangent lines: the tangent at x0 meets the x-axis at x1, the tangent at x1 meets it at x2, and the points close in on the root. On the right, the curve y = x cubed - 2x + 2 with the tangent at x = 0 meeting the axis at x = 1 and the tangent at x = 1 meeting it at x = 0, a cycle that never reaches the root near -1.77.
Square roots the old way
For f(x)=x2−N, Newton's formula becomes
xn+1=21(xn+xnN).
Take the average of a guess and N divided by the guess. This rule was used in Babylon and described by Heron of Alexandria, long before Newton.
Why it is so fast
Expand f about xn, with en=xn−α and f(α)=0. A careful calculation gives
en+1≈2f′(α)f′′(α)en2.
The new error is about a constant times the square of the old one. This is quadratic convergence. An error of 10−3 becomes about 10−6, then 10−12. Compare fixed-point iteration, where the error is only multiplied by a fixed factor.
When Newton's method fails
A flat tangent. If f′(xn)=0, the tangent never meets the axis. A start near a turning point sends x1 far away.
A cycle. For f(x)=x3−2x+2 and x0=0: x1=1 and x2=0, for ever.
Running away. For f(x)=3x, the formula gives xn+1=−2xn. Each step doubles the distance from the root 0.
The wrong root. A start between two roots may land near a third one.
A repeated root. At a double root, f′(α)=0 too. The method still converges, but only linearly, with the error about halving at each step.
So a good method is to locate the root by a change of sign first, then let Newton finish the job.
Practise with new numbers
Try it in Python
The program runs Newton's method and prints the error at each step, so you can watch the correct digits double.
Common mistakes
Dividing by f instead of f′. The step is f′(xn)f(xn).
Degrees on a calculator. Trigonometric functions in calculus use radians.
Trusting a far start. Newton's method is fast near a root, not from anywhere.
Expecting quadratic speed at a double root. There it is only linear.
Remember these
xn+1=xn−f′(xn)f(xn).
Square root: xn+1=21(xn+xnN).
Near a simple root, en+1≈Cen2: digits double.
It fails at flat tangents, in cycles and for 3x, and slows at repeated roots.
Check yourself
Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.
Use one Newton step on x2−5=0 from x0=2.
Write Newton's formula for f(x)=x−cosx.
Why can Newton's method not start at x0=0 for f(x)=x2−4?
An error of 2×10−3 at one Newton step becomes about 3×10−6 at the next. Roughly what will the error be after one more step?
. Near a simple root it is very fast: the number of correct digits roughly doubles at each step. It can fail: a flat tangent, a start that sends it to another root or round a cycle, a function like
3x
, and a repeated root, where it slows to linear speed.
about 25 min1 simulation2 practice questions1 Python program5 questions to check yourself
This lesson is free to read. Enroll free to keep your place and take the tests.