1.4 Numerical Round-Off

Given that a basic algorithm has failed us, it is reasonable to do a survey of our basic tools. We are most interested in finding a graphing algorithm which we may program a modern computer to perform. One immediate concern is the representation such machines use for real numbers. The representation often used is analagous to scientific notation, keeping a fixed number of digits for any given quantity. This can lead to further difficulties.

Consider graphing the equation

by sampling n(x), limiting ourselves to three digits of precision. A transcription of the computations performed, while sampling n(x) at x = -1,0,1,2, and 3 follows:

It is clear that for all x, our computations result in or , due to numerical round-off. It is equally clear that

so that .

Most calculations introduce some numerical round-off. With complicated equations, there will be long sequences of calculations, which allows numerical round-off to accumulate. For such equations, the generated graph may differ significantly from the actual graph.

Jeff Tupper

March 1996