Least-Squares Polynomials

• Because polynomials can be readily manipulated, fitting such functions to data that do not plot linearly is common.
• It will turn out that the normal equations are linear for this situation, which is an added advantage.
• $n$ as the degree of the polynomial and N as the number of data pairs. If $N = n + 1$, the polynomial passes exactly through each point and the methods discussed earlier apply, so we will always have $N > n+ l$ in the following. We assume the functional relationship
  

  $$y=a_0+a_1x+a_2x^2+\ldots+a_nx^n$$ (5)

  
  
  with errors defined by
  
  $$e_i=Y_i - y_i=Y_i- a_0-a_1x-a_2x^2-\ldots-a_nx^n$$
  
  
  We again use $Y_i$ to represent the observed or experimental value corresponding to $x_i$, with $x_i$ free of error. We minimize the sum of squares;
  
  $$S=\sum_{i=1}^N e_i^2=\sum_{i=1}^N (Y_i - a_0 - a_1x - a_2x^2 - \ldots - a_nx^n)^2$$
  
  
  At the minimum, all the partial derivatives $\partial S/\partial a_0,\partial S/\partial a_n$ vanish. Writing the equations for these gives $n + 1$ equations:
  
  $$\begin{array}{l} \frac{\partial S}{\partial a_0}=0=\sum_{i=1... ..._i- a_0-a_1x_i-a_2x_i^2-\ldots-a_ix_i^n)(-x_i^n)\\ \end{array}$$
  
  
  Dividing each by $-2$ and rearranging gives the $n + 1$ normal equations to be solved simultaneously:
  

  $$\begin{array}{rl} a_0N+a_1\sum x_i+a_2 \sum x_i^2+\ldots+a_n... ...i^{n+2}+\ldots+a_n\sum x_i^{2n}= & \sum x_i^nY_i\\ \end{array}$$ (6)

  
  
  Putting these equations in matrix form shows the coefficient matrix;
  

  $$\left[ \begin{array}{rrrrrl} N & \sum x_i & \sum x_i^2 & \sum x_i... ...\sum x_iY_i\\ \sum x_i^2Y_i\\ \vdots \\ \sum x_i^n Y_i\\ \end{array}\right]$$ (7)

  
  
  All the summatins in Eqs. 6 and 7 run from 1 to $N$. We will let B stand for the coefficient matrix.
• Equation 7 represents a linear system. However, you need to know that this system is ill-conditioned and round-off errors can distort the solution: the $a$'s of Eq. 5. Up to degree-3 or -4, the problem is not too great. Special methods that use orthogonal polynomials are a remedy. Degrees higher than 4 are used very infrequently. It is often better to fit a series of lower-degree polynomials to subsets of the data.
• Matrix $B$ of Eq. 7 is called the normal matrix for the least-squares problem. There is another matrix that corresponds to this, called the design matrix. It is of the form;
  
  $$A=\left[ \begin{array}{rrrrr} 1 & 1 & 1 & 1 & 1 \\ x_1 & ... ... x_1^n & x_2^n & x_3^n & \ldots & x_N^n\\ \end{array} \right]$$
  
  
  $AA^T$ is just the coefficient matrix of Eq. 7. It is easy to see that $Ay$, where $y$ is the column vector of $Y$-values, gives the right-hand side of Eq. 7. We can rewrite Eq. 7 in matrix form, as
  
  $$AA^Ta = Ba= Ay$$
  
  

• It is illustrated the use of Eqs. 6 to fit a quadratic to the data of Table 1. Figure 7 shows a plot of the data. The data are actually a perturbation of the relation $y=1 -x+0.2 x^2$.
  

  Table 1: Data to illustrate curve fitting.

  
  
  

  Figure 7: Figure for the data to illustrate curve fitting.

  
  To set up the normal equations, we need the sums tabulated in Table 1. The equations to be solved are:
  
  $$\begin{array}{rl} 11a_0+6.01a_1+4.6545a_2& =5.905 \\ 6.01... ...839 \\ 4.6545a_0+4.1150a_1+3.9161a_2& =1.3357 \\ \end{array}$$
  
  
  The result is $a_0= 0.998$, $a_2=-1.018$,$a_3 = 0.225$, so the least- squares method gives
  
  $$y = 0.998 - 1.018x + 0.225x^2$$
  
  
  which we compare to $y=1 -x+0.2 x^2$. Errors in the data cause the equations to differ.