9 A quadratic equation is of the form *y* = *ax*^{2} + *bx* + *c*.

What if your data is actually more complex than a simple straight line? Surprisingly,

you can actually use a linear model to fit nonlinear data. A simple way to do this is to

add powers of each feature as new features, then train a linear model on this extended

set of features. This technique is called *Polynomial Regression*.

Let’s look at an example. First, let’s generate some nonlinear data, based on a simple

*quadratic equation*^{9} (plus some noise; see Figure 4-12):

`m`` ``=`` ``100`

`X`` ``=`` ``6`` ``*`` ``np``.``random``.``rand``(``m``, ``1``) ``-`` ``3`

`y`` ``=`` ``0.5`` ``*`` ``X``**``2`` ``+`` ``X`` ``+`` ``2`` ``+`` ``np``.``random``.``randn``(``m``, ``1``)`

*Figure 4-12. Generated nonlinear and noisy dataset*

`nomialFeatures` class to transform our training data, adding the square (2^{nd}-degree

polynomial) of each feature in the training set as new features (in this case there is

just one feature):

**>>> ****from**` `**sklearn.preprocessing**` `**import**` ``PolynomialFeatures`

**>>> **`poly_features`` ``=`` ``PolynomialFeatures``(``degree``=``2``, ``include_bias``=``False``)`

**>>> **`X_poly`` ``=`` ``poly_features``.``fit_transform``(``X``)`

**>>> **`X``[``0``]`

`array([-0.75275929])`

**>>> **`X_poly``[``0``]`

`array([-0.75275929, 0.56664654])`

`X_poly` now contains the original feature of `X` plus the square of this feature. Now you

can fit a `LinearRegression` model to this extended training data (Figure 4-13):

**>>> **`lin_reg`` ``=`` ``LinearRegression``()`

**>>> **`lin_reg``.``fit``(``X_poly``, ``y``)`

**>>> **`lin_reg``.``intercept_``, ``lin_reg``.``coef_`

`(array([ 1.78134581]), array([[ 0.93366893, 0.56456263]]))`

*Figure 4-13. Polynomial Regression model predictions*

Not bad: the model estimates *y* = 0 . 56*x*1

2 + 0 . 93*x*1 + 1 . 78 when in fact the original

function was *y* = 0 . 5*x*1

2 + 1 . 0*x*1 + 2 . 0 + Gaussian noise.

Note that when there are multiple features, Polynomial Regression is capable of find‐

ing relationships between features (which is something a plain Linear Regression

model cannot do). This is made possible by the fact that `PolynomialFeatures` also

adds all combinations of features up to the given degree. For example, if there were

features *a*^{2}, *a*^{3}, *b*^{2}, and *b*^{3}, but also the combinations *ab*, *a*^{2}*b*, and *ab*^{2}.

`PolynomialFeatures(degree=d)` transforms an array containing *n*

features into an array containing ^{n}^{ +}^{ d}^{ !}

*d*!* n*! ^{ features, where }^{n}^{! is the}

*factorial* of *n*, equal to 1 × 2 × 3 × ⋯ × *n*. Beware of the combinato‐

rial explosion of the number of features!

A Completed Introduction to Neural Networks for BeginnersAn introduction to neural networks We can describe a neural network as a mathematical model for information processing. As discussed in Chapter 1, Machine Learning – an Introduction, this is a good way to describe any ML algorithm, but, in this chapter, well give it a specific meaning in the context of neural networks. A […]