Htykuut

I was just wondering what the differences between gradient and rate of change are. Are they the same thing?

The instantaneous rate of change of a real value function $f:mathbb{R}rightarrowmathbb{R}$ is given by the derivative:
$$frac{df}{dx}(x_0) = lim_{hrightarrow 0}frac{f(x_0 + h) - f(x_0)}{h}$$

From this we find that:

$$f(x_0 + h) approx f(x_0) + frac{df}{dx}(x_0)h ;;(1)$$

Where (1) is a linear function of the form $f(h)= a + sh$.

For a multi-variable function, we can calculate a rate of change as well:

Let $f:mathbb{R}^nrightarrowmathbb{R}$ be multivariable function and, $mathbf{g}:tinmathbb{R}rightarrowmathbb{R}^n$ a line in the n-dimensional space.

$$mathbf{g}(t) = mathbf{x}_0 + mathbf{h}t$$

Then we can take the derivative using one-dimensional calculus, by composing $f$ with $mathbf{g}$:

$$frac{df circ mathbf{g}}{dt}(0) = lim_{trightarrow 0}frac{f(mathbf{x}_0 + mathbf{h}t) - f(mathbf{x}_0)}{t};;(2)$$
Where (2) is the instantaneous rate of change of $f$ in the direction of $mathbf{h}$.

Now we can postulate that we have a linear approximation of $f(mathbf{x}_0 + mathbf{h})$ ( a multi-variable linear function ):

$$f(mathbf{x}_0 + mathbf{h})approx f(mathbf{x}_0) + sum_i s_i h_i = f(mathbf{x}_0) + left<mathbf{s}, mathbf{h}right>$$

Where $mathbf{s}$ will be the generalized gradient and $<.,.>$ is the inner product in $mathbb{R}^n$ (a.k.a the dot product).

To find $mathbf{s}$, just do the following, set $mathbf{h} = hmathbf{e}_i$, where $mathbf{e}_i$ is a vector of the standard $mathbb{R}^n$ basis.

$$f(mathbf{x}_0 + mathbf{h})approx f(mathbf{x}_0) + hleft <mathbf{s}, mathbf{e}_i right > = f(mathbf{x}_0) + hs_i$$

$$lim_{h rightarrow 0} frac{f(mathbf{x}_0 + hmathbf{e}_i) - f(mathbf{x}_0)}{h} = s_i$$

Basically the left side of the equation is a partial derivative of $f$, hence:

$$s_i = frac{partial f(mathbf{x}_0)}{partial x_i};;(3)$$
$$mathbf{s} = nabla f(mathbf{x}_0)$$

Finally taking (2) and (3), we find that the instantaneous rate of change of $f$ in the $mathbf{h}$ direction is:

$$frac{df circ mathbf{g}}{dt}(0) = lim_{trightarrow 0}frac{f(mathbf{x}_0 + mathbf{h}t) - f(mathbf{x}_0)}{t} = lim_{trightarrow 0}frac{f(mathbf{x}_0) + left<nabla f(mathbf{x}_0), mathbf{h}right> t - f(mathbf{x}_0)}{t} $$

$$frac{df circ mathbf{g}}{dt}(0) = left<nabla f(mathbf{x}_0), mathbf{h}right>$$

Conclusion

In order to compute the directional derivative or directional rate of change you need the gradient. That said the gradient can be interpreted as a multi-variable rate of change in the sense that it is a vector with the rates of changes for each coordinate.