Chapter 30 — Exercises

38 problems on the multivariable chain rule, the gradient, directional derivatives, tangent planes, and gradient descent. ⭐ to ⭐⭐⭐⭐.

Work these by hand unless a problem explicitly invites Python. Keep a unit vector unit; keep your tree diagrams honest. Answers to odd-numbered problems are in appendices/answers-to-selected.md.

Difficulty tiers: ⭐ routine drill · ⭐⭐ standard · ⭐⭐⭐ challenging · ⭐⭐⭐⭐ deep / multi-step.

(Exact tier shown next to each problem below.)

Part A — The Multivariable Chain Rule (§30.2)

1. ⭐ Let $z = x^2 + y$ with $x = t$, $y = t^3$. Use the Case 1 chain rule to find $\dfrac{dz}{dt}$, then check by substituting first.

2. ⭐ Let $z = xy$ with $x = e^t$, $y = e^{-t}$. Find $\dfrac{dz}{dt}$. What does your answer say about $z$ along this path?

3. ⭐ For $z = \sin(xy)$ with $x = t^2$, $y = t$, write $\dfrac{dz}{dt}$ using the chain rule (you may leave it in terms of $x, y, t$).

4. ⭐ Let $w = x^2 + y^2 + z^2$ with $x = t$, $y = 2t$, $z = 3t$. Find $\dfrac{dw}{dt}$ at $t = 1$.

5. ⭐⭐ Let $z = x^2 - y^2$ with $x = s + t$, $y = s - t$. Find $\dfrac{\partial z}{\partial s}$ and $\dfrac{\partial z}{\partial t}$ by the Case 2 chain rule, then verify by substitution.

6. ⭐⭐ Let $z = \ln(x^2 + y^2)$ with $x = s\cos t$, $y = s\sin t$. Find $\dfrac{\partial z}{\partial s}$ and $\dfrac{\partial z}{\partial t}$. Simplify using $x^2 + y^2 = s^2$.

7. ⭐ Draw the tree diagram for $w = f(x, y)$ where $x = x(u, v)$ and $y = y(u, v)$. Label every edge with the correct partial derivative and read off $\dfrac{\partial w}{\partial u}$.

8. ⭐ Draw the tree diagram for $w = f(x, y, z)$ where each of $x, y, z$ depends on a single variable $t$. Read off $\dfrac{dw}{dt}$.

9. ⭐⭐ Using the implicit chain-rule formula $\dfrac{dy}{dx} = -\dfrac{F_x}{F_y}$ (Case 3), find $\dfrac{dy}{dx}$ for $x^2 + xy + y^2 = 7$ at a general point.

10. ⭐⭐ A particle moves so that $x(t) = 1 + 2t$, $y(t) = 3 - t$. The temperature field is $T(x, y) = x^2 y$. At $t = 0$ (so $(x, y) = (1, 3)$), is the particle warming or cooling, and at what rate? Use the chain rule.

11. ⭐⭐ Let $z = f(x, y)$ with $x = r\cos\theta$, $y = r\sin\theta$ (polar). Show that $$\frac{\partial z}{\partial r} = f_x\cos\theta + f_y\sin\theta, \qquad \frac{\partial z}{\partial \theta} = -f_x\, r\sin\theta + f_y\, r\cos\theta.$$

12. ⭐⭐⭐ Continue Problem 11. Show the squared-gradient identity $$\left(\frac{\partial z}{\partial r}\right)^2 + \frac{1}{r^2}\left(\frac{\partial z}{\partial \theta}\right)^2 = f_x^2 + f_y^2 = \|\nabla f\|^2.$$ (This is why $\|\nabla f\|$ is coordinate-independent.)

13. ⭐⭐⭐ Let $F(x, y, z) = 0$ define $z = z(x, y)$ implicitly. Use the chain rule to derive $\dfrac{\partial z}{\partial x} = -\dfrac{F_x}{F_z}$, stating the condition under which the formula is valid.

Part B — The Gradient (§30.3)

14. ⭐ Compute $\nabla f$ for $f(x, y) = 3x^2 - 4xy + y^3$.

15. ⭐ Compute $\nabla f$ for $f(x, y, z) = xyz$ and evaluate it at $(1, 2, 3)$.

16. ⭐ Compute $\nabla f$ for $f(x, y) = e^{x}\cos y$ at $(0, 0)$.

17. ⭐⭐ Compute $\nabla f$ for $f(x, y) = \dfrac{x}{x^2 + y^2}$ at $(1, 1)$.

18. ⭐⭐ For $f(x, y) = \sqrt{x^2 + y^2}$ (distance from the origin), show that $\nabla f = \dfrac{\langle x, y\rangle}{\sqrt{x^2 + y^2}}$ — a unit vector pointing radially outward — wherever $(x, y) \neq (0,0)$.

19. ⭐⭐ Find all points where $\nabla f = \mathbf{0}$ for $f(x, y) = x^2 - 2x + y^2 + 4y$. (These are the candidate critical points studied in Chapter 31.)

Part C — Directional Derivatives (§30.5)

20. ⭐ For $f(x, y) = x^2 + y^2$ at $(3, 4)$, compute $D_{\mathbf{u}} f$ in the direction $\mathbf{u} = \langle 0, 1\rangle$. Confirm it equals $f_y$.

21. ⭐⭐ For $f(x, y) = x^2 y$ at $(2, 1)$, compute the directional derivative in the direction of the vector $\langle 3, 4\rangle$. (Remember to normalize.)

22. ⭐⭐ For $f(x, y) = e^{xy}$ at $(1, 0)$, find $D_{\mathbf{u}} f$ in the direction toward the point $(4, 4)$.

23. ⭐⭐ For $f(x, y, z) = x^2 + y^2 - z$ at $(1, 1, 1)$, find $D_{\mathbf{u}} f$ in the direction $\mathbf{u} = \tfrac{1}{3}\langle 2, 2, -1\rangle$. (Check that $\mathbf{u}$ is a unit vector.)

24. ⭐⭐⭐ At the point $(1, 2)$, a function $f$ has $D_{\langle 1,0\rangle} f = 3$ and $D_{\langle 0,1\rangle} f = -1$. Find $\nabla f(1,2)$, then find the directional derivative in the direction $\mathbf{u} = \tfrac{1}{\sqrt2}\langle 1, 1\rangle$.

25. ⭐⭐⭐ Suppose $D_{\mathbf{u}} f(P) = 6$ when $\mathbf{u} = \langle 1, 0\rangle$ and $D_{\mathbf{v}} f(P) = 10$ when $\mathbf{v} = \tfrac{1}{\sqrt2}\langle 1, 1\rangle$. Recover $\nabla f(P)$.

Part D — Steepest Ascent, Max Rate, and Level Sets (§30.4–30.5)

26. ⭐⭐ For $f(x, y) = x^2 + xy + y^2$ at $(1, 1)$: (a) in what direction does $f$ increase fastest? (b) What is the maximum rate of increase? (c) In what direction does $f$ decrease fastest?

27. ⭐⭐ The temperature on a metal plate is $T(x, y) = 100 - x^2 - 2y^2$. A bug sits at $(2, 1)$. Which way should it crawl to warm up fastest, and how fast does $T$ rise per unit distance in that best direction?

28. ⭐⭐⭐ For $f(x, y) = x^2 - y^2$, find a unit vector $\mathbf{u}$ at the point $(2, 1)$ along which $D_{\mathbf{u}} f = 0$. Interpret your answer geometrically using the level curve through $(2,1)$.

29. ⭐⭐⭐ Show that for $f(x, y) = x^2 + y^2$, the gradient at any point $(a, b) \neq (0,0)$ is perpendicular to the circle (level curve) $x^2 + y^2 = a^2 + b^2$ through that point. (Hint: parametrize the circle and dot with $\nabla f$.)

Part E — Tangent Planes and Normal Lines (§30.6)

30. ⭐⭐ Find the tangent plane to the surface $z = x^2 + y^2$ at the point $(1, 2, 5)$ using the graph formula. (Tangent planes for graphs were introduced in Chapter 29; here the gradient re-derives them.)

31. ⭐⭐ Find the tangent plane and the normal line to the sphere $x^2 + y^2 + z^2 = 9$ at $(2, 1, 2)$.

32. ⭐⭐⭐ Find the tangent plane to the ellipsoid $\dfrac{x^2}{4} + \dfrac{y^2}{9} + z^2 = 3$ at $(2, 3, 1)$. (Check the point lies on the surface first.)

33. ⭐⭐⭐ The surfaces $z = x^2 + y^2$ and $z = 8 - x^2 - y^2$ intersect. At the point $(1, 1, 2)$ (on both), find the angle between their tangent planes by computing the angle between their normal vectors.

Part F — Gradient Descent and Applications (§30.8–30.10)

34. ⭐⭐⭐ For $f(x, y) = x^2 + 4y^2$, start at $(2, 2)$ with learning rate $\eta = 0.1$. Carry out two gradient-descent steps by hand, listing $(x_1, y_1)$ and $(x_2, y_2)$. Compute $f$ at each iterate and confirm it is decreasing.

35. ⭐⭐⭐⭐ (Data science — learning-rate stability.) For $f(x) = ax^2$ (one variable, $a > 0$), gradient descent is $x_{k+1} = x_k - \eta(2a x_k) = (1 - 2a\eta)x_k$. (a) Show the iterates form a geometric sequence with ratio $r = 1 - 2a\eta$. (b) Prove the iteration converges to $0$ iff $|1 - 2a\eta| < 1$, i.e. $0 < \eta < 1/a$. (c) For $f(x,y) = x^2 + 10y^2$, the two coordinates decouple with $a = 1$ and $a = 10$. Find the largest $\eta$ for which both coordinates converge, and explain why the steep $y$-direction is the binding constraint (the "poor conditioning" of §30.8).

36. ⭐⭐⭐⭐ (Data science — linear regression by hand.) Fit $y = wx$ (no intercept) to the two data points $(x, y) = (1, 1)$ and $(2, 3)$ by minimizing the mean-squared-error loss $$L(w) = \tfrac12\big[(w\cdot 1 - 1)^2 + (w\cdot 2 - 3)^2\big].$$ (a) Compute $\dfrac{dL}{dw}$ symbolically. (b) Starting from $w_0 = 0$ with $\eta = 0.1$, perform two gradient-descent steps by hand. (c) Solve $L'(w) = 0$ exactly for the optimal $w^\star$ and confirm your iterates are heading toward it.

37. ⭐⭐⭐⭐ (Physics — force as negative gradient.) A particle moves in the potential $\Phi(x, y) = x^2 + 3y^2$. (a) Compute the force field $\mathbf{F} = -\nabla\Phi$. (b) At $(1, 1)$, show $\mathbf{F}$ points toward lower potential by checking the angle between $\mathbf{F}$ and the displacement from $(1,1)$ to the origin (the potential's minimum). (c) Along which directions from $(1,1)$ does the particle feel no change in potential (i.e. $D_{\mathbf{u}}\Phi = 0$)? Relate this to a level curve of $\Phi$.

38. ⭐⭐⭐⭐ (Synthesis — chain rule meets gradient descent.) A model output is $\hat y = \sigma(wx + b)$ where $\sigma(s) = \dfrac{1}{1 + e^{-s}}$ is the logistic function, and the loss for one data point is $L = \tfrac12(\hat y - y)^2$. (a) Using the single-variable chain rule (Chapter 7) and $\sigma'(s) = \sigma(s)\big(1 - \sigma(s)\big)$, show $$\frac{\partial L}{\partial w} = (\hat y - y)\,\hat y(1 - \hat y)\,x, \qquad \frac{\partial L}{\partial b} = (\hat y - y)\,\hat y(1 - \hat y).$$ (b) Assemble $\nabla_{(w,b)} L$ and write one gradient-descent update for $(w, b)$. (c) Explain in one or two sentences how this hand computation is exactly what backpropagation (the reverse-mode chain rule of §30.2) automates for a network with billions of such links.

Python Exploration (optional)

These invite you to verify hand work, not replace it. Hand-compute the answer first, then confirm.

P1. Use sympy to compute $\nabla f$ for $f = x^2 + 2y^2 + 3z^2$ and confirm the pattern of Problem 15 generalizes.

P2. Implement the one-line gradient-descent loop from §30.8 for $f(x, y) = x^2 + 10y^2$. Sweep $\eta \in \{0.04, 0.09, 0.11\}$ for 30 steps and confirm by experiment the stability boundary $\eta < 0.1$ you proved in Problem 35.

P3. Make a contour plot (plt.contour) of $f(x, y) = x^2 + 4y^2$ and overlay the gradient-descent trajectory from Problem 34. Confirm visually that the steps cross the contours nearly perpendicularly, as Superpower 2 predicts.