Multivariable calculus

Second partials of ln(e^x + e^y)

Q: Why is the discriminant of ln(e^x + e^y) always zero?

Because f(x+c,y+c) = f(x,y) + c, the function is linear along the diagonal, so the Hessian has a zero eigenvalue in that direction. Concretely all three second partials share the magnitude e^{x}e^{y}/S^{2}, and f_{xx}f_{yy} cancels f_{xy}^2.

Q: How large can f_xx get?

At most 1/4. Since f_{xx} = f_x(1-f_x) with f_x = e^{x}/(e^{x}+e^{y}) strictly between 0 and 1, the product is largest when f_x = 1/2, which happens exactly on the line x = y.

For f(x,y) = ln(e^x + e^y) write S = e^x + e^y. Then f_xx = f_yy = e^x e^y / S^2 and the mixed partial f_xy = f_yx = -e^x e^y / S^2. Every entry of the Hessian has the same magnitude, so the discriminant f_xx f_yy - f_xy^2 is exactly zero at every point and the function is convex but flat along one direction.

f(x,y) = \ln\bigl(e^{x} + e^{y}\bigr)

\frac{\partial f}{\partial x} = \frac{e^{x}}{e^{x} + e^{y}}

\frac{\partial f}{\partial y} = \frac{e^{y}}{e^{x} + e^{y}}

Second and mixed partials

\frac{\partial^2 f}{\partial x^2} = \frac{e^{x}e^{y}}{\left(e^{x} + e^{y}\right)^{2}}

\frac{\partial^2 f}{\partial y\,\partial x} = \frac{-e^{x}e^{y}}{\left(e^{x} + e^{y}\right)^{2}}

\frac{\partial^2 f}{\partial y^2} = \frac{e^{x}e^{y}}{\left(e^{x} + e^{y}\right)^{2}}

Differentiating the log-sum-exp

Write $S = e^{x} + e^{y}$ , which is positive for all real inputs, so the logarithm is defined everywhere. The chain rule gives inner over outer, and only one term of $S$ survives each partial derivative.

\frac{\partial f}{\partial x} = \frac{e^{x}}{S}, \qquad \frac{\partial f}{\partial y} = \frac{e^{y}}{S}

Those two first partials are the softmax weights: each is between $0$ and $1$ , and they add to exactly $1$ . Differentiate $e^{x}/S$ in $x$ with the quotient rule, remembering that $S$ contains $e^{x}$ .

\frac{\partial^2 f}{\partial x^2} = \frac{e^{x}S - e^{x}e^{x}}{S^{2}} = \frac{e^{x}\left(S - e^{x}\right)}{S^{2}} = \frac{e^{x}e^{y}}{S^{2}}

For the mixed partial the numerator $e^{x}$ is constant in $y$ , so only the denominator contributes, and it contributes with a minus sign.

\frac{\partial^2 f}{\partial y\,\partial x} = -\frac{e^{x}\cdot e^{y}}{S^{2}}

The mistake: treating the denominator as a constant

The tempting wrong move is to say that since $f_x = e^{x}/S$ looks like an exponential, differentiating again just returns $e^{x}/S$ . That ignores the $e^{x}$ hiding inside $S$ . The wrong answer climbs toward $1$ as $x$ pulls ahead of $y$ , whereas the true curvature decays to $0$ out there and never exceeds $1/4$ anywhere.

A useful check is the identity $f_{xx} = f_x(1 - f_x)$ , which follows because $f_x = e^x/S$ and $1 - f_x = e^y/S$ . Since $f_x$ lies strictly between $0$ and $1$ , the curvature is always positive and peaks at $1/4$ on the line $x = y$ .

The other slip is the sign of the mixed partial. It has to be negative: raising $y$ increases $S$ , which shrinks the weight $e^{x}/S$ , so the slope in the $x$ direction goes down.

A Hessian with determinant zero

All three second partials have the same magnitude $e^{x}e^{y}/S^{2}$ , with a minus sign only on the mixed one, so the discriminant collapses.

D = f_{xx}f_{yy} - f_{xy}^{\,2} = \left(\frac{e^{x}e^{y}}{S^{2}}\right)^{2} - \left(\frac{e^{x}e^{y}}{S^{2}}\right)^{2} = 0

The reason is a symmetry you can see without any calculus. Adding the same constant $c$ to both inputs multiplies $S$ by $e^{c}$ , so $f(x+c, y+c) = f(x,y) + c$ . The function is exactly linear along the diagonal direction, which means zero curvature there and forces a zero eigenvalue in the Hessian.

The other eigenvalue is $2e^{x}e^{y}/S^{2}$ , which is positive, so the surface curves upward across the diagonal. A Hessian that is positive semidefinite everywhere makes this function convex, which is why log-sum-exp is a standard smooth stand-in for the maximum of $x$ and $y$ .

Frequently asked questions

Why is the discriminant of ln(e^x + e^y) always zero?

Because $f(x+c,y+c) = f(x,y) + c$ , the function is linear along the diagonal, so the Hessian has a zero eigenvalue in that direction. Concretely all three second partials share the magnitude $e^{x}e^{y}/S^{2}$ , and $f_{xx}f_{yy}$ cancels $f_{xy}^2$ .

How large can f_xx get?

At most $1/4$ . Since $f_{xx} = f_x(1-f_x)$ with $f_x = e^{x}/(e^{x}+e^{y})$ strictly between $0$ and $1$ , the product is largest when $f_x = 1/2$ , which happens exactly on the line $x = y$ .