Noteworthy Uses:: *[[Noteworthy Uses]]* Proved By:: [[derivative of the dot product]] Intuition:: *[[Intuition]]* ----- > [!proposition] Proposition. ([[derivative of a quadratic form]]) > The [[derivative]] of the function $f: \rr ^{d} \to \rr$ given by $f(x)=x^{\top} Ax + b^{\top}x + c$ is $D(x^{\top}Ax)+ D(b^{\top}x)+ D(c),$ where $D(x^{\top}Ax)=x^{\top}(A + A^{\top})$, $D(b^{\top}x)=b^{\top}$, and $D(c)=0$. > > **Remark.** If $A$ is a [[symmetric matrix]], part of the result simplifies to $D(x^{\top}Ax)=2x^{\top}A$. > [!proposition] Corollary. (Gradient of a quadratic form) > The [[gradient|gradient]] of the above [[quadratic form]] is $\nabla(x^{\top}Ax+b^{\top}x+c)=(A+A^{\top})x+b + 0.$ > [!proof]- Proof. ([[derivative of a quadratic form]]) > The computation is straightforward but a little messy. Using linearity of the [[derivative]] we will compute $D(x^{\top}Ax)$, $D(b^{\top}x)$, and $D(c)$ one at a time and sum the result. Recalling the relationship between the [[dot product]] and [[matrix transpose]] and specifically the [[derivative of the dot product]], we have $\begin{align}D(x^{\top}Ax) = & D(x \cdot Ax) \\= & x^{\top}D(Ax) + (Ax)^{\top} D(Ix) \\ = & x^{\top}A + x^{\top}{ A^{\top}}I \\ = & x^{\top}(A+A^{\top}); \end{align}$ this is the claimed form of $D(x^{\top}Ax)$. \ Next consider $D(b^{\top}x)$. This is just a constant [[matrix]] times $x$ so the answer is $b^{\top}$. \ Finally, it is obvious that $D(c)=0$. ----- #### ---- #### References > [!backlink] > ```dataview > TABLE rows.file.link as "Further Reading" > FROM [[]] > FLATTEN file.tags as Tag > WHERE Tag = "#definition" OR Tag = "#theorem" OR Tag = "#MOC" OR Tag = "#proposition" OR Tag = "#axiom" > GROUP BY Tag > ``` > [!frontlink] > ```dataview > TABLE rows.file.link as "Further Reading" > FROM outgoing([[]]) > FLATTEN file.tags as Tag > WHERE Tag = "#definition" OR Tag = "#theorem" OR Tag = "#MOC" OR Tag = "#proposition" OR Tag = "#axiom" > GROUP BY Tag > ```