Skip to content
Snippets Groups Projects
Commit 059fe20b authored by Erik Strand's avatar Erik Strand
Browse files

Add answer for 4.4

parent b33252ff
Branches
No related tags found
No related merge requests found
......@@ -335,6 +335,35 @@ rates.
{:.question}
Calculate the differential entropy of a Gaussian process.
Since we're integrating the Gaussian over the whole real line, translation is irrelevant. So without
loss of generality I'll calculate the differential entropy of a Gaussian with zero mean.
$$
\begin{align*}
H(N(\mu, \sigma^2))
&= -\int_{-\infty}^\infty \frac{1}{\sqrt{2 \pi \sigma^2}} e^{\frac{-x^2}{2 \sigma^2}} \log \left(
\frac{1}{\sqrt{2 \pi \sigma^2}} e^{\frac{-x^2}{2 \sigma^2}} \right) \mathrm{d} x \\
&= -\int_{-\infty}^\infty \frac{1}{\sqrt{2 \pi \sigma^2}} e^{\frac{-x^2}{2 \sigma^2}} \left(
-\frac{1}{2} \log(2 \pi \sigma^2) - \frac{x^2}{2 \sigma^2} \right) \mathrm{d} x \\
&= \frac{1}{2} \log(2 \pi \sigma^2) \int_{-\infty}^\infty \frac{1}{\sqrt{2 \pi \sigma^2}}
e^{\frac{-x^2}{2 \sigma^2}} \mathrm{d} x + \int_{-\infty}^\infty \frac{1}{\sqrt{2 \pi \sigma^2}}
\frac{x^2}{2 \sigma^2} e^{\frac{-x^2}{2 \sigma^2}} \mathrm{d} x \\
&= \frac{1}{2} \log(2 \pi \sigma^2) + \frac{1}{2} \int_{-\infty}^\infty
\left( \frac{x}{\sqrt{2 \pi \sigma^2}} \right) \left( \frac{x}{\sigma^2}
e^{\frac{-x^2}{2 \sigma^2}} \right) \mathrm{d} x \\
&= \frac{1}{2} \log(2 \pi \sigma^2) + \frac{1}{2} \int_{-\infty}^\infty
\frac{1}{\sqrt{2 \pi \sigma^2}} e^{\frac{-x^2}{2 \sigma^2}} \mathrm{d} x \\
&= \frac{1}{2} \log(2 \pi \sigma^2) + \frac{1}{2} \\
&= \frac{1}{2} \log(2 \pi e \sigma^2)
\end{align*}
$$
The second term was integrated by parts. Note that
$$
\frac{d}{dx} e^{\frac{-x^2}{2 \sigma^2}} = -\frac{x}{\sigma^2} e^{\frac{-x^2}{2 \sigma^2}}
$$
## (4.5)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment