Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Lecture 14 Notes - Power Spectral Analysis and Time Frequency Analysis

Moving Average

Examples of spectra of white noise, moving average, and second-order autoregressive process

For the moving average, let’s consider a causal moving average (remember, this is the version where we only consider values from the past in our moving average):

xt=wt+θwt−1x_t = w_t + \theta w_{t-1}
(1)

The autocovariance function for this is:

γ(h)={(1+θ2)σ2h=0θσ2∣h∣=10∣h∣>1\gamma(h) = \begin{cases} (1+\theta^2)\sigma^2 & h=0\\ \theta\sigma^2 & |h|=1\\ 0 & |h| > 1 \end{cases}
(2)

The spectral density is therefore:

f(ω)=(1+θ2)σ2+θσ2(e−2πiω+e2πiω)=σ2(1+θ2+2θcos⁡(2πω))\begin{aligned} f(\omega) &= (1+\theta^2)\sigma^2 + \theta\sigma^2(e^{-2\pi i \omega}+e^{2\pi i \omega}) \\ &= \sigma^2(1+\theta^2+2\theta\cos(2\pi\omega)) \end{aligned}
(3)

in the second line, we used cos⁡(θ)=(eiθ+e−iθ)/2\cos(\theta) = (e^{i\theta}+e^{-i\theta})/2 from Euler’s formula eiθ=cos⁡(θ)+isin⁡(θ)e^{i\theta}=\cos(\theta)+i\sin(\theta).

What results here is that the MA process has a spectral density that decays from zero, with larger θ\theta corresponding to a steeper decay from ω=0\omega=0 to ω=1/2\omega=1/2.

An overview of the DFT and periodogram

An intuitive relationship between the DFT and the periodogram is that the DFT gives you a complex value that incorporates both the amplitude and phase of each frequency present in the signal, whereas the periodogram represents only the real-valued power at that frequency, removing the phase information. Let’s try this with a tiny example:

Suppose we have x1=2x_1=2, x2=3x_2=3, x3=1x_3=1, and x4=4x_4=4. We can use these to see how the periodogram is derived from squaring the DFT.

The DFT at frequency j/nj/n is:

d(j/n)=1n∑t=1nxte−i2πtj/nd(j/n) = \frac{1}{\sqrt{n}}\sum_{t=1}^n x_t e^{-i2\pi t j/n}
(4)

The periodogram is the rescaled squared modulus:

P(j/n)=(4/n)∣d(j/n)∣2P(j/n) = (4/n) |d(j/n)|^2
(5)

Since we have n=4n=4 time points, the Fourier frequencies are j/n=0/4,1/4,2/4,3/4j/n = 0/4, 1/4, 2/4, 3/4. Since by symmetry P(j/n)=P(1−j/n)P(j/n) = P(1-j/n) we only need j=0,1,2j=0,1,2.

We can now look at what the sine and cosine waves look like at each frequency.

Cosine and sine basis functions

Now let’s compute the DFT for each jj:

For j=0j=0:

d(0)=14∑t=14xte−i2πt0=14∑t=14xt=12(2+3+1+4)=5\begin{aligned} d(0) &= \frac{1}{\sqrt{4}}\sum_{t=1}^4 x_t e^{-i2\pi t 0} \\ &= \frac{1}{\sqrt{4}}\sum_{t=1}^4 x_t\\ &= \frac{1}{2}(2+3+1+4) = 5\\ \end{aligned}
(6)

For j=1j=1:

d(1/4)=14∑t=14xte−i2πt1/4d(1/4) = \frac{1}{\sqrt{4}}\sum_{t=1}^4 x_t e^{-i2\pi t 1/4}
(7)
te−iπt/2e^{-i\pi t/2}cos⁡(πt/2)\cos(\pi t/2)sin⁡(πt/2)\sin(\pi t/2)
1−i-i0-1
2-1-10
3ii01
4110
d(1/4)=12(2(−i)+3(−1)+1(i)+4(1))=12(1−i)=0.5−0.5id(1/4) = \frac{1}{2}(2(-i)+3(-1)+1(i)+4(1)) = \frac{1}{2}(1-i) = 0.5 - 0.5i
(8)

For j=2j=2:

d(2/4)=14∑t=14xte−i2πt2/4=12∑t=14xte−iπt=12∑t=14xt(−1)t=12(2(−1)+3(1)+1(−1)+4(1))=12(4)=2\begin{aligned} d(2/4) &= \frac{1}{\sqrt{4}}\sum_{t=1}^4 x_t e^{-i2\pi t 2/4}\\ &= \frac{1}{2}\sum_{t=1}^4 x_t e^{-i\pi t}\\ &= \frac{1}{2}\sum_{t=1}^4 x_t (-1)^t\\ &= \frac{1}{2}(2(-1)+3(1)+1(-1)+4(1))\\ &= \frac{1}{2}(4) = 2 \end{aligned}
(9)

Now to go to the periodogram from the DFT, we can do:

P(j/n)=(4/n)∣d(j/n)∣2P(0/4)=(4/4)∣5∣2=25P(1/4)=(4/4)∣0.5−0.5i∣2=0.5P(2/4)=(4/4)∣2∣2=4P(3/4)=P(1/4)\begin{aligned} P(j/n) &= (4/n) |d(j/n)|^2\\ P(0/4) &= (4/4) | 5 | ^2 = 25\\ P(1/4) &= (4/4) | 0.5 - 0.5i|^2 = 0.5\\ P(2/4) &= (4/4) | 2 | ^2 = 4\\ P(3/4) &= P(1/4)\\ \end{aligned}
(10)

The way to interpret this is that the DFT is a complex number that encodes both amplitude and phase at each frequency.

Periodogram demo

Read Chapter 4.4 for more info on periodogram smoothing.

Week 8
Lecture 14 - Power Spectral Analysis and Time Frequency Analysis
Week 8
Lecture 14