Convolution

Convolution combines two functions into a third. It answers a question that comes up all over engineering: if a system responds to a single kick in a known way, what does it do when kicked continuously by some input?

The definition looks heavier than the idea:

Read it as a recipe. Fix a time $t$. Flip $g$ backwards and slide it so it sits at $t$. Multiply it point by point with $f$. Add up the area of the result. That single number is the output at time $t$, and repeating this for every $t$ draws the whole output.

That is what the plot shows. The top panel has $f$ standing still, the flipped $g$ sliding across it, and their product shaded. The bottom panel draws that area as it changes.

Try this: press Play on two equal rectangles. The overlap grows steadily, then shrinks, so the output is a triangle.

The flip is the part that puzzles most people. It exists because of the direction of time.

Think of $g$ as a memory of the past. The value of $g$ at an age of 2 seconds says how much something that happened 2 seconds ago still counts now. To work out the output now, you look back along the input. The most recent input is matched with age zero, and older input with larger ages. Laid out against the input, that memory runs backwards.

Try this: with a slow decay as g, notice that the tall edge of the flipped decay leads, on the side of the newest input. Drag along the plot to move it by hand.

An impulse is a very short kick with an area of 1. Convolving anything with an impulse gives the same thing back, as if nothing had happened.

Now suppose a system is linear, so double the input gives double the output, and time invariant, so it behaves the same today as yesterday. Any input can be thought of as a long row of tiny impulses, each scaled by the input at that moment. Each impulse sets off its own copy of the system’s impulse response $h$, and the output is the sum of all those copies:

This is the reason convolution matters. One curve, the impulse response, describes everything a linear system will ever do with any input.

Try this: an impulse as g returns $f$ exactly. Then change $g$’s shape and watch the output become a smeared version of $f$.

Once you see convolution, you see it everywhere.

A low-pass filter in electronics is a resistor and capacitor, whose impulse response is a decaying exponential. Its output is a running average that weights recent input most. Try a rectangle through an RC filter: the sharp edges become smooth curves.

Blur in an image is convolution with a small Gaussian: each pixel becomes a weighted average of its neighbours. Try blurring a sharp step. The edge spreads out, which is exactly what an out-of-focus edge looks like.

Reverb in audio is convolution of a dry recording with the impulse response of a room, which you can capture by firing a starting pistol in a cathedral.

Probability uses it too. The distribution of the sum of two independent random numbers is the convolution of their distributions. This is why two Gaussians give a wider Gaussian, and why adding many random things tends towards one.

The order does not matter, so $f\ast g=g\ast f$. Sliding $f$ across $g$ gives the same answer, as you can check by swapping the shapes.

The areas multiply. The area under the output equals the area of $f$ times the area of $g$, which is why the tool shows all three.

Widths add, roughly. The output of two rectangles lasts as long as both put together. Convolution almost always spreads things out, and very rarely sharpens them.

Convolving with a step integrates. The output at any moment is the area of $f$ so far, as this step shows.

Convolution in time becomes plain multiplication in frequency:

This is the convolution theorem, and it is one of the most useful results in engineering. It explains why a filter is described by a frequency response: each frequency in the input is multiplied by the filter’s gain at that frequency. A rectangle in time has a spectrum with a wide spread of frequencies, and the RC filter turns down the high ones, which is the smoothing you saw.

It also makes convolution fast. Done directly, it takes a number of operations that grows with the square of the signal length. Using the fast Fourier transform, you transform both, multiply, and transform back, which is why long reverbs can run in real time.

Choose “Your formula” and write a function of t. You can use + - * / ^, brackets, the constants pi and e, and sin cos tan exp log sqrt abs sign min max. Two extras are useful for building pulses: step(t) is 0 before time zero and 1 after, and rect(t) is 1 between -0.5 and 0.5.

Try this: a damped oscillation through a moving average, or a pair of pulses, like two samples blurred together.

Everything here is continuous in name but computed on a grid of 0.02 s, as a sum. Edges are placed to the nearest grid point, so sharp results are accurate to about one step.

The window runs from -6 to 6 s. Both functions are treated as zero outside it: shapes such as a step are cut off where the window ends, and a formula counts only inside it. A long decay can lose a very small amount of its tail for that reason.

An impulse is drawn as a single tall spike with the area you set, and the plot is scaled to that area, not to the height of the spike. Real signals are usually discrete, where the integral becomes a sum, but the picture of flip, slide and add is the same.

The test suite checks this tool against known results: two rectangles give a triangle, Gaussian variances add, the order does not change the answer, areas multiply, and an impulse leaves a function unchanged.

• A. V. Oppenheim and A. S. Willsky, Signals and Systems, chapters on linear time-invariant systems
• R. N. Bracewell, The Fourier Transform and Its Applications