You must always avoid the temptation to “connect the dots” (to naively interpolate between samples) when working with discrete signals.¹ Let’s recall the example from the previous section, and see how connecting the dots can get us into trouble.

Imagine someone asks you what the altitude of the plane was at 65 minutes into the flight. How should you respond? You don’t actually have a sample for the altitude at 65 minutes, but you do have measurements for the altitude at 60 minutes and 70 minutes. You might feel tempted to draw a line between these two samples, perform some simple linear interpolation, and infer that the altitude was about 31,000 feet. This sort of temptation is completely natural, but really unhealthy when working with discrete signals. The most appropriate response is simply to say, “I don’t know”. Anything else would be a fib.

Given our measurements and context, we cannot confidently report an altitude for 65 minutes into the flight. Our discrete signal tells us nothing about the altitude of the plane at 65 minutes into the flight. Think about it like this: our discrete signal can actually represent many potential flight histories, most of which have different altitudes at 65 minutes into the flight. In actuality, there are an infinite number of possible flight histories which intersect with the samples of our discrete signal but assume decidedly different altitudes at the 65 minute mark. Click the Play button to see four examples. I hope you’ll notice that connecting the dots in a naive way can be extremely misleading.

We would call the red, blue, green, and orange curves aliases of one another since they are indistinguishable from one another when sampled with a period of 10 minutes per sample. In other words, the red, green, blue, and orange signals all look exactly the same after being sampled every ten minutes.

Whenever there is a gap between samples, uncertainty can creep into our measurements. As the gaps get larger, we become less confident that our discrete signal faithfully represents the physical phenomenon that it was meant to measure. Rapid fluctuations and movements which occur between samples will be lost. We can improve the fidelity of our discrete signals by shortening the gap between samples - by reducing the sampling period.

Imagine that instead of sampling every ten minutes, we had decided to sample the plane’s altitude every 5 minutes. This simple choice would allow us to rule out the orange, green, and red curves as possible aliases of our true signal based upon the sample at 65 minutes. It would be clear that the blue curve is the only possible candidate which still intersects with our samples.

Unfortunately, there is a cost associated with sampling. Each sample must be stored somewhere, and space (e.g. memory) is not free. We cannot reduce our sampling period without incurring some overhead. The fundamental trick to sampling is understanding precisely how often you must sample in order to avoid information loss. When we sample more often than is necessary, we say that we are oversampling. Generally, oversampling implies that memory or computational resources are being wasted.² When we sample too infrequently and lose information we say that we are undersampling.

Proper understanding of sampling theory is much easier when we understand the notion of frequency. In the next few sections we'll leave altitudes behind and look at a class of signals for which frequency is a more natural concept. Sound waves.