Audio & DSP

Name: Audio & DSP
Availability: InStock

Learn digital signal processing by building it from scratch in Python: generate and sample signals, implement the DFT and FFT yourself, window and analyze spectra, design FIR and IIR filters by hand, build delay, reverb and modulation effects, detect pitch, and assemble a working synthesizer. The math, not the library calls.

11 projects, 275 hands-on levels, run in your browser.

Syllabus

Foundations: code through audio: Never written code before? Start here. You will learn the absolute basics of Python using sounds, samples, frequencies, and waveforms as your playground. By the end you are ready for Project 1.
Signals & Sampling: Sound is a continuous pressure wave, but a computer can only hold a list of numbers. This project builds the bridge: generate the basic waveforms by hand, sample them at a chosen rate, and confront the two facts that govern all of digital audio, aliasing above the Nyquist frequency and the quantization noise of finite bit depth. Everything later in the track is built on these samples.
Synthesis: A synthesizer turns numbers into notes. This project builds the pieces of a voice: the ADSR envelope that shapes a note's loudness over time, additive synthesis that stacks harmonics into rich tones, amplitude and frequency modulation, the wavetable oscillator at the heart of most digital synths, and the mixing that lets many notes sound at once. By the end you can synthesize a chord from scratch.
The Fourier Transform: Every signal is a sum of sinusoids, and the Fourier transform finds them. This project builds the Discrete Fourier Transform straight from its definition, reads off the magnitude and phase of each frequency bin, inverts the transform to get the signal back, and finally implements the Fast Fourier Transform, the divide-and-conquer algorithm that makes all of modern spectral audio possible. You will never call np.fft the same way again.
Spectral Analysis: A single FFT of a whole song is useless: it tells you which frequencies appear, but not when. Real analysis chops the signal into short overlapping frames, tapers each with a window to tame spectral leakage, and transforms them one at a time. This project builds the windows, the framing, the Short-Time Fourier Transform and the spectrogram it produces, and the parabolic interpolation that pinpoints a peak between bins.
Convolution & FIR Filters: A filter shapes which frequencies pass through, and convolution is the operation that applies it. This project builds convolution from its definition, then designs Finite Impulse Response filters: the moving average, the windowed-sinc low-pass that is the workhorse of audio, high-pass by spectral inversion, and band-pass by cascading. You will compute a filter's frequency response and see exactly what it does to a spectrum.
IIR Filters: Where a FIR filter only looks at past inputs, an IIR filter feeds its own past outputs back in. That feedback buys steep filtering with very few coefficients, at the cost of possible instability. This project builds one-pole smoothers, the general difference equation, the second-order biquad that is the building block of every parametric equalizer, the RBJ cookbook coefficients, and the pole-zero analysis that tells you whether a filter will sing or blow up.
Audio Effects: Now make it sound good. This project builds the effects rack: delay and feedback echo, the Schroeder reverb that stacks comb and allpass filters into a sense of space, the modulation effects (chorus, flanger, vibrato) that all come from one fractionally-interpolated delay line, distortion by waveshaping, and the dynamics processors (noise gate, envelope follower, compressor) that control loudness. These are the boxes on every guitarist's pedalboard and every mixing engineer's channel strip.
Pitch & Time: How does a tuner know the note, and how does a DAW change a singer's pitch without changing the tempo? This project answers both. It detects pitch in the time domain with zero-crossings and autocorrelation (the YIN idea), resamples to shift pitch, overlap-adds frames to stretch time independently of pitch, and maps frequencies onto the musical scale of notes and cents.
Features & Coding: Machines do not listen to waveforms; they listen to features. This project builds the descriptors that drive speech recognition, music tagging, and audio codecs: loudness measures, the spectral-shape features (centroid, rolloff, flux) that capture timbre, the perceptual mel scale and its triangular filterbank, mu-law companding that squeezes audio into fewer bits, and onset detection that finds where the beats fall.
Capstone: A Subtractive Synthesizer: Everything comes together as an instrument. A subtractive synth starts with a harmonically-rich oscillator, carves it with a filter, shapes its loudness with an ADSR envelope, and sweetens it through effects. This capstone builds that signal chain end to end, wires it into a voice that plays a MIDI note, renders a melody, and proves it works by detecting the pitch back out of the synthesized sound. You will have built a real synthesizer from first principles.

Key concepts

Additive synthesis: Building a tone by summing harmonics with chosen amplitudes. A sawtooth is harmonics falling off as 1/k; a square uses only the odd ones.
ADSR: The four stages of a classic synth envelope: Attack, Decay, Sustain, Release. They define how a note rises, settles, holds, and fades.
Aliasing: When a frequency above the Nyquist limit is sampled, it masquerades as a different, lower frequency. The single most important pitfall in digital audio, preven…
Allpass filter: A filter with constant steady-state magnitude response and frequency-dependent phase. It can change transient shape and interference when mixed with another pa…
Autocorrelation: A measure of similarity to delayed versions of a signal. Periodic peaks can support pitch estimation. YIN-style estimation instead uses a normalized squared-di…
Band-pass filter: A filter that passes a band of frequencies between a low and a high cutoff, made by cascading a high-pass and a low-pass.
Biquad: A filter section with numerator and denominator polynomials of degree at most two. It can have up to two poles and two zeros; coefficients or cancellations can…
Bit depth: The number of bits representing a sample. Uniform quantization gains roughly 6 dB per bit; ideal full-scale sine SNR is approximately 6.02B + 1.76 dB under the…
Chorus: Mixing a signal with one or more slightly delayed, pitch-modulated copies to sound like several players at once.
Clipping: Limiting sample amplitudes with a hard threshold or a smooth saturating curve such as tanh. Both alter the spectrum; neither guarantees a particular perceived…
Comb filter: A delayed feedforward or feedback path producing regularly spaced response peaks or notches. Stable feedback combs provide decaying echoes used in reverberatio…
Companding: Compressing amplitude before quantization and expanding it afterward. Uniform steps in the compressed domain become nonuniform amplitude steps, trading small-…
Convolution: A sum of shifted, scaled copies of a sequence. For a zero-state linear time-invariant system, convolving the input with its impulse response gives the output.
Crest factor: Peak absolute amplitude divided by RMS for a nonzero signal. A coherent sine has crest factor sqrt(2); noise crest factor depends on distribution and observati…
Cutoff frequency: A specified reference frequency for a filter, often a -3 dB point or a design boundary. Real filters have transition bands; cutoff does not mean attenuation su…
Decibel (dB): A logarithmic ratio: 20 log10(A/A_ref) for positive amplitude ratios, or 10 log10(P/P_ref) for power. Doubling amplitude adds about 6 dB; the reference must be…
Delay line: A buffer that holds past samples so they can be read back later. The foundation of echo, reverb, chorus, and flanging.
DFT: The discrete Fourier transform maps a finite sample sequence to complex frequency-bin coefficients. Magnitude and phase describe those coefficients; physical a…
Distortion: Reshaping the waveform to add harmonics, from a hard digital clip to the smooth saturation of a tube. Also called waveshaping.
Echo: A delayed, attenuated copy of a signal. A feedback echo repeats and fades, each repeat quieter than the last.
Envelope: The contour of a sound's amplitude over time. Shaping it is what turns a raw tone into a plucked, swelled, or sustained note.
Envelope follower: A circuit or algorithm that tracks a signal's amplitude over time with a fast attack and slow release. The level detector inside compressors and auto-wah.
Feedback: Routing a system's output back into its input. It gives IIR filters and echoes their sustain and resonance, but too much makes them unstable.
FFT: A family of efficient algorithms for the DFT, commonly requiring O(N log N) operations. The recursive radix-2 exercise assumes a power-of-two length.
FIR filter: A finite impulse response filter uses a finite weighted sum of input samples. Finite coefficients ensure BIBO stability; suitable symmetry or antisymmetry give…
Flanger: A chorus with a very short, swept delay mixed near 50/50, producing a sweeping, jet-like comb-filter sound.
FM synthesis: Frequency modulation: one oscillator bends the frequency of another, generating rich sidebands from very few operators. The sound of 1980s digital synths.
Frequency: How many cycles a wave completes per second, in hertz, perceived as pitch. Doubling the frequency raises the pitch by an octave.
Frequency bin: A DFT coefficient indexed by k with grid spacing sample_rate/N. Two-tone resolving ability also depends on the observation length and window; zero padding only…
Fundamental frequency: The reciprocal of a periodic signal's shortest repetition period. Harmonics are integer multiples; the fundamental spectral component need not itself be pr…
Harmonic: A sinusoid at an integer multiple of a fundamental frequency. The mix of harmonic amplitudes gives an instrument its timbre.
High-pass filter: A filter that passes higher frequencies and attenuates lower ones. The amount of rejection depends on the response; lower frequencies are not generally all rem…
IIR filter: An Infinite Impulse Response filter: it feeds past outputs back in. Steep filtering from very few coefficients, but it can be unstable.
Impulse response: A system's zero-state response to a unit impulse. It characterizes a linear time-invariant system; for an FIR filter it is the finite coefficient sequence.
LFO: A low-frequency oscillator used to vary parameters such as amplitude, pitch or cutoff. It usually operates below the audio range, but modulation can create aud…
Low-pass filter: A filter that passes frequencies below the cutoff and attenuates those above. The most common filter in subtractive synthesis.
Magnitude spectrum: The absolute value of each complex DFT bin: how much of each frequency is present. What a spectrum analyzer plots.
Mel scale: An approximate perceptual frequency scale. This track uses mel = 2595 log10(1 + f/700); it is a conventional model rather than a universal law of equal perceiv…
MFCC: Mel-Frequency Cepstral Coefficients: a compact description of a spectrum's shape on the mel scale, the classic feature set behind speech recognition.
Mu-law: A logarithmic companding curve giving finer effective amplitude steps near zero after quantization. The continuous formula taught here is not the complete disc…
Nyquist frequency: Half the sample rate. For unambiguous reconstruction, band-limit continuous input strictly below this boundary; sampling exactly at it can lose phase informati…
Onset detection: Finding where notes start by detecting sudden rises in energy or spectral flux, then picking peaks. The first step in rhythm and tempo analysis.
Phase: Where in its cycle a sinusoid starts, in radians or degrees. Two waves of the same frequency can reinforce or cancel depending on their relative phase.
Pitch: The perceived highness of a sound. Fundamental frequency is a major cue for periodic tones, but ambiguity, harmonics and noise can defeat simple frequency esti…
Pole: A root of the reduced transfer-function denominator. A causal rational filter is BIBO stable when all its poles lie strictly inside the unit circle; near-unit…
Q factor: How sharp and resonant a filter's peak is. Higher Q means a narrower, taller peak that rings longer.
Quantization: Mapping samples to a finite set of levels. The error may be correlated distortion or noise-like, depending on the signal and use of dither.
Resampling: Computing samples on a new time grid. Changing playback speed at fixed output sample rate changes pitch and duration together. Downsampling needs bandwidth con…
Resonance: A pronounced peak in a filter's response near the cutoff, set by the Q. High resonance makes the squelchy filter sweeps of electronic music.
Reverb: The sound of a space: thousands of overlapping echoes. The Schroeder design builds it from parallel comb filters and series allpass filters.
RMS: Root mean square: sqrt(mean(samples squared)). It measures effective amplitude; frequency weighting, duration and context also affect perceived loudness.
Sample: A single measurement of a sound wave's amplitude at one instant. A digital signal is just a list of samples taken at a steady rate.
Sample rate: How many samples are taken per second, in hertz. CD audio is 44,100 Hz. The rate sets the highest frequency that can be captured, the Nyquist frequency.
Sinusoid: A sine or cosine wave specified by frequency, amplitude and phase. The DFT represents any finite sample sequence using a finite complex sinusoidal basis.
Spectral centroid: A frequency-weighted mean of spectral magnitudes in this track. It is a useful brightness cue, with interpretation dependent on weighting and the signal; silen…
Spectral flux: A measure of change between successive spectra. Peaks can suggest onsets but may also reflect other changes; the precise difference and normalization conventio…
Spectral leakage: When a frequency does not fall exactly on a bin, its energy smears across neighboring bins. Tapering each frame with a window function reduces it.
Spectrogram: A time-frequency image of STFT magnitude or power, often on a logarithmic scale. State the normalization and whether values are magnitude or squared magnitude.
STFT: The Short-Time Fourier Transform: chop the signal into short overlapping windowed frames and FFT each, giving frequency content as it changes over time.
Synthesis: Generating sound from scratch with math. Additive stacks sinusoids; subtractive filters a rich waveform; FM modulates one oscillator with another.
Timbre: The tone color that distinguishes a flute from a trumpet at the same pitch and loudness. It comes from the harmonic spectrum and how it evolves over time.
Time stretching: Changing duration while aiming to preserve pitch. The raw overlap-add exercise changes frame placement but does not guarantee pitch preservation or smooth reco…
Tremolo: Modulating a signal's amplitude with an LFO, a periodic swell in loudness.
Vibrato: Modulating pitch with an LFO by reading from a wobbling delay line, which needs fractional-delay interpolation. The basis of chorus and flanging too.
Waveform: The shape of a signal over time. Sine, square, sawtooth, and triangle are the classic synthesis waveforms, each with a distinct harmonic spectrum.
Wavetable: A table of waveform samples read cyclically to generate an oscillator. Interpolation and appropriate bandwidth control affect its quality.
Window function: A frame weighting used before spectral analysis. Tapers trade lower sidelobes for a broader main lobe. Symmetric Hann reaches zero at its endpoints; Hamming do…
Zero: A root of the transfer-function numerator. A zero on the unit circle produces a response null at its angle, absent cancellation; off-circle zeros shape the res…
Zero-crossing rate: How often a waveform crosses zero. A cheap pitch and noisiness cue: low for voiced tones, high for hiss and noise.