Bit depth, sample rate and dither (noise shaping, theoretical SNR)

Definition

A PCM signal is described by two quantities.

Sample rate fs: the number of samples per unit of time, expressed in Hz. The sampling theorem states that a signal band-limited below fs/2 (the Nyquist frequency) can be reconstructed exactly; components above fs/2 fold back into the band (aliasing) and must be removed with an anti-aliasing filter before sampling or before a downward conversion.

Bit depth N: the number of bits per sample; the available levels are 2^N and the quantization step is q = 2·FS / 2^N. With complex signals the quantization error is uniform within ±q/2 with power q²/12. The theoretical signal-to-noise ratio for a full-scale sine wave is SNR = 6.02·N + 1.76 dB: 98.1 dB at 16 bit, 146.2 dB at 24 bit. 32-bit floating point has a 24-bit mantissa (23 stored plus 1 implicit) and an 8-bit exponent: 24-bit precision relative to the signal level and a representable range greater than 1500 dB.

Dither is noise added before a bit depth reduction to make the quantization error independent of the signal. TPDF dither (triangular probability density function) with an amplitude of ±1 LSB eliminates distortion and noise modulation; the total noise power goes from q²/12 to q²/4 (+4.77 dB). Noise shaping filters the quantization error, moving its energy toward 15–20 kHz, where the ear is less sensitive: perceived noise decreases by 10–20 dB, while the total noise power increases.

Parameters and units

• Sample rates: 44.1 kHz (music, CD), 48 kHz (video), 88.2/96 kHz (production), 176.4/192 kHz.
• Dither types: RPDF (rectangular, ±0.5 LSB), TPDF (triangular, ±1 LSB), with noise shaping (curves of varying aggressiveness).
• Noise at 16 bit with TPDF dither: −96.3 dBFS RMS (mathematical convention).
• Noise at 24 bit with TPDF dither: −144.5 dBFS RMS, below the noise of any converter.

Musical function

Bit depth and sample rate are the container of the sound: when recording, 24 bit offers enough margin to work with 12–18 dB of headroom without audible noise; higher sample rates reduce the aliasing of nonlinear processors and the latency, at the cost of more CPU and storage. Dither preserves the detail of the quietest passages (fade-outs, reverb tails, pianissimo) when going down to 16 bit, turning quantization distortion into a constant, very low hiss.

Practical example

• Piano fade-out exported at 16 bit: without dither, below −80 dBFS the tail becomes grainy and distorted, because the error follows the signal; with TPDF dither the tail stays clean over a constant hiss at −96.3 dBFS RMS; with noise shaping the hiss becomes less audible and concentrates above 15 kHz.
• Quantization step at 16 bit with FS = 1: q = 2/65,536 = 3.05·10⁻⁵; noise without dither q/√12 = 8.8·10⁻⁶ = −101.1 dBFS RMS; SNR relative to the full-scale sine wave (−3.01 dBFS RMS) = 98.1 dB.
• File size: 3:30 of stereo at 44.1 kHz takes up 37.0 MB at 16 bit and 55.6 MB at 24 bit.

Common mistakes

• Applying dither more than once or before the limiter, normalization and fades.
• Using aggressive noise shaping on files that will be processed further: noise pushed above 15 kHz is amplified by later EQs.
• Converting to 16 bit by truncation, without dither.
• Applying dither to a 32-bit floating-point export: it is not needed.
• Assuming that 96 kHz always sounds better than 44.1 kHz: the benefit concerns nonlinear processors, not playback.
• Converting the sample rate with low-quality algorithms or more than once in the chain.

How to check it

• Export settings: check bit depth, sample rate and dither in the DAW; in FL Studio, dither is enabled in the render window for 16-bit exports.
• App analysis: after a 16-bit export with noise shaping, the energy per band of the Ultra-highs (16–20 kHz) in the quiet passages shows the presence of the noise curve; the Sample peak and the True peak (×4, BS.1770) confirm that the bit depth reduction did not create overs.
• Listening to the fade: boost the tail of the exported track by 30–40 dB (for checking only) and verify that it is a constant hiss, not distortion.
• Null test: subtracting the 24-bit file from the aligned 16-bit file leaves only the dither noise.