The Spectrogram — chunks in, rows out¶

A stream arrives in whatever pieces the transport made of it; a spectrogram wants rows of nfft bins, one every hop samples, every one of them, and the same rows however the stream was cut. This page is the design of the object that does it: what it is for, what it must guarantee, what was unknown until it was measured, and which existing primitives it is built from rather than re-implements.

The dated record behind it — what each step measured, and the guesses that were wrong — is the measurement record. Section numbers are shared between the two, so a §5.1 in an issue or a code comment is the same entry on both.

1. The use cases¶

What those callers do today is the evidence the object is missing, not a list of things to port:

• the only spectrograms in the tree are whole-array loops in two examples, one of which drops its last frame;
• the spectrum analyzer carries two copies of the same re-blocking loop in its sources, and its engine deliberately emits only the newest window, which is right for a display that cannot keep up and exactly wrong for a spectrogram;
• PSD drops a trailing partial frame, has no hop, and returns an average rather than rows;
• chunk invariance is tested only object by object, in private copies (doppler_channel, wfm_synth, the resampler), each a page of loop bounds and none a property another object could join; so a streaming object that has no such copy has no such test.

2. Design goals, and what is not yet known¶

The goals, each of which is a thing a test can fail:

What is not known, and is therefore what the characterization measures rather than confirms (all six are now answered, in the record's §5.4 to §5.9 and §5.15):

• U1 — the carry's copy against a bypass. ANSWERED (§5.6, §5.15): no bypass. Every sample goes through the ring, and that copy costs 0.06–0.24 ns per sample: at most 3.3% of a sample's cost at any nfft from 256 to 65536 and hop of nfft/4, nfft/2 or nfft, under the 5% that would have justified a second path.
• U2 — row latency. ANSWERED at nfft 1024 (§5.7, §5.15). None in samples: a row comes back with the push that delivers its last sample. In time, a push costs c_s + c_p/C per sample, with a per-push overhead of about 4.6 ns whatever the hop. So chunks of 1 cost 4% more than one block at hop 64, 17% at 256 and 66–67% at 1024, and from 256 samples on a chunked push is within 0.7% of one block. Chunks between 1 and 256 are not measured.
• U3 — where a row's time goes. ANSWERED (§5.8): the dB conversion dominates. It is 70–83% of a row, at about 5.3 ns per bin: a divide, a clamp, a double log10 and a cast, which the bench does not separate. The FFT is 12–24%. So power rows are the default (#1968, the owner's decision on #2074): a dB row is asked for by name, and a display converts only the bins it draws. A faster dB conversion for the rows that still want it is #2094's.
• U4 — rows per second one core sustains. ANSWERED (§5.9). At nfft 1024 and hop 256, about 141,000 rows per second (±4%), a 36 MSa/s stream, on one core of the fastest class. A 10 MSa/s stream takes 28% of it.
• U5 — the dB floor. ANSWERED (§5.4). A dB row reads no lower than −200 dB per bin: a tone under that, and an all-zero frame, give the same row. Wideband noise reaches it about 10·log10(nfft) sooner, and float sources in the tree can produce such samples. That is the dB mode's floor by design; a caller who must tell zero from tiny takes power rows, the default, whose only floor is float32's.
• U6 — what callers want at the end of a stream. ANSWERED (§5.5). flush stays explicit: only the caller knows where its own stream ends, and the transports' end-of-stream marker can be dropped (PUB/SUB) or repeated (PUSH/PULL), so a flush implied by it could lose the last row or emit one mid-stream.

3. The prototype¶

A throwaway numpy implementation, in a scratch directory and not committed, asked the three questions the design rests on before any C existed:

1. Does a carry of fewer than nfft samples reproduce the one-shot rows for every split?
2. Is the flushed row the row a one-shot run would produce over the input zero-padded to that row's end — so that it sits on the hop grid and exists only when it holds an uncovered sample?
3. How far apart are a spectrogram built from fft plus 10·log10|X|² and the PSD's convention (divide by the window's coherent gain squared)?

The answers are in §5.3: yes, yes, and by 20·log10(Σw) — 54.18 dB for a 1024-point Hann window — so a second kernel is not a small inconsistency but a wrong level. That is why the design promotes PSD's kernel instead of writing one. The prototype also mis-reported its own contract on the first run, through a comparison bug of its own; the record keeps that.

4. The shape¶

Three existing things, composed. The object adds almost nothing of its own, which is the design.

4.1 The ring owns the carry¶

The ring already does "any chunk in, a contiguous frame out, with a hop" (write_some, peek, consume), but every consumer writes that loop itself. The framer is that loop once, a face of the ring beside its element-typed face:

The framer owns its ring exclusively, which is what makes framer_feed the only way in and the bound a fact rather than a hope. Its own section belongs on the ring page, because it is a face of the ring.

4.2 PSD's kernel is the spectrum¶

A spectrogram is PSD per frame, no averaging. PSD already owns every step that makes a frame a spectrum — the window (Hann, Kaiser, Blackman-Harris, and rectangular), the zero-pad, the FFT, the DC-centred power, and the dBFS reference that makes a full-scale tone read 0 dB. Those steps are one struct, dp_psd_kernel_t (filled by dp_psd_kernel_init), and one transform over it that accumulate and the per-frame calls (dp_psd_frame_power, dp_psd_frame_linear, dp_psd_frame_db) share, and one reading of power against that reference — linear, or dB — that the averaged readouts and the single-frame ones share. PSD holds a kernel by value and folds each frame into its average; the Spectrogram holds one by value and emits each frame.

The kernel stays in the psd component. psd_core.c builds it, so the Spectrogram links all of PSD to use the kernel alone: its averaging code and acc_trace. What the Spectrogram stopped holding (§5.10) is memory, not build graph. A component of its own would drop only those two from one link line, for a new component, manifest entry and test target to keep. The Spectrogram already uses the ring and framer itself. It moves when a consumer needs the kernel without PSD's dependencies, and none does (#2211). The three meters would be the next. Holding a kernel and an averager instead of a PSD would drop PSD's ring, which is address space only (§5.10). But it needs PSD's readouts moved below the PSD and a larger kernel API. That is sized L and belongs with this move; it is not started.

The alternative, fft plus dp_magnitude_db_cf32, is rejected on the prototype's number (§3), not on taste.

4.3 The object¶

dp_spectrogram_create (nfft, hop, window, beta, mode);
size_t dp_spectrogram_push (s, const cf32 *in, size_t n_in,
                            float *out, size_t max_out);  /* floats written */
size_t dp_spectrogram_push_max_out (s, size_t n_in);  /* rows_for * nfft  */
size_t dp_spectrogram_consumed (s);        /* input the last push took    */
size_t dp_spectrogram_rows_for (s, size_t n_in);  /* exact, in rows       */
int64_t dp_spectrogram_flush (s, float *row, size_t row_len); /* nfft, 0 if none owed, -1 if row_len != nfft */
size_t dp_spectrogram_pending (s);
/* + reset, destroy, and the state triplet */

It owns a ring, a framer, and PSD's per-frame kernel — the kernel alone, not a PSD, whose averager, dB buffer, ring and framer would be dead here (28·nfft allocated bytes, §5.10). push loops framer_feed → framer_next → the kernel, so any chunk size works whatever the ring's capacity. A sample is taken unless it would complete a row out has no room for — the framer's own feed contract, applied to rows. So a short out stops at a whole row and consumed() says where to resume, input that completes no row is always taken (even with max_out 0), and push_max_out(n) is exactly the room that makes a push take all of n.

nfft is a power of two and 1 <= hop <= nfft, a hop of 0 meaning nfft (PSD's rule, resolved at create, so the blob holds nfft): frame length and row width are then the same number, which a frame PSD zero-pads to more bins than samples would not give (separating the two is #1966). Rows are linear power by default (DP_SPECTROGRAM_POWER, dp_psd_frame_linear), and dBFS when the caller names DP_SPECTROGRAM_DB (dp_psd_frame_db). Both are PSD's readings of one normalised power, so the two modes share one reference: a full-scale tone reads 1.0 in one and 0 dB in the other. Power is the default because the dB conversion is 70–83% of a dB row (§5.8, U3): a display converts only what it draws, and an averaging consumer folds power rows, since the mean of dB rows is not the dB of the mean. Nothing in the tree gets dB rows from a default; every dB call site names its mode (#1968). Every row is DC-centred exactly as PSD's kernel emits it, bin k at index nfft/2 + k: bin order has one home, and an FFT-order option, if one is ever wanted, belongs to that kernel (#1988). The state is the spectrogram's envelope around the configuration a row depends on (mode, nfft, hop, window and the Kaiser beta) and the framer's snapshot. The configuration is there to be checked, not restored: a blob from another configuration would mean something else, so it is refused (#2102). The plan and the scratch are rebuilt by create.

The Python face, doppler.spectral.Spectrogram, is declarative: jm renders all of it from objects/spectrogram.toml, and no hand-written binding stands in. push returns (rows, nfft) through jm's out_cols (just-buildit/just-makeit#2115), sized from its input by the two-argument push_max_out (state, n_in). mode defaults to "power", and "db" is asked for by name, as in C: both faces name the same constants, bound by name ([[enum]] spectrogram_mode), so their order is the header's alone. Two gaps are jm's: a short out cannot reach push (just-buildit/just-makeit#2230), and flush takes the caller's row until a count-less method can return one (#2181, waiting on just-buildit/just-makeit#2233).

4.4 What it composes, and what it does not re-implement¶

5. What is measured¶

Only what has a number today. Each item is a claim; the dated entry, its method and its caveats are in the record under the same number.

• §5.1 — Promoting PSD's kernel left every readout byte-identical to the version before it, over 72 configurations (3 windows × 3 lengths, one not a power of two, × 2 pads × 4 averaging modes, complex and real input).
• §5.2 — accumulate is bimodal on this machine, about 1.79 or 2.71 ns/sample, with the same medians before and after the promotion: a memory-layout effect that the unchanged code shows too.
• §5.3 — The prototype: the contract holds on 392 cases, and a second kernel would be wrong by 20·log10(Σw).
• §5.4 — The dB floor: a row reads no lower than −200 dB, so a tone under it and an all-zero frame give the same row bit for bit. Noise vanishes about 10·log10(nfft) sooner: at nfft 1024 a total of −180 dBFS leaves on average 1 to 6 bins of 1024 above the floor, depending on the window, and the exponential-bin model predicts every count to within 2.3 bins.
• §5.5 — End of stream: the only caller flushes explicitly, and the transports' marker is not reliable enough to imply a flush.
• §5.6 — The carry's copy is 0.2–3.3% of a sample's cost, so it gets no bypass.
• §5.7 — No row latency in samples. In time, a 4.6 ns per-push overhead makes chunks of 1 cost 17% more than one block, at hop 256.
• §5.8 — The dB conversion is 70–83% of a row (#2074).
• §5.9 — One core sustains about 36 MSa/s at nfft 1024 and hop 256, about 141,000 rows per second (±4%).
• §5.10 — Holding PSD's kernel instead of a PSD left every output and state blob of PSD, the Spectrogram and six consumers byte-identical, at the default and native flags on one x86-64 machine; a Spectrogram allocates 28·nfft fewer bytes, virtual and not resident (derived: 12 heap plus the ring's 16 memfd, counted once), and no benchmark row moves by more than its own round-to-round spread.
• §5.15 — At nfft 1024, the per-push overhead does not depend on hop: about 4.6 ns at hop 64, 256 and 1024. So a chunk of 1 costs 4% more per sample than one block at hop 64, and 66–67% more at 1024. From 256 samples per push on, a chunked push is within 0.7% of one block at every hop measured. The carry's copy is 0.7–1.4% of a sample's cost at hop = nfft/2 too.

The framer's cost against the hand-written loop is a property of the ring, and is recorded with it in the ring's measurements.

Its header's claims are certified in C, where they are stated; the Python face is generated over the same C and pinned by its own tests. native/validation/spectrogram_certify.c measures and src/doppler/spectral/tests/validation/spectrogram/results.md is the generated report, with the header's twenty-seven claims inventoried against their pins and the sabotage that turned each red.

6. The order of work¶

One slice at a time, each its own change, each with its own review. The live state — which of these is merged — is the tracking issue #1894, not this page.

7. What this is not¶

• Not an averager. Rows out; folding them is AccTrace's, composed by the caller.
• Not a decimator or a front end. It consumes baseband at the rate it runs; the DDC upstream is the caller's.
• Not a display. No colour map, no scrolling, no dB range; those are the consumer's.
• Not a detector. It stops at the row.
• Not a transport. It never learns where its samples came from.
• Not real-input or double-precision yet. Complex float32 first; the rest is a follow-up with its own measurements.

See also¶

• Streaming Spectrograms — the user guide, and the tested example
• The Ring Buffer — the framer's home
• The measurement record
• Spectral analysis with PSD — the estimator whose kernel this reuses
• Spectrum Analyzer — the display this serves, and why its engine keeps only the newest window
• State Serialization — the envelope the carry uses