Scenes — composing in time¶

A scene is segments in sequence: what plays, for how long, what follows, and what varies per repeat. This page covers the spec format, then the two things you do with a finished scene — sweep it cheaply with Plan, or stream it in real time.

What each waveform is belongs to Waveforms; the object model is on the guide index.

A scene is more than one waveform: sources mixed at the same time, segments sequenced in time, repeats, and reproducible randomness. This is the Segment / Timeline / Composer rungs of the ladder in practice. Both the CLI (--from-file SPEC.json) and the Python Composer drive the same C engine, so their output is byte-identical for the same parameters.

Composer parameter reference¶

Sequencing segments in time¶

--from-file SPEC.json sequences segments — each a waveform plus an optional trailing off-time — and can repeat or run forever.

{
  "version": 1,
  "segments": [
    { "type": "tone", "fs": 1e6, "freq": 1e5, "snr": 100.0,
      "num_samples": 10000, "off_samples": 5000 },
    { "type": "qpsk", "fs": 1e6, "snr": 9.0, "snr_mode": "esno",
      "sps": 8, "num_samples": 40000 }
  ]
}

wfmgen --from-file scenario.json -o scenario.cf32

from doppler.wfm import Segment, Timeline, Composer

# The same two segments as the JSON spec above: a tone, then a QPSK burst.
timeline = Timeline([
    Segment("tone", fs=1e6, freq=1e5, snr=100.0,
            num_samples=10000, off_samples=5000),
    Segment("qpsk", fs=1e6, snr=9.0, snr_mode="esno",
            sps=8, num_samples=40000),
])
iq = Composer(timeline).compose()   # complex64 — byte-identical to the CLI

type and snr_mode are strings in JSON; every other field is numeric and falls back to the engine default if omitted. A segment's shape is delay | on | off: delay_samples is a leading gap (arrival delay), num_samples the on-time, off_samples a trailing gap — use off for inter-burst spacing and delay for arrival jitter. Gaps are not silence by default: a noisy segment's AWGN keeps running through its delay and trailing gap (the channel's noise floor — gh-409), while a clean scene's gaps stay exact zeros; gap_noise: "off" forces hard zeros. repeat loops the sequence; continuous never finishes (for streaming).

Rather than write the JSON schema from memory, dump a ready-to-edit example with wfmgen json-template and edit it down:

wfmgen json-template > scenario.json   # or: wfmgen json-template scenario.json
wfmgen --from-file scenario.json -o scenario.cf32

The template is a representative spec — an inline tone, an RRC-shaped QPSK-from-bits burst with a trailing gap, and a two-source sum mix — that is valid by construction: it round-trips through --from-file unchanged, so it doubles as a working starting point, not just documentation.

The schema itself is published alongside these pages: wfmgen.schema.json (JSON Schema 2020-12). It is the normative description of every key used below — a template shows one valid spec, the schema says what the whole space of them is, which is what an editor or a validator needs.

Mixing sources (sum) and sequencing them (add)¶

A segment can hold several sources mixed at the same time — a signal of interest plus interferers plus a noise floor — instead of just one. The two composition verbs are orthogonal:

• .sum() mixes sources over the same span (one receiver → one sample rate, one shared noise floor). SNR lives on a source; the floor is resolved once, in C, so the Python, JSON, and CLI faces are byte-identical.
• .add() sequences segments in time, back-to-back — the timeline above, built fluently.

from doppler.wfm import Composer, Segment, qpsk, tone

# A scene: a −12 dB QPSK SoI at +50 kHz over a CW interferer, at 15 dB Es/No.
scene = Segment.sum(
    qpsk(snr=15, snr_mode="esno", level=-12),  # the anchor sets the floor
    tone(freq=5e4),                             # an interferer (level 0 dBFS)
    num_samples=65536,
)

# Sequence a clean preamble tone, then the scene:
timeline = Segment("tone", freq=1e5, num_samples=2000, off_samples=500).add(scene)
iq = Composer(timeline).compose()

Rules of the floor (resolved per segment): an explicit noise(level=N) source fixes it at N dBFS; otherwise the first source carrying snr is the anchor and the floor is level(anchor) − SNR_fs(anchor). Other sources place themselves with level (a plain dBFS offset); giving a non-anchor both snr and level is a spec error. A single-source segment keeps its bundled AWGN untouched, so it is byte-identical to the pre-composition path.

In the JSON schema, a mixed segment replaces the inline source fields with a sum array (each entry is a source; fs/num_samples/off_samples stay on the segment):

{ "fs": 1e6, "num_samples": 65536, "off_samples": 0,
  "sum": [
    { "type": "qpsk", "snr": 15.0, "snr_mode": "esno", "sps": 8, "level": -12.0 },
    { "type": "tone", "freq": 5e4 }
  ] }

Seed control on repeat¶

A repeated stream should be a stream, not the same bytes over and over. By default repeats are byte-identical (the seed is fixed). The --seed-advance knob (spec field seed_advance, honoured by --from-file and Composer.from_json) chooses how much of the seed advances on each loop:

The level is ordered and cumulative — noise keeps the signal, all lets everything change. For pn/bpsk/qpsk the code and data come from the same PN LFSR (one seed), so they advance together under all. Under noise, a fixed preamble or sync code re-acquires every burst while the channel noise changes — ideal for a soak test, a live receiver feed, or a rotating-file recorder. The first loop is always the unmodified seed for every mode, so a finite single-pass run is unaffected and --record stays byte-reproducible.

What creates the loops is --repeat: it plays the whole sequence again after the last segment, indefinitely. --continuous is the stronger form — it never finishes and implies --repeat, so every block comes back full. Reach for --repeat when you want the scene's own segment boundaries preserved in a looping capture, and --continuous when you are feeding something that just wants samples forever.

# The scene, on a loop, with fresh noise each time round.
wfmgen --from-file burst.json --repeat --seed-advance noise -o loop.cf32

# A PN preamble + payload, repeating forever, fresh noise each burst:
wfmgen --from-file burst.json --continuous --realtime --seed-advance noise \
       -o stream.cf32

Ranged values (lo:hi)¶

The advancing seed re-rolls the noise (and PN data), but the parameters — frequency, SNR, level, on/off lengths — stay put. To vary a parameter too, give it a range: a numeric field accepts either a scalar (used as-is) or a [lo, hi] pair drawn uniformly on each segment repeat.

{ "type": "bpsk", "fs": 1e6, "sps": 8, "pn_length": 7,
  "freq":        [11200, 12800],   // carrier offset re-drawn every burst
  "snr":         [8, 14],          // a fresh SNR each burst
  "num_samples": 8192,
  "off_samples": [4000, 5600] }    // jittered trailing gap → code phase walks

On the CLI the same fields take LO:HI in place of a scalar — --freq 11200:12800, --off 4000:5600, --snr 8:14, --level -12:-3 — and a bare number is still just that number.

A ranged freq moves the carrier and nothing else. For a source whose clock is moving — symbol and chip rates scaling with the carrier, as a real pass does — see Clock Doppler; doppler/doppler_rate are ranged fields on this same list.

The draw is reproducible without RNG state: each value is a hash of (seed, repeat index, segment index, source index, field), so --record stores the range and --from-file replays the identical sequence of draws byte-for-byte. Ranges compose with the advancing seed and with chirp's freq/f-end (a sweep whose endpoints jitter per burst).

# Endless bursts, each at a random carrier offset and a jittered gap:
wfmgen --type bpsk --fs 1e6 --sps 8 --pn-length 7 \
       --freq 11200:12800 --count 8192 --off 4000:5600 \
       --continuous --realtime -o stream.cf32

Burst trains (repeats)¶

A segment can play itself repeats times back-to-back — each instance is delay | on | off (arrival delay, burst, trailing gap) — before the timeline advances. That turns "N bursts, randomly placed with a minimum gap" into one declaration instead of N copy-pasted segments:

{ "type": "dsss", "fs": 4e6, "sps": 4,
  "snr": 10.0, "snr_mode": "esno",
  "acq_code": "…", "acq_reps": 5, "data_code": "…",
  "payload": "…",
  "off_samples": [15000, 40000],   // jittered gap, min 15k — per instance
  "repeats": 5 }                    // → a 5-burst train

Instance semantics are exactly what a burst train wants:

• Ranged fields re-draw per instance — with off_samples: [lo, hi] every gap is a fresh draw (lo = the guaranteed minimum gap), and a ranged delay_samples jitters each burst's arrival. The draw key extends the ranged hash with the instance index, so instance 0 renders byte-identically to a repeats-less segment and old scenes are unchanged.
• The AWGN is always fresh per instance — two instances never share a noise realization, regardless of --seed-advance.
• The signal is fixed — codes, payload, PN phase, and every non-ranged parameter repeat exactly. (To vary the signal too, that is what --seed-advance all on a looped timeline is for.)

repeats is per-segment instancing; --repeat loops the whole timeline (advancing the epoch seed). They compose: a two-segment spec with "repeats": 5 on the first plays 5 bursts, then the second segment, then — under --repeat — the whole thing again with the next epoch's draws.

On the CLI the single-segment face is --repeats N; in Python it is Segment(..., repeats=5). See DSSS bursts for the worked burst-train walkthrough.

Reproducible runs (--record)¶

--record run.json writes the fully-resolved spec — every value after defaulting (the auto-selected MLS polynomial, the resolved SNR mode, a summed segment's cleaned anchor + explicit noise floor) and the --headroom. Feed it straight back with --from-file and you get a byte-identical stream:

wfmgen --type bpsk --count 50000 --sps 4 --headroom 6 --record run.json -o a.iq
wfmgen --from-file run.json -o b.iq      # a.iq and b.iq are identical

The recorded --headroom is reapplied on replay; an explicit --headroom on the --from-file run overrides it. Use --record to document a capture next to its data, or to pin an exact scenario in a test. The resolved spec also round-trips through JSON in Python — Composer.from_json(c.to_json()) reproduces the stream.

A frame the caller built¶

The framing keys on a source — sync, acq_code, crc, rs_depth, randomise, asm, conv, interleave — spell the frames doppler already knows, at the positions doppler already puts them. A layout outside that shape has one more key: frame, a description you build yourself.

It is two lists, ordered independently, because position and coverage are independent axes: fields in wire order, stages in application order, each stage carrying the span of fields it covers. (In a CCSDS CADU the marker is inserted third and covered by the stage applied fourth, which a single ordered list cannot say — see A CCSDS CADU.)

Present, frame wins: it is the frame, and the flat keys beside it are not merged in. They keep working exactly as before on every scene that does not carry one.

{
  "version": 1,
  "segments": [
    {
      "type": "bits", "fs": 1e6, "sps": 4, "modulation": "bpsk",
      "pattern": "101010101010101010101010",
      "snr": 100.0, "snr_mode": "fs", "num_samples": 224,
      "frame": {
        "fields": [
          { "name": "hdr", "lit": "0101110001011100" },
          { "name": "payload", "lit": "101010101010101010101010" },
          { "name": "crc", "bits": 16, "derived_by": 1 }
        ],
        "stages": [
          { "kind": "crc16", "first_field": 1, "n_fields": 2 }
        ]
      }
    }
  ]
}

That description says the same thing as --sync 0101110001011100 --crc crc16, so the two are byte-identical — which is how you can tell the flat keys really are sugar for building one of these:

wfmgen --from-file frame.json -o from_desc.cf32
wfmgen --type bits --bits 101010101010101010101010 \
       --sync 0101110001011100 --crc crc16 \
       --fs 1e6 --sps 4 --snr 100 --count 224 -o from_flags.cf32
python3 - <<'EOF'
import pathlib
import sys

a = pathlib.Path("from_desc.cf32").read_bytes()
b = pathlib.Path("from_flags.cf32").read_bytes()
print("byte-identical" if a == b else "DIFFER")
sys.exit(0 if a == b else 1)
EOF

byte-identical

What the description buys is everything past that: a field of your own bits at a position you choose, and a stage covering a span you name. hdr above is neither a preamble nor a sync word, and it sits outside the CRC's cover — a receiver has to find it before it can check anything.

error: this frame description does not lay out: a field that declares a
length but supplies no bits is DERIVED and must name the stage that
fills it (`derived_by` = the stage's index plus one), …

The same description from C and from Python¶

The C struct is the primary interface — wfm_source_t.frame takes a wfm_frame_desc_t *, and wfm_frame_add_field() / wfm_frame_add_derived() / wfm_frame_add_stage() build one by name rather than by index. The gallery page is a whole program.

Python reaches the same description through the scene JSON rather than through a keyword, because Composer.from_json() runs the same C code wfmgen --from-file does:

import json

from doppler.wfm import Composer

seg = {
    "type": "bits", "fs": 1e6, "sps": 4, "modulation": "bpsk",
    "pattern": "101010101010101010101010",
    "snr": 100.0, "snr_mode": "fs", "num_samples": 224,
    "frame": {
        "fields": [
            {"name": "hdr", "lit": "0101110001011100"},
            {"name": "payload", "lit": "101010101010101010101010"},
            {"name": "crc", "bits": 16, "derived_by": 1},
        ],
        "stages": [{"kind": "crc16", "first_field": 1, "n_fields": 2}],
    },
}
x = Composer.from_json(json.dumps({"version": 1, "segments": [seg]})).compose()
print(len(x))

224

A description also goes back out through to_json(), so a scene read and re-written keeps its frame rather than silently reverting to a derived one.

Kernels stay in C, by design. A stage names a kind, and the code that runs it is a wfm_frame_ops_t entry. kind accepts a name for the ones doppler ships (crc16, rs, randomise, conv, interleave) or a raw integer for your own, allocated from WFM_STAGE_USER (4096) up — doppler promises never to allocate at or above that. A kind with no kernel is a refusal, never a silent skip.

Prepare once, sweep many — Plan¶

Evaluating a system — a detector, a demodulator, a synchroniser — means feeding it the same scene at many operating points: a detection or BER curve is a sweep over SNR; a robustness check nudges a gain or a phase; a Monte-Carlo run repeats one scene under fresh noise. Re-composing from scratch at every point is wasteful, because a composed scene is already a linear form,

and the expensive DSP — spreading, root-raised-cosine pulse shaping, the local oscillator — lives entirely in the signal terms. Those do not change when you sweep a level, a phase, the SNR, or the noise seed. Only cheap coefficients do.

prepare(scene) renders and caches each source once, returning a Plan. Every subsequent render is a cheap re-weighted sum of the cache — and bit-for-bit identical to a full compose. It is not a fifth rung on the object-model ladder; it is a cache over a finished scene, for when you need that scene many times.

Preparing a scene¶

Build a scene exactly as you would for compose(), then prepare it. The baseline render() (no overrides) reproduces Composer(scene).compose() exactly:

import numpy as np
from doppler.wfm import Composer, Segment, prepare, qpsk

scene = Composer(Segment.sum(
    qpsk(snr=8.0, seed=7, sps=8, pn_length=9),        # the wanted user (anchor)
    qpsk(seed=101, sps=8, pn_length=9, level=-6.0),   # a co-channel interferer
    fs=1e6, num_samples=4096,
))

plan = prepare(scene)                                 # render + cache ONCE
assert np.array_equal(plan.render(), scene.compose())  # baseline is bit-exact
len(plan), plan.n_sources                             # samples, signal sources

len(plan) is the sample count; plan.n_sources counts the signal sources (the resolved noise floor is separate). prepare(scene) is shorthand for Plan(scene) — either works, and a Plan is a context manager if you want to free its cache promptly.

The overridable axes¶

render() takes five optional overrides. Omit them all for the baseline; pass any subset to vary that axis. The three per-source axes are lists in scene order, length n_sources:

# one wanted user, interferer pulled down 6 dB and rotated 90°, floor at 3 dB
x = plan.render(gains=[0.0, -12.0], phases=[0.0, np.pi / 2], snr=3.0)
x.shape, x.dtype

Every override composes: gains and phases and enable and snr and seed can all be set in one call. Anything you leave out keeps its resolved value from the scene.

Which method to reach for¶

render() is the general form. Three convenience methods wrap it for the common campaigns — reach for whichever names your intent:

• at(snr, seed=None) — the scalar fast path (no JSON round-trip), the hot loop of a sweep. seed defaults to the anchor seed, which reproduces a full compose at that SNR.
• sweep(snrs, seed=None) — yields (snr, samples) across an SNR list at a held noise seed, so only the floor moves. The natural stimulus for a Pd/BER-vs-SNR curve.
• monte_carlo(snr, n, seed0=0) — yields n independent noise realizations at a fixed SNR; the signal is identical across draws, only the noise differs.

# a held-seed SNR curve — same noise realization, only the floor moves
curve = {snr: x for snr, x in plan.sweep([-3.0, 0.0, 3.0, 6.0, 9.0])}

# 16 independent noise draws at 6 dB — identical signal, different noise
draws = list(plan.monte_carlo(6.0, 16, seed0=1000))
assert len({d.tobytes() for d in draws}) == 16       # every realization differs

Recipe — a detection / BER curve¶

Sweep the channel SNR and measure a per-point statistic. Here a light matched-filter peak-SNR against a clean copy of the wanted user (itself produced by disabling the interferer — see the next recipe). A real campaign averages each point over Monte-Carlo draws:

# a clean, interference-free copy of the wanted user = the matched filter
template = plan.render(enable=[True, False])[: len(plan) // 2]
template = template / (np.linalg.norm(template) + 1e-30)

def peak_snr(x):
    c = np.abs(np.correlate(x, template, mode="valid"))
    pk = int(c.argmax())
    off = np.delete(c, slice(max(0, pk - 4), pk + 5))
    return 10 * np.log10(c[pk] ** 2 / (np.mean(off ** 2) + 1e-30))

snrs = np.arange(-6.0, 13.0, 3.0)
# each point: mean peak-SNR over 8 independent noise draws
detect = [np.mean([peak_snr(plan.at(s, 2000 + j)) for j in range(8)])
          for s in snrs]
len(detect) == len(snrs)

The measured curve climbs with channel SNR and then flattens as the multiple-access interference floor takes over — and the cache reproduces the precise noise power the resolver placed at every point.

Recipe — isolate or recombine sources¶

enable drops a source as an exact gain = 0 term, so you can pull any subset out of a scene without rebuilding it — a clean reference, an interference-only capture, a jammer-free template:

wanted_only = plan.render(enable=[True, False])       # signal + noise, no MAI
interferer_only = plan.render(enable=[False, True])   # co-channel only + noise

Recipe — a gain-imbalance or phase sweep¶

Because gains and phases are cheap post-multiplies on the cache, a sensitivity sweep over a relative level or phase is nearly free:

# sweep the interferer's level from 0 down to -18 dB (wanted user fixed at 0)
gain_sweep = [plan.render(gains=[0.0, g]) for g in range(0, -19, -3)]

# sweep its carrier phase across a full turn
phase_sweep = [plan.render(phases=[0.0, ph])
               for ph in np.linspace(0, 2 * np.pi, 8, endpoint=False)]
len(gain_sweep), len(phase_sweep)

Why it is fast (and exact)¶

The cost of prepare() is one full render of every source. After that, each render()/at() is a handful of scaled vector adds over the cache — no LFSR, no convolution, no transcendentals. So a campaign of P points costs roughly "one compose + P cheap sums" instead of "P full composes", and the speedup grows with the number of sources and the sample count, since that is exactly the signal work the cache elides.

Exactness is guaranteed by construction: the composer's accumulate is Σₖ gainₖ·synthₖ in source order with gainₖ = 10^(levelₖ/20), and synthₖ depends on everything except level — so a re-weighted sum of the cached renders is bitwise identical to a full compose at those gains. render() with no overrides equals compose() to the last bit, which is the standing test contract (checked in both harnesses). Phase is a defined render-time rotation (φ = 0 is the identity, skipped entirely), exact by construction rather than reproduced.

Scope and limits¶

prepare() needs a scene whose length is fixed and whose per-source weights are fixed, because that is what makes a re-render a re-weighted sum instead of a re-synthesis. Two things break that, and they are the two it refuses:

Everything else the scene can say is fine — including three things this page used to claim were rejected. Multi-segment scenes, repeats, and ranged timing (off_samples, delay_samples, the per-instance jitter a burst train wants) all prepare, which is what DSSS bursts relies on. So does a lone bundled noisy source, whose private RNG is fused into the signal.

>>> from doppler.wfm import Composer, Segment, prepare
>>> plan = prepare(Composer([Segment(type="qpsk", snr=6.0, num_samples=512,
...                                  off_samples=(100, 300), repeats=2)]))
>>> len(plan) > 0                     # ranged TIMING + repeats: prepared
True
>>> try:                              # a ranged SIGNAL field: refused
...     prepare(Composer([Segment(type="qpsk", snr=(4.0, 8.0),
...                               num_samples=512)]))
... except ValueError:
...     print("rejected")
rejected

A prepared Plan can be serialized — plan.save() → bytes, plan.dump(path) → a file, restored with PlanFromBlob / PlanFromFile — and the restored cache reproduces the stimulus bit-for-bit. But reach for it only to checkpoint or resume a live Plan, not to move one between processes: the blob is the whole rendered cache — roughly n_sources × len(plan) × 8 bytes — so it grows with the scene, and for anything large, restoring a multi-gigabyte blob is slower than re-rendering it from scratch.

For a hand-off, persist the scene's compact spec JSON (Composer.to_json()) and re-prepare() on the far side — the re-render is almost always cheaper than shipping and deserializing the cache, and the spec is kilobytes, not gigabytes. The rule of thumb: transport the recipe, not the rendered signal.

Frequency (Doppler) and delay (multipath) are planned follow-ups on the same frame — additive axes, not a rewrite.

Streaming — real-time pacing¶

By default wfmgen emits as fast as the CPU allows — fs is only metadata (the BLUE xdelta, the NATS header). Add --realtime to throttle the output to fs, so blocks leave on an epoch + n/fs schedule — mimicking a hardware sample clock feeding the sink. This is what you want when a downstream consumer expects samples to arrive at the real rate (a live spectrum display, an SDR playback emulation):

# Stream QPSK to a live receiver at the true 1 MS/s, not as fast as possible.
# Requires a nats-server reachable at the endpoint.
wfmgen --type qpsk --fs 1e6 --sps 8 --continuous --realtime \
       --output nats://127.0.0.1:4222/iq

The schedule is drift-free: each deadline is recomputed from the cumulative sample count against a fixed epoch, so sleep jitter never accumulates — the long-run rate is exactly fs. Pacing does not alter the samples; a file written with and without --realtime is byte-identical.

If the producer can't keep up (a block takes longer than its N/fs period — an underrun), wfmgen keeps the absolute timeline and prints a summary to stderr at exit (wfmgen: 3 underrun(s) — worst 1.2 ms behind real time). Use --realtime-resync instead to re-anchor the clock to "now" on each underrun, staying near real time going forward at the cost of an inserted gap.

# Same stream, but never try to "catch up" after a stall: a live receiver
# would rather see a gap than a burst of stale samples arriving too fast.
wfmgen --type qpsk --fs 1e6 --sps 8 --continuous --realtime --realtime-resync \
       --output nats://127.0.0.1:4222/iq

Which one you want is a question about the consumer, not about the generator. Keep the default when the absolute timeline is the truth (you are going to line the capture up against something else afterwards); use --realtime-resync when staying current matters more than the total elapsed count.

The same clock in Python — SampleClock¶

The same C core is exposed as SampleClock, which paces and timestamps a stream against an ideal fs-Hz clock — throttle a producer to real time and tag blocks with their ideal timestamp:

from doppler.wfm import Composer, SampleClock, StreamSink

# Requires a nats-server reachable at the endpoint.
comp = Composer(type="qpsk", sps=8, continuous=True)
clk = SampleClock(fs=1e6)
with StreamSink("nats://127.0.0.1:4222/iq") as sink:
    while True:
        blk = comp.execute(4096)
        ts = clk.stamp()              # ideal ns timestamp of this block
        sink.send(blk, fs=1e6, fc=0.0)
        clk.pace(len(blk))            # sleep to epoch + n/fs (GIL released)

The schedule is drift-free (deadlines come from the cumulative sample count, not summed sleeps); underruns are counted in clk.underruns / clk.max_lateness, and SampleClock(fs, resync=True) re-anchors to "now" on each underrun. SampleClock and StreamSink are POSIX-only. See the Python API for the full class surface.

See also¶

• Waveforms — what goes into a segment.
• Python API — the same scenes from Python.
• Gallery: scene composition.