Reading captures¶

doppler.wfm.Reader is the dual of Writer: it opens a capture, works out what file type it is, and streams unit-scale complex64 samples out of it.

import pathlib
import tempfile

from doppler.wfm import Composer, Reader, Segment, Writer

tmpdir = tempfile.TemporaryDirectory()
tmp = pathlib.Path(tmpdir.name)
x = Composer([Segment("qpsk", sps=8, snr=15, fs=2.4e6,
                      num_samples=8192)]).compose()
with Writer(tmp / "capture.blue", file_type="blue", sample_type="ci16",
            fs=2.4e6, fc=1.2e9) as w:
    w.write(x)

with Reader(tmp / "capture.blue") as r:
    print(r.file_type, r.sample_type, r.fs, r.fc)
    total = 0
    while len(block := r.read(4096)):
        total += len(block)
assert total == len(x)

What each file type actually carries¶

A capture is samples plus the metadata needed to interpret them, and the file types differ enormously in how much of that second part they can hold. This table is the honest state of it — read down your file type's column before assuming a field will survive a round trip.

● carried by the file · ○ not carried — supplied by you, or lost

The ○ rows are the ones that bite. raw and csv are headerless: the sample type and byte order you pass to Reader are hints, and nothing in the file can confirm or contradict them. A wrong hint does not raise — it returns plausible garbage at the wrong stride. See Wrong hints and truncation.

num_samples is now populated for every file type. A CSV has no header to declare its length, so the first read of the property scans the file once and counts rows exactly the way read parses them; later reads are free, and the read position is untouched.

The file type is detected from content, not from the name¶

Detection order, first match wins:

1. the BLUE magic at byte 0
2. a .sigmf-data name, whose .sigmf-meta sidecar is then required
3. a first line that scans as I,Q → CSV
4. otherwise raw, at the sample_type/endian you passed

So a CSV saved as capture.dat still reads as CSV, and a BLUE file saved as capture.csv still reads as BLUE. The name only breaks ties the content cannot: a .det payload (headerless by construction — its header sibling describes it) and a CSV whose first line is a column header.

Nothing is refused for looking unfamiliar. An unrecognised file opens as raw, because a partial or truncated recording is a real thing and a reader that rejects it is useless.

Centre frequency, and why fc_source exists¶

0.0 is a legitimate centre frequency. A genuine baseband capture and a capture whose frequency Reader could not find both report fc == 0.0, so the number alone cannot be trusted — fc_source is what separates them:

with Reader(tmp / "capture.blue") as r:
    if r.fc_source == "none":
        origin = "nothing declares it; fc is a default, not a reading"
    else:
        origin = f"declared as {r.fc} Hz by the {r.fc_source} keyword"
print(origin)
assert origin.startswith("declared")   # this capture carries FREQ

BLUE type-1000 has no header field for centre frequency — the adjunct's xstart/xdelta/xunits describe the abscissa (time), not the RF. So an RF capture conveys it as a keyword, and which tag it uses is X-Midas convention rather than anything the format mandates: BLUE 1.1 §3.1.2.6.4.4 defines FREQ only as a type-6000 column name, under a heading stating those names "are not keyword names".

Reader therefore tries the conventional tags in order — FREQ, RF_FREQ, CENTER_FREQ, F_C — and reports which one answered. Both encodings are accepted, because captures in the wild use both:

• ASCII, in the HCB keyword area (§3.1.1.24.1: KEY=VALUE\0 text at offset 164, no type field). This is where real X-Midas captures put it.
• Typed, in the extended header (§3.3.1), where a D keeps full double precision.

A value that is not a bare number is left alone rather than guessed at: FREQ=2.4 GHz yields fc_source == "none", and the string stays visible in .keywords for a caller who knows the convention. Reading it as 2.4 would be wrong by a factor of a billion.

Writer writes both copies for a non-zero fc. The typed extended-header one is §3.4-compliant and authoritative; the ASCII mirror is what an X-Midas reader will actually look for. §3.4 reserves that 92-byte area for six standard keywords and warns that X-Midas may delete a user keyword found there to make room for IO/VER — which is exactly why it is the mirror and not the original. Reader prefers the typed copy, so the pair can never be read as disagreeing.

Random access¶

seek(index) moves the read position to a sample, so you can go straight to the part of a capture you care about instead of decoding everything in front of it. The index is absolute and in samples — 0 is the first sample, and there is no whence:

with Reader(tmp / "capture.blue") as r:
    r.seek(4096)                    # straight to sample 4096
    burst = r.read(1024)
    assert r.position == 5120       # every read advances the position
    r.reset()                       # reset() is seek(0)
    assert r.position == 0

Samples rather than seconds, because samples are the unit every container can answer. For raw and blue-style strided files this is one seek; a CSV is delimited rather than strided, so it has no byte arithmetic to do and the seek is a scan. The scan runs forward from where you already are, so a loop that walks forward in steps costs one pass over the file, not one per step.

Seeking to num_samples is legal and lands at the end. Past it — or below zero — raises, rather than quietly leaving you at EOF, and a refused seek does not move the read position:

with Reader(tmp / "capture.blue") as r:
    r.seek(100)
    try:
        r.seek(r.num_samples + 1)
    except ValueError as exc:
        assert "past the end" in str(exc)
    assert r.position == 100        # the refusal cost nothing

Seeking by time, and when it refuses¶

seek_time(seconds) is seek(round(seconds * fs)), with seconds measured from the first sample of the capture — not from the UNIX epoch. Its job is the refusal: a capture whose fs_source is "none" reports fs == 0.0, so writing that multiplication out yourself lands on sample 0 for every time, silently. That is raw and csv, always.

with Writer(tmp / "timed.blue", 1e6, file_type="blue",
            sample_type="cf32") as w:
    w.write(x)

with Reader(tmp / "timed.blue") as r:
    r.seek_time(250e-6)             # 250 us at 1 MHz
    assert r.position == 250

with Writer(tmp / "untimed.raw", 0.0, file_type="raw",
            sample_type="cf32", sidecar=False) as w:
    w.write(x)

with Reader(tmp / "untimed.raw", sample_type="cf32") as r:
    assert (r.fs, r.fs_source) == (0.0, "none")
    try:
        r.seek_time(250e-6)         # nothing declared a rate
    except ValueError as exc:
        assert "declares no sample rate" in str(exc)
    r.seek(250)                     # the sample index needs no metadata
    assert r.position == 250

For an absolute time, subtract the capture's own start: `r.seek_time(t_unix

• r.t0), and checkt0_sourcefirst — it is"none"on every capture doppler writes, sot0` is 0.0 there and the subtraction would be meaningless.

Wrong hints and truncation¶

trailing_bytes is the payload bytes left over after the last whole sample. It is 0 for any capture whose declared sample type and mode match its content, and always 0 for CSV (delimited, not strided).

Non-zero means one of two things, and the reader cannot tell which:

• the sample_type/endian hint is wrong for a headerless file type, or
• the capture is truncated — cut mid-sample.

Either way the leftover bytes are dropped; read stops at the last complete sample. For a headerless file type this is the only signal there is:

# a ci8 capture, deliberately read back with the wrong hint
with Writer(tmp / "capture.raw", fs=1e6, file_type="raw",
            sample_type="ci8") as w:
    w.write(x[:5])                       # 5 samples, 2 bytes each = 10 bytes

with Reader(tmp / "capture.raw", sample_type="cf32") as r:
    if r.trailing_bytes:
        print(f"{r.trailing_bytes} bytes do not fit — wrong sample_type, "
              f"or the capture is truncated")
    assert r.trailing_bytes == 2         # 10 bytes is one cf32 sample + 2

Note it cannot catch every wrong hint: reading a ci16 file as ci8 gets the stride wrong but still divides evenly, so trailing_bytes stays 0. It catches the mismatches that leave a remainder, which is most of them, and it never reports a false alarm.

BLUE specifics¶

.header is the whole 512-byte header control block as a dict, under the names the format itself uses (version, data_start, data_size, format, keylength, xstart, xdelta, xunits, …). Nothing is renamed or dropped, so what you see is what the file holds.

.keywords merges both keyword blocks — the HCB's own area and the extended header — into one {tag: value} dict, so a caller cannot tell which block carried a key. Values follow the keyword's type: str for A, int or float for a single-element numeric, a list for a multi-element one. An HCB- area value is always a str, since that area has no type field.

Detached captures (detached = 1) split into a header file and a .det payload. Open either one: given the header, Reader resolves the collocated .det; given the .det, it looks for the header as .hdr, .prm or .tmp. The extension never decides — the HCB's detached field does.

SigMF specifics¶

A SigMF capture is a pair: <base>.sigmf-data holds the samples and <base>.sigmf-meta holds the JSON without which they cannot be decoded. Both halves are found by name, so the name is part of the format — Writer requires a path ending in .sigmf-data and emits the sidecar itself at close:

with Writer(tmp / "capture.sigmf-data", file_type="sigmf",
            sample_type="ci16", fs=2e6, fc=1.2e9) as w:
    w.write(x)          # capture.sigmf-meta is written on close

assert (tmp / "capture.sigmf-meta").exists()
with Reader(tmp / "capture.sigmf-data") as r:
    assert (r.sample_type, r.fs, r.fc) == ("ci16", 2e6, 1.2e9)

tmpdir.cleanup()

fc comes from captures[0]["core:frequency"], and fc_source reports it as "core:frequency".

For a capture with per-segment annotations — ground truth for scoring a detector — build it through Composer, which knows the scene; a plain Writer has nothing to annotate and emits an empty annotations array. The sidecar's schema is documented in Output & file types.