File dp_tlm_core.h¶

FileList > dp_tlm > dp_tlm_core.h

Go to the source code of this file

Lightweight scalar telemetry taps for running DSP objects. More...

• #include "buffer/buffer.h"
• #include "clib_common.h"
• #include "jm_perf.h"

Classes¶

Public Types¶

Public Functions¶

Public Static Functions¶

Macros¶

Detailed Description¶

A dp_tlm_t context lets a hot loop publish named scalar time series (tracking-loop stress, AGC gain, lock metrics, ...) without perturbing the signal path:

• Detached (the default): an instrumented object holds a NULL dp_tlm_t *; every probe site is a single pointer load and a predicted-not-taken branch, and only at event rate (per recovered symbol, per gain update) — never per input sample. Consumers who want literal zero can compile with -DDP_TLM_DISABLE, which turns the DP_TLM() probe macro into ((void) 0).
• Attached: each emit is a per-probe decimation check plus one 16-byte record written into a lock-free VM-mirrored SPSC ring (buffer/buffer.h). The write never blocks and never allocates; on overrun the record is dropped and counted, so a slow (or absent) reader can never stall the DSP thread.

Drops are preventable, not merely countable¶

Dropping is the ring's fallback, not the intended steady state. Because no probe can emit more than once per input sample, a block of N inputs emits at most dp_tlm_probe_count() * N records — see dp_tlm_block_bound(). A ring sized to that bound and drained to empty at every block boundary therefore cannot overflow, which is what dp_tlm_capture_open() (dp_tlm_capture/dp_tlm_capture_core.h) sets up for you. Prefer a capture to a hand-rolled drain loop: guessing a ring size and hoping the reader keeps up is the failure mode this bound exists to retire.

Threading contract¶

The ring is single-producer / single-consumer:

• All objects attached to one context must step on ONE producer thread (true of any doppler pipeline). Use one context per pipeline/thread.
• dp_tlm_read() may run concurrently on one consumer thread — that hand-off is the ring's whole design.
• Probe registration (dp_tlm_probe, i.e. obj_set_telemetry) must complete before the producer starts stepping: the probe table is written unlocked at setup time.

Timestamps¶

Records carry a caller-maintained sample index now (stamp it once per block from the pipeline's dp_sample_clock_t via dp_tlm_set_now). If never stamped it stays 0 and consumers index by record order — fine for per-symbol series.

dp_tlm_t *tlm = dp_tlm_create (1 << 14);
int id = dp_tlm_probe (tlm, "agc.gain_db", 1);
...
DP_TLM (tlm, id, gain_db);            // in the hot loop, per event
...
dp_tlm_rec_t recs[512];
size_t n = dp_tlm_read (tlm, 512, recs, 512);   // on the consumer side
dp_tlm_destroy (tlm);

Public Types Documentation¶

typedef dp_tlm_capture_t¶

typedef struct dp_tlm_capture dp_tlm_capture_t;

Opaque lossless capture (dp_tlm_capture_core.h); see dp_tlm_set_now.

typedef dp_tlm_rec_fits_slot¶

typedef char dp_tlm_rec_fits_slot[sizeof(dp_tlm_rec_t)==2 *sizeof(uint64_t) ? 1 :-1];

typedef dp_tlm_state_t¶

jm's spelling of dp_tlm_t .

typedef dp_tlm_t dp_tlm_state_t;

jm derives an object's state struct as <component>_state_t with no override (just-makeit#797), and this type predates jm by years — it is in the signature of every instrumented object's *_set_telemetry, so renaming it is not on the table. An alias costs one line and nothing at runtime.

Not a second type: dp_tlm_t remains the name to write. This exists so the generated binding compiles, and it goes away when jm#797 lands state_type.

typedef dp_tlm_t¶

Telemetry context: probe registry + SPSC record ring.

typedef struct dp_tlm dp_tlm_t;

Public (not opaque) because the emit path is inline; treat the fields as read-only outside dp_tlm_core.c and dp_tlm_emit.

capture is deliberately LAST: the emit hot path touches ring, now and probes, and appending here leaves their cache layout untouched.

Public Functions Documentation¶

function dp_tlm_avail¶

Records currently readable, without consuming them.

size_t dp_tlm_avail (
    const dp_tlm_t * t
) 

The consumer-side head/tail snapshot. Safe to call from the consumer thread while the producer runs: the true count can only GROW after the snapshot, so the value is a lower bound and never over-reports.

function dp_tlm_block_bound¶

Records this context can emit while processing block_samples inputs — the number that makes drops preventable.

size_t dp_tlm_block_bound (
    const dp_tlm_t * t,
    size_t block_samples
) 

probe_count * block_samples, and that is a genuine upper bound rather than an estimate: no probe can emit more than once per input sample. Verified across every object with a *_set_telemetry — the interpolating ones are not counterexamples, because a cascade that produces several outputs from one input collapses them into a single emitted |= strobe (ratesync_core.h, mpsk_receiver_core.h), so one input yields at most one flush. Each probe belongs to exactly one object, so summing over objects is just the context-wide probe count.

Size a ring to this and drain it to empty every block and the ring cannot overflow — no scheduling assumption, no safety factor. Registering more probes raises the bound, which is why a capture re-checks it at each boundary.

Returns:

The bound, or 0 for a NULL context / zero block / no probes. Saturates at SIZE_MAX rather than wrapping.

function dp_tlm_capacity¶

Authoritative ring capacity in records (post page rounding).

size_t dp_tlm_capacity (
    const dp_tlm_t * t
) 

function dp_tlm_create¶

Creates a telemetry context with a ring of ring_records slots.

dp_tlm_t * dp_tlm_create (
    size_t ring_records
) 

Parameters:

• ring_records Requested ring capacity in records. MUST be a power of 2. Sub-page requests are rounded up to the page minimum (buffer.h semantics) — read the authoritative value back with dp_tlm_capacity().

Returns:

New context, or NULL on invalid size / allocation failure.

function dp_tlm_demux¶

Splits recs into per-probe value (and sample-index) buffers.

void dp_tlm_demux (
    const dp_tlm_rec_t * recs,
    size_t n,
    float *const * values,
    uint64_t *const * index,
    const size_t * caps,
    size_t nbuf
) 

Fill half of the demux, also ONE pass: every record is placed on the visit that reads it, so the whole split is O(n) rather than O(n * probes) — the reason this is not just a filter called once per probe.

values and index are arrays of nbuf destination pointers indexed by probe id, exactly as dp_tlm_demux_counts() indexes counts, so the counting pass sizes the buffers this pass fills. A NULL entry in either table skips that probe's component; index may itself be NULL when only the values are wanted. Writes stop at each probe's capacity, so a buffer sized from a stale count truncates rather than overruns.

Parameters:

• recs Records to split; borrowed, only read.
• n Records in recs.
• values Per-probe float destinations, indexed by id; may be NULL.
• index Per-probe uint64_t sample-index destinations; may be NULL.
• caps Per-probe capacities in records, indexed by id.
• nbuf Entries in values, index and caps.

float v0[2], v1[1];
uint64_t n0[2], n1[1];
float *values[2]    = { v0, v1 };
uint64_t *index[2]  = { n0, n1 };
size_t caps[2]      = { 2, 1 };
dp_tlm_demux (recs, 3, values, index, caps, 2);
// v0 == { 1.0f, 3.0f }, n0 == { 0, 2 }, v1 == { 2.0f }, n1 == { 1 }

function dp_tlm_demux_counts¶

Counts each probe's records in recs , in ONE pass.

void dp_tlm_demux_counts (
    const dp_tlm_rec_t * recs,
    size_t n,
    size_t * counts,
    size_t ncounts
) 

Sizing half of the demux. Probe ids are registry slots (dp_tlm_probe_id_at()), so counts is indexed directly by id and needs no map: counts[id] is that probe's record count. Ids at or beyond ncounts are skipped rather than treated as an error — a caller sizing from dp_tlm_probe_count() is asking about the probes it knows, and a blob from another context may legitimately carry more.

Takes a plain array, not a context, so it serves dp_tlm_read()'s output, dp_tlm_capture_records(), and a .tlm16 file read straight off disk.

Parameters:

• recs Records to scan; may be NULL when n is 0.
• n Records in recs.
• counts Destination, zeroed by this call, indexed by probe id.
• ncounts Entries in counts.

dp_tlm_rec_t recs[3] = { { 0, 1.0f, 0, 0 },
                         { 1, 2.0f, 1, 0 },
                         { 2, 3.0f, 0, 0 } };
size_t counts[2];
dp_tlm_demux_counts (recs, 3, counts, 2);
// counts[0] == 2, counts[1] == 1

function dp_tlm_destroy¶

Destroys a context. NULL-safe. Detach all objects first.

void dp_tlm_destroy (
    dp_tlm_t * t
) 

function dp_tlm_dropped¶

Total records dropped on ring overrun (monotonic).

uint64_t dp_tlm_dropped (
    const dp_tlm_t * t
) 

function dp_tlm_emit¶

Records one scalar for probe id . The hot-path primitive.

JM_FORCEINLINE void dp_tlm_emit (
    dp_tlm_t * t,
    int32_t id,
    double v
) 

Detached (t NULL) this is one branch — the entire disabled cost. Attached: bump the probe's decimation phase, and on the decim-th event write one 16-byte record (value narrowed to float, stamped with the context's current now). Never blocks, never allocates; on ring overrun the record is dropped and counted.

id must come from a successful dp_tlm_probe() on this context — an object's set_telemetry fails the whole attach otherwise.

The bound checked here is the ARRAY's, not the registry's. probes is a fixed DP_TLM_MAX_PROBES array, so the unguarded indexing this used to do turned any out-of-range id into an out-of-bounds write — reachable from a language binding, where the id is whatever the caller passed, and Telemetry.emit(1000000, 1.0) segfaulted the interpreter. Comparing against the compile-time constant (unsigned, so a negative id fails it too) needs no memory and measures free. Comparing against n_probes instead would also reject an in-range-but-unregistered id, but it loads a field on the early-return path and cost ~16% of the decimated case (bench_telemetry_core, ABBA-interleaved) — so that check belongs at the binding boundary, where the id is untrusted, not in the hot loop, where the caller holds an id dp_tlm_probe() gave it.

Parameters:

• t Context; NULL is a no-op (the detached case).
• id Probe id from dp_tlm_probe() on THIS context.
• v The scalar, narrowed to float by the ring record.

The Python face binds dp_tlm_emit_checked() instead, which additionally refuses an id the registry never issued — see its docs for why the hot path does not.

>>> from doppler.telemetry import Telemetry
>>> tlm = Telemetry(1 << 12)
>>> pid = tlm.probe("rx.snr_db")
>>> tlm.emit(pid, 12.5)
>>> float(tlm.read()[0]["value"])
12.5

An id the registry never issued is refused, not written:

>>> tlm.emit(pid + 1, 1.0)
Traceback (most recent call last):
ValueError: emit failed (rc=-4)

function dp_tlm_emit_checked¶

Validating dp_tlm_emit() : refuses an id the registry never issued.

int dp_tlm_emit_checked (
    dp_tlm_t * t,
    int32_t id,
    double v
) 

The out-of-line twin of the inline hot-path emit, for callers whose id did not come from dp_tlm_probe() on this context — in practice, a language binding, where the id is whatever the caller passed. dp_tlm_emit() checks only the ARRAY bound (see its docs: checking n_probes there costs ~16% of the decimated path), so an in-range but unregistered id reaches it and emits a record against a probe nobody registered. Here that is an error.

C hot loops keep calling dp_tlm_emit() directly and pay nothing for this.

Parameters:

• t Context. NULL is rejected.
• id Probe id from dp_tlm_probe() on THIS context.
• v The scalar, narrowed to float by the ring record.

Returns:

DP_OK, or DP_ERR_INVALID on a NULL context or an id outside the registry.

>>> from doppler.telemetry import Telemetry
>>> tlm = Telemetry(1 << 12)
>>> pid = tlm.probe("rx.snr_db")
>>> tlm.emit(pid, 12.5)
>>> float(tlm.read()[0]["value"])
12.5

An id the registry never issued is refused, not written:

>>> tlm.emit(pid + 1, 1.0)
Traceback (most recent call last):
ValueError: emit failed (rc=-4)

function dp_tlm_emitted¶

Records written for probe id (post-decimation, post-drop).

uint64_t dp_tlm_emitted (
    const dp_tlm_t * t,
    int id
) 

Reconcile against dp_tlm_dropped() to account for losses: what a probe emitted is what reached the ring, not what the call sites offered it.

Parameters:

• t Context.
• id Probe id from dp_tlm_probe().

Returns:

Records written for that probe, 0 for an unknown id.

>>> from doppler.telemetry import Telemetry
>>> tlm = Telemetry(1 << 12)
>>> eid = tlm.probe("sync.e", decim=2)
>>> for i in range(4):
...     tlm.emit(eid, i / 10)
>>> tlm.emitted(eid)            # decim=2: half the events
2
>>> tlm.dropped
0

function dp_tlm_probe¶

Registers (or re-registers) a named probe. Setup path, not hot.

int dp_tlm_probe (
    dp_tlm_t * t,
    const char * name,
    uint32_t decim
) 

Idempotent by name: registering an existing name returns its id and updates decim (re-attach after a reset keeps ids stable). The decimation phase is primed so the FIRST event after registration emits.

Parameters:

• t Context.
• name Probe name, e.g. "agc.gain_db". Must be shorter than DP_TLM_NAME_MAX.
• decim Emit every decim-th event; >= 1.

Returns:

Probe id (>= 0), or DP_ERR_INVALID on NULL/overlong name, decim == 0, or a full table.

>>> from doppler.telemetry import Telemetry
>>> tlm = Telemetry(1 << 12)
>>> tlm.probe("sync.e", decim=4)
0
>>> tlm.probe("sync.e")     # same name: same id, decim retuned
0
>>> tlm.probe_count
1

function dp_tlm_probe_count¶

Number of registered probes.

size_t dp_tlm_probe_count (
    const dp_tlm_t * t
) 

function dp_tlm_probe_id¶

Looks up a probe id by name; DP_ERR_INVALID if unknown.

int dp_tlm_probe_id (
    const dp_tlm_t * t,
    const char * name
) 

Parameters:

• t Context.
• name Probe name as passed to dp_tlm_probe().

Returns:

Probe id (>= 0), or DP_ERR_INVALID if no such probe.

>>> from doppler.telemetry import Telemetry
>>> tlm = Telemetry(1 << 12)
>>> _ = tlm.probe("agc.gain_db")
>>> tlm.probe_id("agc.gain_db")
0
>>> tlm.probe_id("never.registered")
Traceback (most recent call last):
KeyError: 'no probe by that name (rc=-4)'

function dp_tlm_probe_id_at¶

Probe id at registry slot i . Alwaysi — ids ARE slots.

int dp_tlm_probe_id_at (
    const dp_tlm_t * t,
    size_t i
) 

Exists so a {name: id} mapping can be built from the plain-C triple (dp_tlm_probe_count, dp_tlm_probe_name, this) without the caller needing to know that the identity holds.

function dp_tlm_probe_name¶

Probe name for id , or NULL if out of range.

const char * dp_tlm_probe_name (
    const dp_tlm_t * t,
    int id
) 

function dp_tlm_read¶

Drains records into out . Non-blocking.

size_t dp_tlm_read (
    dp_tlm_t * t,
    size_t n,
    dp_tlm_rec_t * out,
    size_t max_out
) 

Consumer side of the SPSC ring: safe to call from a different thread than the producer. Returns immediately with whatever is available (possibly 0) — never spins.

Parameters:

• t Context.
• n Records wanted; 0 means "everything available".
• out Destination.
• max_out Capacity of out, in records.

n and max_out are separate because the binding allocates out from dp_tlm_read_max_out() and then resizes to what came back — so the request and the buffer are genuinely two numbers, and the read is clamped to the smaller.

Returns:

Number of records copied out.

>>> from doppler.telemetry import Telemetry
>>> tlm = Telemetry(1 << 12)
>>> eid = tlm.probe("sync.e")
>>> for i in range(5):
...     tlm.emit(eid, i / 10)
>>> recs = tlm.read(2)          # take two
>>> recs.shape, recs.dtype.names
((2,), ('n', 'value', 'probe', 'flags'))
>>> tlm.read().shape            # 0 means "everything left"
(3,)
>>> tlm.read().shape            # drained
(0,)

function dp_tlm_read_max_out¶

Upper bound on what dp_tlm_read() can return right now.

size_t dp_tlm_read_max_out (
    dp_tlm_t * t
) 

Simply the available count: a caller sizing a destination cannot know the request will be smaller, and jm's generated binding allocates this much, reads, then resizes to what actually came back.

function dp_tlm_resize¶

Replaces the ring with one holding at least records .

int dp_tlm_resize (
    dp_tlm_t * t,
    size_t records
) 

Rounds records up to a power of two (buffer.h requires it) and then to the page minimum. A no-op returning DP_OK when the ring is already big enough, so it is cheap to call speculatively at every boundary.

Warning:

Destroys whatever the ring holds and is unsynchronised with the producer. Legal only where the producer is quiescent AND the ring has been drained — i.e. a block boundary. dp_tlm_capture_block() is the only caller that needs it; call it yourself only if you own the same guarantee.

Returns:

DP_OK, or DP_ERR_INVALID on NULL / allocation failure (in which case the existing ring is left intact).

function dp_tlm_set_decim¶

Retunes an EXISTING probe's decimation, by name.

int dp_tlm_set_decim (
    dp_tlm_t * t,
    const char * name,
    uint32_t decim
) 

Distinct from dp_tlm_probe(), which registers on a miss: this refuses an unknown name rather than quietly creating a probe nothing emits to, which is what a typo in a retune call deserves.

Parameters:

• t Context.
• name Name of an ALREADY registered probe.
• decim Emit every decim-th event; >= 1.

Returns:

DP_OK, or DP_ERR_INVALID on NULL, an unknown name, or decim == 0.

>>> from doppler.telemetry import Telemetry
>>> tlm = Telemetry(1 << 12)
>>> _ = tlm.probe("sync.e", decim=1)
>>> tlm.set_decim("sync.e", 8)      # retune the existing probe
>>> tlm.set_decim("typo.e", 8)      # refused, not silently created
Traceback (most recent call last):
ValueError: set_decim failed (rc=-4)

function dp_tlm_stats¶

Snapshots the context's counters. Zeroed for a NULL context.

dp_tlm_stats_t dp_tlm_stats (
    const dp_tlm_t * t
) 

Parameters:

• t Context, or NULL for an all-zero record.

Returns:

The four counters as one dp_tlm_stats_t value.

>>> from doppler.telemetry import Telemetry
>>> tlm = Telemetry(1 << 12)
>>> pid = tlm.probe("agc.gain_db")
>>> tlm.emit(pid, -3.5)
>>> tlm.stats()
doppler.telemetry.TelemetryStats(dropped=0, emitted=1, capacity=4096, probes=1)
>>> tlm.stats().emitted
1

Public Static Functions Documentation¶

function dp_tlm_set_now¶

Stamps the sample index carried by subsequent records, and — when a capture is open — closes out the block just finished.

static inline void dp_tlm_set_now (
    dp_tlm_t * t,
    uint64_t n
) 

Call once per block from whoever owns the pipeline's sample clock (dp_tlm_set_now (tlm, clk->n)). NULL-safe so pipeline glue can call it unconditionally.

Callers already place this at the top of the block loop, before stepping, which makes it exactly the boundary a lossless capture needs: delegating here drains the PREVIOUS block, leaving the ring empty as the next one starts. That is the invariant dp_tlm_block_bound() is sized against, so an existing set_now / steps / read loop becomes lossless by opening a capture and changing nothing else.

With no capture open the behaviour is byte-identical to a bare assignment. The delegation is a cold branch on a per-block call, never a per-sample one, so it is nowhere near the hot loops dp_tlm_emit() cares about.

Parameters:

• t Context; NULL is a no-op.
• n Sample index stamped into every subsequent record.

>>> from doppler.telemetry import Telemetry
>>> tlm = Telemetry(1 << 12)
>>> pid = tlm.probe("agc.gain_db")
>>> tlm.set_now(1000)           # top of the block, before stepping
>>> tlm.emit(pid, -3.5)
>>> rec = tlm.read()[0]
>>> int(rec["n"]), float(rec["value"])
(1000, -3.5)

Macro Definition Documentation¶

define DP_TLM¶

Probe-site wrapper around dp_tlm_emit() .

#define DP_TLM (
    ctx,
    id,
    v
) `dp_tlm_emit ((ctx), (id), (v))`

Instrumented hot loops use this form so a consumer building with -DDP_TLM_DISABLE compiles every probe site out entirely.

define DP_TLM_MAX_PROBES¶

#define DP_TLM_MAX_PROBES `64`

Maximum probes per context. Registration fails once full.

define DP_TLM_NAME_MAX¶

#define DP_TLM_NAME_MAX `32`

Maximum probe-name length including the NUL terminator.

define DP_TLM_REC_DTYPE_JSON¶

dp_tlm_rec_t as a numpy-style dtype, in JSON, for a sidecar.

#define DP_TLM_REC_DTYPE_JSON `/* multi line expression */`

A record file is just these 16 bytes repeated, so the dtype is what makes it self-describing — np.fromfile needs nothing else and no doppler code at all. It is a macro rather than a string in each writer because there are two of them (dp_tlm_capture's <path>-meta, and the doppler:telemetry global dp_event_log puts in a SigMF sidecar), and two spellings of one struct's layout is a drift waiting for the day a field moves.

The documentation for this class was generated from the following file native/inc/dp_tlm/dp_tlm_core.h