| 1 | # BNC regression tests
|
|---|
| 2 |
|
|---|
| 3 | Run BNC headless on fixed inputs and compare its outputs with stored reference
|
|---|
| 4 | outputs. No network access is needed.
|
|---|
| 5 |
|
|---|
| 6 | ```
|
|---|
| 7 | tests/run_tests.py # all cases (parallel), exit code 1 on failure
|
|---|
| 8 | tests/run_tests.py ssr_ rinex_qc # only cases whose name contains a pattern
|
|---|
| 9 | tests/run_tests.py --list
|
|---|
| 10 | tests/run_tests.py --update NAME # accept current output as new reference
|
|---|
| 11 | # (removes references of outputs no longer produced)
|
|---|
| 12 | ```
|
|---|
| 13 |
|
|---|
| 14 | Build `./bnc` first. Each case runs in `tests/work/<case>/`,
|
|---|
| 15 | which has an `Input` symlink to `Example_Configs/Input` and an empty `Output`
|
|---|
| 16 | directory, so the example configurations run unchanged. On a failure, the
|
|---|
| 17 | full diff is at `tests/work/<case>/<output>.diff` and the exact command line
|
|---|
| 18 | is in `tests/work/<case>/command.txt`.
|
|---|
| 19 |
|
|---|
| 20 | ## Layout
|
|---|
| 21 |
|
|---|
| 22 | | Path | Content |
|
|---|
| 23 | |---|---|
|
|---|
| 24 | | `run_tests.py` | runner; the `case.json` format is documented in its header |
|
|---|
| 25 | | `cases/<name>/case.json` | config, extra `-key` arguments, raw capture, outputs to compare |
|
|---|
| 26 | | `cases/<name>/expected/*.gz` | normalised reference outputs |
|
|---|
| 27 | | `data/*.raw` | short raw captures (BNC `rawOutFile` format) replayed via `--file` |
|
|---|
| 28 | | `tools/fakecaster.py` | local Ntrip caster recording an upload as a BNC raw file (case key `fakecaster`); optionally TLS and serving a stream |
|
|---|
| 29 | | `tools/bncraw.py` | `info` (streams, time span, RTCM3 message types), `slice` and `crs` (decode 1300/1301/1302) for raw captures |
|
|---|
| 30 | | `tools/mkcrafted.py` | writes `data/crafted_limits.raw`: crafted frames at the upper field limits |
|
|---|
| 31 | | `tools/ssr_harness.sh` | builds `tools/ssr_harness.cpp` with the SSR decoders under ASan/UBSan and runs it |
|
|---|
| 32 | | `tools/ssl_check.py` | TLS checks of the Ntrip 2s client and upload (not part of `run_tests.py`) |
|
|---|
| 33 | | `work/` | scratch output, not versioned |
|
|---|
| 34 |
|
|---|
| 35 | Before comparing, the runner removes content that changes with each run or
|
|---|
| 36 | BNC version: wall-clock log prefixes, `Start BNC` lines, the BNC version
|
|---|
| 37 | string, the RINEX `PGM / RUN BY / DATE` line, and the `Run by`/`Date` lines
|
|---|
| 38 | in reqc logs.
|
|---|
| 39 |
|
|---|
| 40 | ## Cases
|
|---|
| 41 |
|
|---|
| 42 | * File-based examples: `rinex_concat`, `rinex_qc`, `sp3_compare`, `ppp_postproc`,
|
|---|
| 43 | `ppp_osm` (04, 05, 21, 11, 12), plus the command-line variants `*_cli` and
|
|---|
| 44 | `sp3_compare_exclude` (scripts 24, 23, 28). The QC cases use the 1 h BRUX
|
|---|
| 45 | files instead of the full-day file.
|
|---|
| 46 | * `rinex_edit_minele`: Edit/Concatenate with a 10 deg elevation mask (1 h BRUX
|
|---|
| 47 | files), covering the ephemeris selection for the elevation mask.
|
|---|
| 48 | * Stream replays of the real-time examples (`replay_*`), all from the 5 min
|
|---|
| 49 | slice `data/wtzr_cnes_5min.raw` (BCEP00BKG0, WTZR00DEU0, SSRA00CNE1):
|
|---|
| 50 | `replay_rinex_obs` (01, skeleton not mandatory as there is no caster offline),
|
|---|
| 51 | `replay_rinex_eph` (02), `replay_feed_engine` (07, IP port disabled) and
|
|---|
| 52 | `replay_save_sp3` (14, with SSRA00CNE1 instead of SSRA00BKG1).
|
|---|
| 53 | * `replay_save_sp3_has`: 14 with Galileo HAS (SSRA00EUH0) only; its epochs are
|
|---|
| 54 | off the SP3/CLK sampling grid and are written at the preceding grid epoch.
|
|---|
| 55 | * Real-time PPP replays (`ppp_rt_*`) from 11 min slices of the CNES and WHU
|
|---|
| 56 | captures (`data/wtzr_cnes_11min.raw`, `data/wtzr_whu_11min.raw`):
|
|---|
| 57 | `ppp_rt_float` (08), `ppp_rt_quickstart` (10, NMEA port disabled), `ppp_rt_ar_cnes` (29) and `ppp_rt_ar_whu`
|
|---|
| 58 | (30), compared exactly.
|
|---|
| 59 | * `ppp_rt_net` (09, two stations) and `ppp_rt_has` (13, Galileo HAS via
|
|---|
| 60 | SSRA00EUH0) from `data/merged_260930_11min.raw`, a slice of two simultaneous
|
|---|
| 61 | recordings merged by time (BCEP00BKG0, WTZR00DEU0, FFMJ00DEU0, SSRA00CNE1,
|
|---|
| 62 | SSRA00EUH0).
|
|---|
| 63 | * Combination (`combi`, `combi_ppp`; 17 and 18 unchanged) from
|
|---|
| 64 | `data/combi_261005_11min.raw` (SSRA00BKG1, SSRA00CNE1, SSRA00CHC1, BCEP00BKG0,
|
|---|
| 65 | FFMJ00DEU0), upload to the caster disabled, SP3/CLK/BIA written to files.
|
|---|
| 66 | * `combi_multi`: 17 with the 7 ACs of an operational combination (BKG1, CHC1,
|
|---|
| 67 | CNE1, GFZ1, GMV0, NRC0, SHA1; IGS-SSR and RTCM-SSR) from
|
|---|
| 68 | `data/combi_multi_261002_5min.raw`. Covers legacy and erroneous epoch
|
|---|
| 69 | encodings, missing reference-signal biases, inconsistent biases and
|
|---|
| 70 | persistent outliers. The longest case (about 4 min).
|
|---|
| 71 | * `combi_refsig`: `combi` with non-default reference signals (`ssrRefSignals
|
|---|
| 72 | C:1P&5P`): BDS clock referral and code biases from the Bias SINEX file on
|
|---|
| 73 | C1P/C5P (generic DCB derivation), CLK/BIA and the combination log compared.
|
|---|
| 74 | * Upload (`upload_*`; 16, 19, 20) from `data/upload_rtnet_5min.raw` (RTNET engine
|
|---|
| 75 | output + BCEP00BKG0): BNC uploads to a local fake caster
|
|---|
| 76 | (`tools/fakecaster.py`, Ntrip 1/2, chunked), and a `post` step decodes the
|
|---|
| 77 | received stream again: `upload_rtnet` (IGS-SSR encoding, SP3/CLK/BIA files,
|
|---|
| 78 | decoded `.ssr`, CRS messages 1300/1302 decoded by `bncraw.py crs` in a
|
|---|
| 79 | `tool` post step), `upload_eph` (RTCM3 ephemerides to RINEX nav) and
|
|---|
| 80 | `upload_raw` (raw forwarding to RINEX nav). `upload_rtnet_rtcm` encodes the same input to RTCM-SSR
|
|---|
| 81 | (message numbers 1057-1068, 1240-1270) and decodes it again.
|
|---|
| 82 | * SSR decoding (`ssr_*`): 3 min slices of real streams replayed through
|
|---|
| 83 | `03_BrdcCorr.bnc`. They compare the `.ssr` files and the `BNC_DEBUG_SSR`
|
|---|
| 84 | stderr trace. On a build without that trace, the trace comparison is
|
|---|
| 85 | reported as SKIP.
|
|---|
| 86 | * `ssr_igs_SSRA00CNE1`: IGS-SSR 4076, `SsrCorrIgs`
|
|---|
| 87 | * `ssr_rtcm_SSRA00CNE0`, `ssr_rtcm_WHU1_OSBC`: RTCM 1059-1270; these test
|
|---|
| 88 | whichever class `RTCM3Decoder.cpp` selects for non-4076 SSR.
|
|---|
| 89 | * `ssr_weekchange`: 2 min across the GPS week change (SSRA00CNE1, SSRA00CHC1);
|
|---|
| 90 | corrections of Saturday received after midnight must keep their epochs.
|
|---|
| 91 | * `ssr_qc`, `ssr_qc_select`: SSR QC (31_SsrQC, one file per stream, 5 min interval;
|
|---|
| 92 | all streams resp. `ssrQcStreams SSRA00CNE1,SSRA00GMV0`) of the 7 AC streams of
|
|---|
| 93 | `data/combi_multi_261002_5min.raw`. `ssr_qc_bias` adds the CAS DCB file as
|
|---|
| 94 | 'QC Bias File' (code bias check against the other ACs and the file).
|
|---|
| 95 |
|
|---|
| 96 | * `decoder_limits`: crafted, CRC-valid frames from `tools/mkcrafted.py`
|
|---|
| 97 | using the largest values the formats allow: VTEC with degree/order 16
|
|---|
| 98 | (IGS-SSR 4076/201 and RTCM-SSR 1264, 2 and 4 layers) and RTCM 1300/1301/1302
|
|---|
| 99 | with 31-character names, listed by the RTCM scan in the log. The
|
|---|
| 100 | coefficients are C(d,o) = (100*d+o)/200 TECU and S = -C, both `.ssr` files
|
|---|
| 101 | are identical apart from the station ID. These frames overflowed the
|
|---|
| 102 | coefficient and name arrays; the 4-layer VTEC crashed BNC.
|
|---|
| 103 |
|
|---|
| 104 | ## Sanitizer check of the SSR decoders
|
|---|
| 105 |
|
|---|
| 106 | ```
|
|---|
| 107 | tests/tools/ssr_harness.sh # crafted_limits.raw
|
|---|
| 108 | tests/tools/ssr_harness.sh tests/data/*.raw # real captures
|
|---|
| 109 | ```
|
|---|
| 110 |
|
|---|
| 111 | In BNC all SSR target structs are members of one `RTCM3coDecoder` object,
|
|---|
| 112 | so a write beyond one struct corrupts the next member without any
|
|---|
| 113 | AddressSanitizer report. The harness decodes every frame with all three
|
|---|
| 114 | format classes into separately allocated structs, so ASan reports it. It
|
|---|
| 115 | also covers the new RTCM-SSR metadata (message 60 in
|
|---|
| 116 | `crafted_limits.raw`: 7 entries with additional parameters of 40 and 0 bits,
|
|---|
| 117 | and 31 entries, which must be rejected), which `RTCM3Decoder.cpp` does not
|
|---|
| 118 | dispatch at present. Every decoded VTEC and metadata message is encoded
|
|---|
| 119 | again with the `Make*` function of the same class and decoded once more; the
|
|---|
| 120 | result must be identical. Exit code 0 only without sanitizer errors and
|
|---|
| 121 | differences.
|
|---|
| 122 |
|
|---|
| 123 | ## TLS check
|
|---|
| 124 |
|
|---|
| 125 | ```
|
|---|
| 126 | tests/tools/ssl_check.py [--bnc PATH]
|
|---|
| 127 | ```
|
|---|
| 128 |
|
|---|
| 129 | Runs BNC against `tools/fakecaster.py` with TLS on port 443 (BNC uploads with
|
|---|
| 130 | Ntrip 2s always to 443) inside an unprivileged user and network namespace
|
|---|
| 131 | (`unshare -rn`), with a test CA, server and client certificates created by
|
|---|
| 132 | openssl in `work/ssl_check/`. Checks for upload and download: certificate of
|
|---|
| 133 | an unknown CA rejected, accepted with the CA in `sslCaCertPath`, accepted with
|
|---|
| 134 | `sslIgnoreErrors` and the ignored errors logged; for upload also a
|
|---|
| 135 | certificate for another host name (rejected) and a client certificate
|
|---|
| 136 | required by the caster, with and without `sslIgnoreErrors`. Takes about 3 min.
|
|---|
| 137 |
|
|---|
| 138 | ## Adding a replay case
|
|---|
| 139 |
|
|---|
| 140 | ```
|
|---|
| 141 | tests/tools/bncraw.py info capture.raw
|
|---|
| 142 | tests/tools/bncraw.py slice capture.raw tests/data/new.raw --stations SSRA00XYZ0 \
|
|---|
| 143 | --start 2026-09-28T12:00:00 --end 2026-09-28T12:03:00
|
|---|
| 144 | mkdir tests/cases/new && $EDITOR tests/cases/new/case.json # copy an ssr_* case
|
|---|
| 145 | tests/run_tests.py --update new
|
|---|
| 146 | ```
|
|---|
| 147 |
|
|---|
| 148 | Before accepting a reference with `--update`, check it by hand; for decoder
|
|---|
| 149 | cases, hand-decode the raw bits. The reference records what BNC does, not what
|
|---|
| 150 | is correct.
|
|---|
| 151 |
|
|---|
| 152 | ## Reproducibility of replays
|
|---|
| 153 |
|
|---|
| 154 | In batch post-processing (`--file`), BNC delivers decoded ephemerides and
|
|---|
| 155 | corrections synchronously to the PPP (`t_bncCore::decoderConnectionType`) and
|
|---|
| 156 | starts reading only after all PPP instances are connected
|
|---|
| 157 | (`t_pppThread::waitUntilReady`). Replays are therefore reproducible run to run,
|
|---|
| 158 | including PPP with ambiguity resolution. Live operation is not affected.
|
|---|
