Changeset 11064 in ntrip

Timestamp:

Oct 2, 2026, 1:53:41 PM (4 hours ago)

Author:

stuerze

Message:

test suite description is added

File:

• trunk/BNC/src/bnchelp.html (modified) (2 diffs)

trunk/BNC/src/bnchelp.html

@@ -97 +97 @@
     &nbsp; &nbsp; &nbsp; 1.6 <a href="#introConf">Configuration</a><br>
     &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1.6.1 <a href="#introExamples">Examples</a><br>
+    &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1.6.2 <a href="#introTests">Regression Test Suite</a><br>
     &nbsp; &nbsp; &nbsp; 1.7 <a href="#introLimit">Limitations</a><br>
     &nbsp; &nbsp; &nbsp; 1.8 <a href="#introLBack">Looking Back</a><br><br>
@@ -2306 +2307 @@
     </li>
   </ol>
+  </p>
+
+  <p>
+  <h4 id="introTests">1.6.2 Regression Test Suite</h4>
+  </p>
+  <p>
+    The source code distribution contains in directory 'tests' a regression test suite which is based on the
+    configuration examples described above. It runs BNC headless ('--nw') on fixed input data and compares the
+    output files with stored reference outputs. Hence, it shows whether a modification of the source code changes
+    BNC's results - intended or not. No network access is needed: real-time examples are tested by replaying short
+    raw captures of real streams ('--file' option, see section 'Raw Output File'), uploads are received by a local
+    fake Ntrip caster. The test suite requires Python 3; it is intended for developers and is not needed to operate
+    BNC.
+  </p>
+  <p>
+    <b>Usage</b><br>
+    Compile BNC first, the test suite uses the program './bnc' in the BNC root directory. Then call from that
+    directory:
+  </p>
+  <pre>
+tests/run_tests.py                   # run all test cases in parallel
+tests/run_tests.py ssr_ rinex_qc     # run only cases whose name contains one of the patterns
+tests/run_tests.py --list            # list all cases with a short description
+tests/run_tests.py -j 4              # number of cases run in parallel (default: number of CPUs)
+tests/run_tests.py --bnc /path/bnc   # test another BNC binary
+tests/run_tests.py --update NAME     # accept the current outputs of case NAME as new reference
+</pre>
+  <p>
+    Each case is reported as PASS, FAIL or SKIP; the exit code is 1 if a case failed. For a failed case the first
+    differing lines are printed. The full difference is saved in 'tests/work/&lt;case&gt;/&lt;output&gt;.diff' and
+    the exact BNC command line in 'tests/work/&lt;case&gt;/command.txt', so that the run can be repeated by hand.
+    Each case runs in its own directory 'tests/work/&lt;case&gt;' which contains a link 'Input' to
+    'Example_Configs/Input' and an empty directory 'Output'. Thus the example configurations are used unchanged and
+    nothing is written into 'Example_Configs'.
+  </p>
+  <p>
+    Before comparing, contents that change with each run or BNC version are removed: wall-clock time stamps of
+    logfile lines, 'Start BNC' lines, the BNC version string, the RINEX header line 'PGM / RUN BY / DATE' and the
+    'Run by' and 'Date' lines of QC logfiles. Replays in batch mode are reproducible from run to run, including PPP
+    with ambiguity resolution; hence most outputs are compared exactly.
+  </p>
+  <p>
+    <b>Test cases</b><br>
+    The test cases cover:
+  </p>
+  <ul>
+    <li>File-based examples (RINEX concatenation, RINEX QC, RINEX editing with elevation mask, SP3 comparison, PPP
+      in post-processing mode and with plots), partly also as command line variants.</li>
+    <li>Stream replays of real-time examples: RINEX observation and navigation files, Feed Engine, SP3/Clock RINEX
+      files from broadcast corrections (also Galileo HAS).</li>
+    <li>Real-time PPP: float, Quick-Start, ambiguity resolution with CNES and WHU corrections, two stations, Galileo
+      HAS.</li>
+    <li>Combination of broadcast corrections, also with PPP using the combined corrections ('INTERNAL').</li>
+    <li>Upload of broadcast corrections (encoded as IGS-SSR), of broadcast ephemerides and of raw streams to a local
+      fake caster; the received streams are decoded again.</li>
+    <li>Decoding of SSR streams in IGS-SSR and RTCM-SSR format; besides the decoded corrections files the decoder
+      trace on stderr is compared, if BNC is compiled with debug option 'BNC_DEBUG_SSR'.</li>
+  </ul>
+  <p>
+    <b>Accepting changes and adding cases</b><br>
+    If a modification changes the results intentionally, the reference outputs of the affected cases are updated
+    with option '--update'. Check the differences carefully before: a reference output records what BNC does, not
+    what is correct. A new case is a directory 'tests/cases/&lt;name&gt;' with a file 'case.json' (configuration
+    file, raw capture, additional command line options, expected exit code and output files to compare; the format
+    is documented in the header of 'run_tests.py') and the reference outputs in sub-directory 'expected'. Short raw
+    captures for replay cases can be prepared with the tool 'tests/tools/bncraw.py':
+  </p>
+  <pre>
+tests/tools/bncraw.py info capture.raw        # streams, time span and RTCM3 message types
+tests/tools/bncraw.py slice capture.raw tests/data/new.raw --stations SSRA00CNE1,BCEP00BKG0 \
+                      --start 2026-09-30T14:00:00 --end 2026-09-30T14:03:00
+tests/run_tests.py --update new
+</pre>
+  <p>
+    Raw captures are recorded by BNC itself with option 'Raw Output File'. Further details are given in
+    'tests/README.md'.
   </p>