Changeset 11190 in ntrip for trunk/BNC

Timestamp:

Nov 10, 2026, 9:21:30 AM (7 hours ago)

Author:

stuerze

Message:

Structured logging improved

Location:

trunk/BNC

Files:

• CHANGELOG.md (modified) (1 diff)
• src/bncgetthread.cpp (modified) (1 diff)
• src/bnchelp.html (modified) (1 diff)
• src/bnclog.cpp (modified) (1 diff)
• src/latencychecker.cpp (modified) (1 diff)
• tests/cases/log_structured/expected/PPPQuickStart.jsonl.gz (modified) ( previous)
• tests/cases/log_structured/expected/PPPQuickStart.logfmt.gz (modified) ( previous)
• tests/tools/logcheck.py (modified) (3 diffs)

trunk/BNC/CHANGELOG.md

@@ -1 +1 @@
 # Changelog
 ## 2.13.8 (2026-11-04)
-- ADDED: **Structured logfile** (General options 'Structured logfile', 'Format' and 'Min. level', keys logFileStructured, logFormatStructured and logLevelStructured): in addition to the unchanged logfile, the same records as JSON Lines ('json', default) or key=value pairs ('logfmt'), one appended file without date for rotation by logrotate & co (reopened if moved or removed); each record has the time in UTC ('time'), for records of a data epoch its GPS Time ('epoch'), level, component, event, mountpoint ('mp'), the event's fields, the logfile text ('msg', without the leading mountpoint and shortened where the fields carry the details, e.g. 'PPP position') and a version ('v'). Events with fields: ppp_position, ppp_epoch_failed, stream_format, stream_content (RTCM scan, with message_type and size of the received message types), stream_timeout, stream_outage_begin/end, stream_corrupted_begin/end, stream_latency; all other messages as event 'message' with the component of their origin (stream, ntrip, bce, ssr, ppp, cmb, upload, ssrqc, bnc) and a level (error: a function does not work; warning: data or configuration problem, BNC continues; info: status); messages of the BNC_DEBUG_BCE/OBS/CMB defines with level 'debug' and component bce/obs/cmb. Test case log_structured and check tests/tools/logcheck.py
+- ADDED: **Structured logfile** (General options 'Structured logfile', 'Format' and 'Min. level', keys logFileStructured, logFormatStructured and logLevelStructured): in addition to the unchanged logfile, the same records as JSON Lines ('json', default) or key=value pairs ('logfmt'), one appended file without date for rotation by logrotate & co (reopened if moved or removed); each record has the time in UTC ('time'), for records of a data epoch its GPS Time ('epoch'), level, component, event, mountpoint ('mp'), the event's fields, the logfile text ('msg', without the leading mountpoint and trailing blanks, shortened where the fields carry the details, e.g. 'PPP position') and a version ('v'). Events with fields: ppp_position, ppp_epoch_failed, stream_format, stream_content (RTCM scan, with message_type and size of the received message types), stream_timeout, stream_outage_begin/end, stream_corrupted_begin/end, stream_latency; all other messages as event 'message' with the component of their origin (stream, ntrip, bce, ssr, ppp, cmb, upload, ssrqc, bnc) and a level (error: a function does not work; warning: data or configuration problem, BNC continues; info: status); messages of the BNC_DEBUG_BCE/OBS/CMB defines with level 'debug' and component bce/obs/cmb. Test case log_structured and check tests/tools/logcheck.py
 - FIXED: Build with BNC_DEBUG_BCE: undeclared message number in the debug messages of the Galileo ephemeris decoder
 - ADDED: **epoch/velocity fields to PPP coordinates file to allow BNC to propagate the reference coordinates** related to a dedicated reference epoch to the current processing epoch internally; coordinates can also be given as latitude/longitude/height

trunk/BNC/src/bncgetthread.cpp

@@ -840 +840 @@
         t_logRecord rec(t_logRecord::info, "stream", "stream_content", _staID,
                         _staID + ": Received message type " + type.toLatin1() +  size.toLatin1());
-        rec.setStructuredMsg("Received message");
+        rec.setStructuredMsg("Received RTCM message");
         rec.add("message_type", int(decoder()->_typeList[ii]._type));
         if (decoder()->_typeList[ii]._size) {

trunk/BNC/src/bnchelp.html

@@ -2818 +2818 @@
     <li>'mp': the mountpoint of the stream or the PPP station, if known,</li>
     <li>the fields of the event,</li>
-    <li>'msg': the text of the record in the logfile, without the leading mountpoint ('&lt;mp&gt;: ') and shortened
-        for events whose fields carry the details: 'PPP position' (ppp_position), 'Received message' (message types
-        of the RTCM scan), 'Mean latency' (stream_latency), 'Failure threshold exceeded, outage' / 'corrupted' and
+    <li>'msg': the text of the record in the logfile, without the leading mountpoint ('&lt;mp&gt;: ') and trailing blanks, shortened
+        for events whose fields carry the details: 'PPP position' (ppp_position), 'Received RTCM message' (message types
+        of the RTCM scan), 'Latency' (stream_latency), 'Failure threshold exceeded, outage' / 'corrupted' and
         'Recovery threshold exceeded, outage ended' / 'corruption ended' (stream_outage_*, stream_corrupted_*),</li>
     <li>'v': the version of the record layout, currently 1.</li>

trunk/BNC/src/bnclog.cpp

@@ -275 +275 @@
     msg.remove(0, _stream.size() + 2);
   }
+  // no trailing blanks (e.g. RTCM scan: "Receiver Serial Number: 1831551 ")
+  while (msg.endsWith(' ')) {
+    msg.chop(1);
+  }
 
   QVector<t_field> all;

trunk/BNC/src/latencychecker.cpp

@@ -567 +567 @@
   }
   t_logRecord rec(t_logRecord::info, "stream", "stream_latency", _staID, text.toLatin1());
-  rec.setStructuredMsg("Mean latency");
+  rec.setStructuredMsg("Latency");
   rec.add("type", l._type)
      .add("mean", mean, 2)

trunk/BNC/tests/tools/logcheck.py

@@ -9 +9 @@
 'epoch' after 'time', GPS Time without 'Z'), that the records belong line by
 line to the classic logfile ('msg' is the classic text without the "<mp>: "
-prefix, shortened for events whose fields carry the details, see matches();
+prefix and trailing blanks, shortened for events whose fields carry the details, see matches();
 classic lines without their "yy-mm-dd hh:mm:ss " prefix; the start line is
 skipped, it goes to the file of the current date) and that 'time' (UTC) is the
@@ -77 +77 @@
 }
 
+SHORT_MSG = {"ppp_position": "PPP position", "stream_latency": "Latency"}
+
 
 def matches(r, line):
@@ -84 +86 @@
     mp, msg, ev = r.get("mp"), r.get("msg"), r.get("event")
     if ev in SHORT:
-        return line.startswith(SHORT[ev](r))
+        return msg == SHORT_MSG[ev] and line.startswith(SHORT[ev](r))
     if ev == "stream_content" and "message_type" in r:
         size = " (size %d)" % r["size"] if "size" in r else " "
-        return msg == "Received message" and \
+        return msg == "Received RTCM message" and \
             line == "%s: Received message type %d%s" % (mp, r["message_type"], size)
     if mp and line.startswith(mp + ": "):
         line = line[len(mp) + 2:]
+    line = line.rstrip(" ")
     if ev.startswith("stream_outage_") or ev.startswith("stream_corrupted_"):
         return line.startswith(msg + " ")