fwt_software/docs/architecture.md

Architecture

Overview

The program rotates a pan gimbal carrying up to four cameras, periodically stops/points it, triggers the cameras, compresses frames to JPEG XL, writes them to disk, and announces each capture over MQTT. Remote operators can override behaviour over MQTT.

The design separates policy (the control logic) from mechanism (the I/O to hardware/broker):

• fgc_core — an SDK-independent static library: typed configuration, path resolution, logging, the telemetry/command parsers, and the CaptureScheduler (control state machine). Depends on nothing proprietary, so it builds and unit-tests anywhere.
• Five interfaces abstract the outside world, each with a real and a mock/null implementation:

Interface	Real	Mock / Null
IMotorController	SerialMotorController (Boost.Asio)	MockMotorController (simulated sweep)
IControlChannel	MqttControlChannel (Paho)	NullControlChannel (no broker)
ICameraSource	VimbaCameraSource (Vimba X)	MockCameraSource (synthetic frames)
IImuSource	MtiImuSource (Xsens MTi, Boost.Asio)	MockImuSource (synthetic orientation)
IEnvSensor	Sht41EnvSensor (SHT41 over I2C)	MockEnvSensor (synthetic temp/humidity)

Both sensors are optional ([Features] enable_imu / enable_env) and feed the Sensors panel: the IMU supplies orientation and the IMU-referenced gimbal pitch setup, the SHT41 the ambient temperature and humidity. Long-running operations (gimbal pitch setup on a worker thread, gimbal diag captured from the firmware) publish progress + results into the UiSnapshot activity strip, polled on the control thread so all geometry/state mutation stays single-threaded.

Application picks real vs mock from config + CLI, wires everything to the ImagePipeline and CaptureScheduler, and runs the loop. Selecting mocks lets the whole system run with no hardware or broker.

                         ┌──────────────── fgc_core (no SDKs) ───────────────┐
   main.cpp ──► Application ──► CaptureScheduler ──► (interfaces below)       │
                  │  Config · Logger · Paths · TelemetryParser · CommandParser│
                  └───────────────────────────────────────────────────────────┘
                         │ builds + owns
        ┌────────────────┼───────────────────────────┬──────────────────────┐
   IMotorController   IControlChannel             ICameraSource          ImagePipeline
   Serial/Mock        Mqtt/Null                   Vimba/Mock      ◄── frames ──┘ encode→.jxl→CamEvent
        │                  │                           │
     serial            MQTT broker                  cameras

Threading model

Thread	Where	Role
Main / control loop	Application::run	10 ms tick: drain UI commands, scheduler.tick(), publish a UiSnapshot
UI input (headless)	HeadlessUi	reads stdin lines into the command sink
UI render + input (TUI)	TuiUi	FTXUI event loop + 10 Hz refresher; pulls snapshots, pushes commands
Serial I/O	SerialMotorController	Boost.Asio io_context; async read-until parses telemetry
Image worker	ImagePipeline	drains the frame queue: rotate → encode → write → publish
MQTT client	Paho (internal)	delivers callbacks; auto-reconnects (no busy-wait loop)
Camera acquisition	Vimba X (internal)	free-runs at the mode's frame rate and delivers frames via the observer, which keeps the latest complete one

Shared state is mutex-guarded: latest MotorTelemetry (serial), RemoteCommand queue (channel), the frame queue (pipeline), the console command queue, and the latest UiSnapshot. The CaptureScheduler runs only on the main thread; the UI is a pure observer + command source (it copies a snapshot to render and pushes command strings back through the same queue the console uses, so it never touches live logic objects). In TUI mode all Logger output is diverted to an on-screen pane via a Logger::setSink callback so the screen is never corrupted.

Data flow

1. Startup — main() parses CLI, resolves + loads config, constructs Application, which builds the motor/channel/camera (real or mock), the ImagePipeline, and the CaptureScheduler.
2. Telemetry — the real motor controller streams firmware ST Y:...[ P:...] lines; parseTelemetryLine turns each into a per-axis MotorTelemetry snapshot (state + encoder counts + flags). The mock synthesizes one. Encoder counts are mapped to/from degrees by Geometry ([Motor] calibration).
3. Mode — auto, pan or manual (OperatingMode.h), changed only by the mode command (console, dashboard keys, or MQTT Command) through Application::setMode, which publishes it on StatusCode. The same header's allowedIn() gates every command by mode.
4. Capture cycle (CaptureScheduler::tick, per 10 ms) — a move → settle → wait → keep machine:
   • auto: MOVE <yaw>,<pitch> to the next ScanGrid waypoint (front to back, then wrap).
   • pan: MOVE to the next of panHeadings() (left to right, then back to the left edge) at the pan's pitch, converted to counts via Geometry.
   • manual: nothing, bar a requested snapshot or continuous capture (capture on), which skip the move and wait only for standstill.
   • Once both axes report standstill at the target, wait settle_wait_ms, then keep the newest frame and advance the grid or pan position.
5. Acquisition — the camera free-runs at the active profile's AcquisitionFrameRate with its own auto-exposure tracking the light; trigger() hands over the freshest complete frame. See Image capture below.
6. Frame handling — the kept Frame goes to the callback, which submit()s it to the ImagePipeline. submit() samples the gimbal pose there and then — yaw from the encoder, pitch from the IMU — and the pose travels through the queue with the frame, so encode latency cannot attribute a later pose to earlier pixels. The worker rotates it by Camera.rotation_deg, encodes JPEG XL, writes <output_dir>/<label>/<unix_ms>.jxl, and publishes a CamEvent carrying that pose.

Capture state machine

        interval elapsed AND mode auto or pan AND not already moving
              │
              ▼
   ┌──────────────────────┐  auto → next ScanGrid (yaw,pitch)
   │ MOVE <yaw>,<pitch>   │  pan  → next pan heading, at the pan's pitch
   └──────────┬───────────┘  deg→counts via Geometry; set moving, reset timer
              │ (both axes standstill AND |xenc − target| ≤ tol)
              ▼
   ┌──────────────────────┐  the camera keeps streaming throughout; this pause is
   │ wait settle_wait_ms  │  what makes the kept frame one exposed at rest
   └──────────┬───────────┘
              ▼
   ┌──────────────────────┐  on success: clear moving, advance the grid / pan position
   │  camera.trigger()    │  no fresh frame: retry next tick
   └──────────────────────┘  still none after frame_timeout_ms: warn, skip, advance

Triggering only at the settled target replaces the original's trigger-while-moving behaviour (see known-issues.md #7). The scheduler is unit-tested with mock doubles and an injected clock (tests/test_scheduler.cpp).

Note the scheduler knows nothing about image quality. It asks for an image and advances when it gets one; everything below is an implementation detail of the camera source. That is what keeps it testable.

Image capture

The model: free-run, then hold still

The camera runs continuously at the mode's frame rate with its own auto-exposure and auto-gain in Continuous. Almost every frame it produces is thrown away. Their job is not to be saved — it is to keep the camera's exposure controller converged on the current light, so that the one frame we do keep is already correct without anyone waiting for it.

At each waypoint the scheduler therefore:

1. moves, and waits for both axes to report standstill at the target;
2. waits a further settle_wait_ms (1 s in standard, 2.5 s in night);
3. keeps the newest complete frame.

The wait is the whole mechanism. It guarantees the kept frame was exposed with the gimbal at rest (no slew blur) and after the auto-exposure has seen this waypoint's light for several frames. It also sets the floor on a profile's frame rate: a profile must produce comfortably more than one frame inside its own settle wait, which is why night at 1 Hz waits 2.5 s where standard at 10 Hz waits 1 s.

If no fresh frame has arrived by frame_timeout_ms past the wait, the waypoint is logged and skipped. A stalled camera costs single images, never the whole sweep.

Camera profiles

Everything on the camera comes from a profile: a settings XML exported from the VimbaX Viewer, at <profile_dir>/<name>.xml. The program hands that file to the Vimba X SDK (LoadSettings, restricted to the export's RemoteDevice block) and nothing else writes camera settings.

The reason is validity, not convenience. GenICam writability is stateful — AcquisitionFrameRate is only writable once AcquisitionFrameRateEnable is set, Width's maximum depends on binning, IntensityAutoPrecedence takes Minimizenoise but not MinimizeNoise — so a list of features applied line by line can be individually plausible and collectively wrong, and a line that fails leaves the camera quietly on its old value. The Viewer greys out invalid combinations as the operator configures; replaying its export is how that guarantee is inherited. An earlier design applied a hand-written INI feature list; it was removed rather than repaired, because the failure mode was silent.

Beside each XML the program writes <name>.txt, a generated human-readable record of the same settings. It configures nothing — it exists so a profile can be read and diffed without opening the Viewer. At start-up the XML is hashed and compared with the hash stored in the record; a mismatch regenerates the record and logs every setting that changed, so a camera behaving differently from the last run says so rather than being discovered later. Every *.xml in the directory is loaded at start-up, so a profile nobody uses today still fails at launch rather than the first night someone switches to it.

Each camera has its own [Camera.<name>] block and its own profile directory, so profiles are per camera rather than shared. Switch live with set camera profile [<camera>] <name>; that stops acquisition, reloads the XML and restarts, because binning, ROI and pixel format cannot be changed on a running stream.

Why not judge each frame instead

An earlier design software-triggered one frame per waypoint, measured it (mean luma, clipping, sharpness), corrected exposure and re-shot until it passed, seeded from a per-angle exposure store. It was removed. It duplicated — and fought — the camera's own exposure controller, which meters continuously in hardware and does not need the scan to stop for it. Its premise was that the camera's auto-exposure needs several frames to converge and a scan changes scene at every waypoint; free-running at 10 Hz supplies those frames for free, during the slew, at no cost to scan rate.

Bandwidth: why both modes bin 2×2

At PixelFormat = RGB8 a full-resolution 4512×4512 frame is 61 MB. Under the 250 MB/s DeviceLinkThroughputLimit that caps out at ~4 fps, and even at the interface maximum only ~7 — so 10 Hz at full resolution is not reachable in RGB8 at all. (The datasheet's "21 fps at full resolution" is 450 MB/s Mono8.) With BinningHorizontal/Vertical = 2 the frame is 2256×2256 ≈ 5 MP, 15.3 MB, and 10 Hz needs ~153 MB/s — comfortably inside the cap. Binning also sums four photosites, which is worth having at night. Raising the throughput limit to 450 is not the fix: it produced incomplete frames on this host (known-issues.md).

Full parameter reference: [Camera] in configuration.md.

Why this shape

Decoupling the control logic from the SDKs makes the core testable and the binary buildable/runnable without proprietary dependencies, while preserving the original real-time behaviour. Persistence remains deliberately limited to image files plus fire-and-forget MQTT.