fwt_software/docs/architecture.md

Architecture

Overview

The program rotates a pan gimbal carrying up to four cameras, periodically stops/points it, triggers the cameras, compresses frames to JPEG XL, writes them to disk, and announces each capture over MQTT. Remote operators can override behaviour over MQTT.

The design separates policy (the control logic) from mechanism (the I/O to hardware/broker):

• fgc_core — an SDK-independent static library: typed configuration, path resolution, logging, the telemetry/command parsers, and the CaptureScheduler (control state machine). Depends on nothing proprietary, so it builds and unit-tests anywhere.
• Five interfaces abstract the outside world, each with a real and a mock/null implementation:

Interface	Real	Mock / Null
IMotorController	SerialMotorController (Boost.Asio)	MockMotorController (simulated sweep)
IControlChannel	MqttControlChannel (Paho)	NullControlChannel (no broker)
ICameraSource	VimbaCameraSource (Vimba X)	MockCameraSource (synthetic frames)
IImuSource	MtiImuSource (Xsens MTi, Boost.Asio)	MockImuSource (synthetic orientation)
IEnvSensor	Sht41EnvSensor (SHT41 over I2C)	MockEnvSensor (synthetic temp/humidity)

Both sensors are optional ([Features] enable_imu / enable_env) and feed the Sensors panel: the IMU supplies orientation and the IMU-referenced gimbal calib, the SHT41 the ambient temperature and humidity (also published on the MQTT Env topic). Long-running operations (gimbal calib on a worker thread, gimbal diag captured from the firmware) publish progress + results into the UiSnapshot activity strip, polled on the control thread so all geometry/state mutation stays single-threaded.

Application picks real vs mock from config + CLI, wires everything to the ImagePipeline and CaptureScheduler, and runs the loop. Selecting mocks lets the whole system run with no hardware or broker.

                         ┌──────────────── fgc_core (no SDKs) ───────────────┐
   main.cpp ──► Application ──► CaptureScheduler ──► (interfaces below)       │
                  │  Config · Logger · Paths · TelemetryParser · CommandParser│
                  └───────────────────────────────────────────────────────────┘
                         │ builds + owns
        ┌────────────────┼───────────────────────────┬──────────────────────┐
   IMotorController   IControlChannel             ICameraSource          ImagePipeline
   Serial/Mock        Mqtt/Null                   Vimba/Mock      ◄── frames ──┘ encode→.jxl→CamEvent
        │                  │                           │
     serial            MQTT broker                  cameras

Threading model

Thread	Where	Role
Main / control loop	Application::run	10 ms tick: drain UI commands, scheduler.tick(), publish a UiSnapshot
UI input (headless)	HeadlessUi	reads stdin lines into the command sink
UI render + input (TUI)	TuiUi	FTXUI event loop + 10 Hz refresher; pulls snapshots, pushes commands
Serial I/O	SerialMotorController	Boost.Asio io_context; async read-until parses telemetry
Image worker	ImagePipeline	drains the frame queue: rotate → encode → write → publish
MQTT client	Paho (internal)	delivers callbacks; auto-reconnects (no busy-wait loop)
Camera acquisition	Vimba X (internal)	free-runs at the mode's frame rate and delivers frames via the observer, which keeps the latest complete one

Shared state is mutex-guarded: latest MotorTelemetry (serial), ControlCommand (channel), the frame queue (pipeline), the console command queue, and the latest UiSnapshot. The CaptureScheduler runs only on the main thread; the UI is a pure observer + command source (it copies a snapshot to render and pushes command strings back through the same queue the console uses, so it never touches live logic objects). In TUI mode all Logger output is diverted to an on-screen pane via a Logger::setSink callback so the screen is never corrupted.

Data flow

1. Startup — main() parses CLI, resolves + loads config, constructs Application, which builds the motor/channel/camera (real or mock), the ImagePipeline, and the CaptureScheduler.
2. Telemetry — the real motor controller streams firmware ST Y:...[ P:...] lines; parseTelemetryLine turns each into a per-axis MotorTelemetry snapshot (state + encoder counts + flags). The mock synthesizes one. Encoder counts are mapped to/from degrees by Geometry ([Motor] calibration).
3. Control input — IControlChannel::poll() returns the latest ControlCommand (control code + target heading), clearing its "available" flags so each update is acted on once.
4. Capture cycle (CaptureScheduler::tick, per 10 ms) — a move → settle → wait → keep machine:
   • ControlCode 0: MOVE <yaw>,<pitch> to the next ScanGrid waypoint (front to back, then wrap).
   • ControlCode 1: MOVE yaw to target_HDG (pitch held), converted to counts via Geometry.
   • Once both axes report standstill at the target, wait settle_wait_ms, then keep the newest frame and advance the grid.
5. Acquisition — the camera free-runs at the active mode's AcquisitionFrameRate with its own auto-exposure tracking the light; trigger() hands over the freshest complete frame. See Image capture below.
6. Frame handling — the kept Frame goes to the callback, which submit()s it to the ImagePipeline. The worker rotates it by Camera.rotation_deg, encodes JPEG XL (or copies the demo image), writes <output_dir>/<label>/<unix_ms>.jxl, and publishes a CamEvent (yaw + pitch from the encoders).

Capture state machine

        interval elapsed AND capture active AND not already moving
              │
              ▼
   ┌──────────────────────┐  ControlCode 0 → next ScanGrid (yaw,pitch)
   │ MOVE <yaw>,<pitch>   │  ControlCode 1 → target_HDG (pitch held)
   └──────────┬───────────┘  deg→counts via Geometry; set moving, reset timer
              │ (both axes standstill AND |xenc − target| ≤ tol)
              ▼
   ┌──────────────────────┐  the camera keeps streaming throughout; this pause is
   │ wait settle_wait_ms  │  what makes the kept frame one exposed at rest
   └──────────┬───────────┘
              ▼
   ┌──────────────────────┐  on success: clear moving, advance grid (ControlCode 0)
   │  camera.trigger()    │  no fresh frame: retry next tick
   └──────────────────────┘  still none after frame_timeout_ms: warn, skip, advance

Triggering only at the settled target replaces the original's trigger-while-moving behaviour (see known-issues.md #7). The scheduler is unit-tested with mock doubles and an injected clock (tests/test_scheduler.cpp).

Note the scheduler knows nothing about image quality. It asks for an image and advances when it gets one; everything below is an implementation detail of the camera source. That is what keeps it testable.

Image capture

The model: free-run, then hold still

The camera runs continuously at the mode's frame rate with its own auto-exposure and auto-gain in Continuous. Almost every frame it produces is thrown away. Their job is not to be saved — it is to keep the camera's exposure controller converged on the current light, so that the one frame we do keep is already correct without anyone waiting for it.

At each waypoint the scheduler therefore:

1. moves, and waits for both axes to report standstill at the target;
2. waits a further settle_wait_ms (1 s in standard, 2.5 s in night);
3. keeps the newest complete frame.

The wait is the whole mechanism. It guarantees the kept frame was exposed with the gimbal at rest (no slew blur) and after the auto-exposure has seen this waypoint's light for several frames. It also sets the floor on mode frame rate: a mode must produce comfortably more than one frame inside its own settle wait, which is why night at 1 Hz waits 2.5 s where standard at 10 Hz waits 1 s.

If no fresh frame has arrived by frame_timeout_ms past the wait, the waypoint is logged and skipped. A stalled camera costs single images, never the whole sweep.

Camera modes

Everything on the camera comes from a mode: one INI file per mode, <mode_dir>/<name>.ini, holding an ordered list of Vimba X features written verbatim at start-up and on every switch. There is no C++ struct of camera settings and no translation layer — a feature the camera exposes can be set by adding a line, and Feature::GetDataType() tells the applier whether to write it as an int, float, enum, string or bool.

Two properties of the file are load-bearing, and both are why the mode loader does not reuse the ordinary config parser (which flattens into a std::map):

• Order is preserved. GenICam selectors are stateful: BinningHorizontal means nothing until BinningSelector is set, and binning must precede Width/Height because it changes their maxima.
• Repeated keys are kept. Writing both white-balance ratios requires BalanceRatioSelector = Red, BalanceRatio = …, BalanceRatioSelector = Blue, BalanceRatio = ….

A feature this camera does not expose is a warning, not a fatal error — that warning is the feedback loop for a hand-edited mode file. A structural error (unknown section, unknown [Mode] key, a mode with no features) is fatal, and every mode file in the directory is parsed at start-up, so a typo in a mode nobody uses today still fails at launch rather than the first night someone switches to it.

Ships with standard (10 Hz daytime) and night (1 Hz, long exposures, high gain ceiling). Switch live with set camera mode <name>; that stops acquisition, rewrites the features and restarts, because binning, ROI and pixel format cannot be changed on a running stream.

Why not judge each frame instead

An earlier design software-triggered one frame per waypoint, measured it (mean luma, clipping, sharpness), corrected exposure and re-shot until it passed, seeded from a per-angle exposure store. It was removed. It duplicated — and fought — the camera's own exposure controller, which meters continuously in hardware and does not need the scan to stop for it. Its premise was that the camera's auto-exposure needs several frames to converge and a scan changes scene at every waypoint; free-running at 10 Hz supplies those frames for free, during the slew, at no cost to scan rate.

Bandwidth: why both modes bin 2×2

At PixelFormat = RGB8 a full-resolution 4512×4512 frame is 61 MB. Under the 250 MB/s DeviceLinkThroughputLimit that caps out at ~4 fps, and even at the interface maximum only ~7 — so 10 Hz at full resolution is not reachable in RGB8 at all. (The datasheet's "21 fps at full resolution" is 450 MB/s Mono8.) With BinningHorizontal/Vertical = 2 the frame is 2256×2256 ≈ 5 MP, 15.3 MB, and 10 Hz needs ~153 MB/s — comfortably inside the cap. Binning also sums four photosites, which is worth having at night. Raising the throughput limit to 450 is not the fix: it produced incomplete frames on this host (known-issues.md).

Full parameter reference: [Camera] in configuration.md.

Why this shape

Decoupling the control logic from the SDKs makes the core testable and the binary buildable/runnable without proprietary dependencies, while preserving the original real-time behaviour. Persistence remains deliberately limited to image files plus fire-and-forget MQTT.