fwt_software/docs/architecture.md

Architecture

Overview

The program rotates a pan gimbal carrying up to four cameras, periodically stops/points it, triggers the cameras, compresses frames to JPEG XL, writes them to disk, and announces each capture over MQTT. Remote operators can override behaviour over MQTT.

The design separates policy (the control logic) from mechanism (the I/O to hardware/broker):

• fgc_core — an SDK-independent static library: typed configuration, path resolution, logging, the telemetry/command parsers, and the CaptureScheduler (control state machine). Depends on nothing proprietary, so it builds and unit-tests anywhere.
• Five interfaces abstract the outside world, each with a real and a mock/null implementation:

Interface	Real	Mock / Null
IMotorController	SerialMotorController (Boost.Asio)	MockMotorController (simulated sweep)
IControlChannel	MqttControlChannel (Paho)	NullControlChannel (no broker)
ICameraSource	VimbaCameraSource (Vimba X)	MockCameraSource (synthetic frames)
IImuSource	MtiImuSource (Xsens MTi, Boost.Asio)	MockImuSource (synthetic orientation)
IEnvSensor	Sht41EnvSensor (SHT41 over I2C)	MockEnvSensor (synthetic temp/humidity)

Both sensors are optional ([Features] enable_imu / enable_env) and feed the Sensors panel: the IMU supplies orientation and the IMU-referenced gimbal calib, the SHT41 the ambient temperature and humidity (also published on the MQTT Env topic). Long-running operations (gimbal calib on a worker thread, gimbal diag captured from the firmware) publish progress + results into the UiSnapshot activity strip, polled on the control thread so all geometry/state mutation stays single-threaded.

Application picks real vs mock from config + CLI, wires everything to the ImagePipeline and CaptureScheduler, and runs the loop. Selecting mocks lets the whole system run with no hardware or broker.

                         ┌──────────────── fgc_core (no SDKs) ───────────────┐
   main.cpp ──► Application ──► CaptureScheduler ──► (interfaces below)       │
                  │  Config · Logger · Paths · TelemetryParser · CommandParser│
                  └───────────────────────────────────────────────────────────┘
                         │ builds + owns
        ┌────────────────┼───────────────────────────┬──────────────────────┐
   IMotorController   IControlChannel             ICameraSource          ImagePipeline
   Serial/Mock        Mqtt/Null                   Vimba/Mock      ◄── frames ──┘ encode→.jxl→CamEvent
        │                  │                           │
     serial            MQTT broker                  cameras

Threading model

Thread	Where	Role
Main / control loop	Application::run	10 ms tick: drain UI commands, scheduler.tick(), publish a UiSnapshot
UI input (headless)	HeadlessUi	reads stdin lines into the command sink
UI render + input (TUI)	TuiUi	FTXUI event loop + 10 Hz refresher; pulls snapshots, pushes commands
Serial I/O	SerialMotorController	Boost.Asio io_context; async read-until parses telemetry
Image worker	ImagePipeline	drains the frame queue: rotate → encode → write → publish
MQTT client	Paho (internal)	delivers callbacks; auto-reconnects (no busy-wait loop)
Camera acquisition	Vimba X (internal)	delivers frames via the observer; the gate's shoot/judge/retry loop runs on the control thread, so a slow waypoint delays the tick rather than racing it

Shared state is mutex-guarded: latest MotorTelemetry (serial), ControlCommand (channel), the frame queue (pipeline), the console command queue, and the latest UiSnapshot. The CaptureScheduler runs only on the main thread; the UI is a pure observer + command source (it copies a snapshot to render and pushes command strings back through the same queue the console uses, so it never touches live logic objects). In TUI mode all Logger output is diverted to an on-screen pane via a Logger::setSink callback so the screen is never corrupted.

Data flow

1. Startup — main() parses CLI, resolves + loads config, constructs Application, which builds the motor/channel/camera (real or mock), the ImagePipeline, and the CaptureScheduler.
2. Telemetry — the real motor controller streams firmware ST Y:...[ P:...] lines; parseTelemetryLine turns each into a per-axis MotorTelemetry snapshot (state + encoder counts + flags). The mock synthesizes one. Encoder counts are mapped to/from degrees by Geometry ([Motor] calibration).
3. Control input — IControlChannel::poll() returns the latest ControlCommand (control code + target heading), clearing its "available" flags so each update is acted on once.
4. Capture cycle (CaptureScheduler::tick, per 10 ms) — a move → settle → trigger machine:
   • ControlCode 0: MOVE <yaw>,<pitch> to the next ScanGrid waypoint (ping-pong).
   • ControlCode 1: MOVE yaw to target_HDG (pitch held), converted to counts via Geometry.
   • Trigger the cameras once both axes report standstill at the target, then advance the grid.
5. Acquisition + quality gate (GatedCameraSource) — trigger() does not simply take a picture. It seeds exposure/gain from the per-angle store, software-triggers one frame, measures it, corrects and re-shoots until it passes or the budget runs out. See Image capture below.
6. Frame handling — the accepted Frame goes to the callback, which submit()s it to the ImagePipeline. The worker rotates it 90° CCW, encodes JPEG XL (or copies the demo image), writes <output_dir>/<label>/<unix_ms>.jxl, and publishes a CamEvent (yaw + pitch from the encoders).

Capture state machine

        interval elapsed AND capture active AND not already moving
              │
              ▼
   ┌──────────────────────┐  ControlCode 0 → next ScanGrid (yaw,pitch)
   │ MOVE <yaw>,<pitch>   │  ControlCode 1 → target_HDG (pitch held)
   └──────────┬───────────┘  deg→counts via Geometry; set moving, reset timer
              │ (both axes standstill AND |xenc − target| ≤ tol)
              ▼
   ┌──────────────────────┐
   │  camera.trigger()    │  on success: clear moving, advance grid (ControlCode 0)
   └──────────────────────┘  on failure: retry next tick, do NOT advance

Triggering only at the settled target replaces the original's trigger-while-moving behaviour (see known-issues.md #7). The scheduler is unit-tested with mock doubles and an injected clock (tests/test_scheduler.cpp).

Note the scheduler knows nothing about image quality. It asks for an image and advances when it gets one; everything below is an implementation detail of the camera source. That is what keeps it testable.

Image capture

Why not just take a picture

The first implementation streamed continuously at 1 fps and kept whichever frame arrived most recently ("paced free-run + keep-latest"). Two problems followed. The frame saved at a waypoint was captured at some unknown moment before the axes finished settling, and its exposure had been metered for wherever the camera happened to be pointing at the time — so brightness swung wildly between waypoints as the horizon changed. And nothing checked that the saved image was usable at all.

The camera's own continuous auto-exposure cannot fix this: it needs several frames to converge every time the scene changes, and a scan changes scene at every waypoint. Waiting for it costs exactly the time the scan is trying to save.

The current model

One deliberate frame per attempt, judged before it is kept:

   seed exposure/gain from the per-angle store
              │
              ▼
   ┌──────────────────────────┐
   │ setExposure / setGain    │
   │ TriggerSoftware  → 1 frame│   attempt 1..max_attempts
   └────────────┬─────────────┘
                ▼
   ┌──────────────────────────┐   mean luma, clipped %, dark %, sharpness
   │ analyzeFrame  (metrics)  │   + gain read back from the camera
   └────────────┬─────────────┘
                ▼
   ┌──────────────────────────┐   accept → deliver
   │ evaluate  (verdict)      │   reject → corrected settings, shoot again
   └──────────────────────────┘   budget spent → deliver BEST attempt, flag degraded

Layering, all behind ICameraSource so the scheduler is unaffected:

Piece	Role
ImageQuality	Pure metrics over the raw buffer. No OpenCV, so it lives in fgc_core and is testable on synthetic frames
ExposurePolicy	Pure control law: verdict + corrected settings + a score for ranking attempts
ExposureStore	Per-angle memory of what worked, persisted as CSV
GatedCameraSource	Decorator wiring the three around a real camera

Because the gate depends only on the interface plus pure modules, the whole retry loop is unit-tested against a scripted fake camera — no hardware, no OpenCV (tests/test_gatedcamera.cpp).

The reasoning behind each rule

Clipping outranks brightness. A bright sky can saturate while the mean still looks perfectly reasonable, and blown highlights are unrecoverable — a slightly dark frame is not. So clipping is checked first and forces exposure down regardless of the mean.

Exposure before gain when brightening; gain before exposure when darkening. Gain buys brightness with noise, so it is the last resort on the way up and the first thing surrendered on the way down.

The exposure ceiling is the motion-blur budget. A long exposure on a wind-loaded tower is itself a blur source, so [Camera] exposure_max_us bounds both. This couples the two checks: the cure for darkness cannot be allowed to cause blur. It matters more with a long lens, where angular blur scales with focal length — a telephoto needs more light but tolerates less exposure time, and the resolution of that tension is gain (and its noise), not time.

Blur is relative, never absolute. A foggy or featureless horizon is legitimately low-detail; a fixed sharpness threshold would retry forever on it. Each frame is compared against the same angle's own recent sharpness, which is scene-independent: fog stays fog, but a gust or an early trigger shows as a sudden collapse. Blur never re-meters — reshooting is the fix, not a different exposure.

Pinned at the limits means accept, not retry. When the proposed correction would not move the camera (exposure and gain both at their caps), the verdict is saturated and the frame is taken: further attempts could not do better, and the budget is better spent elsewhere.

A waypoint is never lost. If the budget runs out, the best-scoring attempt is delivered anyway and marked degraded — in the log, in the TUI c panel, and on the MQTT CamEvent — so coverage stays complete while a degraded frame is never mistaken for a good one.

Per-angle memory beats the camera's auto. Accepted settings are stored per quantised heading, so the next visit starts from a value known to work there instead of re-converging from scratch. Seeding falls through three tiers: this angle's fresh entry → the last settings accepted anywhere (outdoor light drifts, so a recent reading one heading over predicts current light better than an hour-old reading at this exact heading) → the configured defaults. Keying by angle rather than waypoint index means it survives grid edits and also serves ControlCode 1 directed moves.

White balance stays on the camera. It is far less time-critical than exposure and is correctable in post, unlike a blown or blurred frame.

Full parameter reference: [Capture] in configuration.md.

Why this shape

Decoupling the control logic from the SDKs makes the core testable and the binary buildable/runnable without proprietary dependencies, while preserving the original real-time behaviour. Persistence remains deliberately limited to image files plus fire-and-forget MQTT.