201 lines
13 KiB
Markdown
201 lines
13 KiB
Markdown
# Architecture
|
||
|
||
## Overview
|
||
|
||
The program rotates a pan gimbal carrying up to four cameras, periodically stops/points it, triggers the
|
||
cameras, compresses frames to JPEG XL, writes them to disk, and announces each capture over MQTT. Remote
|
||
operators can override behaviour over MQTT.
|
||
|
||
The design separates **policy** (the control logic) from **mechanism** (the I/O to hardware/broker):
|
||
|
||
- **`fgc_core`** — an SDK-independent static library: typed configuration, path resolution, logging, the
|
||
telemetry/command parsers, and the `CaptureScheduler` (control state machine). Depends on nothing
|
||
proprietary, so it builds and unit-tests anywhere.
|
||
- **Five interfaces** abstract the outside world, each with a real and a mock/null implementation:
|
||
|
||
| Interface | Real | Mock / Null |
|
||
|-----------|------|-------------|
|
||
| `IMotorController` | `SerialMotorController` (Boost.Asio) | `MockMotorController` (simulated sweep) |
|
||
| `IControlChannel` | `MqttControlChannel` (Paho) | `NullControlChannel` (no broker) |
|
||
| `ICameraSource` | `VimbaCameraSource` (Vimba X) | `MockCameraSource` (synthetic frames) |
|
||
| `IImuSource` | `MtiImuSource` (Xsens MTi, Boost.Asio) | `MockImuSource` (synthetic orientation) |
|
||
| `IEnvSensor` | `Sht41EnvSensor` (SHT41 over I2C) | `MockEnvSensor` (synthetic temp/humidity) |
|
||
|
||
Both sensors are optional (`[Features] enable_imu` / `enable_env`) and feed the Sensors panel: the IMU
|
||
supplies orientation and the IMU-referenced `gimbal calib`, the SHT41 the ambient temperature and
|
||
humidity (also published on the MQTT `Env` topic). Long-running operations (`gimbal calib` on a worker thread, `gimbal diag` captured from
|
||
the firmware) publish progress + results into the `UiSnapshot` activity strip, polled on the control
|
||
thread so all geometry/state mutation stays single-threaded.
|
||
|
||
`Application` picks real vs mock from config + CLI, wires everything to the `ImagePipeline` and
|
||
`CaptureScheduler`, and runs the loop. Selecting mocks lets the whole system run with **no hardware or broker**.
|
||
|
||
```
|
||
┌──────────────── fgc_core (no SDKs) ───────────────┐
|
||
main.cpp ──► Application ──► CaptureScheduler ──► (interfaces below) │
|
||
│ Config · Logger · Paths · TelemetryParser · CommandParser│
|
||
└───────────────────────────────────────────────────────────┘
|
||
│ builds + owns
|
||
┌────────────────┼───────────────────────────┬──────────────────────┐
|
||
IMotorController IControlChannel ICameraSource ImagePipeline
|
||
Serial/Mock Mqtt/Null Vimba/Mock ◄── frames ──┘ encode→.jxl→CamEvent
|
||
│ │ │
|
||
serial MQTT broker cameras
|
||
```
|
||
|
||
## Threading model
|
||
|
||
| Thread | Where | Role |
|
||
|--------|-------|------|
|
||
| Main / control loop | `Application::run` | 10 ms tick: drain UI commands, `scheduler.tick()`, publish a `UiSnapshot` |
|
||
| UI input (headless) | `HeadlessUi` | reads stdin lines into the command sink |
|
||
| UI render + input (TUI) | `TuiUi` | FTXUI event loop + 10 Hz refresher; pulls snapshots, pushes commands |
|
||
| Serial I/O | `SerialMotorController` | Boost.Asio `io_context`; async read-until parses telemetry |
|
||
| Image worker | `ImagePipeline` | drains the frame queue: rotate → encode → write → publish |
|
||
| MQTT client | Paho (internal) | delivers callbacks; auto-reconnects (no busy-wait loop) |
|
||
| Camera acquisition | Vimba X (internal) | delivers frames via the observer; the gate's shoot/judge/retry loop runs on the **control thread**, so a slow waypoint delays the tick rather than racing it |
|
||
|
||
Shared state is mutex-guarded: latest `MotorTelemetry` (serial), `ControlCommand` (channel), the frame queue
|
||
(pipeline), the console command queue, and the latest `UiSnapshot`. The `CaptureScheduler` runs only on the
|
||
main thread; the UI is a pure observer + command source (it copies a snapshot to render and pushes command
|
||
strings back through the same queue the console uses, so it never touches live logic objects). In TUI mode
|
||
all `Logger` output is diverted to an on-screen pane via a `Logger::setSink` callback so the screen is never
|
||
corrupted.
|
||
|
||
## Data flow
|
||
|
||
1. **Startup** — `main()` parses CLI, resolves + loads config, constructs `Application`, which builds the
|
||
motor/channel/camera (real or mock), the `ImagePipeline`, and the `CaptureScheduler`.
|
||
2. **Telemetry** — the real motor controller streams firmware `ST Y:...[ P:...]` lines; `parseTelemetryLine`
|
||
turns each into a per-axis `MotorTelemetry` snapshot (state + encoder counts + flags). The mock synthesizes
|
||
one. Encoder counts are mapped to/from degrees by `Geometry` (`[Motor]` calibration).
|
||
3. **Control input** — `IControlChannel::poll()` returns the latest `ControlCommand` (control code + target
|
||
heading), clearing its "available" flags so each update is acted on once.
|
||
4. **Capture cycle** (`CaptureScheduler::tick`, per 10 ms) — a move → settle → trigger machine:
|
||
- ControlCode 0: `MOVE <yaw>,<pitch>` to the next `ScanGrid` waypoint (ping-pong).
|
||
- ControlCode 1: `MOVE` yaw to `target_HDG` (pitch held), converted to counts via `Geometry`.
|
||
- Trigger the cameras once **both axes report standstill at the target**, then advance the grid.
|
||
5. **Acquisition + quality gate** (`GatedCameraSource`) — `trigger()` does not simply take a picture. It
|
||
seeds exposure/gain from the per-angle store, software-triggers **one** frame, measures it, corrects and
|
||
re-shoots until it passes or the budget runs out. See *Image capture* below.
|
||
6. **Frame handling** — the accepted `Frame` goes to the callback, which `submit()`s it to the
|
||
`ImagePipeline`. The worker rotates it 90° CCW, encodes JPEG XL (or copies the demo image), writes
|
||
`<output_dir>/<label>/<unix_ms>.jxl`, and publishes a `CamEvent` (yaw + pitch from the encoders).
|
||
|
||
## Capture state machine
|
||
|
||
```
|
||
interval elapsed AND capture active AND not already moving
|
||
│
|
||
▼
|
||
┌──────────────────────┐ ControlCode 0 → next ScanGrid (yaw,pitch)
|
||
│ MOVE <yaw>,<pitch> │ ControlCode 1 → target_HDG (pitch held)
|
||
└──────────┬───────────┘ deg→counts via Geometry; set moving, reset timer
|
||
│ (both axes standstill AND |xenc − target| ≤ tol)
|
||
▼
|
||
┌──────────────────────┐
|
||
│ camera.trigger() │ on success: clear moving, advance grid (ControlCode 0)
|
||
└──────────────────────┘ on failure: retry next tick, do NOT advance
|
||
```
|
||
|
||
Triggering only at the settled target replaces the original's trigger-while-moving behaviour (see
|
||
[known-issues.md](known-issues.md) #7). The scheduler is unit-tested with mock doubles and an injected clock
|
||
([tests/test_scheduler.cpp](../tests/test_scheduler.cpp)).
|
||
|
||
Note the scheduler knows nothing about image quality. It asks for an image and advances when it gets one;
|
||
everything below is an implementation detail of the camera source. That is what keeps it testable.
|
||
|
||
## Image capture
|
||
|
||
### Why not just take a picture
|
||
|
||
The first implementation streamed continuously at 1 fps and kept whichever frame arrived most recently
|
||
("paced free-run + keep-latest"). Two problems followed. The frame saved at a waypoint was captured at some
|
||
unknown moment *before* the axes finished settling, and its exposure had been metered for wherever the camera
|
||
happened to be pointing at the time — so brightness swung wildly between waypoints as the horizon changed.
|
||
And nothing checked that the saved image was usable at all.
|
||
|
||
The camera's own continuous auto-exposure cannot fix this: it needs **several frames to converge** every time
|
||
the scene changes, and a scan changes scene at every waypoint. Waiting for it costs exactly the time the scan
|
||
is trying to save.
|
||
|
||
### The current model
|
||
|
||
One deliberate frame per attempt, judged before it is kept:
|
||
|
||
```
|
||
seed exposure/gain from the per-angle store
|
||
│
|
||
▼
|
||
┌──────────────────────────┐
|
||
│ setExposure / setGain │
|
||
│ TriggerSoftware → 1 frame│ attempt 1..max_attempts
|
||
└────────────┬─────────────┘
|
||
▼
|
||
┌──────────────────────────┐ mean luma, clipped %, dark %, sharpness
|
||
│ analyzeFrame (metrics) │ + gain read back from the camera
|
||
└────────────┬─────────────┘
|
||
▼
|
||
┌──────────────────────────┐ accept → deliver
|
||
│ evaluate (verdict) │ reject → corrected settings, shoot again
|
||
└──────────────────────────┘ budget spent → deliver BEST attempt, flag degraded
|
||
```
|
||
|
||
Layering, all behind `ICameraSource` so the scheduler is unaffected:
|
||
|
||
| Piece | Role |
|
||
|---|---|
|
||
| [`ImageQuality`](../include/fgc/ImageQuality.h) | Pure metrics over the raw buffer. No OpenCV, so it lives in `fgc_core` and is testable on synthetic frames |
|
||
| [`ExposurePolicy`](../include/fgc/ExposurePolicy.h) | Pure control law: verdict + corrected settings + a score for ranking attempts |
|
||
| [`ExposureStore`](../include/fgc/ExposureStore.h) | Per-angle memory of what worked, persisted as CSV |
|
||
| [`GatedCameraSource`](../include/fgc/GatedCameraSource.h) | Decorator wiring the three around a real camera |
|
||
|
||
Because the gate depends only on the interface plus pure modules, the whole retry loop is unit-tested against
|
||
a scripted fake camera — no hardware, no OpenCV ([tests/test_gatedcamera.cpp](../tests/test_gatedcamera.cpp)).
|
||
|
||
### The reasoning behind each rule
|
||
|
||
**Clipping outranks brightness.** A bright sky can saturate while the mean still looks perfectly reasonable,
|
||
and blown highlights are unrecoverable — a slightly dark frame is not. So clipping is checked first and
|
||
forces exposure down regardless of the mean.
|
||
|
||
**Exposure before gain when brightening; gain before exposure when darkening.** Gain buys brightness with
|
||
noise, so it is the last resort on the way up and the first thing surrendered on the way down.
|
||
|
||
**The exposure ceiling is the motion-blur budget.** A long exposure on a wind-loaded tower is itself a blur
|
||
source, so `[Camera] exposure_max_us` bounds both. This couples the two checks: the cure for darkness cannot
|
||
be allowed to cause blur. It matters more with a long lens, where angular blur scales with focal length — a
|
||
telephoto needs *more* light but tolerates *less* exposure time, and the resolution of that tension is gain
|
||
(and its noise), not time.
|
||
|
||
**Blur is relative, never absolute.** A foggy or featureless horizon is legitimately low-detail; a fixed
|
||
sharpness threshold would retry forever on it. Each frame is compared against *the same angle's own* recent
|
||
sharpness, which is scene-independent: fog stays fog, but a gust or an early trigger shows as a sudden
|
||
collapse. Blur never re-meters — reshooting is the fix, not a different exposure.
|
||
|
||
**Pinned at the limits means accept, not retry.** When the proposed correction would not move the camera
|
||
(exposure and gain both at their caps), the verdict is `saturated` and the frame is taken: further attempts
|
||
could not do better, and the budget is better spent elsewhere.
|
||
|
||
**A waypoint is never lost.** If the budget runs out, the best-scoring attempt is delivered anyway and marked
|
||
`degraded` — in the log, in the TUI `c` panel, and on the MQTT `CamEvent` — so coverage stays complete while
|
||
a degraded frame is never mistaken for a good one.
|
||
|
||
**Per-angle memory beats the camera's auto.** Accepted settings are stored per quantised heading, so the next
|
||
visit starts from a value known to work there instead of re-converging from scratch. Seeding falls through
|
||
three tiers: this angle's fresh entry → the last settings accepted *anywhere* (outdoor light drifts, so a
|
||
recent reading one heading over predicts current light better than an hour-old reading at this exact heading)
|
||
→ the configured defaults. Keying by angle rather than waypoint index means it survives grid edits and also
|
||
serves ControlCode 1 directed moves.
|
||
|
||
**White balance stays on the camera.** It is far less time-critical than exposure and is correctable in post,
|
||
unlike a blown or blurred frame.
|
||
|
||
Full parameter reference: `[Capture]` in [configuration.md](configuration.md).
|
||
|
||
## Why this shape
|
||
|
||
Decoupling the control logic from the SDKs makes the core testable and the binary buildable/runnable without
|
||
proprietary dependencies, while preserving the original real-time behaviour. Persistence remains deliberately
|
||
limited to image files plus fire-and-forget MQTT.
|