189 lines
13 KiB
Markdown
189 lines
13 KiB
Markdown
# Architecture
|
||
|
||
## Overview
|
||
|
||
The program rotates a pan gimbal carrying up to four cameras, periodically stops/points it, triggers the
|
||
cameras, compresses frames to JPEG XL, writes them to disk, and announces each capture over MQTT. Remote
|
||
operators can override behaviour over MQTT.
|
||
|
||
The design separates **policy** (the control logic) from **mechanism** (the I/O to hardware/broker):
|
||
|
||
- **`fgc_core`** — an SDK-independent static library: typed configuration, path resolution, logging, the
|
||
telemetry/command parsers, and the `CaptureScheduler` (control state machine). Depends on nothing
|
||
proprietary, so it builds and unit-tests anywhere.
|
||
- **Five interfaces** abstract the outside world, each with a real and a mock/null implementation:
|
||
|
||
| Interface | Real | Mock / Null |
|
||
|-----------|------|-------------|
|
||
| `IMotorController` | `SerialMotorController` (Boost.Asio) | `MockMotorController` (simulated sweep) |
|
||
| `IControlChannel` | `MqttControlChannel` (Paho) | `NullControlChannel` (no broker) |
|
||
| `ICameraSource` | `VimbaCameraSource` (Vimba X) | `MockCameraSource` (synthetic frames) |
|
||
| `IImuSource` | `MtiImuSource` (Xsens MTi, Boost.Asio) | `MockImuSource` (synthetic orientation) |
|
||
| `IEnvSensor` | `Sht41EnvSensor` (SHT41 over I2C) | `MockEnvSensor` (synthetic temp/humidity) |
|
||
|
||
Both sensors are optional (`[Features] enable_imu` / `enable_env`) and feed the Sensors panel: the IMU
|
||
supplies orientation and the IMU-referenced `gimbal pitch setup`, the SHT41 the ambient temperature and
|
||
humidity. Long-running operations (`gimbal pitch setup` on a worker thread, `gimbal diag` captured from
|
||
the firmware) publish progress + results into the `UiSnapshot` activity strip, polled on the control
|
||
thread so all geometry/state mutation stays single-threaded.
|
||
|
||
`Application` picks real vs mock from config + CLI, wires everything to the `ImagePipeline` and
|
||
`CaptureScheduler`, and runs the loop. Selecting mocks lets the whole system run with **no hardware or broker**.
|
||
|
||
```
|
||
┌──────────────── fgc_core (no SDKs) ───────────────┐
|
||
main.cpp ──► Application ──► CaptureScheduler ──► (interfaces below) │
|
||
│ Config · Logger · Paths · TelemetryParser · CommandParser│
|
||
└───────────────────────────────────────────────────────────┘
|
||
│ builds + owns
|
||
┌────────────────┼───────────────────────────┬──────────────────────┐
|
||
IMotorController IControlChannel ICameraSource ImagePipeline
|
||
Serial/Mock Mqtt/Null Vimba/Mock ◄── frames ──┘ encode→.jxl→CamEvent
|
||
│ │ │
|
||
serial MQTT broker cameras
|
||
```
|
||
|
||
## Threading model
|
||
|
||
| Thread | Where | Role |
|
||
|--------|-------|------|
|
||
| Main / control loop | `Application::run` | 10 ms tick: drain UI commands, `scheduler.tick()`, publish a `UiSnapshot` |
|
||
| UI input (headless) | `HeadlessUi` | reads stdin lines into the command sink |
|
||
| UI render + input (TUI) | `TuiUi` | FTXUI event loop + 10 Hz refresher; pulls snapshots, pushes commands |
|
||
| Serial I/O | `SerialMotorController` | Boost.Asio `io_context`; async read-until parses telemetry |
|
||
| Image worker | `ImagePipeline` | drains the frame queue: rotate → encode → write → publish |
|
||
| MQTT client | Paho (internal) | delivers callbacks; auto-reconnects (no busy-wait loop) |
|
||
| Camera acquisition | Vimba X (internal) | free-runs at the mode's frame rate and delivers frames via the observer, which keeps the latest complete one |
|
||
|
||
Shared state is mutex-guarded: latest `MotorTelemetry` (serial), `ControlCommand` (channel), the frame queue
|
||
(pipeline), the console command queue, and the latest `UiSnapshot`. The `CaptureScheduler` runs only on the
|
||
main thread; the UI is a pure observer + command source (it copies a snapshot to render and pushes command
|
||
strings back through the same queue the console uses, so it never touches live logic objects). In TUI mode
|
||
all `Logger` output is diverted to an on-screen pane via a `Logger::setSink` callback so the screen is never
|
||
corrupted.
|
||
|
||
## Data flow
|
||
|
||
1. **Startup** — `main()` parses CLI, resolves + loads config, constructs `Application`, which builds the
|
||
motor/channel/camera (real or mock), the `ImagePipeline`, and the `CaptureScheduler`.
|
||
2. **Telemetry** — the real motor controller streams firmware `ST Y:...[ P:...]` lines; `parseTelemetryLine`
|
||
turns each into a per-axis `MotorTelemetry` snapshot (state + encoder counts + flags). The mock synthesizes
|
||
one. Encoder counts are mapped to/from degrees by `Geometry` (`[Motor]` calibration).
|
||
3. **Control input** — `IControlChannel::poll()` returns the latest `ControlCommand` (control code + target
|
||
heading), clearing its "available" flags so each update is acted on once.
|
||
4. **Capture cycle** (`CaptureScheduler::tick`, per 10 ms) — a move → settle → wait → keep machine:
|
||
- ControlCode 0: `MOVE <yaw>,<pitch>` to the next `ScanGrid` waypoint (front to back, then wrap).
|
||
- ControlCode 1: `MOVE` yaw to `target_HDG` (pitch held), converted to counts via `Geometry`.
|
||
- Once **both axes report standstill at the target**, wait `settle_wait_ms`, then keep the newest
|
||
frame and advance the grid.
|
||
5. **Acquisition** — the camera free-runs at the active mode's `AcquisitionFrameRate` with its own
|
||
auto-exposure tracking the light; `trigger()` hands over the freshest complete frame. See *Image
|
||
capture* below.
|
||
6. **Frame handling** — the kept `Frame` goes to the callback, which `submit()`s it to the
|
||
`ImagePipeline`. `submit()` samples the gimbal pose there and then — yaw from the encoder, pitch from the IMU —
|
||
and the pose travels through the queue with the frame, so encode latency cannot attribute a later pose to earlier
|
||
pixels. The worker rotates it by `Camera.rotation_deg`, encodes JPEG XL (or copies the demo image), writes
|
||
`<output_dir>/<label>/<unix_ms>.jxl`, and publishes a `CamEvent` carrying that pose.
|
||
|
||
## Capture state machine
|
||
|
||
```
|
||
interval elapsed AND capture active AND not already moving
|
||
│
|
||
▼
|
||
┌──────────────────────┐ ControlCode 0 → next ScanGrid (yaw,pitch)
|
||
│ MOVE <yaw>,<pitch> │ ControlCode 1 → target_HDG (pitch held)
|
||
└──────────┬───────────┘ deg→counts via Geometry; set moving, reset timer
|
||
│ (both axes standstill AND |xenc − target| ≤ tol)
|
||
▼
|
||
┌──────────────────────┐ the camera keeps streaming throughout; this pause is
|
||
│ wait settle_wait_ms │ what makes the kept frame one exposed at rest
|
||
└──────────┬───────────┘
|
||
▼
|
||
┌──────────────────────┐ on success: clear moving, advance grid (ControlCode 0)
|
||
│ camera.trigger() │ no fresh frame: retry next tick
|
||
└──────────────────────┘ still none after frame_timeout_ms: warn, skip, advance
|
||
```
|
||
|
||
Triggering only at the settled target replaces the original's trigger-while-moving behaviour (see
|
||
[known-issues.md](known-issues.md) #7). The scheduler is unit-tested with mock doubles and an injected clock
|
||
([tests/test_scheduler.cpp](../tests/test_scheduler.cpp)).
|
||
|
||
Note the scheduler knows nothing about image quality. It asks for an image and advances when it gets one;
|
||
everything below is an implementation detail of the camera source. That is what keeps it testable.
|
||
|
||
## Image capture
|
||
|
||
### The model: free-run, then hold still
|
||
|
||
The camera **runs continuously** at the mode's frame rate with its own auto-exposure and auto-gain in
|
||
`Continuous`. Almost every frame it produces is thrown away. Their job is not to be saved — it is to keep
|
||
the camera's exposure controller converged on the current light, so that the one frame we do keep is
|
||
already correct without anyone waiting for it.
|
||
|
||
At each waypoint the scheduler therefore:
|
||
|
||
1. moves, and waits for both axes to report standstill at the target;
|
||
2. waits a further `settle_wait_ms` (1 s in `standard`, 2.5 s in `night`);
|
||
3. keeps the newest complete frame.
|
||
|
||
The wait is the whole mechanism. It guarantees the kept frame was exposed with the gimbal **at rest** (no
|
||
slew blur) and after the auto-exposure has seen this waypoint's light for several frames. It also sets the
|
||
floor on mode frame rate: a mode must produce comfortably more than one frame inside its own settle wait,
|
||
which is why `night` at 1 Hz waits 2.5 s where `standard` at 10 Hz waits 1 s.
|
||
|
||
If no fresh frame has arrived by `frame_timeout_ms` past the wait, the waypoint is logged and skipped. A
|
||
stalled camera costs single images, never the whole sweep.
|
||
|
||
### Camera profiles
|
||
|
||
Everything on the camera comes from a **profile**: a settings XML exported from the VimbaX Viewer, at
|
||
`<profile_dir>/<name>.xml`. The program hands that file to the Vimba X SDK (`LoadSettings`, restricted to
|
||
the export's `RemoteDevice` block) and nothing else writes camera settings.
|
||
|
||
The reason is validity, not convenience. GenICam writability is **stateful** — `AcquisitionFrameRate` is
|
||
only writable once `AcquisitionFrameRateEnable` is set, `Width`'s maximum depends on binning,
|
||
`IntensityAutoPrecedence` takes `Minimizenoise` but not `MinimizeNoise` — so a list of features applied
|
||
line by line can be individually plausible and collectively wrong, and a line that fails leaves the camera
|
||
quietly on its old value. The Viewer greys out invalid combinations as the operator configures; replaying
|
||
its export is how that guarantee is inherited. An earlier design applied a hand-written INI feature list;
|
||
it was removed rather than repaired, because the failure mode was silent.
|
||
|
||
Beside each XML the program writes `<name>.txt`, a generated human-readable record of the same settings.
|
||
**It configures nothing** — it exists so a profile can be read and diffed without opening the Viewer. At
|
||
start-up the XML is hashed and compared with the hash stored in the record; a mismatch regenerates the
|
||
record and logs every setting that changed, so a camera behaving differently from the last run says so
|
||
rather than being discovered later. Every `*.xml` in the directory is loaded at start-up, so a profile
|
||
nobody uses today still fails at launch rather than the first night someone switches to it.
|
||
|
||
Each camera has its own `[Camera.<name>]` block and its own profile directory, so profiles are per camera
|
||
rather than shared. Switch live with `set camera profile [<camera>] <name>`; that stops acquisition,
|
||
reloads the XML and restarts, because binning, ROI and pixel format cannot be changed on a running stream.
|
||
|
||
### Why not judge each frame instead
|
||
|
||
An earlier design software-triggered one frame per waypoint, measured it (mean luma, clipping, sharpness),
|
||
corrected exposure and re-shot until it passed, seeded from a per-angle exposure store. It was removed. It
|
||
duplicated — and fought — the camera's own exposure controller, which meters continuously in hardware and
|
||
does not need the scan to stop for it. Its premise was that the camera's auto-exposure needs several frames
|
||
to converge and a scan changes scene at every waypoint; free-running at 10 Hz supplies those frames for
|
||
free, during the slew, at no cost to scan rate.
|
||
|
||
### Bandwidth: why both modes bin 2×2
|
||
|
||
At `PixelFormat = RGB8` a full-resolution 4512×4512 frame is **61 MB**. Under the 250 MB/s
|
||
`DeviceLinkThroughputLimit` that caps out at ~4 fps, and even at the interface maximum only ~7 — so 10 Hz at
|
||
full resolution is not reachable in RGB8 at all. (The datasheet's "21 fps at full resolution" is 450 MB/s
|
||
**Mono8**.) With `BinningHorizontal/Vertical = 2` the frame is 2256×2256 ≈ 5 MP, 15.3 MB, and 10 Hz needs
|
||
~153 MB/s — comfortably inside the cap. Binning also sums four photosites, which is worth having at night.
|
||
Raising the throughput limit to 450 is not the fix: it produced incomplete frames on this host
|
||
([known-issues.md](known-issues.md)).
|
||
|
||
Full parameter reference: `[Camera]` in [configuration.md](configuration.md).
|
||
|
||
## Why this shape
|
||
|
||
Decoupling the control logic from the SDKs makes the core testable and the binary buildable/runnable without
|
||
proprietary dependencies, while preserving the original real-time behaviour. Persistence remains deliberately
|
||
limited to image files plus fire-and-forget MQTT.
|