fwt_software/docs/configuration.md

# Configuration & Commands

Behaviour is controlled by three layers: the **`config.ini`** file, **command-line flags** (which override the
config), and **interactive console commands** typed while running.

## Config file resolution

There are no hardcoded paths. The file is resolved in this order ([src/core/Paths.cpp](../src/core/Paths.cpp)):

1. `--config <path>` CLI flag
2. `$FGC_CONFIG` environment variable
3. `./config.ini` (current directory)
4. `<executable dir>/config.ini`
5. `$XDG_CONFIG_HOME/fire_gimbal_control/config.ini` (else `~/.config/...`)

If none exist, the program prints every location it searched and exits. Start from the template:

```bash
cp config/config.example.ini config.ini
```

## `config.ini` keys

Parsed and validated by `ConfigLoader` ([src/core/Config.cpp](../src/core/Config.cpp)) into a typed
`AppConfig`. Invalid types/values fail fast with a clear message.

| Section | Key | Type | Default | Meaning |
|---------|-----|------|---------|---------|
| `General` | `tower_name` | string | `Unnamed` | Tower identity; used in all MQTT topics/payloads |
| `General` | `image_interval` | int > 0 | `5` | Seconds between captures (→ `image_rate = 1/interval`) |
| `General` | `debug` | bool | `false` | Start with debug-level logging |
| `Network` | `zkms_server_ip` | string | `127.0.0.1` | MQTT broker address |
| `Network` | `mqtt_user` / `mqtt_pw` | string | — | MQTT credentials (see secrets below) |
| `Serial` | `device` | string | `/dev/ttyACM0` | Motor-controller serial device |
| `Serial` | `baud` | int | `115200` | Serial baud rate |
| `Camera` | `id_Cam1`..`id_Cam4` | string | — | Camera IDs (GigE IP or USB `DEV_...`); non-empty ones used in order |
| `Camera` | `binning` | int ≥ 1 | `2` | Digital binning; `1` = full res, `2` = 2×2 (~5 MP, ~15 MB — the reliable default) |
| `Camera` | `offset_x`/`offset_y` | int | `0` | ROI origin (0 = full frame) |
| `Camera` | `width`/`height` | int | `0` | ROI size in pixels; `0` = sensor maximum |
| `Camera` | `pixel_format` | string | `RGB8` | On-camera format (RGB8 keeps de-Bayer + white balance on-camera) |
| `Camera` | `throughput_mbytes` | int > 0 | `250` | `DeviceLinkThroughputLimit` (MByte/s); do **not** max it (≥450 drops frames) |
| `Camera` | `stream_fps` | double > 0 | `1.0` | Paced acquisition rate; keeps on-camera auto converged. Keep low (~2 fps of 15 MB frames stalls this USB3 host; 1 fps sustains) |
| `Camera` | `exposure_auto` | bool | `true` | `ExposureAuto=Continuous` (adapt to changing light) |
| `Camera` | `exposure_max_us` | double | `0` | `ExposureAutoMax` cap (µs); `0` = camera default |
| `Camera` | `gain_auto` | bool | `true` | `GainAuto=Continuous` |
| `Camera` | `gain_max_db` | double | `0` | `GainAutoMax` cap (dB); `0` = camera default |
| `Camera` | `white_balance_auto` | bool | `true` | `BalanceWhiteAuto=Continuous` |
| `Camera` | `jxl_distance` | double ≥ 0 | `0.8` | JPEG XL distance (`0` = lossless, `~0.8` = near-lossless) |
| `Camera` | `jxl_effort` | int 1..9 | `4` | JPEG XL effort (higher = slower/smaller) |
| `Paths` | `output_dir` | string | `$XDG_DATA_HOME/fire_gimbal_control/images` | Image output dir; supports `~`/`$ENV` |
| `Features` | `enable_mqtt` | bool | `true` | Use MQTT (vs null channel) |
| `Features` | `enable_camera` | bool | `true` | (reserved) |
| `Features` | `enable_serial` | bool | `true` | (reserved) |
| `Features` | `enable_imu` | bool | `false` | Use the Xsens MTi orientation/IMU |
| `Features` | `mock_camera` | bool | `false` | Use the simulated camera |
| `Features` | `mock_serial` | bool | `false` | Use the simulated motor controller |
| `Features` | `mock_imu` | bool | `false` | Use the simulated IMU instead of the MTi |
| `Features` | `enable_env` | bool | `false` | Use the ambient temp/humidity sensor (SHT41) |
| `Features` | `mock_env` | bool | `false` | Use the simulated env sensor instead of the SHT41 |
| `IMU` | `device` | string | — | MTi serial device (see `[IMU]` note); required when `enable_imu` |
| `IMU` | `baud` | int | `115200` | MTi serial baud rate |
| `Env` | `i2c_device` | string | `/dev/i2c-1` | SHT41 I2C bus device (see `[Env]` note); required when `enable_env` |
| `Env` | `i2c_addr` | int | `68` (`0x44`) | SHT41 I2C slave address; decimal or `0x`-prefixed hex |
| `Env` | `period_ms` | int | `2000` | Sample interval |
| `Logging` | `level` | enum | `info` | Linear log level (`--log-level` overrides) |
| `Logging` | `trace` | csv | — | Wire-trace categories, off by default (`--trace` overrides) |
| `UI` | `enable_tui` | bool | `false` | Full-screen terminal dashboard (`--tui`/`--no-tui` override; needs `WITH_TUI=ON`) |
| `Motor` | `yaw_counts_per_deg` / `pitch_counts_per_deg` | float | `983.33` / — | Encoder counts per degree (**calibrate**; may be negative to flip) |
| `Motor` | `yaw_zero_count` / `pitch_zero_count` | int | `500000` / `0` | `xenc` value that = 0° |
| `Motor` | `yaw_min_deg`/`yaw_max_deg`/`pitch_*` | float | `-90`/`90`/… | Soft clamp on commanded degrees |
| `Scan` | `grid_file` | string | — | CSV of `yaw_deg,pitch_deg` waypoints; empty → generate |
| `Scan` | `yaw_intervals` | int | `56` | Generated yaw positions across `[yaw_min_deg, yaw_max_deg]` |
| `Scan` | `yaw_min_deg`/`yaw_max_deg` | float | `-90`/`90` | Generated yaw arc |
| `Scan` | `pitch_levels` | csv | `0` | Generated pitch elevations (deg) |

### `[Motor]` calibration & `[Scan]` grid

The firmware reports only **encoder counts**; `[Motor]` maps them to the heading/elevation degrees
used by MQTT (`target_HDG`) and `CamEvent`. Calibrate `*_counts_per_deg` / `*_zero_count` against
real `xenc` readings after homing (`MOVE` a known angle, read the resulting `xenc`), **or** run the
automated IMU-referenced `gimbal calib` (below).

#### `gimbal calib` — automated IMU-referenced calibration

`gimbal calib` (requires the IMU) fits each axis's `counts_per_deg` / `zero_count` by sweeping it and
correlating encoder counts with the MTi's measured angle. The sequence is deliberately ordered so the
**yaw fit is not spoiled by the IMU's heading drift** (see the no-magnetometer discussion below):

1. **Home if needed** — if the gimbal is not already `READY`, it runs the endstop-finding home first.
2. **Pitch at the first yaw position** — yaw moves to the start of its travel and holds there while
   pitch is swept across its soft-limit travel; at each step it dwells and records the
   gravity-referenced IMU **pitch** (stable, absolute), then least-squares fits pitch.
3. **Pitch → 0°** — using the just-fitted pitch map.
4. **Switch to a no-magnetometer XKF profile** — picked from the device's own available-profiles list
   (a `*nomag*` profile, else a `VRU` one), so the IMU's heading stops chasing the stepper-distorted
   magnetic field. Sent in the MTi's Config state, so it **persists on the device** across power-cycles.
   Best-effort: if the device reports no magnetometer-free profile, this step is skipped (logged) and
   calibration continues.
5. **Drift-correct + zero the heading** — holding the gimbal still, it runs the MTi **no-rotation**
   gyro-bias update (cuts yaw drift), then a **heading reset** so the current pose becomes yaw 0. This
   happens right before the yaw sweep, so any drift accrued during the slow pitch sweep is discarded.
6. **Yaw sweep** — yaw is swept from that first (now zero-heading) position and fitted.

After calibration the IMU is left on the no-mag profile (this is intentional — the gimbal's homed
encoders are the absolute heading reference; the IMU only needs stable roll/pitch and short-term
yaw-rate). The `i` view's **XKF profile** row will show the new selection (press `r` to refresh if you
changed it outside calibration).

Each axis fit reports an R² (shown in the gimbal `g` view + activity strip). The result is applied to
the live session immediately; in the TUI the activity strip then offers to save it to `[Motor]` as the
new default (`y`/`n`). `gimbal stop` cancels a run. Timings (dwell, no-rotation duration, etc.) are the
`CalibParams` defaults in [CalibrationRoutine.h](../include/fgc/CalibrationRoutine.h).

The capture **scan grid** is the ordered `(yaw,pitch)` waypoints auto-sweep visits (ping-pong). Set
`[Scan] grid_file` to an editable CSV ([config/scan.csv](../config/scan.csv)) to define exact
coordinates, or leave it blank to generate `yaw_intervals × pitch_levels` points.

Camera index → output subfolder defaults to `RGB`, `ACR`, `NIR` (`CameraConfig::labels`).

### `[IMU]` — Xsens MTi orientation sensor

Enable with `[Features] enable_imu = true` and point `[IMU] device` at the MTi's serial node. On the
LattePanda the MTi is wired to the **RS-232 header**, i.e. an onboard hardware UART — a stable
`/dev/ttyS*` node (e.g. `/dev/ttyS4`), **not** a USB device and not `/dev/ttyUSB0-3` (those are the
modem). Find it with `ls /dev/ttyS*` / `dmesg | grep -iE 'ttyS|LPSS'`, or probe for the `0xFA`-framed
stream. At startup the host reconfigures the MTi (`GoToConfig → SetOutputMode → SetOutputSettings →
GoToMeasurement`) to a 100 Hz Euler + calibrated stream — so the device will start streaming even if it
was left in Config state. Yaw is reported as a **0..360 heading** (not the MTi's native −180..180). Set
`[Features] mock_imu = true` to use a synthetic IMU on dev machines (no hardware). Protocol/units are
documented in the modules reference (`MtiProtocol`).

During that same Config-state handshake the host also **reads back the device configuration**
(`ReqProductCode`, `ReqDID`, `ReqFWRev`, `ReqPeriod`, `ReqOutputMode`, `ReqOutputSettings`, and the
filter-profile / **XKF profile** via `ReqFilterProfile` + `ReqAvailableFilterProfiles`). The decoded
values — product code, firmware, device ID, output mode/format, calibration channels, sample rate, and
the list of supported **Xsens Kalman Filter (XKF) profiles** with the active one marked — are surfaced
in the **IMU CONFIG** section of the expanded Sensors view (press `i`). This is read-only: it reports
what the device is actually configured to do, which is the place to confirm the active XKF profile
(e.g. `General` vs `VRU_general`) when diagnosing yaw drift. On the legacy MTi the same concept is
called a "scenario" in the device manual (e.g. `Machine_nomagfield`); it is the same setting and shares
the wire MIDs. Older MTi firmware that does not answer the `Req*` queries simply leaves the section
absent (the host logs `no configuration acks received`).

The config is read **once at startup** and cached, so if you change the XKF profile externally the view
stays stale until you **refresh** it: press `r` (or type `refresh`). That re-queries the device (briefly
pausing the stream) and also requests a fresh firmware dump for the gimbal `g` view.

### `[Env]` — SHT41 ambient temperature/humidity sensor

Enable with `[Features] enable_env = true` and point `[Env] i2c_device` at the LattePanda's **own
native I2C bus** — this is a separate physical link from the motor Leonardo's serial connection, not
the Arduino's D2 pin (the original DHT11 plan). Bus number varies by board; confirm with
`i2cdetect -y N` (the SHT41 answers at `0x44`/`68`). `i2c_addr` accepts either form — `0x44` hex or
`68` decimal. The backend (`Sht41EnvSensor`) polls at `period_ms`, discarding any sample that fails
the SHT4x CRC-8 check (decode lives in the pure, unit-tested
[Sht41Protocol](../include/fgc/sensors/Sht41Protocol.h)); once nothing valid arrives within **2× the
period**, `sample()` reports absence rather than a frozen value, which is what turns the panel from
`live` to `no fix`. Set `[Features] mock_env = true` (or pass `--mock-env`, which also implies
`enable_env`) to use a synthetic reading on dev machines with no hardware. The sensor sits behind the
generic `IEnvSensor` interface
([include/fgc/IEnvSensor.h](../include/fgc/IEnvSensor.h)), so swapping in a different ambient sensor
later means adding a new backend, not touching `Application`, the UI, or MQTT.

### Secrets

`mqtt_user` / `mqtt_pw` are read from the environment variables **`FGC_MQTT_USER` / `FGC_MQTT_PW`** first,
falling back to the config file. Keep credentials out of `config.ini` (which is gitignored anyway) by exporting
them or using a systemd `EnvironmentFile`.

## Command-line flags

Parsed by Boost.Program_options ([main.cpp](../main.cpp)). Flags override `[Features]`.

| Flag | Effect |
|------|--------|
| `-h, --help` | Show help and exit |
| `-c, --config <path>` | Explicit config file path |
| `-i, --init` | Run the endstop-finding init sequence before the loop |
| `-s, --start` | Start capture automatically |
| `-d, --demo` | Demo mode: copy the placeholder image instead of encoding |
| `--no-mqtt` | Disable MQTT (use the null channel) |
| `--mock-camera` | Use the simulated camera |
| `--mock-serial` | Use the simulated motor controller |
| `--mock-env` | Use the simulated ambient temp/humidity sensor (implies `[Features] enable_env`) |
| `--tui` | Show the full-screen terminal dashboard (overrides `[UI] enable_tui`) |
| `--no-tui` | Force the headless line console (overrides config; wins over `--tui`) |
| `--log-level <lvl>` | `trace`/`debug`/`info`/`warn`/`error`/`off` |
| `--trace <cats>` | Verbatim wire trace; comma list `serial,mqtt,camera,control,all,none` |

Typical headless dev run: `scripts/run.sh --mock-serial --mock-camera --no-mqtt --start`.

### Init sequence (`--init`)

Sends `ENABLE Y`, `ENABLE P`, `HOME`, then polls telemetry until both axes report `READY`
(firmware `ST` state `A`), bounded by the firmware's 60 s homing timeout, then sets `SPEED Y/P`
([src/core/Application.cpp](../src/core/Application.cpp), `runInitSequence`).

## Interactive console commands

Lines on stdin are parsed by `parseCommand` ([src/core/CommandParser.cpp](../src/core/CommandParser.cpp)) — a
whitespace tokenizer (`<verb> [device] [option] [value]`) that replaced the old fragile Boost.Spirit grammar.
Handled in `Application::Impl::handleCommand`.

| Type this | Meaning |
|-----------|---------|
| `start` | Start camera acquisition + capture |
| `stop` | Stop acquisition |
| `debug` | Toggle debug logging |
| `trace <cat> [on\|off]` | Toggle a wire-trace category (`serial`/`mqtt`/`camera`/`control`) |
| `trace all` / `trace off` | Enable every category / silence all |
| `set camera jxlq <v>` | JPEG XL distance (0 = lossless) |
| `set camera jxle <v>` | JPEG XL effort |
| `set camera display <0\|1>` | Toggle OpenCV preview window |
| `set camera fps <v>` | Camera acquisition frame rate (real camera only) |
| `set fps <v>` | Capture interval rate (images/second) |
| `set motorctl <cmd>` | Forward a raw command to the motor controller (e.g. `set motorctl MOVE Y 20000`) |
| `test [<subsystem> [<leaf>]] [<profile>]` | Run the hardware self-test suite (see below) |
| `test list` / `test baseline` | List profiles / pin the last report as the comparison baseline |
| `exit` | Quit (Ctrl-D also works) |

### `test` — hardware self-test suite

`test` validates the gimbal/IMU/host on real hardware, writes a text report to
`logs/test_*.log`, and compares each run to a pinned baseline and the previous run.
It is modular (`test gimbal`, `test imu drift`, `test host`, …) and driven by
**profiles** configured under `[Test]` / `[TestProfile.<name>]` in `config.ini`
(repetition counts, sampling windows, pass thresholds; default `standard`). While a
test (or calibration, or homing) runs, interfering commands are ignored and **Esc**
cancels. Full reference: [test-command.md](test-command.md). Replaces the former
`gimbal diag` (now folded into `test gimbal encoder`).

## Terminal dashboard (TUI)

An **optional** full-screen interface (`--tui`, or `[UI] enable_tui = true`) renders the tower
status as sectioned, colored panels updated in place, with a scrolling log pane and a nano-style
key bar. It is built on [FTXUI](https://github.com/ArthurSonzogni/FTXUI) (fetched via
[cmake/Ftxui.cmake](../cmake/Ftxui.cmake) when `WITH_TUI=ON`, the default) and is **fully decoupled
from application logic**: the control loop publishes a plain `UiSnapshot`
([include/fgc/ui/UiSnapshot.h](../include/fgc/ui/UiSnapshot.h)) that the UI renders, and the UI
forwards keystrokes/typed commands back through the same command queue the console uses. Headless
operation is unchanged and remains the default — the same binary runs under systemd/ssh/pipes with
logs on stdout.

Dashboard panels: **Gimbal** (per-axis state, heading, encoder counts, flag badges, target),
**Sensors** (the **Xsens MTi** shows live roll/pitch/yaw once `enable_imu`; the ambient
**SHT41** shows live temp/humidity once `enable_env`. The SHT41's is the **only** temperature on the main window: the MTi's
device-internal temp is diagnostic detail and appears solely in the expanded view), **Camera** (count, capture state, rate, last capture), **Connectivity** (MQTT state,
broker, tower, control mode, target heading). Adding a panel later is a struct in `UiSnapshot.h` plus
one node in [src/ui/TuiUi.cpp](../src/ui/TuiUi.cpp).

**Expanded takeover views** replace the dashboard body full-screen (`Esc` or the same key closes):

- **Gimbal** (`g`) — both axes side by side: live telemetry, the decoded firmware register **dump**
  (`d` requests a fresh one), the homing limits, and the **last calibration** result (per-axis
  `counts_per_deg` / `zero_count` / R² / age).
- **Sensors** (`i`) — both sensors, in headed sections so every reading is attributable.
  **MTi**: every channel with units — orientation (°), acceleration (m/s²), rate-of-turn
  (rad/s), magnetic field (a.u.), `TEMP (internal)`, sample counter, plus an **IMU CONFIG** section
  (read back from the device at startup): product code, firmware, device ID, output mode/format,
  calibration channels, sample rate, and the **Xsens Kalman Filter (XKF) profile** list — every
  profile the device supports, by name, with the active one marked `●` (selected).
  **SHT41**: `TEMP (ambient)`, humidity, the age of the last sample, and the configured I2C bus
  (device, address, poll period — hidden for the mock). Empty states name what to enable
  (`enable_env` / `--mock-env`) or, when enabled but silent, the staleness window that was missed.

**Activity strip** — a compact section between the log and the key bar that shows the
currently-running special operation with live progress (`test`, `gimbal calib`, homing, capture
scan) **and persists the last test/calibration result** (PASS/FAIL, age) so it doesn't scroll
away in the log. It only appears once something has run. While a procedure is running, **Esc**
cancels it.

When a `gimbal calib` completes it is applied to the live session and the strip shows a highlighted
yes/no prompt — **`Save this calibration to config as the new default? (y / n)`**. Press **`y`** to
write the full `[Motor]` per-axis map — `*_counts_per_deg`, `*_zero_count`, and `*_min_deg`/`*_max_deg`
— back into the `config.ini` the program was launched with (replacing those keys in place, preserving
everything else) so it persists across restarts; press **`n`** to keep it for this session only. The keys are active only while the prompt is
showing. (Until you answer `y`, calibration remains session-only — see
[known-issues.md](known-issues.md).)

Keys (bottom bar): `s` start · `x` stop · `h` home · `g` gimbal view · `i` IMU view · `r` refresh
(re-read IMU config + firmware dump) · arrow keys nudge the gimbal (yaw ±5 % / pitch ±10 %) · `:` open
a command line (any console/`gimbal …`
command) · `?` help · `q` quit. Plain letters are used rather than Ctrl chords so terminal
flow-control (`Ctrl-S`/`Ctrl-Q` XON/XOFF) can't swallow them. In TUI mode all log output is diverted
from stdout into the on-screen log pane via a `Logger` sink, so the screen is never corrupted.

Build without it (`-DWITH_TUI=OFF`) for a smaller, dependency-free binary; `--tui` then warns and
runs headless.

## Logging: level vs. wire-trace categories

Two **independent** controls, each settable via config (`[Logging]`), CLI, and a console command:

- **Linear level** — `[Logging] level`, `--log-level`, console `debug`. Filters ordinary messages
  (`trace<debug<info<warn<error<off`). Default `info`; an active capture is nearly silent at `info`.
- **Wire-trace categories** — `[Logging] trace`, `--trace`, console `trace`. Each enabled category
  (`serial`, `mqtt`, `camera`, `control`) logs **every** message exchanged with that subsystem,
  verbatim. They are **off by default** and **independent of the level**: `--trace serial` prints all
  firmware serial traffic even at `info`. Only level `off` silences them. The `camera` category is
  high-rate (one line per frame, metadata only — no pixel data).

Precedence at startup: config applies first, then CLI overrides (`--trace` replaces the config set,
it does not merge). At runtime the `trace` console command edits categories incrementally.

Trace lines carry a category tag and a `TX`/`RX` direction so one subsystem is easy to follow/grep:

```
[SERIAL]  TX MOVE -90,0                     (command to firmware: yaw,pitch counts)
[SERIAL]  RX ST Y:A,982,969,80084000,0,8,8,Se P:A,...  (status from firmware)
[MQTT]    PUB GGS/FWT/Tower/StatusCode 0   /   [MQTT] RX GGS/FWT/Tower/target_HDG 180
[CAMERA]  TX trigger cam0                  /   [CAMERA] RX frame cam0 1936x1216 7064576B
[CONTROL] sweep -> grid yaw=-90 pitch=0    (scheduler decisions + inbound console commands)
```

## Motor command vocabulary (emitted by the software)

The firmware speaks **full-word, newline-terminated** commands in absolute **encoder counts**
(see `../firmware/docs/protocol.md`). The host converts degrees↔counts via the `[Motor]` calibration.

| Command | When | Meaning |
|---------|------|---------|
| `ENABLE Y` / `ENABLE P` | init | energize the axis coils |
| `HOME` | init | home all axes (firmware runs it non-blocking; watch `ST` state `R→H→A`) |
| `SPEED Y\|P <vel>` | init | set the production move speed (VMAX, counts/s) |
| `MOVE <yaw>,<pitch>` | each capture point | drive both axes to absolute counts (combined form) |
| `MOVE Y\|P <pos>` | — | single-axis absolute move (`set motorctl`) |
| `STOP Y\|P\|ALL` | — | ramp to a controlled stop |

Capture is a **move → settle → trigger** cycle: the scheduler issues a `MOVE`, waits until both
axes report standstill at the target, then triggers the cameras. **ControlCode 0** walks the scan
grid (ping-pong); **ControlCode 1** drives yaw to `target_HDG` (pitch held). Telemetry arrives as
firmware `ST` lines (per-axis state + encoder counts), parsed by
[TelemetryParser](../src/core/TelemetryParser.cpp).