152 lines
14 KiB
Markdown
152 lines
14 KiB
Markdown
# Known Issues — Status
|
||
|
||
This tracks the reproduction blockers and robustness issues identified in the original code and what the
|
||
refactor did about them.
|
||
|
||
## Resolved
|
||
|
||
| # | Original issue | Resolution |
|
||
|---|----------------|------------|
|
||
| 1 | Hardcoded config path (`/home/ggs/...`) | Config search order: `--config` → `$FGC_CONFIG` → `./config.ini` → exe dir → XDG ([src/core/Paths.cpp](../src/core/Paths.cpp)) |
|
||
| 2 | Hardcoded image output path | `[Paths] output_dir` with `~`/`$ENV` expansion + sensible default |
|
||
| 3 | Startup scripts with `~/projects/...` | Replaced by path-independent [scripts/run.sh](../scripts/run.sh) + [systemd unit](../scripts/fire-gimbal-control.service) |
|
||
| 4 | Vimba X required to build | `WITH_VIMBA` CMake option (default ON); `OFF` builds a mock-only binary |
|
||
| 5 | No hardware path; exits if MQTT down | Mock implementations + runtime toggles; MQTT failure now logs and continues |
|
||
| 8 | Plaintext MQTT credentials | `$FGC_MQTT_USER`/`$FGC_MQTT_PW` env override; `config.ini` gitignored |
|
||
| 9 | MQTT busy-wait (`while(running);`) | Gone — Paho async client + `set_automatic_reconnect`; no spin thread |
|
||
| 10 | `parser()` missing return | Old parser removed; `parseTelemetryLine` returns `std::optional` cleanly |
|
||
| 11 | Fragile Boost.Spirit command grammar | Replaced by `parseCommand` whitespace tokenizer ([src/core/CommandParser.cpp](../src/core/CommandParser.cpp)) |
|
||
| 12 | Two divergent `config.ini` files | Single committed `config/config.example.ini`; real configs gitignored |
|
||
| 6 | Telemetry field order: humidity before temperature | **Obsolete** — migrated to the current firmware's `ST` protocol (encoder counts, no environmental fields); humidity/temp/fan are gone |
|
||
| 7 | Trigger fires while `is_moving == 1` (not when stopped) | **Fixed** by the protocol migration — capture is now move → **settle** → trigger; the camera fires only once both axes report standstill at the target ([CaptureScheduler.cpp](../src/core/CaptureScheduler.cpp)) |
|
||
|
||
Also added along the way: a leveled logger, typed/validated config, an SDK-independent core library, and a
|
||
doctest unit-test suite (`ctest`).
|
||
|
||
## Open / needs hardware confirmation
|
||
|
||
| # | Issue | Status |
|
||
|---|-------|--------|
|
||
| 13 | `[Motor]` degrees↔counts calibration | The `config.example.ini` values are **placeholders**. Calibrate `*_counts_per_deg` / `*_zero_count` against real `xenc` readings after homing on the rig. |
|
||
| 14 | Capture sweep on hardware | **Mechanically verified on the rig (2026-08-07).** A full `--init --start` run homed both axes and completed `MOVE → settle → trigger` for 15/15 waypoints with 0 dropped frames, 0 stalls and 0 acquisition failures. `kSettleTolCounts` (600) and the per-interval timing in [CaptureScheduler.cpp](../src/core/CaptureScheduler.cpp) have **not** been tuned against observed `ST` behaviour yet, and the settle tolerance has not been checked against a sharp image (see #16). |
|
||
| 15 | **Exposure convergence unverified** | The quality gate has never metered a properly-lit scene. On the rig the lens was covered, so every frame pinned at the exposure/gain ceilings and returned `saturated`; the correction path ran (dark → corrected → saturated) but never *converged onto* `target_mean`. Re-run one sweep with the lens uncovered and confirm the mean lands within `mean_tolerance` in ≤2 attempts at most waypoints. |
|
||
| 16 | **Blur check never exercised** | The relative-sharpness test needs a reference score from a genuinely good frame at that angle; the stored values were all noise-level (sharpness ~1–26 on black frames). `blur_relative_floor` (0.5) is therefore an untuned guess. Verify on a lit scene, and specifically that a deliberate disturbance (nudge the rig during a shot, or shorten `settle_delay_ms` to 0) is actually caught. |
|
||
| 17 | **`degraded` path never fired on hardware** | Every rig capture either passed or hit `saturated`, so the budget-exhausted branch — best-effort frame + `degraded` flag through log/TUI/`CamEvent` — is covered only by unit tests. Confirm once thresholds are tuned. |
|
||
| 18 | **`[Camera] exposure_max_us` / `gain_max_db` unset on the device** | The deployed `config.ini` leaves both at `0`, so the gate falls back to hardcoded 20 ms / 12 dB. Those need real values from a lit scene, and matter more than usual on this rig: the long optical zoom needs more light while tolerating less exposure time (blur scales with focal length). |
|
||
| 19 | **USB preconditions do not survive a reboot** | `usbcore.usbfs_memory_mb` reverts to 16 (needs 1000) and USB3 LPM re-enables on the camera's port. **Both were wrong on the LattePanda when tested on 2026-08-07**, and frame delivery stalls without them regardless of application code. Needs the udev rule + kernel cmdline persistence described under *Camera acquisition* below. |
|
||
|
||
## Camera acquisition (USB3) — root cause + approach
|
||
|
||
Real-camera capture (Alvium 1800 U-2040c, Sony IMX541, 20.4 MP) was brought up on the LattePanda this
|
||
session. Two problems were root-caused (both **host/USB3-side**, reproduced in Allied Vision's own
|
||
`vmbpy` — the camera keeps acquiring while the host stops receiving; **no kernel errors**):
|
||
|
||
1. **Camera shipped in hardware-trigger mode** (`TriggerSource=Line0`) → produced no frames. Fixed by
|
||
configuring **software trigger** explicitly in-session in [VimbaCameraSource.cpp](../src/camera/VimbaCameraSource.cpp)
|
||
(relying on the camera's persisted user set alone did not work).
|
||
2. **Acquisition stall** — frame delivery stops after a few frames:
|
||
- **USB3 hardware LPM (U1/U2) was enabled** on the camera's link (the camera logs
|
||
`Enable of device-initiated U2 failed`) → silent bulk-transfer stalls under load. Disable per-port:
|
||
write `0` to `/sys/devices/.../usb2/2-0:1.0/usb2-port<N>/usb3_lpm_permit`, then re-enumerate
|
||
(the per-device `power/usb3_hardware_lpm_u*` files are read-only). **Make persistent** via a udev
|
||
rule matched on idVendor `1ab2`.
|
||
- **The host xHCI cannot reliably move large single frames.** Measured (LPM off, paced ~1 fps): RGB8
|
||
full-res **61 MB stalls in 3–5 frames**; **≤ ~20 MB frames sustain when *paced*** (BayerRG8 full-res
|
||
20 MB → 21 frames/20 s; 3.8 MB → 41/20 s). Freerun at max rate stalls even for medium frames.
|
||
- Also raise `usbcore.usbfs_memory_mb` to 1000 (persist via kernel cmdline; resets to 16 on reboot).
|
||
|
||
**Approach (config-driven, see `[Camera]`/`[Capture]` in [configuration.md](configuration.md)):** RGB8 +
|
||
**2×2 binning** (~5 MP, ~15 MB) keeps de-Bayering **and** white balance on-camera (the on-camera pipeline
|
||
applies white balance before de-Bayering). `DeviceLinkThroughputLimit` is kept conservative (~250 MB/s;
|
||
≥450 caused incomplete frames). Full 20 MP is possible via BayerRG8 (20 MB) + a host de-Bayer step if
|
||
reduced resolution proves insufficient.
|
||
|
||
**Acquisition is now per-waypoint software trigger** (`[Capture] mode = trigger`), not the paced stream.
|
||
Note what the measurements above do and do not show: they bound **frame size and pacing**, not trigger
|
||
mode. The earlier conclusion that "software trigger stalls this host" was drawn from an implementation
|
||
that triggered only `cameras[0]`, never set `TriggerSource`/`TriggerMode` at all, fired a blind 4×400 ms
|
||
burst, and ran *before* the binning fix — i.e. it was pushing 61 MB frames down the path that stalls for
|
||
unrelated reasons. One frame per waypoint is a **lower** USB load than a 1 fps stream. `mode = freerun`
|
||
restores the old paced-stream behaviour if a host ever does misbehave under trigger.
|
||
|
||
**Status (rig-tested 2026-08-07).** The `[Camera]`/`[Capture]` schema, in-session imaging config,
|
||
near-lossless JPEG XL, the quality gate and unit tests are in place. On the LattePanda with the real Alvium:
|
||
|
||
**Confirmed working** — software trigger sustains a full sweep (15/15 waypoints, **0 dropped, 0 stalls, 0
|
||
acquisition failures**, across four runs), which settles the load-bearing assumption behind `mode = trigger`.
|
||
Camera configuration was verified *independently* through Allied Vision's own `vmbpy` (`RGB8`, binning 2,
|
||
2256×2256, `ExposureAuto=Off`, `GainAuto=Off`, `TriggerMode=On`, `TriggerSource=Software`,
|
||
`TriggerSelector=FrameStart`), and `ExposureTime` writes land within quantisation (`want 20000 →
|
||
readback 19980.6`). Real-motor integration, per-angle store persistence + warm start, and clean shutdown
|
||
(camera not left wedged) all verified.
|
||
|
||
**Not confirmed** — the lens was covered for the whole test, so every frame was black (mean ~1/255) and the
|
||
gate could only ever report `saturated`. Everything that depends on seeing a real scene is still open: see
|
||
**#15–#18** in *Open / needs hardware confirmation* above, plus **#19** for the USB preconditions.
|
||
|
||
A useful diagnostic for the "is it the optics or the code?" question: with frame status validated and gain 0,
|
||
a **2-second** exposure produced mean 3.6/255 (brightest pixel 80). A lit room at gain 0 should be well
|
||
exposed near 10–30 ms, so that is ~1000× too little light — conclusively optical, not software. `vmbpy`
|
||
scripts for this are worth keeping; note that wrapping them in `timeout` SIGTERMs Python and skips
|
||
`stop_streaming()`, which leaves the camera unusable until the next USB reset.
|
||
|
||
## Verification caveats
|
||
|
||
- **Full build verified on the LattePanda**: a `WITH_VIMBA=ON WITH_MQTT=ON` build compiles and links on the
|
||
device (real Vimba X SDK + Paho fetched). The MQTT wrapper also builds on the dev box (GCC 16 / CMake 4 — see
|
||
`cmake/Paho.cmake` for the toolchain-compat shims).
|
||
- **Serial protocol verified live against real firmware** (LattePanda, this session): `ENABLE`/`HOME`/`SPEED`
|
||
reach the firmware, the `ST` telemetry parses with zero unparsed lines across a full session, and a live
|
||
`--init` drove a clean re-home of both axes to `READY`.
|
||
- **Real-camera capture verified on the rig (2026-08-07)**: a live `--init --start` run with the real Alvium
|
||
and real motor captured 15/15 waypoints, one software-triggered frame each, 0 dropped and 0 stalls. What
|
||
this does **not** cover is image *content* — the lens was covered, so exposure/blur behaviour on a real
|
||
scene remains open (#15–#17).
|
||
- **Makefile parity** is moot: the Makefile was removed in favour of CMake, and the full
|
||
`WITH_VIMBA=ON WITH_MQTT=ON` CMake build now runs on the device.
|
||
- **Demo mode** copies `bin/x64/Release/test_smoke.jxl`, resolved relative to the working directory. Run from a
|
||
directory where that path exists, or extend `ImagePipeline::Params::demo_image`.
|
||
|
||
## Possible follow-ups (not done)
|
||
|
||
- **Spurious `could not set AcquisitionFrameRateEnable=false` warning** on every camera start.
|
||
`AcquisitionFrameRateEnable` and `TriggerMode` lock each other: while `TriggerMode=On` the frame-rate
|
||
control is read-only, and vice versa. Re-configuring an already-triggered camera therefore tries to write a
|
||
locked feature that is *already* at the wanted value. All four state transitions were traced and the
|
||
ordering in `configureCamera` is correct — this is cosmetic — but it is misleading noise that would mask a
|
||
genuine failure of that constraint. Fix by skipping the write when the value already matches.
|
||
- **Per-angle white balance.** WB is deliberately left on the camera (`BalanceWhiteAuto=Continuous`) since it
|
||
is correctable in post; storing and seeding it per angle like exposure is the natural next step once
|
||
exposure gating is proven on a lit scene.
|
||
- **Decaying sharpness reference.** `ExposureStore` keeps the *last accepted* sharpness as the blur
|
||
reference. If a scene changes character (fog rolling in), that baseline is stale in the wrong direction. A
|
||
recent-window or decaying value would be more robust — worth revisiting with real data from #16.
|
||
- Graceful shutdown on SIGINT (currently exit via `exit`/Ctrl-D; a pending `getline` can delay shutdown).
|
||
- Make the camera index→label map fully config-driven. (JPEG XL distance/effort and the on-camera imaging
|
||
settings are now config-driven under `[Camera]` — see [configuration.md](configuration.md).)
|
||
- Reintroduce optional image upload to the ground station, config-driven (the old hardcoded NFS/SMB upload was
|
||
removed).
|
||
- **`gimbal calib` persistence**: the fitted `counts_per_deg`/`zero_count` are always applied to the live
|
||
session (display, manual moves, MQTT heading, **and** the capture scheduler) and written to
|
||
`logs/calib_*.log`. In the **TUI**, the activity strip then prompts `Save … as the new default? (y/n)`
|
||
— `y` writes the full per-axis `[Motor]` map (`*_counts_per_deg`, `*_zero_count`,
|
||
`*_min_deg`/`*_max_deg`) into `config.ini` (persists across restarts), `n` keeps them
|
||
session-only. In the **headless** console there is no prompt, so calibration stays session-only there;
|
||
copy the logged values into `[Motor]` by hand to keep them.
|
||
- **`gimbal nudge`** moves a fixed fraction of the **homed** endstop-to-endstop travel (from the
|
||
firmware dump), so it is independent of the degrees↔counts calibration. If no dump has been captured
|
||
yet (not homed / no `gimbal dump`), nudge requests one and does nothing that press — it never falls
|
||
back to the configured degree clamps (which, if `*_min_deg`/`*_max_deg` were unset, produced absurd
|
||
±100000° steps).
|
||
- Ambient temperature/humidity (SHT41, `IEnvSensor`/`Sht41EnvSensor`/`MockEnvSensor`) is fully
|
||
integrated and **validated against the physical sensor on the LattePanda** — config-gated
|
||
(`[Features] enable_env`, `[Env]`), Sensors panel + expanded `i` view + MQTT `Env` topic wired.
|
||
It is **off by default** (`enable_env = false`), so a config without an `[Env]` section shows the
|
||
panel as `pending` — that status means "no driver constructed", not "sensor unreachable"; an
|
||
enabled-but-silent sensor reads `no fix` instead. Dev machines without the hardware should use
|
||
`--mock-env` (implies `enable_env`) rather than editing config.
|
||
- `[Env] i2c_addr` used to be parsed with a decimal-only `std::stoi`, so the `0x44` form every
|
||
datasheet and `i2cdetect` uses silently became address `0x00` and the sensor never answered. It now
|
||
accepts decimal or `0x` hex (`getIntAutoBase` in `Config.cpp`, covered by a test). Other integer
|
||
config keys stay decimal-only on purpose, so a leading zero never turns into octal.
|