fwt_software/docs/known-issues.md

Known Issues — Status

This tracks the reproduction blockers and robustness issues identified in the original code and what the refactor did about them.

Resolved

#	Original issue	Resolution
1	Hardcoded config path (/home/ggs/...)	Config search order: --config → $FGC_CONFIG → ./config.ini → exe dir → XDG (src/core/Paths.cpp)
2	Hardcoded image output path	[Paths] output_dir with ~/$ENV expansion + sensible default
3	Startup scripts with ~/projects/...	Replaced by path-independent scripts/run.sh + systemd unit
4	Vimba X required to build	WITH_VIMBA CMake option (default ON); OFF builds a mock-only binary
5	No hardware path; exits if MQTT down	Mock implementations + runtime toggles; MQTT failure now logs and continues
8	Plaintext MQTT credentials	$FGC_MQTT_USER/$FGC_MQTT_PW env override; config.ini gitignored
9	MQTT busy-wait (while(running);)	Gone — Paho async client + set_automatic_reconnect; no spin thread
10	parser() missing return	Old parser removed; parseTelemetryLine returns std::optional cleanly
11	Fragile Boost.Spirit command grammar	Replaced by parseCommand whitespace tokenizer (src/core/CommandParser.cpp)
12	Two divergent config.ini files	Single committed config/config.example.ini; real configs gitignored
6	Telemetry field order: humidity before temperature	Obsolete — migrated to the current firmware's ST protocol (encoder counts, no environmental fields); humidity/temp/fan are gone
7	Trigger fires while is_moving == 1 (not when stopped)	Fixed by the protocol migration — capture is now move → settle → trigger; the camera fires only once both axes report standstill at the target (CaptureScheduler.cpp)

Also added along the way: a leveled logger, typed/validated config, an SDK-independent core library, and a doctest unit-test suite (ctest).

Open / needs hardware confirmation

#	Issue	Status
13	[Motor] degrees↔counts calibration	The config.example.ini values are placeholders. Calibrate *_counts_per_deg / *_zero_count against real xenc readings after homing on the rig.
14	Capture sweep on hardware	Mechanically verified on the rig (2026-08-07). A full --init --start run homed both axes and completed MOVE → settle → trigger for 15/15 waypoints with 0 dropped frames, 0 stalls and 0 acquisition failures. kSettleTolCounts (600) and the per-interval timing in CaptureScheduler.cpp have not been tuned against observed ST behaviour yet, and the settle tolerance has not been checked against a sharp image (see #16).
15	Exposure convergence unverified	The quality gate has never metered a properly-lit scene. On the rig the lens was covered, so every frame pinned at the exposure/gain ceilings and returned saturated; the correction path ran (dark → corrected → saturated) but never converged onto target_mean. Re-run one sweep with the lens uncovered and confirm the mean lands within mean_tolerance in ≤2 attempts at most waypoints.
16	Blur check never exercised	The relative-sharpness test needs a reference score from a genuinely good frame at that angle; the stored values were all noise-level (sharpness ~1–26 on black frames). blur_relative_floor (0.5) is therefore an untuned guess. Verify on a lit scene, and specifically that a deliberate disturbance (nudge the rig during a shot, or shorten settle_delay_ms to 0) is actually caught.
17	degraded path never fired on hardware	Every rig capture either passed or hit saturated, so the budget-exhausted branch — best-effort frame + degraded flag through log/TUI/CamEvent — is covered only by unit tests. Confirm once thresholds are tuned.
18	[Camera] exposure_max_us / gain_max_db unset on the device	The deployed config.ini leaves both at 0, so the gate falls back to hardcoded 20 ms / 12 dB. Those need real values from a lit scene, and matter more than usual on this rig: the long optical zoom needs more light while tolerating less exposure time (blur scales with focal length).
19	USB preconditions do not survive a reboot	usbcore.usbfs_memory_mb reverts to 16 (needs 1000) and USB3 LPM re-enables on the camera's port. Both were wrong on the LattePanda when tested on 2026-08-07, and frame delivery stalls without them regardless of application code. Needs the udev rule + kernel cmdline persistence described under Camera acquisition below.

Camera acquisition (USB3) — root cause + approach

Real-camera capture (Alvium 1800 U-2040c, Sony IMX541, 20.4 MP) was brought up on the LattePanda this session. Two problems were root-caused (both host/USB3-side, reproduced in Allied Vision's own vmbpy — the camera keeps acquiring while the host stops receiving; no kernel errors):

1. Camera shipped in hardware-trigger mode (TriggerSource=Line0) → produced no frames. Fixed by configuring software trigger explicitly in-session in VimbaCameraSource.cpp (relying on the camera's persisted user set alone did not work).
2. Acquisition stall — frame delivery stops after a few frames:
   • USB3 hardware LPM (U1/U2) was enabled on the camera's link (the camera logs Enable of device-initiated U2 failed) → silent bulk-transfer stalls under load. Disable per-port: write 0 to /sys/devices/.../usb2/2-0:1.0/usb2-port<N>/usb3_lpm_permit, then re-enumerate (the per-device power/usb3_hardware_lpm_u* files are read-only). Make persistent via a udev rule matched on idVendor 1ab2.
   • The host xHCI cannot reliably move large single frames. Measured (LPM off, paced ~1 fps): RGB8 full-res 61 MB stalls in 3–5 frames; ≤ ~20 MB frames sustain when paced (BayerRG8 full-res 20 MB → 21 frames/20 s; 3.8 MB → 41/20 s). Freerun at max rate stalls even for medium frames.
   • Also raise usbcore.usbfs_memory_mb to 1000 (persist via kernel cmdline; resets to 16 on reboot).

Approach (config-driven, see [Camera]/[Capture] in configuration.md): RGB8 + 2×2 binning (~5 MP, ~15 MB) keeps de-Bayering and white balance on-camera (the on-camera pipeline applies white balance before de-Bayering). DeviceLinkThroughputLimit is kept conservative (~250 MB/s; ≥450 caused incomplete frames). Full 20 MP is possible via BayerRG8 (20 MB) + a host de-Bayer step if reduced resolution proves insufficient.

Acquisition is now per-waypoint software trigger ([Capture] mode = trigger), not the paced stream. Note what the measurements above do and do not show: they bound frame size and pacing, not trigger mode. The earlier conclusion that "software trigger stalls this host" was drawn from an implementation that triggered only cameras[0], never set TriggerSource/TriggerMode at all, fired a blind 4×400 ms burst, and ran before the binning fix — i.e. it was pushing 61 MB frames down the path that stalls for unrelated reasons. One frame per waypoint is a lower USB load than a 1 fps stream. mode = freerun restores the old paced-stream behaviour if a host ever does misbehave under trigger.

Status (rig-tested 2026-08-07). The [Camera]/[Capture] schema, in-session imaging config, near-lossless JPEG XL, the quality gate and unit tests are in place. On the LattePanda with the real Alvium:

Confirmed working — software trigger sustains a full sweep (15/15 waypoints, 0 dropped, 0 stalls, 0 acquisition failures, across four runs), which settles the load-bearing assumption behind mode = trigger. Camera configuration was verified independently through Allied Vision's own vmbpy (RGB8, binning 2, 2256×2256, ExposureAuto=Off, GainAuto=Off, TriggerMode=On, TriggerSource=Software, TriggerSelector=FrameStart), and ExposureTime writes land within quantisation (want 20000 → readback 19980.6). Real-motor integration, per-angle store persistence + warm start, and clean shutdown (camera not left wedged) all verified.

Not confirmed — the lens was covered for the whole test, so every frame was black (mean ~1/255) and the gate could only ever report saturated. Everything that depends on seeing a real scene is still open: see #15–#18 in Open / needs hardware confirmation above, plus #19 for the USB preconditions.

A useful diagnostic for the "is it the optics or the code?" question: with frame status validated and gain 0, a 2-second exposure produced mean 3.6/255 (brightest pixel 80). A lit room at gain 0 should be well exposed near 10–30 ms, so that is ~1000× too little light — conclusively optical, not software. vmbpy scripts for this are worth keeping; note that wrapping them in timeout SIGTERMs Python and skips stop_streaming(), which leaves the camera unusable until the next USB reset.

Verification caveats

• Full build verified on the LattePanda: a WITH_VIMBA=ON WITH_MQTT=ON build compiles and links on the device (real Vimba X SDK + Paho fetched). The MQTT wrapper also builds on the dev box (GCC 16 / CMake 4 — see cmake/Paho.cmake for the toolchain-compat shims).
• Serial protocol verified live against real firmware (LattePanda, this session): ENABLE/HOME/SPEED reach the firmware, the ST telemetry parses with zero unparsed lines across a full session, and a live --init drove a clean re-home of both axes to READY.
• Real-camera capture verified on the rig (2026-08-07): a live --init --start run with the real Alvium and real motor captured 15/15 waypoints, one software-triggered frame each, 0 dropped and 0 stalls. What this does not cover is image content — the lens was covered, so exposure/blur behaviour on a real scene remains open (#15–#17).
• Makefile parity is moot: the Makefile was removed in favour of CMake, and the full WITH_VIMBA=ON WITH_MQTT=ON CMake build now runs on the device.
• Demo mode copies bin/x64/Release/test_smoke.jxl, resolved relative to the working directory. Run from a directory where that path exists, or extend ImagePipeline::Params::demo_image.

Possible follow-ups (not done)

• Spurious could not set AcquisitionFrameRateEnable=false warning on every camera start. AcquisitionFrameRateEnable and TriggerMode lock each other: while TriggerMode=On the frame-rate control is read-only, and vice versa. Re-configuring an already-triggered camera therefore tries to write a locked feature that is already at the wanted value. All four state transitions were traced and the ordering in configureCamera is correct — this is cosmetic — but it is misleading noise that would mask a genuine failure of that constraint. Fix by skipping the write when the value already matches.
• Per-angle white balance. WB is deliberately left on the camera (BalanceWhiteAuto=Continuous) since it is correctable in post; storing and seeding it per angle like exposure is the natural next step once exposure gating is proven on a lit scene.
• Decaying sharpness reference. ExposureStore keeps the last accepted sharpness as the blur reference. If a scene changes character (fog rolling in), that baseline is stale in the wrong direction. A recent-window or decaying value would be more robust — worth revisiting with real data from #16.
• Graceful shutdown on SIGINT (currently exit via exit/Ctrl-D; a pending getline can delay shutdown).
• Make the camera index→label map fully config-driven. (JPEG XL distance/effort and the on-camera imaging settings are now config-driven under [Camera] — see configuration.md.)
• Reintroduce optional image upload to the ground station, config-driven (the old hardcoded NFS/SMB upload was removed).
• gimbal calib persistence: the fitted counts_per_deg/zero_count are always applied to the live session (display, manual moves, MQTT heading, and the capture scheduler) and written to logs/calib_*.log. In the TUI, the activity strip then prompts Save … as the new default? (y/n) — y writes the full per-axis [Motor] map (*_counts_per_deg, *_zero_count, *_min_deg/*_max_deg) into config.ini (persists across restarts), n keeps them session-only. In the headless console there is no prompt, so calibration stays session-only there; copy the logged values into [Motor] by hand to keep them.
• gimbal nudge moves a fixed fraction of the homed endstop-to-endstop travel (from the firmware dump), so it is independent of the degrees↔counts calibration. If no dump has been captured yet (not homed / no gimbal dump), nudge requests one and does nothing that press — it never falls back to the configured degree clamps (which, if *_min_deg/*_max_deg were unset, produced absurd ±100000° steps).
• Ambient temperature/humidity (SHT41, IEnvSensor/Sht41EnvSensor/MockEnvSensor) is fully integrated and validated against the physical sensor on the LattePanda — config-gated ([Features] enable_env, [Env]), Sensors panel + expanded i view + MQTT Env topic wired. It is off by default (enable_env = false), so a config without an [Env] section shows the panel as pending — that status means "no driver constructed", not "sensor unreachable"; an enabled-but-silent sensor reads no fix instead. Dev machines without the hardware should use --mock-env (implies enable_env) rather than editing config.
• [Env] i2c_addr used to be parsed with a decimal-only std::stoi, so the 0x44 form every datasheet and i2cdetect uses silently became address 0x00 and the sensor never answered. It now accepts decimal or 0x hex (getIntAutoBase in Config.cpp, covered by a test). Other integer config keys stay decimal-only on purpose, so a leading zero never turns into octal.