fwt_software/docs/tower-installation.md

# Installing a tower from scratch

How a new fire-watch tower host goes from a freshly imaged machine to a running,
calibrated `fire_gimbal_control`. Written while doing exactly that on the IOTA
tower at `10.11.12.121` (2026-09-15) and kept honest about which steps have
actually been exercised and which are still on paper.

> [deployment.md](deployment.md) is the *update* loop for a tower that already
> works. This is the *first* install, including the parts that only happen once:
> the board's preconditions, the Vimba X SDK, `hardware.ini`, and the motor
> firmware.

**Status legend** — each step says where it stands:
`[verified]` run end to end on a real tower and the result checked ·
`[partial]` run, with something still open · `[untested]` written from the
design, not yet exercised.

---

## What you need before starting

| | |
|---|---|
| The tower host | Powered, on the network, reachable by IP, with an OS installed and an `ssh` server running |
| An account on it | `ggs`, in the `sudo` and `dialout` groups |
| A dev box | This repository checked out, plus a copy of the Vimba X SDK (login-walled, so it cannot be fetched by a script) |
| The hardware | Camera plugged in, motor MCU plugged in, and a note of **what was assembled**: which axes, which IMU, which ambient sensor |

Two facts decide most of what follows, so establish them first:

```bash
ssh ggs@<tower-ip> 'cat /sys/class/dmi/id/product_name; . /etc/os-release && echo "$PRETTY_NAME"'
```

| Board | MCU | Setup kit | PlatformIO env |
|---|---|---|---|
| LattePanda 3 Delta / Sigma | ATmega32U4 (Leonardo), 5 V | *not written - the Delta fleet predates the kits* | `leonardo` |
| LattePanda IOTA | RP2040, 3.3 V | [`hardware/lattepanda-iota/`](../hardware/lattepanda-iota/) | `iota_rp2040` |

---

## 1. SSH by key `[verified]`

Everything after this is driven over ssh from the dev box, and `deploy.sh` runs
several ssh commands per invocation, so password auth is not workable.

```bash
# on the dev box
ssh-copy-id -i ~/.ssh/id_ed25519_ggs_pedro.pub ggs@<tower-ip>
```

Then give the host a block in `~/.ssh/config`, so every tool picks the right key
without being told:

```
Host 10.11.12.121
    User ggs
    IdentityFile ~/.ssh/id_ed25519_ggs_pedro
```

Verify with `ssh ggs@<tower-ip> hostname` - it must not prompt.

> **`ssh-copy-id` needs a terminal.** Run it in a real shell, not through a tool
> that gives it no tty: without one, ssh looks for a GUI `ssh-askpass`, and on a
> machine that has none it fails with `ssh_askpass: exec(...): No such file or
> directory` and then `Permission denied` - which looks like a rejected key
> rather than a prompt that never appeared.

## 2. Survey the machine before changing it `[verified]`

The setup kit can describe the machine without touching it, and this is the
cheapest moment to notice that the tower is not what you assumed:

```bash
rsync -a hardware/ ggs@<tower-ip>:~/projects/fwt_2a/software/hardware/
ssh ggs@<tower-ip> '~/projects/fwt_2a/software/hardware/lattepanda-iota/install.sh --probe'
```

```
board            : lattepanda-iota
os               : Ubuntu 26.04.1 LTS (7.0.0-31-generic)
motor MCU        : /dev/serial/by-id/usb-MicroPython_Board_in_FS_mode_df61a863dfa02932-if00
camera DEV_ id   : NOT FOUND (needs the Vimba X SDK installed)
alvium on usb    : 1 device(s)
usbfs_memory_mb  : 16 (want >= 1000)
camera lpm_permit: u1_u2 (want 0)
```

Read every line. On tower 121 this pass found that the MCU was still running the
MicroPython image it shipped with, and that both USB preconditions were at their
(wrong) defaults.

**Check what is already in `~/projects/`.** A machine cloned from another tower,
or one that has been used for development, can arrive with a `config.ini` that
belongs to a different rig. Tower 121 had a byte-identical copy of the dev box's
`config.ini` - `tower_name = dev`, broker `127.0.0.1` - which is worse than no
config at all, because **none of those values trip the placeholder guard in
`deploy.sh --run`**. That guard only catches `ExampleTower`, `CHANGE_ME` and
`DEV_0000000000`.

```bash
ssh ggs@<tower-ip> 'grep -E "tower_name|zkms_server_ip|^id" ~/projects/fwt_2a/software/config.ini'
```

## 3. Host preconditions and build dependencies `[verified]`

Both come from the board's setup kit, in one step, on the device:

```bash
ssh ggs@<tower-ip>
cd ~/projects/fwt_2a/software/hardware/lattepanda-iota
sudo ./install.sh --deps --preconditions
```

`--deps` installs [`packages.txt`](../hardware/lattepanda-iota/packages.txt): the
toolchain, OpenCV, Boost, libjxl, tmux, and the Python bits PlatformIO needs.

`--preconditions` installs two files and applies them immediately:

| File | Why |
|---|---|
| `/etc/udev/rules.d/99-fwt-alvium-usb.rules` | Disables USB3 hardware LPM on the camera's port, which the host otherwise re-enables on every enumeration. With U1/U2 on, the camera logs link errors and frame delivery stalls (known-issues #19) |
| `/etc/tmpfiles.d/10-fwt-usbfs.conf` | Raises `usbfs_memory_mb` from 16 to 1000 at every boot. A single full-resolution frame does not fit in 16 MB |

Both are settings that **revert on reboot** if nobody persists them, which is the
whole of known-issue #19. The script re-reads them after applying and says so:

```
>> usbfs_memory_mb = 1000
>> camera USB3 LPM disabled (2-1)
```

If it instead warns that a value did not take, stop and find out why - a camera
on a host with these wrong looks like a flaky camera.

Confirmed on tower 121 (2026-09-15): both went from wrong to right in one run,
`usbfs_memory_mb` 16 -> 1000 and `lpm_permit` `u1_u2` -> `0`. Re-check after the
first reboot, since surviving one is the entire point.

> **Neither step can be driven from a tool without a terminal.** `sudo` needs a
> tty to prompt, so this runs from a real terminal window:
>
> ```bash
> ssh -t ggs@<tower-ip> 'cd ~/projects/fwt_2a/software/hardware/lattepanda-iota && sudo ./install.sh --deps --preconditions'
> ```

> **The Delta fleet does not have these applied.** Checked on `10.11.12.111`
> 2026-09-15: `usbfs_memory_mb` is 16 and the kernel cmdline is stock. The
> preconditions were documented in known-issues but never persisted anywhere.

## 4. Vimba X SDK `[verified]`

Proprietary and login-walled, so it is staged from a machine that already has it
rather than downloaded on the tower:

```bash
# dev box
rsync -az ~/Programs/VimbaX_2025-3 ggs@<tower-ip>:~/
# tower
sudo ./install.sh --vimba ~ggs/VimbaX_2025-3
```

That copies it to `/opt/VimbaX_<version>`, points `/opt/VimbaX` at it, runs the
SDK's `Install_GenTL_Path.sh` (which registers `GENICAM_GENTL64_PATH` in
`/etc/profile.d` and writes the `1ab2` udev rule), registers `api/lib` with the
dynamic loader, and then asks the SDK to list cameras as a check.

**The SDK asks for a reboot before the transport layers are usable.** If it
finds nothing, reboot and try again before suspecting the camera. On tower 121
no reboot was needed - the camera answered immediately.

Two things about that check, both found the hard way here:

- **The binary is `ListCameras_VmbC`**, in `/opt/VimbaX/bin/`. There is no plain
  `ListCameras`.
- **Run it with `GENICAM_GENTL64_PATH` set**, or it fails with `Could not start
  system. Error code: -16` on a perfectly good installation - that is "no
  transport layer found", not "no camera":

  ```bash
  GENICAM_GENTL64_PATH=/opt/VimbaX/cti /opt/VimbaX/bin/ListCameras_VmbC
  ```

> **The SDK ships a camera simulator, and it is on by default.** On tower 121
> `ListCameras_VmbC` reported **four** cameras: the real Alvium plus three
> synthetic ones from `VimbaCameraSimulatorTL.cti`, with ids like `DEV_Cam1`. A
> tower configured with one of those would open its "camera" happily and capture
> synthetic images forever. Take the id only from a block whose transport layer
> path is `VimbaUSBTL` or `VimbaGigETL` - which is what `install.sh --probe`
> does, after this bit me.

> Non-interactive ssh shells do not source `/etc/profile.d`, which is why
> `deploy.sh` resolves `GENICAM_GENTL64_PATH` itself before launching the app.
> Getting this wrong shows up as `VmbStartup()` failing with *"Could not start
> Vimba X API"* even though the viewer works when you are logged in.

## 5. `hardware.ini` - what this machine IS `[verified]`

```bash
sudo ./install.sh --hardware-ini --axes yaw --imu none --env none --cameras rgb
```

This writes `/etc/fire_gimbal_control/hardware.ini`, **outside** the directory
`deploy.sh` rsyncs, so no deploy can overwrite or delete it.

The probed half - board model, firmware env, the MCU's stable `by-id` path, the
camera's `DEV_` id - the script fills in. The declared half is the flags, and
they are not a formality: the program degrades gracefully around missing
hardware, so **a part that was never fitted and a part that has failed look
identical** unless the file says which was expected. Declare what was assembled.

Check the result, especially the serial path:

```bash
ssh ggs@<tower-ip> 'cat /etc/fire_gimbal_control/hardware.ini'
```

> `[Serial] device` must be a `/dev/serial/by-id/` path, never `/dev/ttyACM0`.
> Tower 121 has a 5G modem that claims three `ttyUSB` nodes, and enumeration
> order is not stable across reboots.
>
> **The by-id path changes when you flash the MCU**, because the name comes from
> the running image: `usb-MicroPython_Board_in_FS_mode_*` before, a Pico/Arduino
> name after. Re-run `--probe` after step 6 and fix the file if it moved.

## 6. Motor firmware `[verified]`

The firmware is built and flashed **on the tower** - it is the machine wired to
the MCU - so PlatformIO is installed there, not on the dev box.

```bash
# tower: PlatformIO core, in its own venv under ~/.platformio. No sudo needed.
curl -fsSL https://raw.githubusercontent.com/platformio/platformio-core-installer/master/get-platformio.py -o /tmp/get-platformio.py
python3 /tmp/get-platformio.py
~/.platformio/penv/bin/pio --version      # check it actually runs
```

`[verified]` on tower 121: PlatformIO Core 6.2.0 on the system Python 3.14.4.

> The installer ends with a `FileNotFoundError` traceback out of
> `multiprocessing/util.py` about a missing `/tmp/.piocore-installer-*` path.
> It is a temp-directory cleanup finalizer running after the work is done -
> cosmetic. Judge the install by whether `pio --version` answers.

`deploy.sh` looks for `pio` in the usual places, but a non-interactive ssh shell
does not have `~/.platformio/penv/bin` on PATH, so name it explicitly:

```ini
REMOTE_PIO=/home/ggs/.platformio/penv/bin/pio
```

Then, from the dev box, in `firmware/`:

```ini
# .deploy.env
REMOTE_HOST=ggs@10.11.12.121
REMOTE_DIR=/home/ggs/projects/fwt_2a/firmware
PIO_ENV=iota_rp2040          # leonardo on a Delta
SERIAL_PORT=/dev/serial/by-id/usb-...
```

```bash
./deploy.sh --compile-only     # build on the tower, flash nothing
./deploy.sh                    # build + upload
./deploy.sh --monitor          # watch it boot
```

**Confirm the pin map before flashing a board that is wired to motors.** The map
lives in `src/platform_iota.h` and a chip-select on the wrong pin writes into a
driver that was never meant to hear it. The numbers there are the loom as built
on tower 121, which is **not** the map proposed in the porting plan.

### Getting a factory board to accept its first flash

`[verified]` on tower 121, and this is the step that will bite you. These boards
ship running MicroPython, and **the normal upload does not work against it**:

```
Forcing reset using 1200bps open/close on port /dev/ttyACM0
No new RPxxxx device found yet, waiting..          (x20, then it gives up)
```

| What | Factory board (MicroPython) | Our firmware |
|---|---|---|
| 1200-baud touch, i.e. a plain upload | no reset at all | resets, but **reboots into the application** |
| `picotool reboot -f -u` | refused: no reset interface | - |
| MicroPython REPL | **works** | n/a |

So drop into MicroPython's own REPL and tell it to enter the bootloader. Write
this to a file and run it with `~/.platformio/penv/bin/python` (it has pyserial):

```python
import serial, time
s = serial.Serial("/dev/serial/by-id/usb-MicroPython_Board_in_FS_mode_XXXX-if00", 115200, timeout=1)
s.write(b"\x03\x03"); time.sleep(0.3)        # Ctrl-C: interrupt whatever is running
s.write(b"import machine\r\n"); time.sleep(0.3)
s.write(b"machine.bootloader()\r\n")         # the port drops here - that is success
```

```bash
lsusb | grep 2e8a        # expect: 2e8a:0003 Raspberry Pi RP2 Boot
```

Then run `./deploy.sh` and it flashes. A physical BOOTSEL press does the same
job if you are standing at the board.

**Every board needs exactly one physical BOOTSEL press in its life** - the one
that installs the first image carrying the `BOOTSEL` command (below). After
that, updates are remote.

**`sudo ./install.sh --preconditions` must have run first**, or picotool finds
the board and refuses it:

```
RP2040 device at bus 1, address 4 appears to be in BOOTSEL mode, but picotool
    was unable to connect. Maybe try 'sudo' or check your permissions.
```

### Re-flashing later: the `BOOTSEL` command

`[verified]` on tower 121. Neither documented remote path works on this board -
the 1200-baud touch resets the firmware but returns it to the application, and
`picotool reboot -f -u` answers `Unable to locate reset interface on the
device`. So the firmware carries a command that does it deliberately:

```bash
# over the serial port, or from the app: gimbal raw "BOOTSEL"
BOOTSEL   -> OK entering bootloader - reflash to return
lsusb | grep 2e8a        # 2e8a:0003 Raspberry Pi RP2 Boot
./deploy.sh              # flashes and reboots into the new firmware
```

The whole loop was run end to end from the dev box with nobody at the tower.
`BOOTSEL` disables every axis first, since the bootrom does not run the control
loop and unsupervised energised drivers would hold torque.

> **It takes the gimbal out of service and the firmware cannot undo it** - once
> in the bootrom there is no command loop left to change its mind. Recovery is
> `deploy.sh`, or a power cycle: entering the bootrom erases nothing, so the
> existing image is still in flash. Anything that can write to the serial port
> can send it, including `gimbal raw`. Near misses (`BOOT`, `BOOTSE`) are
> rejected as unknown commands.

### The identity changes when you flash

| | by-id path | USB |
|---|---|---|
| before | `usb-MicroPython_Board_in_FS_mode_<serial>-if00` | `2e8a:0005` |
| after | `usb-Raspberry_Pi_Pico_<SERIAL>-if00` | `2e8a:000a` |

So generate `hardware.ini` (step 5) **after** flashing, or re-run
`install.sh --hardware-ini --force` to pick up the new path.

### Verify the flash

Read the boot banner over the serial port:

```
BOOT build=aab1925-dirty
IOIN Y=0x3000001E version=0x30
ST Y:R,500000,18,80084000,0,8,8,Sd
```

`version=0x30` is the TMC5160 answering over SPI - the single most useful line
here, because it proves the pin map, the 3.3 V logic level and the driver board
all at once. `BOOT build=` must match the commit you flashed. One `Y` line and
no `P` line is a yaw-only build.

> **Flashing an RP2040 remotely is not the same as a Leonardo.** The documented
> flow is *hold BOOTSEL, tap RST, drag a `.uf2`* - a person at the board. The
> two remote paths (the 1200-baud touch that the Arduino tooling already uses,
> and `picotool reboot -f -u`) both need the **running** image to present a USB
> CDC port. That is why `setup()` brings `Serial` up before any hardware init:
> an image that hangs before USB comes up cannot be reflashed without a site
> visit, while an interrupted flash recovers by itself - the bootrom falls back
> to mass storage when flash holds no valid image.

## 7. Deploy the software `[verified]`

On the dev box, in `software/`:

```ini
# .deploy.env
REMOTE_HOST=ggs@10.11.12.121
REMOTE_DIR=/home/ggs/projects/fwt_2a/software
CMAKE_ARGS="-DWITH_MQTT=ON -DWITH_VIMBA=ON"
RUN_ARGS="--start"
```

```bash
./deploy.sh --check-deps    # must print "remote deps OK"
./deploy.sh                 # rsync + remote cmake + build
```

The first deploy seeds `config.ini` from `config/config.example.ini` **only if
the device has none** - see the warning in step 2 about a machine that arrived
with somebody else's config.

`[verified]` on tower 121: builds clean on GCC 15 / Ubuntu 26.04 with a single
`-Wformat-truncation` warning in `utcStampNow()`, which is a false positive (the
compiler cannot prove `tm_year` is four digits; the real output is 17 bytes into
a 32-byte buffer).

## 8. Camera profiles `[verified]`

**Without a profile the program does not start** - `no camera profiles (*.xml)
in config/camera_profiles/rgb` is a startup error, by design. A fresh tower has
an empty profile directory, so this step is not optional.

The documented way to make one is the VimbaX Viewer, which is a GUI - awkward on
a headless tower and impossible unattended. `scripts/export_camera_profile.py`
does the same job through the SDK's Python API:

```bash
# once per tower: the SDK ships the wheel
python3 -m venv ~/.venvs/vmbpy
~/.venvs/vmbpy/bin/pip install /opt/VimbaX/api/python/vmbpy-*.whl

cd ~/projects/fwt_2a/software
export GENICAM_GENTL64_PATH=/opt/VimbaX/cti
~/.venvs/vmbpy/bin/python scripts/export_camera_profile.py --profile standard --dry-run
~/.venvs/vmbpy/bin/python scripts/export_camera_profile.py --profile standard
~/.venvs/vmbpy/bin/python scripts/export_camera_profile.py --profile night
```

It writes each recipe value to the real camera **in dependency order**, lets the
camera validate it, and exports what was actually accepted. That preserves the
property the Viewer workflow exists to provide: GenICam validity is stateful, so
a hand-written XML can be individually plausible and collectively wrong. Here
the camera is the judge, and anything it refused is printed rather than assumed.

Read the summary line it prints - it is the number that decides whether the
profile can be delivered at all:

```
result  : 2664x1520 RGB8  12.1 MB/frame  4.95 fps  = 60 MB/s
```

### Profiles do not transfer between cameras

`[verified]` the hard way. The fleet runs several Alvium generations and their
feature vocabularies differ, so copying another tower's XML does not work:

| | Delta tower (1800 U-1240c) | IOTA tower (1800 U-1620c) |
|---|---|---|
| `SensorBitDepth` | `Bpp8` | **no `Bpp8`** - only `Adaptive` / `Bpp12` |
| `IntensityAutoPrecedence` | `Minimizeexposuretime` / `Minimizenoise` | **`MinimizeBlur` / `MinimizeNoise`** |
| Sensor | 12 MP | 16 MP, 5328x3040 |

### Binning is not optional, and the frame-rate ceiling is not what it looks like

Unbinned RGB8 on the 16 MP sensor is a **48.6 MB frame** - over the ~20 MB
ceiling in known-issues #19. 2x2 binning brings it to 12.1 MB.

The frame rate is then capped by `DeviceLinkThroughputLimit`, but the limit is
accounted against the **unbinned** payload, not the binned frame that crosses
the wire. Measured on tower 121:

| `DeviceLinkThroughputLimit` | max `AcquisitionFrameRate` | actual USB traffic |
|---|---|---|
| 250 MB/s (the fleet's value) | 4.95 fps | ~60 MB/s |
| 450 MB/s (this camera's max) | 8.79 fps | ~106 MB/s |

48.6 MB x 4.95 = 240 MB/s, which is the 250 limit - so the camera is budgeting
for data that binning discards. **The recipe ships the conservative 250 MB/s**,
giving ~5 fps rather than the Delta's 10, because the USB stalls behind
known-issues #19 are a host-side finding that has not been re-measured on the
N150 yet (porting plan phase 5). Raising it is a one-line change in the recipe
once that measurement exists.

> **Export `standard` after `night`, not the other way round, or use the current
> script.** A refused write is not a no-op: the first version of this script
> asked for 10 fps, the camera refused, and the 1 fps left over from the night
> profile stayed - producing a "standard" profile that ran at night speed. The
> script now clamps an out-of-range numeric value to the camera's limit and says
> `~ ... (clamped)`.

## 9. `config.ini` - what this tower DOES `[partial]`

Device-local, never overwritten by a deploy. Edit it on the tower:

```ini
[General]
tower_name = <this tower>          ; drives every MQTT topic
[Network]
zkms_server_ip = <broker>
```

Credentials belong in `/etc/fire_gimbal_control/credentials.env`, not here.

Anything `hardware.ini` now owns - `[Serial] device`, `[IMU]`, `[Env]`, the
camera ids - should be **deleted** from `config.ini` rather than left to rot:
the program warns at startup about every key set in both.

`[Features] enable_imu` and `enable_env` matter more than they look. Declaring
`imu = none` in `hardware.ini` does **not** yet switch them off - deriving them
is section 12 of the config plan and is not implemented. On tower 121 the first
clean-looking run did this:

```
[INFO ] MTi IMU started on /dev/ttyS0 @ 115200
[ERROR] I2C: failed to open /dev/i2c-2
[ERROR] SHT41: failed to open I2C bus; sensor disabled
```

It "started" an IMU on a tower that has none - `/dev/ttyS0` exists and accepts a
serial open whether or not anything is attached to it. Set both flags to `false`
by hand until reconciliation lands.

## 10. First run: bring the subsystems up one at a time `[partial]`

Do **not** start a new tower with everything switched on. Each phase below adds
exactly one subsystem, so that a failure names its own cause instead of arriving
as a wall of unrelated errors. It also keeps the motors - the only part that can
break itself - until you have proved the boring parts work.

Each phase says what good looks like and what noise to expect.

### Phase 0 - the host alone, no hardware `[verified]`

```bash
./build/fire_gimbal_control --mock-serial --mock-camera --no-mqtt --no-tui
```

Proves config resolution, `hardware.ini` layering, the single-instance lock, the
scan grid and the output directory - with nothing physical involved. Neither a
camera nor its profile XMLs need exist:

```
[INFO ] camera profiles not loaded: no real camera in this run
```

`[Features] enable_camera = false` does the same thing from the config file, for
a tower that has no camera fitted yet, and says so rather than pretending:

```
[WARN ] camera disabled ([Features] enable_camera = false); using a mock source
        - no images come from hardware in this run
```

```
[INFO ] hardware: /etc/fire_gimbal_control/hardware.ini  board lattepanda-iota  axes yaw
[INFO ] Loaded config: config.ini
[INFO ] scan grid: 15 waypoints
```

Any `config.ini still sets X, which hardware.ini overrides` warning belongs here
and should be cleared before moving on (step 9).

### Phase 1 - motors `[partial]`

Real serial link, nothing else:

```bash
./build/fire_gimbal_control --mock-camera --no-mqtt --no-tui
```

What proves the link is a **live `DUMP` from the board**, not just an open port:

```
[INFO ] Serial controller started on /dev/serial/by-id/usb-Raspberry_Pi_Pico_...
DUMP BEGIN build=aab1925-dirty uptime=7925212 mcusr=0x3 free_ram=252712
DUMP Y state=1 ... speed=3000 center=500000 has_encoder=1
```

Check `build=` matches the firmware you flashed and `speed=` matches the
drivetrain values you intended. Then, and only then, move the thing:

```
gimbal home              # the endstop-finding sequence - see the warning below
gimbal diag y            # firmware self-test, reports PASS/FAIL
gimbal move 10,0         # a small deliberate move
```

**Expected noise in this phase, neither of which is a fault:**

| Message | Why |
|---|---|
| `Motor controller: ERR axis not present` | The host configures both axes at startup; a yaw-only firmware refuses the pitch half. Harmless until presence reconciliation lands (config plan section 10) |
| `encoder tracking error on yaw: 500366 counts (limit 400)` | Before homing, the firmware presets `XACTUAL` to the centre count while the encoder sits at its own zero, so they disagree by the whole range. Homing resyncs them. **If it persists after a successful home, that is real** |

> **On an unfamiliar drivetrain, homing is the dangerous step** - it finds the
> endstop by driving into it. Start from a deliberately slow `homing_speed` and
> walk it up, watching the free-running encoder gap that `ENC_DEVIATION_LIMIT` is
> measured against. Be at the rig with a hand on the power for the first one.

### Phase 2 - camera `[verified]`

Add the camera, still no broker:

```bash
./build/fire_gimbal_control --no-mqtt --no-tui
```

```
[INFO ] Opened camera 'rgb' DEV_1AB22C0B7D4E (1800 U-1620c)
[INFO ] camera profile 'standard' applied from .../standard.xml (17206 settings)
```

Then `start`, and confirm `.jxl` files appear under the configured output dir.

> **A camera-free phase needs no profile XMLs** - but this only became true on
> 2026-09-16. Before that, `[Features] enable_camera` was parsed and never used,
> and the profile directory was validated even for a mock camera, so a tower
> whose camera had not been configured yet could not start at all. If you are
> running an older binary, export the profiles (step 8) first or it will refuse
> with `no camera profiles (*.xml)`.

### Phase 3 - the broker

Set `[Features] enable_mqtt = true` and point `zkms_server_ip` at the broker.
Without one, the app says so and carries on:

```
[ERROR] MQTT connect failed (tcp://10.11.12.121:1883): TCP/TLS connect failure
[WARN ] Control channel not connected; continuing in degraded mode
```

**Degraded mode is not a failure to notice later** - the tower captures images
and reports to nobody. Confirm topics arrive at the broker before calling the
phase done.

Each tower runs its own RabbitMQ, in Docker, with the `rabbitmq_mqtt` and
`rabbitmq_shovel` plugins - the shovel is what forwards this tower's messages to
the ground station. That lives in `~/zkms-client/`, outside this repository.

### Phase 4 - IMU and ambient sensor

Only if fitted. `hardware.ini` should already say so, and `[Features]
enable_imu` / `enable_env` have to agree with it (see step 9 - declaring
`imu = none` does not yet switch the flag off by itself).

### Phase 5 - unattended

Only once every phase above is green:

```bash
scripts/fgc start --start     # detached, inside tmux
scripts/fgc status            # running? since when? which PID?
```

Close the ssh session, reconnect, and check it is still running. Then reboot the
tower and confirm the USB preconditions survived (`install.sh --probe`) and the
app comes back.

### Calibration

Per tower, and **cannot be inherited** - least of all from a tower with a
different gear ratio, where `yaw_counts_per_deg` is derived from a
`steps_per_rev` that no longer applies:

```
gimbal heading setup     # ties yaw to true north by sighting landmarks; no IMU needed
gimbal pitch setup       # needs an IMU; skip on a rig without one
```

## 11. Verification checklist

| Check | How |
|---|---|
| Key-only ssh | `ssh ggs@<tower> hostname`, no prompt |
| Preconditions survive a reboot | reboot, then `install.sh --probe`: `usbfs_memory_mb 1000`, `lpm_permit 0` |
| Camera enumerates | `/opt/VimbaX/bin/ListCameras` reports the `DEV_` id in `hardware.ini` |
| Declared hardware matches | startup logs no presence warnings |
| Firmware identity | `gimbal dump` reports the `GIT_REV` you flashed |
| Homing repeatable | `gimbal home` several times, endstop counts within `home_spread_max_counts` |
| Images land | `.jxl` files appear under the configured output dir |
| Survives a disconnect | `scripts/fgc start`, close ssh, reconnect, `scripts/fgc status` |