Rewrite the running and installing guides for how towers are set up now
This commit is contained in:
parent
08c749ca06
commit
91cc8af915
GPG Key ID:
D4A6C394DF13F1D7
|
|
@ -233,7 +233,7 @@ libjxl-dev`, plus the Vimba X SDK under `/opt/VimbaX` for `-DWITH_VIMBA=ON`.
|
|||
|----------|----------|
|
||||
| [docs/architecture.md](docs/architecture.md) | Components, interfaces, threading, data flow, capture state machine |
|
||||
| [docs/build-and-setup.md](docs/build-and-setup.md) | CMake, dependencies, options, Vimba X / Paho, host setup |
|
||||
| [docs/deployment.md](docs/deployment.md) | Step-by-step deploy to the gimbal LattePanda (systemd) |
|
||||
| [docs/deployment.md](docs/deployment.md) | Running and updating a tower: `deploy.sh`, `scripts/fgc`, the boot service, the log |
|
||||
| [docs/configuration.md](docs/configuration.md) | `config.ini` keys, CLI flags, console commands |
|
||||
| [docs/mqtt-api.md](docs/mqtt-api.md) | MQTT topic catalog and payloads |
|
||||
| [docs/test-command.md](docs/test-command.md) | The `test` hardware self-test suite: subsystems, profiles, reports, baselines |
|
||||
|
|
|
|||
|
|
@ -1,204 +1,163 @@
|
|||
# Deploying to the Gimbal LattePanda
|
||||
# Running and updating a tower
|
||||
|
||||
Step-by-step guide to deploy `fire_gimbal_control` on a tower's LattePanda (Linux x86-64). Like the
|
||||
firmware workflow, this is a repeatable **build → install → configure → enable → verify** cycle; the
|
||||
[Update / redeploy](#9-update--redeploy) section at the end is the short loop you run for every new version.
|
||||
How a tower that already works is operated and updated: pushing a new version
|
||||
from the dev box, running the program detached, starting it at boot, reading its
|
||||
log. A **first** install - a freshly imaged board, or a tower not yet migrated -
|
||||
is [tower-installation.md](tower-installation.md), which `scripts/install-tower`
|
||||
automates.
|
||||
|
||||
> **Which LattePanda.** The deployed fleet is the **3 Delta** — read from tower 111's DMI
|
||||
> `product_name`, *not* the Sigma, which this page claimed until 2026-09-18. Tower 121 is a
|
||||
> **LattePanda IOTA**; see [lattepanda-iota.md](lattepanda-iota.md) and its setup kit under
|
||||
> `hardware/lattepanda-iota/`. Nothing in this guide is board-specific — it is x86-64 Linux either way —
|
||||
> so the distinction matters for the device paths in `hardware.ini`, not for these steps.
|
||||
>
|
||||
> Assumes a Debian/Ubuntu-based Linux on the LattePanda. For Arch/Manjaro swap `apt` for `pacman`
|
||||
> (see [build-and-setup.md](build-and-setup.md)). Run as a normal user with `sudo` rights; the service runs
|
||||
> under a dedicated `ggs` user.
|
||||
The fleet: tower **111** is a LattePanda 3 Delta (ATmega32U4 motor MCU, two
|
||||
axes), tower **121** a LattePanda IOTA (RP2040, yaw only). Nothing here differs
|
||||
between them; what differs lives in each tower's `hardware.ini`.
|
||||
|
||||
## 0. Prerequisites (one-time per device)
|
||||
## Where things are on a tower
|
||||
|
||||
Install the toolchain and system libraries:
|
||||
| Path | What | Written by |
|
||||
|---|---|---|
|
||||
| `~ggs/projects/fwt_2a/software/` | this repository, and the program's build in `build/` | `deploy.sh` (sync + build), or the installer |
|
||||
| `~ggs/projects/fwt_2a/firmware/` | the firmware repository, built on the tower | `firmware/deploy.sh` |
|
||||
| `.../software/config.ini` | what the tower DOES: scan, capture, logging, calibration | the installer (seeded once), `gimbal ... setup` (calibration), a person |
|
||||
| `/etc/fire_gimbal_control/hardware.ini` | what the tower IS: board, axes, sensors, pins, drivetrain | the installer; a person when the hardware changes |
|
||||
| `/etc/fire_gimbal_control/credentials.env` | `FGC_MQTT_USER=` / `FGC_MQTT_PW=`, mode 0640 root:ggs | the installer |
|
||||
| `/etc/fire_gimbal_control/install.log` | one line per installer run | the installer |
|
||||
| `~ggs/.local/share/fire_gimbal_control/logs/` | the program's log, test reports | the program |
|
||||
| `.../software/config/camera_profiles/rgb/*.xml` | the camera settings, exported ON the tower from its camera | the installer, or `scripts/export_camera_profile.py` |
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y git cmake build-essential \
|
||||
libopencv-dev libjxl-dev libboost-all-dev
|
||||
```
|
||||
`deploy.sh` never touches `config.ini`, `hardware.ini`, the camera profiles or
|
||||
the images: they exist only on the tower, and `rsync --delete` once erased a
|
||||
freshly exported profile before they were excluded.
|
||||
|
||||
Install the **Allied Vision Vimba X SDK** (proprietary, needed for the cameras):
|
||||
## 1. Updating the program from the dev box
|
||||
|
||||
1. Download Vimba X for Linux x86-64 from
|
||||
<https://www.alliedvision.com/en/products/software/vimba-x-sdk/>.
|
||||
2. Unpack to a fixed location, e.g. `/opt/VimbaX_2024-1`.
|
||||
3. Run the SDK's transport-layer installer so cameras are discoverable, e.g.:
|
||||
```bash
|
||||
sudo /opt/VimbaX_2024-1/cti/Install_GenTL_Path.sh
|
||||
```
|
||||
4. Make its runtime libraries findable at run time:
|
||||
```bash
|
||||
echo /opt/VimbaX_2024-1/api/lib | sudo tee /etc/ld.so.conf.d/vimbax.conf
|
||||
sudo ldconfig
|
||||
```
|
||||
|
||||
Ensure an **MQTT broker** is reachable from the tower (the ground-station/ZKMS broker, or a local Mosquitto
|
||||
for bring-up).
|
||||
|
||||
## 1. Get the code
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/src && sudo chown "$USER" /opt/src
|
||||
cd /opt/src
|
||||
git clone <repo-url> fire_gimbal_control_src
|
||||
cd fire_gimbal_control_src
|
||||
# (or: git pull, for an update)
|
||||
```
|
||||
|
||||
## 2. Build (full, with cameras + MQTT)
|
||||
|
||||
```bash
|
||||
cmake -B build -DCMAKE_BUILD_TYPE=Release \
|
||||
-DWITH_VIMBA=ON -DWITH_MQTT=ON \
|
||||
-DVMB_HOME=/opt/VimbaX_2024-1
|
||||
cmake --build build -j"$(nproc)"
|
||||
```
|
||||
|
||||
- `WITH_MQTT=ON` fetches + builds Eclipse Paho from source on first configure (needs network).
|
||||
- If a camera build isn't needed yet, use `-DWITH_VIMBA=OFF` for a mock binary to test the pipeline.
|
||||
- Smoke-test the core before installing: `ctest --test-dir build` (after adding `-DBUILD_TESTING=ON`).
|
||||
|
||||
## 3. Install to a fixed location
|
||||
|
||||
```bash
|
||||
sudo useradd -r -s /usr/sbin/nologin ggs 2>/dev/null || true
|
||||
sudo mkdir -p /opt/fire_gimbal_control
|
||||
sudo cp build/fire_gimbal_control /opt/fire_gimbal_control/
|
||||
# demo-mode placeholder image (only needed if you ever run --demo)
|
||||
sudo cp bin/x64/Release/test_smoke.jxl /opt/fire_gimbal_control/ 2>/dev/null || true
|
||||
```
|
||||
|
||||
## 4. Configure
|
||||
|
||||
```bash
|
||||
sudo cp config/config.example.ini /opt/fire_gimbal_control/config.ini
|
||||
sudo nano /opt/fire_gimbal_control/config.ini
|
||||
```
|
||||
|
||||
Set at least:
|
||||
On the dev box, in `software/`, with a `.deploy.env` naming the tower:
|
||||
|
||||
```ini
|
||||
[General]
|
||||
tower_name = ExampleTower ; this tower's name (drives all MQTT topics)
|
||||
image_interval = 3
|
||||
|
||||
[Network]
|
||||
zkms_server_ip = 10.11.12.13 ; the broker address
|
||||
|
||||
[Serial]
|
||||
device = /dev/ttyACM0 ; the motor-controller MCU
|
||||
baud = 115200
|
||||
|
||||
[Camera.rgb]
|
||||
id = DEV_1AB22C0AADED ; or a GigE IP like 192.168.11.101
|
||||
label = RGB
|
||||
profile_dir = config/camera_profiles/rgb
|
||||
profile = standard
|
||||
REMOTE_HOST=ggs@10.11.12.121
|
||||
REMOTE_DIR=/home/ggs/projects/fwt_2a/software
|
||||
CMAKE_ARGS="-DWITH_MQTT=ON -DWITH_VIMBA=ON"
|
||||
RUN_ARGS="--start"
|
||||
```
|
||||
|
||||
**Credentials** go in the environment, not the config file. Create a root-only env file for the service:
|
||||
|
||||
```bash
|
||||
sudo install -m 600 /dev/null /etc/fire_gimbal_control/credentials.env 2>/dev/null || \
|
||||
{ sudo mkdir -p /etc/fire_gimbal_control && sudo install -m 600 /dev/null /etc/fire_gimbal_control/credentials.env; }
|
||||
printf 'FGC_MQTT_USER=fwt_gimbal\nFGC_MQTT_PW=CHANGE_ME\n' | sudo tee /etc/fire_gimbal_control/credentials.env >/dev/null
|
||||
sudo chmod 600 /etc/fire_gimbal_control/credentials.env
|
||||
./deploy.sh # sync the tree, build on the tower
|
||||
./deploy.sh --check-deps # only check the tower's build dependencies
|
||||
./deploy.sh --clean # wipe the tower's build dir first (fresh configure)
|
||||
```
|
||||
|
||||
Give the `ggs` user access to its files and the image output dir:
|
||||
It syncs the tree (leaving `.git` behind, and writing the revision into
|
||||
`.revision` beside it instead), then configures and builds **on the tower**. The
|
||||
program already running keeps running the old binary until it is restarted:
|
||||
|
||||
```bash
|
||||
sudo chown -R ggs:ggs /opt/fire_gimbal_control
|
||||
# on a tower started by its boot service
|
||||
ssh ggs@<tower> sudo systemctl restart fire-gimbal-control
|
||||
# on a tower started by hand
|
||||
./deploy.sh --stop && ./deploy.sh --run # --run --home adds --init
|
||||
```
|
||||
|
||||
## 5. Hardware access
|
||||
`--run` starts the program detached through `scripts/fgc` (section 2) and first
|
||||
refuses a `config.ini` that still holds the example's placeholders
|
||||
(`ExampleTower`, `CHANGE_ME`, `DEV_0000000000`); `--force` overrides that. On a
|
||||
tower whose service is running, `--run` fails on the single-instance lock
|
||||
(section 3) - restart it through systemd instead.
|
||||
|
||||
**Serial (motor controller):** let the service user open the port.
|
||||
After an update, check it:
|
||||
|
||||
```bash
|
||||
sudo usermod -aG dialout ggs
|
||||
ssh ggs@<tower> 'cd ~/projects/fwt_2a/software && scripts/install-tower --verify'
|
||||
```
|
||||
|
||||
For a stable device name across reboots, add a udev rule keyed on the MCU's USB IDs (find them with
|
||||
`udevadm info -a -n /dev/ttyACM0 | grep -m1 idVendor`) and point `[Serial] device` at the symlink:
|
||||
It changes nothing; it runs `test hardware` inside the running program (pausing
|
||||
and then resuming capture) and checks the files, the firmware's pins and the
|
||||
boot service.
|
||||
|
||||
`--push-config` also overwrites the tower's `config.ini` with the dev box's,
|
||||
backing the tower's copy up first. It exists for the development tower; a
|
||||
tower's `config.ini` holds its own calibration, so do not push one tower's to
|
||||
another. `hardware.ini` is never pushed.
|
||||
|
||||
> **One-time: `git pull` can delete your `.deploy.env`.** It was tracked by
|
||||
> mistake until `b0127db`, which stopped tracking it and added it to
|
||||
> `.gitignore`. Pulling *across* that commit deletes the working copy, credentials
|
||||
> and all, with no warning. Only checkouts that predate `b0127db` are exposed,
|
||||
> and only once: copy it aside first (`cp .deploy.env ../deploy.env.bak`), pull,
|
||||
> copy it back. To recover one already lost: `git show b0127db^:.deploy.env`.
|
||||
|
||||
### Updating the firmware
|
||||
|
||||
**Host first, firmware second**, always: a new firmware asks the host for its
|
||||
drivetrain at boot, and an older host cannot answer. In `firmware/`, with its own
|
||||
`.deploy.env` (`REMOTE_HOST`, `REMOTE_DIR`, `PIO_ENV` = `leonardo` or
|
||||
`iota_rp2040`, `SERIAL_PORT` = the MCU's `/dev/serial/by-id/` path):
|
||||
|
||||
```bash
|
||||
echo 'SUBSYSTEM=="tty", ATTRS{idVendor}=="XXXX", ATTRS{idProduct}=="YYYY", SYMLINK+="ttyGimbal"' | \
|
||||
sudo tee /etc/udev/rules.d/99-gimbal.rules
|
||||
sudo udevadm control --reload && sudo udevadm trigger
|
||||
./deploy.sh --compile-only # sync + build on the tower, flash nothing
|
||||
./deploy.sh # build + flash
|
||||
./deploy.sh --monitor # ... then watch it boot
|
||||
```
|
||||
|
||||
**GigE cameras (if used):** put the host NIC on the camera subnet (e.g. `192.168.11.0/24`), enable jumbo
|
||||
frames, and confirm discovery with the Vimba X `VimbaXViewer`/`ListCameras` tool.
|
||||
The build takes the pins from the tower's `hardware.ini` (`[Pins]`) and fails
|
||||
without them. The flashing is `firmware/tools/flash.sh`, run on the tower: an
|
||||
RP2040 is put in its bootloader by the firmware's own `BOOTSEL` command first; a
|
||||
Leonardo uses its 1200-baud touch. **Stop the program first** - it holds the
|
||||
serial port - and restart it after; the new firmware then asks, the host
|
||||
answers, and `install-tower --verify` confirms the firmware holds what
|
||||
`hardware.ini` declares.
|
||||
|
||||
## 6. First run by hand (verify before enabling the service)
|
||||
## 2. Running detached (`scripts/fgc`)
|
||||
|
||||
The tower is operated over ssh, so the program has to survive the connection
|
||||
closing. `scripts/fgc` runs it inside a tmux session, which owns the pty:
|
||||
|
||||
```bash
|
||||
cd /opt/fire_gimbal_control
|
||||
set -a; . /etc/fire_gimbal_control/credentials.env; set +a
|
||||
./fire_gimbal_control --config ./config.ini --init --start --log-level debug
|
||||
```
|
||||
|
||||
Watch for: `Loaded config`, `Serial controller started`, `Opened camera ...`, `MQTT connected`, then
|
||||
`Saved .../RGB/<timestamp>.jxl` after the first capture. Type `exit` (or Ctrl-D) to stop. To validate without
|
||||
cameras first: add `--mock-camera --mock-serial --no-mqtt`.
|
||||
|
||||
This run is attached to your terminal and dies with the ssh session — fine for a first check, not for
|
||||
leaving it running. For that, use section 6b.
|
||||
|
||||
### 6b. Running detached (`scripts/fgc`)
|
||||
|
||||
The tower is operated over ssh, so the program has to survive the connection closing. `scripts/fgc` runs
|
||||
it inside a tmux session, which owns the pty:
|
||||
|
||||
```bash
|
||||
cd /opt/fire_gimbal_control
|
||||
cd ~/projects/fwt_2a/software
|
||||
scripts/fgc start --start # starts detached, returns as soon as it is up
|
||||
scripts/fgc status # running? since when? which PID?
|
||||
scripts/fgc attach # the dashboard — Ctrl-b d detaches, leaves it running
|
||||
scripts/fgc attach # the dashboard - Ctrl-b d detaches, leaves it running
|
||||
scripts/fgc stop # SIGTERM -> the same clean teardown as `exit`
|
||||
scripts/fgc peek # snapshot of the pane without attaching
|
||||
```
|
||||
|
||||
**`Ctrl-b d` detaches. The dashboard's own `q` key opens an exit prompt** offering `q` (quit the
|
||||
program) or `b` (background — detach and leave it running, the same as `Ctrl-b d`), with `Esc` to
|
||||
cancel. `b` is offered only inside a `scripts/fgc` session; outside one there is nothing to detach
|
||||
into and it is shown dimmed.
|
||||
The boot service (section 4) runs the program the same way, so all of this
|
||||
works on a tower started at boot too.
|
||||
|
||||
`scripts/fgc stop` sends SIGTERM and waits up to 30s; the program handles it with the same shutdown path
|
||||
as the `exit` command (stop capture, close the camera, flush the image queue). Do **not** `kill -9` —
|
||||
an unclean exit leaves the camera wedged for the next run. If it will not stop, attach and look at why.
|
||||
**`Ctrl-b d` detaches. The dashboard's own `q` key opens an exit prompt**
|
||||
offering `q` (quit the program) or `b` (background - detach and leave it
|
||||
running, the same as `Ctrl-b d`), with `Esc` to cancel. `b` is offered only
|
||||
inside a `scripts/fgc` session; outside one there is nothing to detach into and
|
||||
it is shown dimmed.
|
||||
|
||||
`scripts/fgc start` refuses if a run is already up, and prints the previous run's last output if it died
|
||||
rather than silently discarding it.
|
||||
`scripts/fgc stop` sends SIGTERM and waits up to 120 s; the program handles it
|
||||
with the same shutdown path as the `exit` command (stop capture, close the
|
||||
camera, flush the image queue). Do **not** `kill -9` - an unclean exit leaves the
|
||||
camera wedged for the next run. If it will not stop, attach and look at why.
|
||||
|
||||
Choosing **quit** at that prompt while attached ends the tmux session too, so you land straight back in
|
||||
your shell — `exit` then closes ssh. A run that *fails* is the exception: its pane is kept so you can read
|
||||
why (`remain-on-exit failed`), and `fgc status` shows it until `fgc stop` or the next `fgc start` clears it.
|
||||
`scripts/fgc start` refuses if a run is already up, and prints the previous
|
||||
run's last output if it died rather than silently discarding it.
|
||||
|
||||
A kept pane has **no process behind it** — keys and Ctrl-C do nothing, which reads as a hung ssh session.
|
||||
`Ctrl-b d` still detaches (tmux, not the pane, handles it); the session's status line spells that out along
|
||||
the bottom of the screen. As a last resort `Enter ~ .` closes the ssh client itself from your end.
|
||||
Choosing **quit** at that prompt while attached ends the tmux session too, so you
|
||||
land straight back in your shell. A run that *fails* is the exception: its pane
|
||||
is kept so you can read why (`remain-on-exit failed`), and `fgc status` shows it
|
||||
until `fgc stop` or the next `fgc start` clears it. A kept pane has **no process
|
||||
behind it** - keys and Ctrl-C do nothing, which reads as a hung ssh session.
|
||||
`Ctrl-b d` still detaches; as a last resort `Enter ~ .` closes the ssh client.
|
||||
|
||||
> **Why `_run` waits on the app instead of `exec`ing it.** When the app is itself the pane's process, tmux
|
||||
> never records an exit status for it — `#{pane_dead_status}` comes back empty and the process lingers as a
|
||||
> zombie — so `remain-on-exit failed` cannot tell success from failure and keeps *every* finished pane,
|
||||
> including a clean `q`. That is what used to strand an operator on a frozen dead pane. Interposing a shell
|
||||
> that reaps the child and exits with its status gives tmux a real status. Measured on the device: 3/3 panes
|
||||
> kept when exec'd, 4/4 sessions closed cleanly when parented. The wrapper forwards SIGTERM/SIGINT/SIGHUP,
|
||||
> so `fgc stop` still reaches the app and runs the full teardown.
|
||||
MQTT credentials are read from `/etc/fire_gimbal_control/credentials.env` by
|
||||
`fgc` itself, inside the pane, as are the Vimba X paths: a tmux server that
|
||||
already exists hands a new session its own environment, not the caller's.
|
||||
|
||||
### 6c. The single-instance lock
|
||||
> **Why `_run` waits on the app instead of `exec`ing it.** When the app is itself
|
||||
> the pane's process, tmux never records an exit status for it, so
|
||||
> `remain-on-exit failed` cannot tell success from failure and keeps *every*
|
||||
> finished pane. Interposing a shell that reaps the child and exits with its
|
||||
> status gives tmux a real one. Measured on the device: 3/3 panes kept when
|
||||
> exec'd, 4/4 sessions closed cleanly when parented. The wrapper forwards
|
||||
> SIGTERM/SIGINT/SIGHUP, so `fgc stop` still reaches the app.
|
||||
|
||||
Two instances would fight over the motor's serial port and the camera. The binary
|
||||
prevents that itself: at startup it takes an exclusive `flock` on
|
||||
## 3. The single-instance lock
|
||||
|
||||
Two instances would fight over the motor's serial port and the camera. The
|
||||
binary prevents that itself: at startup it takes an exclusive `flock` on
|
||||
|
||||
- `/run/user/<uid>/fire_gimbal_control.lock` when that directory exists, else
|
||||
- `/tmp/fire_gimbal_control-<uid>.lock`
|
||||
|
|
@ -209,24 +168,31 @@ and, if it is already held, exits with status **2** and:
|
|||
Cannot start: another instance is already running (PID 1234); attach to it with: scripts/fgc attach
|
||||
```
|
||||
|
||||
The path is derived from the uid and not from `$XDG_RUNTIME_DIR` on purpose: that variable is set in an
|
||||
interactive ssh shell but not in a non-interactive `ssh host cmd` one, and a lock the two launch routes
|
||||
disagree about would exclude nothing.
|
||||
The path is derived from the uid and not from `$XDG_RUNTIME_DIR` on purpose: that
|
||||
variable is set in an interactive ssh shell but not in a non-interactive
|
||||
`ssh host cmd` one, and a lock the two launch routes disagree about would exclude
|
||||
nothing. For the same reason a tower with a boot service needs **lingering**
|
||||
(section 4).
|
||||
|
||||
Because the check lives in the binary, it covers every route in — `scripts/fgc`, systemd, or a bare
|
||||
`./fire_gimbal_control`. Nothing needs cleaning up after a crash or a power cut: the kernel releases an
|
||||
`flock` when the holder dies, so a leftover lockfile with a dead PID in it never blocks a launch. Use
|
||||
`--no-lock` only to run a second mock-only instance during development, and `--lock-file <path>` to
|
||||
point at a different lockfile.
|
||||
Because the check lives in the binary, it covers every route in - `scripts/fgc`,
|
||||
the service, or a bare `./build/fire_gimbal_control`. Nothing needs cleaning up
|
||||
after a crash or a power cut: the kernel releases an `flock` when the holder
|
||||
dies. Use `--no-lock` only to run a second mock-only instance during
|
||||
development, and `--lock-file <path>` to point at a different lockfile.
|
||||
|
||||
## 7. Install the systemd service
|
||||
## 4. Starting at boot
|
||||
|
||||
`scripts/install-tower` does this (it asks first, and shows the unit it will
|
||||
`scripts/install-tower` installs this (it asks first, and shows the unit it will
|
||||
write). The unit is the template `scripts/fire-gimbal-control.service`, and it
|
||||
runs the program **through `scripts/fgc`, inside tmux** - so a tower started at
|
||||
boot still has its dashboard (`scripts/fgc attach`), and `fgc status` / `peek` /
|
||||
`stop` work from any ssh session. It is not an alternative to the tmux workflow
|
||||
any more; it is how that workflow starts at boot.
|
||||
boot still has its dashboard, and `fgc status` / `peek` / `stop` work from any
|
||||
ssh session.
|
||||
|
||||
```bash
|
||||
sudo systemctl start|stop|restart fire-gimbal-control
|
||||
systemctl is-enabled fire-gimbal-control # starts at boot?
|
||||
sudo systemctl disable fire-gimbal-control # not any more
|
||||
```
|
||||
|
||||
By hand, fill in `@USER@`, `@REPO@` and `@ARGS@` (normally `--init --start`) and:
|
||||
|
||||
|
|
@ -244,73 +210,27 @@ running", and a second copy could start beside the first.
|
|||
|
||||
**`--init` homes the gimbal unattended at every boot.** That is what a tower in
|
||||
service needs - the firmware keeps no position across a power cycle, so without
|
||||
it the gimbal sits un-homed and captures nothing - but it is a deliberate choice,
|
||||
and the installer asks for the arguments.
|
||||
it the gimbal sits un-homed and captures nothing - but it is a deliberate
|
||||
choice, and the installer asks for the arguments.
|
||||
|
||||
MQTT credentials go in `/etc/fire_gimbal_control/credentials.env`
|
||||
(`FGC_MQTT_USER=` / `FGC_MQTT_PW=`, mode 0640 root:ggs). `fgc` reads it inside
|
||||
the tmux pane rather than taking it from the unit's environment: a tmux server
|
||||
that already exists hands a new session its own environment, not the caller's.
|
||||
|
||||
## 8. Enable and verify
|
||||
|
||||
```bash
|
||||
sudo systemctl start fire-gimbal-control
|
||||
scripts/fgc status # running, lockfile under /run/user/<uid>
|
||||
scripts/install-tower --verify # every check, including test hardware
|
||||
```
|
||||
| Tower | Service | Starts with |
|
||||
|---|---|---|
|
||||
| 111 | enabled 2026-09-24 | `--init --start`: homes and captures at boot. Not yet exercised by a reboot |
|
||||
| 121 | enabled 2026-09-23 | no arguments: starts idle, neither homes nor captures (the development tower) |
|
||||
|
||||
`[verified]` on tower 121, 2026-09-23: the service came up by itself after a
|
||||
reboot with nobody logged in, with its lock under `/run/user/1000`; `fgc status`
|
||||
and the dashboard worked from a later ssh session; credentials reached the
|
||||
program intact (a password with `$`, a quote and a space); `systemctl stop` shut
|
||||
it down cleanly; and `install-tower --verify` passed after the reboot, USB
|
||||
preconditions included. 121 runs it with no arguments, so it does not home at
|
||||
boot - see the note on `--init` above.
|
||||
preconditions included.
|
||||
|
||||
Confirm end to end:
|
||||
- Images accumulating under the configured `[Paths] output_dir` (default
|
||||
`~ggs/.local/share/fire_gimbal_control/images/RGB/`).
|
||||
- CamEvents on the broker:
|
||||
`mosquitto_sub -h <broker> -t 'GGS/FWT/<tower>/#' -v`.
|
||||
A tower started this way is in service when:
|
||||
- images accumulate under `[Paths] output_dir` (default
|
||||
`~ggs/.local/share/fire_gimbal_control/images/RGB/`);
|
||||
- CamEvents reach the broker: `mosquitto_sub -h <broker> -t 'GGS/FWT/<tower>/#' -v`.
|
||||
|
||||
## 9. Update / redeploy
|
||||
|
||||
The short loop for each new version (mirrors the firmware flash cycle):
|
||||
|
||||
```bash
|
||||
cd /opt/src/fire_gimbal_control_src
|
||||
git pull
|
||||
cmake --build build -j"$(nproc)" # reuses the configured build dir
|
||||
sudo systemctl stop fire-gimbal-control
|
||||
sudo cp build/fire_gimbal_control /opt/fire_gimbal_control/
|
||||
sudo systemctl start fire-gimbal-control
|
||||
journalctl -u fire-gimbal-control -n 50 -f # confirm healthy
|
||||
```
|
||||
|
||||
If `CMakeLists.txt` or options changed, re-run the `cmake -B build ...` configure step from
|
||||
[section 2](#2-build-full-with-cameras--mqtt) first. To roll back, keep the previous binary
|
||||
(`fire_gimbal_control.prev`) and swap it back.
|
||||
|
||||
> **One-time: `git pull` can delete your `.deploy.env`.** It was tracked by
|
||||
> mistake until `b0127db`, which stopped tracking it and added it to
|
||||
> `.gitignore`. Pulling *across* that commit therefore does what git does to any
|
||||
> file a commit removes — deletes the working copy, credentials and all. It is a
|
||||
> checkout, not a merge conflict, so there is no warning and nothing to resolve.
|
||||
>
|
||||
> Only checkouts that predate `b0127db` are exposed, and only once. Copy it aside
|
||||
> first:
|
||||
>
|
||||
> ```bash
|
||||
> cp .deploy.env ../deploy.env.bak 2>/dev/null # then pull, then copy back
|
||||
> ```
|
||||
>
|
||||
> After that pull the file is ignored and stays put. `deploy.sh` already excludes
|
||||
> it from the rsync, so deploying never overwrites the tower's own copy — this is
|
||||
> purely about the machine you deploy *from*. To recover one already lost:
|
||||
> `git show b0127db^:.deploy.env`.
|
||||
|
||||
## 10. Where the log is
|
||||
## 5. Where the log is
|
||||
|
||||
The program writes a persistent log alongside the self-test and diagnostics
|
||||
reports:
|
||||
|
|
@ -325,7 +245,7 @@ roughly 5 MB/day in steady state.
|
|||
|
||||
This exists because a tower runs detached and, in TUI mode, every line otherwise
|
||||
goes to a 500-line on-screen ring buffer that dies with the tmux session. The
|
||||
file is **more verbose than the dashboard** — `info` on screen, `debug` on disk —
|
||||
file is **more verbose than the dashboard** - `info` on screen, `debug` on disk -
|
||||
so it holds detail the pane never showed.
|
||||
|
||||
```bash
|
||||
|
|
@ -337,9 +257,8 @@ Use `tail -F`, not `-f`: rotation replaces the file and `-f` would follow the ol
|
|||
inode into a file nothing writes to any more.
|
||||
|
||||
Each file opens with a header naming the build, tower, PID, config and hardware
|
||||
paths and the full command line — repeated after every rotation, so an archive
|
||||
found on its own still says where it came from. Lines carry a `YYYY-MM-DD` prefix
|
||||
the console ones do not.
|
||||
paths and the full command line - repeated after every rotation, so an archive
|
||||
found on its own still says where it came from.
|
||||
|
||||
What to grep when a tower will not come up:
|
||||
|
||||
|
|
@ -352,17 +271,22 @@ head -8 "$L" # which build, which
|
|||
|
||||
> Wire traces (`[Logging] trace` / `--trace`) reach this file too. The serial
|
||||
> trace alone takes it from ~5 MB/day to ~41 MB/day, enough to age out the
|
||||
> startup record within days — turn it on to chase something, off afterwards.
|
||||
> startup record within days - turn it on to chase something, off afterwards.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
Start with `scripts/install-tower --verify` on the tower: it checks most of the
|
||||
list below and says which.
|
||||
|
||||
| Symptom | Check |
|
||||
|---------|-------|
|
||||
| `No config.ini found` | `FGC_CONFIG` / `--config` path; service `Environment=` line |
|
||||
| `Failed to open serial port` | MCU plugged in, `dialout` group, `[Serial] device` path / udev symlink |
|
||||
| `No camera found` / Vimba startup error | SDK installed, GenTL path set, `ldconfig`, NIC subnet, camera IDs in config |
|
||||
| `MQTT connect failed` (degraded mode) | broker reachable, credentials env, firewall |
|
||||
| Runs but no images | capture started (`--start` or `start` command), `output_dir` writable by `ggs` |
|
||||
| `another instance is already running` (exit 2) | `scripts/fgc status`; a systemd unit and a tmux session both running (section 6c) |
|
||||
| Program dies when ssh disconnects | started with `run.sh`/`--foreground` instead of `scripts/fgc start` (section 6b) |
|
||||
| `No camera found` only when started via `scripts/fgc` | GenTL path not resolved in the pane — check `scripts/fgc peek`, and `/etc/profile.d/*Vimba*GenTL*.sh` or `VIMBA_CTI_PATH` |
|
||||
| `No config.ini found` | run from `~/projects/fwt_2a/software`, or `--config` / `$FGC_CONFIG` |
|
||||
| `Config error` naming a key | `build/fgc-config check --config config.ini --hardware /etc/fire_gimbal_control/hardware.ini` |
|
||||
| `Failed to open serial port` | MCU plugged in, `ggs` in `dialout`, `[Serial] device` in `hardware.ini` - a `/dev/serial/by-id/` path, which **changes when the MCU is flashed** |
|
||||
| An axis will not move, `ERR need config` | `hardware.ini` is missing a drivetrain value for it; the log names the key |
|
||||
| `No camera found` / Vimba startup error | SDK installed, GenTL path set (`/etc/profile.d/*Vimba*GenTL*.sh` or `VIMBA_CTI_PATH`), `[Camera.rgb] id` in `hardware.ini` |
|
||||
| `MQTT connect failed` (degraded mode) | broker reachable, `credentials.env` readable by `ggs`, firewall |
|
||||
| Runs but no images | capture started (`--start`, or the `start` command) and the gimbal homed; `output_dir` writable |
|
||||
| `another instance is already running` (exit 2) | `scripts/fgc status`; the service and a hand-started run both up (section 3) |
|
||||
| `fgc status` says not running, but the program is | the service started it before anyone logged in, without lingering (section 4) |
|
||||
| Program dies when ssh disconnects | started with `run.sh` / `deploy.sh --foreground` instead of `scripts/fgc start` |
|
||||
|
|
|
|||
|
|
@ -1,56 +1,88 @@
|
|||
# Installing a tower from scratch
|
||||
# Installing a tower
|
||||
|
||||
How a new fire-watch tower host goes from a freshly imaged machine to a running,
|
||||
calibrated `fire_gimbal_control`. Written while doing exactly that on the IOTA
|
||||
tower at `10.11.12.121` (2026-09-15) and kept honest about which steps have
|
||||
actually been exercised and which are still on paper.
|
||||
How a fire-watch tower host goes from a freshly imaged machine - or from an
|
||||
older, unmigrated setup - to a running, calibrated `fire_gimbal_control`.
|
||||
[deployment.md](deployment.md) is what comes after: updating and operating a
|
||||
tower that already works.
|
||||
|
||||
> [deployment.md](deployment.md) is the *update* loop for a tower that already
|
||||
> works. This is the *first* install, including the parts that only happen once:
|
||||
> the board's preconditions, the Vimba X SDK, `hardware.ini`, and the motor
|
||||
> firmware.
|
||||
**`scripts/install-tower` does the whole of this.** The short way is the next
|
||||
section. Everything after it, sections 1 to 11, is the reference for what each
|
||||
step does and why, written while doing each one by hand, and still the way to
|
||||
do one step on its own.
|
||||
|
||||
> **`scripts/install-tower` now does the steps below**, on the tower, in this
|
||||
> order: it surveys the machine, asks for what it cannot work out (every
|
||||
> question can also be answered with `--answer KEY=VALUE`), shows the plan, and
|
||||
> applies it after one yes. `--dry-run` shows the plan and changes nothing;
|
||||
> `--verify` runs the checks only. It never invents a drivetrain value: answer
|
||||
> `unknown` and it writes a `; TODO`, and refuses to flash firmware an axis of
|
||||
> which would then sit in `NEEDCFG`. Homing is offered only at the end, and only
|
||||
> to someone who says they are at the rig.
|
||||
>
|
||||
> `[verified]` as a re-run on the already-configured tower 121 (2026-09-23): it
|
||||
> changed only what was out of date and passed `test hardware`. `[untested]` on a
|
||||
> freshly imaged board - the fresh path has been rehearsed with `--dry-run` only.
|
||||
> What follows stays the reference for what each step does and why.
|
||||
|
||||
**Status legend** — each step says where it stands:
|
||||
**Status legend** - each step says where it stands:
|
||||
`[verified]` run end to end on a real tower and the result checked ·
|
||||
`[partial]` run, with something still open · `[untested]` written from the
|
||||
design, not yet exercised.
|
||||
|
||||
---
|
||||
|
||||
## What you need before starting
|
||||
## The short way: `scripts/install-tower`
|
||||
|
||||
### Before starting
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| The tower host | Powered, on the network, reachable by IP, with an OS installed and an `ssh` server running |
|
||||
| An account on it | `ggs`, in the `sudo` and `dialout` groups |
|
||||
| A dev box | This repository checked out, plus a copy of the Vimba X SDK (login-walled, so it cannot be fetched by a script) |
|
||||
| The hardware | Camera plugged in, motor MCU plugged in, and a note of **what was assembled**: which axes, which IMU, which ambient sensor |
|
||||
| The tower host | Powered, on the network, an OS installed, `ssh` running, reachable by key (section 1) |
|
||||
| An account | `ggs`, in the `sudo` and `dialout` groups |
|
||||
| Both repositories on it | `~/projects/fwt_2a/software` and `~/projects/fwt_2a/firmware` - cloned (`git.ggs-solutions.eu:GGS/fwt_software.git` and `fwt_firmware.git`, which needs a key on the tower), or synced by each repo's `deploy.sh` |
|
||||
| The Vimba X SDK, staged on it | Login-walled, so it cannot be fetched by a script: copy it on (USB stick, `scp`) and give its directory when asked. `none` for a tower with no camera yet |
|
||||
| The facts of the build | Which axes, which IMU, which ambient sensor, how the driver boards are wired, and - per axis - the drivetrain: sense resistor, currents, travel, `steps_per_rev`, `sw_mode`, `enc_deviation`, encoder or not. Anything not yet measured can be answered `unknown` |
|
||||
|
||||
Two facts decide most of what follows, so establish them first:
|
||||
|
||||
```bash
|
||||
ssh ggs@<tower-ip> 'cat /sys/class/dmi/id/product_name; . /etc/os-release && echo "$PRETTY_NAME"'
|
||||
```
|
||||
The installer takes the board from DMI and picks its kit:
|
||||
|
||||
| Board | MCU | Setup kit | PlatformIO env |
|
||||
|---|---|---|---|
|
||||
| LattePanda 3 Delta / Sigma | ATmega32U4 (Leonardo), 5 V | *not written - the Delta fleet predates the kits* | `leonardo` |
|
||||
| LattePanda 3 Delta | ATmega32U4 (Leonardo), 5 V | [`hardware/lattepanda-3-delta/`](../hardware/lattepanda-3-delta/) | `leonardo` |
|
||||
| LattePanda IOTA | RP2040, 3.3 V | [`hardware/lattepanda-iota/`](../hardware/lattepanda-iota/) | `iota_rp2040` |
|
||||
|
||||
### Running it
|
||||
|
||||
On the tower, as `ggs`:
|
||||
|
||||
```bash
|
||||
cd ~/projects/fwt_2a/software
|
||||
scripts/install-tower --dry-run # survey + questions + the plan; changes nothing
|
||||
scripts/install-tower # the same, then does it after one yes
|
||||
scripts/install-tower --verify # the checks only; changes nothing
|
||||
```
|
||||
|
||||
It surveys the machine, asks only what it cannot work out, prints a plan with
|
||||
each step marked `skip`, `create`, `change` or `blocked`, and applies it after
|
||||
one yes. In order:
|
||||
|
||||
1. packages, then the USB preconditions (section 3), then the Vimba X SDK (4)
|
||||
2. builds the host
|
||||
3. writes `hardware.ini` (5) - or, when one exists, only fills its gaps
|
||||
4. seeds `config.ini` (9) and `credentials.env`, or edits only what is out of date
|
||||
5. PlatformIO, then the camera profiles that are missing (8)
|
||||
6. builds and flashes the firmware (6) - **blocked** while any drivetrain value
|
||||
is still a `; TODO`, because that axis would refuse to move
|
||||
7. `test hardware`
|
||||
8. the boot service ([deployment.md](deployment.md) section 4), if wanted
|
||||
9. asks whether someone is at the rig; only on a yes does it home the gimbal and
|
||||
take a short capture - through the service, when the service is what homes it
|
||||
|
||||
Every question has a key, so any can be answered up front - `--answer
|
||||
tower_name=north`, `--answer yaw.r_sense_ohm=0.075` - or from a file of
|
||||
`key=value` lines with `--answers FILE`. `--non-interactive` stops instead of
|
||||
prompting. **It never invents a drivetrain value**: a wrong sense resistor cooks a
|
||||
driver and a wrong homing current shears an endstop, and both have happened on
|
||||
this project. Answer `unknown` and it writes a commented `; TODO` naming what to
|
||||
measure.
|
||||
|
||||
Re-running it is safe, and is how a configured tower is repaired: an existing
|
||||
`hardware.ini` or `config.ini` is never regenerated, only filled in; an existing
|
||||
camera profile is never overwritten; a firmware already running the declared
|
||||
revision and pins is not reflashed; and if it fails after stopping the program,
|
||||
it puts the program back the way it was.
|
||||
|
||||
| Where | Result |
|
||||
|---|---|
|
||||
| Tower 121 (IOTA), 2026-09-23 | `[verified]` re-run on a configured tower: changed only what was out of date, `--verify` green, boot service installed and proven across a reboot |
|
||||
| Tower 111 (Delta), 2026-09-23/24 | `[verified]` migration of a tower that predated all of this: host first, firmware once the drivetrain was known, `test hardware` green on every leaf (section 10b) |
|
||||
| A freshly imaged board | `[untested]` - rehearsed with `--dry-run` and typed answers only. The first flash of a factory MicroPython RP2040 is automated but has not met a factory board |
|
||||
|
||||
---
|
||||
|
||||
## 1. SSH by key `[verified]`
|
||||
|
|
@ -158,9 +190,11 @@ first reboot, since surviving one is the entire point.
|
|||
> ssh -t ggs@<tower-ip> 'cd ~/projects/fwt_2a/software/hardware/lattepanda-iota && sudo ./install.sh --deps --preconditions'
|
||||
> ```
|
||||
|
||||
> **The Delta fleet does not have these applied.** Checked on `10.11.12.111`
|
||||
> 2026-09-15: `usbfs_memory_mb` is 16 and the kernel cmdline is stock. The
|
||||
> preconditions were documented in known-issues but never persisted anywhere.
|
||||
> **The Delta kit carries the same two files** (`hardware/lattepanda-3-delta/`,
|
||||
> as symlinks to these). Tower 111 had neither - `usbfs_memory_mb` 16, camera
|
||||
> LPM `u1_u2`, checked 2026-09-15 and again 2026-09-23 - and got both on
|
||||
> 2026-09-23, reading 1000 and `0` afterwards. Not yet confirmed across a reboot
|
||||
> on a Delta.
|
||||
|
||||
## 4. Vimba X SDK `[verified]`
|
||||
|
||||
|
|
@ -289,7 +323,8 @@ is what catches the second case, so it has to be set from the real travel.
|
|||
> `hardware.ini` is merged *over* `config.ini` and wins, so a shared key would
|
||||
> silently discard whatever `gimbal heading setup` had saved. What it does instead
|
||||
> is work the requirement out from `steps_per_rev` and print it as a `NOTE` in the
|
||||
> block, because the default is wrong for any geared-up rig:
|
||||
> block, because the default is wrong for any geared-up rig - and
|
||||
> `scripts/install-tower` then sets it in `config.ini` when it is too small:
|
||||
>
|
||||
> ```
|
||||
> ; NOTE: half this travel is 605389 counts, so config.ini needs
|
||||
|
|
@ -315,9 +350,23 @@ yaw motor current: run 756 mA (GLOBALSCALER 80, IRUN 31), homing 189 mA (IRUN 7)
|
|||
-- R_sense 0.075 ohm from hardware.ini [Motor.yaw]
|
||||
```
|
||||
|
||||
Omit the block entirely on a tower whose endstops hold against full torque: no
|
||||
`CURRENT` is sent, the firmware keeps its compile-time defaults, and nothing
|
||||
changes.
|
||||
There is no leaving the block out any more. The firmware holds no drivetrain
|
||||
values of its own: at boot every axis stops its driver, asks the host (`NEED`),
|
||||
and refuses to move until every value it asked for has arrived. An axis whose
|
||||
block is missing or incomplete is an axis that will not turn - by design, since a
|
||||
guessed value is how drivers get cooked and endstops sheared. (For a bench board
|
||||
with no host, the firmware's `USE DEFAULTS` command loads deliberately gentle
|
||||
limp-home values instead; `test hardware motor` fails on a tower running them.)
|
||||
|
||||
The kit writes eight keys per axis - the four above plus `steps_per_rev`,
|
||||
`sw_mode`, `enc_deviation` and `has_encoder` - each as a value or a `; TODO`;
|
||||
`--motor AXIS.KEY=VALUE` gives any of them for any axis, where the older
|
||||
`--r-sense`-style flags cover the first axis only. The three speeds
|
||||
(`scan_speed_deg_s`, `homing_speed_deg_s`, `center_speed_deg_s`, all in degrees
|
||||
per second) are written as gentle starting values: **on a tower that is being
|
||||
migrated, set the homing speed to what its endstop detection is proven at**, as
|
||||
was done for tower 111 (section 10b), or a stall-detected endstop may behave
|
||||
differently.
|
||||
|
||||
> **If an axis declares a reduced homing current and the firmware has not adopted
|
||||
> it, the host refuses to home** rather than driving into a stop at full torque.
|
||||
|
|
@ -330,7 +379,27 @@ changes.
|
|||
>
|
||||
> **The by-id path changes when you flash the MCU**, because the name comes from
|
||||
> the running image: `usb-MicroPython_Board_in_FS_mode_*` before, a Pico/Arduino
|
||||
> name after. Re-run `--probe` after step 6 and fix the file if it moved.
|
||||
> name after. `scripts/install-tower` re-probes after flashing and updates
|
||||
> `[Serial] device` itself; by hand, re-run `--probe` after step 6 and fix the
|
||||
> file if it moved.
|
||||
|
||||
### The wiring block: `[Pins]`
|
||||
|
||||
The firmware is **built** from this section - which MCU pin reaches which driver
|
||||
board - by `firmware/tools/gen_wiring.py`, reading the tower's own
|
||||
`/etc/fire_gimbal_control/hardware.ini` when the firmware is built on the tower.
|
||||
There is no copy in the firmware and no fallback: without `[Pins]` the firmware
|
||||
build fails, naming what to add. Each kit's reference loom, with the table saying
|
||||
which wire goes where, is its `pins.ini`; `install.sh --loom standard` copies it
|
||||
in, and `--pin KEY=N` gives pins one at a time for a tower wired differently.
|
||||
|
||||
The generator refuses what the board cannot do rather than building it: on the
|
||||
RP2040, SPI pins outside SPI0's reach (SCK `{2,6,18,22}`, MOSI `{3,7,19,23}`,
|
||||
MISO `{0,4,16,20}`); on the Leonardo, any `spi_*` key at all (its SPI is fixed on
|
||||
the ICSP header) and the ICSP pins as chip-selects; duplicates; and a
|
||||
`hardware.ini` written for the other board. The running firmware reports the
|
||||
pins it was built with, and `test hardware motor` compares them with this
|
||||
section - so a flash built from an older copy shows up there.
|
||||
|
||||
## 6. Motor firmware `[verified]`
|
||||
|
||||
|
|
@ -374,17 +443,20 @@ SERIAL_PORT=/dev/serial/by-id/usb-...
|
|||
./deploy.sh --monitor # watch it boot
|
||||
```
|
||||
|
||||
`deploy.sh` syncs the tree and then runs `firmware/tools/flash.sh` on the tower,
|
||||
which is also what `scripts/install-tower` runs, locally: one implementation of
|
||||
the board-specific bootloader step for both. **Stop the program first**; it holds
|
||||
the serial port.
|
||||
|
||||
**Confirm the pin map before flashing a board that is wired to motors.** A
|
||||
chip-select on the wrong pin writes into a driver that was never meant to hear
|
||||
it. The pins are the `[Pins]` section of this tower's `hardware.ini`: the build
|
||||
reads them from `/etc/fire_gimbal_control/hardware.ini` (`firmware/tools/gen_wiring.py`)
|
||||
and **fails without them**, so add the section before the first build. The
|
||||
reference loom, with the table saying which wire goes where, is
|
||||
[`hardware/lattepanda-iota/pins.ini`](../hardware/lattepanda-iota/pins.ini) -
|
||||
the loom as built on tower 121, which is **not** the map proposed in the porting
|
||||
plan. Copy it in if the tower is wired the same way. The generator refuses pins
|
||||
SPI0 cannot reach, and `test hardware motor` compares the declared pins with the
|
||||
ones the running firmware reports it was built with.
|
||||
it. The build takes the pins from this tower's `hardware.ini` and fails without
|
||||
them - see "The wiring block" in section 5. The IOTA's reference loom is the one
|
||||
as built on tower 121, which is **not** the map proposed in the porting plan.
|
||||
|
||||
`[verified]` on both boards: the RP2040 on tower 121 (2026-09-23), the Leonardo on
|
||||
tower 111 (2026-09-24) - the Leonardo through its own 1200-baud touch, which
|
||||
`flash.sh` leaves alone.
|
||||
|
||||
### Getting a factory board to accept its first flash
|
||||
|
||||
|
|
@ -469,8 +541,9 @@ loop and unsupervised energised drivers would hold torque.
|
|||
| before | `usb-MicroPython_Board_in_FS_mode_<serial>-if00` | `2e8a:0005` |
|
||||
| after | `usb-Raspberry_Pi_Pico_<SERIAL>-if00` | `2e8a:000a` |
|
||||
|
||||
So generate `hardware.ini` (step 5) **after** flashing, or re-run
|
||||
`install.sh --hardware-ini --force` to pick up the new path.
|
||||
So generate `hardware.ini` (step 5) **after** flashing, or update `[Serial]
|
||||
device` once the name has moved - `scripts/install-tower` does that itself,
|
||||
re-probing after every flash.
|
||||
|
||||
### Verify the flash
|
||||
|
||||
|
|
@ -513,10 +586,14 @@ RUN_ARGS="--start"
|
|||
./deploy.sh # rsync + remote cmake + build
|
||||
```
|
||||
|
||||
The first deploy seeds `config.ini` from `config/config.example.ini` **only if
|
||||
The first deploy copies `config/config.example.ini` to `config.ini` **only if
|
||||
the device has none** - see the warning in step 2 about a machine that arrived
|
||||
with somebody else's config.
|
||||
with somebody else's config. That copy still carries the example's placeholders
|
||||
and the keys `hardware.ini` owns; `scripts/install-tower` seeds it instead with
|
||||
`build/fgc-config seed`, which fills in the answers, comments out every key
|
||||
`hardware.ini` sets, and refuses to write a file the program would not load.
|
||||
|
||||
`[verified]` on tower 111: builds on GCC 11 / Ubuntu 22.04.
|
||||
`[verified]` on tower 121: builds clean on GCC 15 / Ubuntu 26.04 with a single
|
||||
`-Wformat-truncation` warning in `utcStampNow()`, which is a false positive (the
|
||||
compiler cannot prove `tm_year` is four digits; the real output is 17 bytes into
|
||||
|
|
@ -610,8 +687,12 @@ zkms_server_ip = <broker>
|
|||
Credentials belong in `/etc/fire_gimbal_control/credentials.env`, not here.
|
||||
|
||||
Anything `hardware.ini` now owns - `[Serial] device`, `[IMU]`, `[Env]`, the
|
||||
camera ids - should be **deleted** from `config.ini` rather than left to rot:
|
||||
the program warns at startup about every key set in both.
|
||||
camera ids - should not be in `config.ini` as well: the program warns at startup
|
||||
about every key set in both. `scripts/install-tower` comments them out (value and
|
||||
all), and `build/fgc-config set config.ini --config config.ini --hardware
|
||||
/etc/fire_gimbal_control/hardware.ini --drop Section.key=note` does one by hand.
|
||||
Every edit through `fgc-config` is loaded the way the program loads it before it
|
||||
is written, and keeps the previous value as a `;prev` comment.
|
||||
|
||||
> **`[Camera.<name>] id` is required even when the camera is disabled.** Config
|
||||
> validation runs before any of the feature flags are consulted, so a tower with
|
||||
|
|
@ -619,20 +700,12 @@ the program warns at startup about every key set in both.
|
|||
> `DEV_0000000000`, is what `deploy.sh`'s placeholder guard refuses. On a tower
|
||||
> that has its camera, the id lives in `hardware.ini` and this never comes up.
|
||||
|
||||
`[Features] enable_imu` and `enable_env` matter more than they look. Declaring
|
||||
`imu = none` in `hardware.ini` does **not** yet switch them off - deriving them
|
||||
is section 12 of the config plan and is not implemented. On tower 121 the first
|
||||
clean-looking run did this:
|
||||
|
||||
```
|
||||
[INFO ] MTi IMU started on /dev/ttyS0 @ 115200
|
||||
[ERROR] I2C: failed to open /dev/i2c-2
|
||||
[ERROR] SHT41: failed to open I2C bus; sensor disabled
|
||||
```
|
||||
|
||||
It "started" an IMU on a tower that has none - `/dev/ttyS0` exists and accepts a
|
||||
serial open whether or not anything is attached to it. Set both flags to `false`
|
||||
by hand until reconciliation lands.
|
||||
`[Features] enable_imu` and `enable_env` are for towers **without** a
|
||||
`hardware.ini`. Where `hardware.ini` declares `imu` / `env`, the declaration
|
||||
decides, and a flag that contradicts it is warned about and ignored. Before that
|
||||
was so, tower 121's first clean-looking run "started" an IMU it does not have -
|
||||
`/dev/ttyS0` accepts a serial open whether or not anything is attached - which is
|
||||
why a declared `none` has to be able to switch a sensor off by itself.
|
||||
|
||||
## 10. First run: bring the subsystems up one at a time `[partial]`
|
||||
|
||||
|
|
@ -690,8 +763,16 @@ DUMP BEGIN build=aab1925-dirty uptime=7925212 mcusr=0x3 free_ram=252712
|
|||
DUMP Y state=1 ... speed=3000 center=500000 has_encoder=1
|
||||
```
|
||||
|
||||
Check `build=` matches the firmware you flashed and `speed=` matches the
|
||||
drivetrain values you intended. Then, and only then, move the thing:
|
||||
Check `build=` matches the firmware you flashed, and that the handshake
|
||||
completed - the log says `firmware config: answered N keys for axis Y`, and the
|
||||
axis leaves state 5 (`NEEDCFG`). The scripted form of all of this, which also
|
||||
compares every value the firmware holds with `hardware.ini`, moves nothing:
|
||||
|
||||
```bash
|
||||
./build/fire_gimbal_control --no-mqtt --test hardware # exit 0 PASS, 4 FAIL, 5 could not start
|
||||
```
|
||||
|
||||
Then, and only then, move the thing:
|
||||
|
||||
```
|
||||
gimbal home # the endstop-finding sequence - see the warning below
|
||||
|
|
@ -703,13 +784,14 @@ gimbal move 10,0 # a small deliberate move
|
|||
|
||||
| Message | Why |
|
||||
|---|---|
|
||||
| `Motor controller: ERR axis not present` | The host configures both axes at startup; a yaw-only firmware refuses the pitch half. Harmless until presence reconciliation lands (config plan section 10) |
|
||||
| `Motor controller: ERR unknown command`, a burst at startup | Only on firmware older than the config handshake: it does not know `SET`. Seen on tower 111 before its flash; harmless there, and gone once the firmware is current |
|
||||
| `encoder tracking error on yaw: 500366 counts (limit 400)` | Before homing, the firmware presets `XACTUAL` to the centre count while the encoder sits at its own zero, so they disagree by the whole range. Homing resyncs them. **If it persists after a successful home, that is real** |
|
||||
|
||||
> **On an unfamiliar drivetrain, homing is the dangerous step** - it finds the
|
||||
> endstop by driving into it. Start from a deliberately slow `homing_speed` and
|
||||
> walk it up, watching the free-running encoder gap that `ENC_DEVIATION_LIMIT` is
|
||||
> measured against. Be at the rig with a hand on the power for the first one.
|
||||
> endstop by driving into it. Start from a deliberately slow
|
||||
> `homing_speed_deg_s` and walk it up, watching the free-running encoder gap that
|
||||
> `enc_deviation` is measured against. Be at the rig with a hand on the power for
|
||||
> the first one - which is why `scripts/install-tower` asks before it homes.
|
||||
|
||||
### Phase 2 - camera `[verified]`
|
||||
|
||||
|
|
@ -753,9 +835,10 @@ the ground station. That lives in `~/zkms-client/`, outside this repository.
|
|||
|
||||
### Phase 4 - IMU and ambient sensor
|
||||
|
||||
Only if fitted. `hardware.ini` should already say so, and `[Features]
|
||||
enable_imu` / `enable_env` have to agree with it (see step 9 - declaring
|
||||
`imu = none` does not yet switch the flag off by itself).
|
||||
Only if fitted, and declared in `hardware.ini` (`imu`, `env`) - the declaration
|
||||
is what switches them on. `test hardware imu` wants a `0xFA`-framed stream from
|
||||
the MTi; `test hardware env` wants a real I2C adapter and a CRC-valid SHT41
|
||||
reading. `[verified]` on tower 111, 2026-09-23: both.
|
||||
|
||||
### Phase 5 - unattended
|
||||
|
||||
|
|
@ -766,9 +849,10 @@ scripts/fgc start --start # detached, inside tmux
|
|||
scripts/fgc status # running? since when? which PID?
|
||||
```
|
||||
|
||||
Close the ssh session, reconnect, and check it is still running. Then reboot the
|
||||
tower and confirm the USB preconditions survived (`install.sh --probe`) and the
|
||||
app comes back.
|
||||
Close the ssh session, reconnect, and check it is still running. Then install
|
||||
the boot service (`scripts/install-tower` asks; [deployment.md](deployment.md)
|
||||
section 4), reboot the tower, and confirm with `scripts/install-tower --verify`
|
||||
that the USB preconditions survived and the program came back by itself.
|
||||
|
||||
### Calibration
|
||||
|
||||
|
|
@ -791,7 +875,31 @@ turning.
|
|||
|
||||
So the order matters. **Host first, firmware second.** Each step below either
|
||||
changes nothing or is reversible, and the step that could strand the tower comes
|
||||
after the one that prevents it.
|
||||
after the one that prevents it. `scripts/install-tower` follows this order by
|
||||
itself, and blocks the flash while any drivetrain value is missing.
|
||||
|
||||
### The worked example: tower 111 `[verified]`
|
||||
|
||||
The Delta tower, which predated `hardware.ini`, the handshake and the kits.
|
||||
2026-09-23 and -24, through the installer with the Delta kit:
|
||||
|
||||
1. **A rollback copy** of the sources, the running binary and `config.ini`, in
|
||||
`~ggs/rollback-20260923/`, before anything changed.
|
||||
2. **The host half.** Preconditions, `hardware.ini` with every drivetrain value
|
||||
a `; TODO`, `config.ini`'s duplicated keys commented out, the missing `night`
|
||||
profile, a new host. `test hardware`: axes, IMU, env and camera PASS; motor
|
||||
FAIL on the old firmware's missing echoes, as expected (step 2 below). Firmware
|
||||
blocked. It homed and resumed capturing on the new host.
|
||||
3. **The drivetrain, without inventing it.** The sense resistors were read off
|
||||
both boards (75 mOhm). Everything else was converted from what the old
|
||||
firmware ran - its revision is in `DUMP BEGIN build=`, and git history at
|
||||
that revision has its register values - through the host's own planners, and
|
||||
round-tripped back to the identical registers before being written. Homing
|
||||
and centring speeds were set to what that firmware used, not the kit's
|
||||
gentle defaults, since its endstop detection was proven at those.
|
||||
4. **The flash**, by the installer: every gap filled, the firmware built from
|
||||
`hardware.ini` and flashed, `test hardware` green on every leaf, and homing
|
||||
within 0.03 % of the old firmware's travel.
|
||||
|
||||
### 0. Record what you are rolling back to
|
||||
|
||||
|
|
@ -806,9 +914,14 @@ the comparison that tells you whether the new one adopted the same values.
|
|||
|
||||
### 1. Update `hardware.ini` — nothing else, and no restart yet
|
||||
|
||||
Add the `[Motor.<axis>]` block (section 5). The five keys the new firmware will
|
||||
not run without are `steps_per_rev`, `enc_deviation`, `sw_mode`, plus the two
|
||||
speeds; `install.sh --stdout` generates the block if you would rather not type it.
|
||||
Add the `[Motor.<axis>]` block (section 5) and `[Pins]`. The firmware asks for
|
||||
eleven values per axis - `steps_per_rev`, `global_scaler`, `run_irun`,
|
||||
`homing_irun`, `center_count`, `soft_margin`, `enc_deviation`, `normal_speed`,
|
||||
`homing_speed`, `center_speed`, `sw_mode` - and the host derives them from the
|
||||
block (currents from milliamps and the sense resistor, speeds from degrees per
|
||||
second, centre and margin from `config.ini`). `install.sh --stdout` generates the
|
||||
block; for values nobody has measured, convert what the old firmware runs, as
|
||||
for tower 111 above.
|
||||
|
||||
Doing this **before** flashing is what keeps the window at zero: the host is ready
|
||||
to answer the moment the firmware starts asking.
|
||||
|
|
@ -825,16 +938,21 @@ scripts/fgc peek
|
|||
test hardware motor
|
||||
```
|
||||
|
||||
Expect: no handshake lines at all, and the motor leaf reporting a **version skew**
|
||||
rather than a mismatch —
|
||||
Expect: no handshake lines at all (and possibly a burst of `ERR unknown
|
||||
command` - the old firmware refusing `SET`), and the motor leaf **failing** on
|
||||
missing echoes, reported as a version skew rather than a mismatch:
|
||||
|
||||
```
|
||||
; yaw_current: yaw: firmware echoes only part of the current (2 of 3 fields
|
||||
absent), so it cannot be confirmed -- it predates the full CURRENT echo. Re-flash it
|
||||
; yaw_pins: yaw: the firmware does not report the pins it was built with, so its
|
||||
wiring cannot be confirmed -- it predates the pin echo. Re-flash it
|
||||
```
|
||||
|
||||
If instead you see wrong *values*, stop: something in `hardware.ini` disagrees
|
||||
with what the tower is really running, and flashing will not fix it.
|
||||
That FAIL is by design - a check that could not happen must not read as a pass -
|
||||
so on old firmware the gate is "every other leaf green, and `hw_motor` failing
|
||||
only on missing echoes". A speed can also differ before the first home, since
|
||||
the host sends it after homing. If instead you see wrong *values* for something
|
||||
the firmware does echo, stop: `hardware.ini` disagrees with what the tower is
|
||||
really running, and flashing will not fix it.
|
||||
|
||||
### 3. Flash
|
||||
|
||||
|
|
@ -923,37 +1041,43 @@ table and lets the tower move while you work out what `hardware.ini` is missing.
|
|||
|
||||
## 11. Verification checklist
|
||||
|
||||
The checks that move nothing can be run as one command, which is what a script
|
||||
should use rather than typing into the dashboard. Stop the app first: it holds
|
||||
the serial port.
|
||||
Most of this is one command, which changes nothing:
|
||||
|
||||
```bash
|
||||
scripts/install-tower --verify
|
||||
```
|
||||
|
||||
It checks the USB preconditions, that `config.ini` and `hardware.ini` load
|
||||
together, that `hardware.ini` has no gaps, that the firmware reports the pins it
|
||||
declares, and the boot service - and runs `test hardware`: inside the running
|
||||
program when it is up (pausing capture, then resuming it), or as a one-shot run
|
||||
when it is not. By hand, the one-shot form is:
|
||||
|
||||
```bash
|
||||
scripts/fgc stop
|
||||
GENICAM_GENTL64_PATH=/opt/VimbaX/cti ./build/fire_gimbal_control --no-mqtt --test hardware
|
||||
echo $? # 0 PASS, 4 FAIL, 5 could not start (the log says why)
|
||||
scripts/fgc start
|
||||
```
|
||||
|
||||
It waits for the firmware to be talking and configured before it looks, so a
|
||||
freshly flashed board is not failed for values the host had not sent yet.
|
||||
`build/fgc-config check --config config.ini --hardware /etc/fire_gimbal_control/hardware.ini`
|
||||
checks that the two files load together, and changes nothing.
|
||||
|
||||
| Check | How | Tower 121 |
|
||||
|---|---|---|
|
||||
| Key-only ssh | `ssh ggs@<tower> hostname`, no prompt | ✅ |
|
||||
| Preconditions survive a reboot | reboot, then `install.sh --probe`: `usbfs_memory_mb 1000`, `lpm_permit 0` | ✅ confirmed across a real reboot |
|
||||
| `hardware.ini` found unaided | run with no `FGC_HARDWARE`: startup names `/etc/fire_gimbal_control/hardware.ini` | ✅ |
|
||||
| Camera enumerates | `ListCameras_VmbC` reports the `DEV_` id in `hardware.ini` | ✅ `DEV_1AB22C0B7D4E` |
|
||||
| Camera profile applies | startup logs `profile 'standard' applied ... (N settings)` | ✅ 17206 settings |
|
||||
| Firmware identity | `gimbal dump` reports the `GIT_REV` you flashed | ✅ `build=aab1925-dirty` |
|
||||
| Driver answers over SPI | boot banner shows `IOIN Y=0x… version=0x30` | ✅ |
|
||||
| Remote re-flash | `BOOTSEL`, then `./deploy.sh` - no hands on the board | ✅ |
|
||||
| Declared hardware matches | startup logs no presence warnings | ✅ no shadowed-key warnings |
|
||||
| Homing repeatable | `gimbal home` several times, endstop counts within `home_spread_max_counts` | ✅ three homes within 0.27 % (1,210,778 / 1,208,358 / 1,207,514 counts) |
|
||||
| Endstop survives homing | inspect the stop after a home; the axis must stall against it, not push through | ✅ at 189 mA, after failing at full torque |
|
||||
| Degree scale derived | startup logs `counts_per_deg` agreeing with the homed travel | ✅ 3552.87, saved with `;prev` history |
|
||||
| Images land | `.jxl` files appear under the configured output dir | ❌ needs a capture run |
|
||||
| Survives a disconnect | `scripts/fgc start`, close ssh, reconnect, `scripts/fgc status` | ✅ runs under the boot service, independent of any ssh session |
|
||||
| Starts at boot | reboot with nobody logged in; `scripts/fgc status` from a later ssh | ✅ 2026-09-23, lock under `/run/user/1000` (lingering on) |
|
||||
| Reports to a broker | topics arrive at the configured broker | ❌ no broker on this tower yet |
|
||||
| Check | How | Tower 121 (IOTA) | Tower 111 (Delta) |
|
||||
|---|---|---|---|
|
||||
| Key-only ssh | `ssh ggs@<tower> hostname`, no prompt | ✅ | ✅ |
|
||||
| Preconditions applied | `--verify`: `usbfs_memory_mb 1000`, camera LPM `0` | ✅ | ✅ 2026-09-23 (were 16 and `u1_u2`) |
|
||||
| Preconditions survive a reboot | reboot, then `--verify` | ✅ confirmed across a real reboot | ❓ not rebooted yet |
|
||||
| `hardware.ini` found unaided | startup names `/etc/fire_gimbal_control/hardware.ini` | ✅ | ✅ |
|
||||
| Declared hardware matches | startup logs `declared assembly confirmed`; `test hardware axes` | ✅ yaw | ✅ yaw, pitch |
|
||||
| IMU / ambient sensor | `test hardware imu` / `env` | none declared | ✅ MTi streaming, SHT41 CRC-valid on a real I2C bus |
|
||||
| Camera enumerates | `ListCameras_VmbC` reports the `DEV_` id in `hardware.ini` | ✅ `DEV_1AB22C0B7D4E` | ✅ `DEV_1AB22C049CD1` |
|
||||
| Camera profile applies | startup logs `profile 'standard' applied ... (N settings)` | ✅ 17206 settings | ✅ 3743 settings |
|
||||
| Firmware identity | `DUMP BEGIN build=` is the revision flashed | ✅ | ✅ `a0b1e4a` |
|
||||
| Driver answers over SPI | `IOIN` reads version `0x30` for every axis | ✅ | ✅ both axes |
|
||||
| Firmware holds `hardware.ini` | `test hardware motor`: values, current, pins | ✅ | ✅ every leaf, 2026-09-24 |
|
||||
| Remote re-flash | no hands on the board | ✅ `BOOTSEL` | ✅ 1200-baud touch |
|
||||
| Homing repeatable | endstop counts across homes | ✅ three homes within 0.27 % | ✅ within 0.03 %, old and new firmware |
|
||||
| Endstop survives homing | the axis must stall against the stop, not push through | ✅ at 189 mA, after failing at full torque | not inspected; full current, as for its whole service life |
|
||||
| Degree scale derived | startup logs `counts_per_deg` agreeing with the homed travel | ✅ 3552.87 | ✅ agrees, calibration left alone |
|
||||
| Images land | `.jxl` files appear under the configured output dir | ❌ needs a capture run | ✅ |
|
||||
| Starts at boot | reboot with nobody logged in; `scripts/fgc status` from a later ssh | ✅ lock under `/run/user/1000` | ❓ service enabled (`--init --start`), not rebooted yet |
|
||||
| Reports to a broker | topics arrive at the configured broker | ❌ no broker on this tower yet | ❓ not checked |
|
||||
|
|
|
|||
|
|
@ -36,7 +36,9 @@ WorkingDirectory=@REPO@
|
|||
ExecStart=@REPO@/scripts/fgc start @ARGS@
|
||||
ExecStop=@REPO@/scripts/fgc stop
|
||||
TimeoutStartSec=120
|
||||
TimeoutStopSec=60
|
||||
# Longer than `fgc stop` itself waits (120 s) for the program's clean shutdown:
|
||||
# past this systemd kills it outright, which wedges the camera for the next run.
|
||||
TimeoutStopSec=150
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
|
|||
Loading…
Reference in New Issue