fwt_software/docs/deployment.md

Deploying to the Gimbal LattePanda

Step-by-step guide to deploy fire_gimbal_control on a tower's LattePanda Sigma (Linux x86-64). Like the firmware workflow, this is a repeatable build → install → configure → enable → verify cycle; the Update / redeploy section at the end is the short loop you run for every new version.

Assumes a Debian/Ubuntu-based Linux on the LattePanda. For Arch/Manjaro swap apt for pacman (see build-and-setup.md). Run as a normal user with sudo rights; the service runs under a dedicated ggs user.

0. Prerequisites (one-time per device)

Install the toolchain and system libraries:

sudo apt update
sudo apt install -y git cmake build-essential \
                    libopencv-dev libjxl-dev libboost-all-dev

Install the Allied Vision Vimba X SDK (proprietary, needed for the cameras):

1. Download Vimba X for Linux x86-64 from https://www.alliedvision.com/en/products/software/vimba-x-sdk/.
2. Unpack to a fixed location, e.g. /opt/VimbaX_2024-1.
3. Run the SDK's transport-layer installer so cameras are discoverable, e.g.:

   sudo /opt/VimbaX_2024-1/cti/Install_GenTL_Path.sh
   

4. Make its runtime libraries findable at run time:

   echo /opt/VimbaX_2024-1/api/lib | sudo tee /etc/ld.so.conf.d/vimbax.conf
   sudo ldconfig
   

Ensure an MQTT broker is reachable from the tower (the ground-station/ZKMS broker, or a local Mosquitto for bring-up).

1. Get the code

sudo mkdir -p /opt/src && sudo chown "$USER" /opt/src
cd /opt/src
git clone <repo-url> fire_gimbal_control_src
cd fire_gimbal_control_src
# (or: git pull, for an update)

2. Build (full, with cameras + MQTT)

cmake -B build -DCMAKE_BUILD_TYPE=Release \
      -DWITH_VIMBA=ON -DWITH_MQTT=ON \
      -DVMB_HOME=/opt/VimbaX_2024-1
cmake --build build -j"$(nproc)"

• WITH_MQTT=ON fetches + builds Eclipse Paho from source on first configure (needs network).
• If a camera build isn't needed yet, use -DWITH_VIMBA=OFF for a mock binary to test the pipeline.
• Smoke-test the core before installing: ctest --test-dir build (after adding -DBUILD_TESTING=ON).

3. Install to a fixed location

sudo useradd -r -s /usr/sbin/nologin ggs 2>/dev/null || true
sudo mkdir -p /opt/fire_gimbal_control
sudo cp build/fire_gimbal_control /opt/fire_gimbal_control/
# demo-mode placeholder image (only needed if you ever run --demo)
sudo cp bin/x64/Release/test_smoke.jxl /opt/fire_gimbal_control/ 2>/dev/null || true

4. Configure

sudo cp config/config.example.ini /opt/fire_gimbal_control/config.ini
sudo nano /opt/fire_gimbal_control/config.ini

Set at least:

[General]
tower_name = ExampleTower           ; this tower's name (drives all MQTT topics)
image_interval = 3

[Network]
zkms_server_ip = 10.11.12.13        ; the broker address

[Serial]
device = /dev/ttyACM0               ; the motor-controller MCU
baud = 115200

[Camera]
id_Cam1 = DEV_1AB22C0AADED          ; or GigE IPs like 192.168.11.101

Credentials go in the environment, not the config file. Create a root-only env file for the service:

sudo install -m 600 /dev/null /etc/fire_gimbal_control/credentials.env 2>/dev/null || \
  { sudo mkdir -p /etc/fire_gimbal_control && sudo install -m 600 /dev/null /etc/fire_gimbal_control/credentials.env; }
printf 'FGC_MQTT_USER=fwt_gimbal\nFGC_MQTT_PW=CHANGE_ME\n' | sudo tee /etc/fire_gimbal_control/credentials.env >/dev/null
sudo chmod 600 /etc/fire_gimbal_control/credentials.env

Give the ggs user access to its files and the image output dir:

sudo chown -R ggs:ggs /opt/fire_gimbal_control

5. Hardware access

Serial (motor controller): let the service user open the port.

sudo usermod -aG dialout ggs

For a stable device name across reboots, add a udev rule keyed on the MCU's USB IDs (find them with udevadm info -a -n /dev/ttyACM0 | grep -m1 idVendor) and point [Serial] device at the symlink:

echo 'SUBSYSTEM=="tty", ATTRS{idVendor}=="XXXX", ATTRS{idProduct}=="YYYY", SYMLINK+="ttyGimbal"' | \
  sudo tee /etc/udev/rules.d/99-gimbal.rules
sudo udevadm control --reload && sudo udevadm trigger

GigE cameras (if used): put the host NIC on the camera subnet (e.g. 192.168.11.0/24), enable jumbo frames, and confirm discovery with the Vimba X VimbaXViewer/ListCameras tool.

6. First run by hand (verify before enabling the service)

cd /opt/fire_gimbal_control
set -a; . /etc/fire_gimbal_control/credentials.env; set +a
./fire_gimbal_control --config ./config.ini --start --log-level debug

Watch for: Loaded config, Serial controller started, Opened camera ..., MQTT connected, then Saved .../RGB/<timestamp>.jxl after the first capture. Type exit (or Ctrl-D) to stop. To validate without cameras first: add --mock-camera --mock-serial --no-mqtt.

This run is attached to your terminal and dies with the ssh session — fine for a first check, not for leaving it running. For that, use section 6b.

6b. Running detached (scripts/fgc)

The tower is operated over ssh, so the program has to survive the connection closing. scripts/fgc runs it inside a tmux session, which owns the pty:

cd /opt/fire_gimbal_control
scripts/fgc start --start        # starts detached, returns as soon as it is up
scripts/fgc status               # running? since when? which PID?
scripts/fgc attach               # the dashboard — Ctrl-b d detaches, leaves it running
scripts/fgc stop                 # SIGTERM -> the same clean teardown as `exit`
scripts/fgc peek                 # snapshot of the pane without attaching

Ctrl-b d detaches. The dashboard's own q key exits the program. That is the one thing to get right while attached.

scripts/fgc stop sends SIGTERM and waits up to 30s; the program handles it with the same shutdown path as the exit command (stop capture, close the camera, save the exposure store). Do not kill -9 — an unclean exit leaves the camera wedged for the next run. If it will not stop, attach and look at why.

scripts/fgc start refuses if a run is already up, and prints the previous run's last output if it died rather than silently discarding it.

Quitting the dashboard with q while attached ends the tmux session too, so you land straight back in your shell — exit then closes ssh. A run that fails is the exception: its pane is kept so you can read why (remain-on-exit failed), and fgc status shows it until fgc stop or the next fgc start clears it.

A kept pane has no process behind it — keys and Ctrl-C do nothing, which reads as a hung ssh session. Ctrl-b d still detaches (tmux, not the pane, handles it); the session's status line spells that out along the bottom of the screen. As a last resort Enter ~ . closes the ssh client itself from your end.

Why _run waits on the app instead of execing it. When the app is itself the pane's process, tmux never records an exit status for it — #{pane_dead_status} comes back empty and the process lingers as a zombie — so remain-on-exit failed cannot tell success from failure and keeps every finished pane, including a clean q. That is what used to strand an operator on a frozen dead pane. Interposing a shell that reaps the child and exits with its status gives tmux a real status. Measured on the device: 3/3 panes kept when exec'd, 4/4 sessions closed cleanly when parented. The wrapper forwards SIGTERM/SIGINT/SIGHUP, so fgc stop still reaches the app and runs the full teardown.

6c. The single-instance lock

Two instances would fight over the motor's serial port, the camera, and the exposure store. The binary prevents that itself: at startup it takes an exclusive flock on

• /run/user/<uid>/fire_gimbal_control.lock when that directory exists, else
• /tmp/fire_gimbal_control-<uid>.lock

and, if it is already held, exits with status 2 and:

Cannot start: another instance is already running (PID 1234); attach to it with: scripts/fgc attach

The path is derived from the uid and not from $XDG_RUNTIME_DIR on purpose: that variable is set in an interactive ssh shell but not in a non-interactive ssh host cmd one, and a lock the two launch routes disagree about would exclude nothing.

Because the check lives in the binary, it covers every route in — scripts/fgc, systemd, or a bare ./fire_gimbal_control. Nothing needs cleaning up after a crash or a power cut: the kernel releases an flock when the holder dies, so a leftover lockfile with a dead PID in it never blocks a launch. Use --no-lock only to run a second mock-only instance during development, and --lock-file <path> to point at a different lockfile.

7. Install the systemd service

An alternative to section 6b, not a companion to it: systemd gives you start-at-boot and restart-on-crash, but no attachable dashboard — you get journalctl instead. Pick one. If you run both by accident the second one to start fails on the single-instance lock (section 6c), so the failure is loud rather than a pair of processes corrupting each other's state.

sudo cp scripts/fire-gimbal-control.service /etc/systemd/system/
sudoedit /etc/systemd/system/fire-gimbal-control.service

Confirm/adjust these lines:

User=ggs
WorkingDirectory=/opt/fire_gimbal_control
ExecStart=/opt/fire_gimbal_control/fire_gimbal_control --start
Environment=FGC_CONFIG=/opt/fire_gimbal_control/config.ini
EnvironmentFile=/etc/fire_gimbal_control/credentials.env

(Use ExecStart=... --init --start for the first power-on if endstops must be found at boot.)

8. Enable and verify

sudo systemctl daemon-reload
sudo systemctl enable --now fire-gimbal-control
systemctl status fire-gimbal-control
journalctl -u fire-gimbal-control -f          # live logs

Confirm end to end:

• Images accumulating under the configured [Paths] output_dir (default ~ggs/.local/share/fire_gimbal_control/images/RGB/).
• CamEvents on the broker: mosquitto_sub -h <broker> -t 'GGS/FWT/<tower>/#' -v.

9. Update / redeploy

The short loop for each new version (mirrors the firmware flash cycle):

cd /opt/src/fire_gimbal_control_src
git pull
cmake --build build -j"$(nproc)"                 # reuses the configured build dir
sudo systemctl stop fire-gimbal-control
sudo cp build/fire_gimbal_control /opt/fire_gimbal_control/
sudo systemctl start fire-gimbal-control
journalctl -u fire-gimbal-control -n 50 -f       # confirm healthy

If CMakeLists.txt or options changed, re-run the cmake -B build ... configure step from section 2 first. To roll back, keep the previous binary (fire_gimbal_control.prev) and swap it back.

Troubleshooting

Symptom	Check
No config.ini found	FGC_CONFIG / --config path; service Environment= line
Failed to open serial port	MCU plugged in, dialout group, [Serial] device path / udev symlink
No camera found / Vimba startup error	SDK installed, GenTL path set, ldconfig, NIC subnet, camera IDs in config
MQTT connect failed (degraded mode)	broker reachable, credentials env, firewall
Runs but no images	capture started (--start or start command), output_dir writable by ggs
another instance is already running (exit 2)	scripts/fgc status; a systemd unit and a tmux session both running (section 6c)
Program dies when ssh disconnects	started with run.sh/--foreground instead of scripts/fgc start (section 6b)
No camera found only when started via scripts/fgc	GenTL path not resolved in the pane — check scripts/fgc peek, and /etc/profile.d/*Vimba*GenTL*.sh or VIMBA_CTI_PATH