Compare commits

..

No commits in common. "cba8856c8516311bc1ea418d2ff283c69406fabc" and "ae53f52131ee1ebe37b80203e377830aa03258a2" have entirely different histories.

29 changed files with 137 additions and 1575 deletions

View File

@ -88,7 +88,7 @@ Everything that makes one tower different from another is in two files:
| File | Where | What goes in it | Who writes it | | File | Where | What goes in it | Who writes it |
|---|---|---|---| |---|---|---|---|
| `hardware.ini` | `/etc/fire_gimbal_control/` | **What the tower IS** - the board, which axes and sensors are fitted, how the motor board is wired, each motor's currents, travel and speeds, the camera's id | the installer, once; a person when the hardware changes | | `hardware.ini` | `/etc/fire_gimbal_control/` | **What the tower IS** - the board, which axes and sensors are fitted, how the motor board is wired, each motor's currents, travel and speeds, the camera's id | the installer, once; a person when the hardware changes |
| `config.ini` | `~/.config/fire_gimbal_control/`, with the scan grid and camera profiles | **What the tower DOES** - its name, the scan grid, capture settings, the broker, logging, and calibration such as which way is north | the installer (a first copy), the calibration commands, a person | | `config.ini` | beside the program | **What the tower DOES** - its name, the scan grid, capture settings, the broker, logging, and calibration such as which way is north | the installer (a first copy), the calibration commands, a person |
Where both set the same thing, `hardware.ini` wins, and the program says so at Where both set the same thing, `hardware.ini` wins, and the program says so at
start-up. Full reference: [docs/configuration.md](docs/configuration.md) and start-up. Full reference: [docs/configuration.md](docs/configuration.md) and
@ -481,7 +481,7 @@ settings the way the real firmware does.
| Flag | Effect | | Flag | Effect |
|---|---| |---|---|
| `-c, --config <path>` | use this `config.ini` (otherwise `$FGC_CONFIG`, then `~/.config/fire_gimbal_control/config.ini`, then `./config.ini`, ...) | | `-c, --config <path>` | use this `config.ini` (otherwise `$FGC_CONFIG`, then `./config.ini`, ...) |
| `--hardware <path>` | use this `hardware.ini` (otherwise `$FGC_HARDWARE`, then `/etc/fire_gimbal_control/hardware.ini`, then beside `config.ini`) | | `--hardware <path>` | use this `hardware.ini` (otherwise `$FGC_HARDWARE`, then `/etc/fire_gimbal_control/hardware.ini`, then beside `config.ini`) |
| `--test <selection>` | run one self-test and exit: 0 PASS, 4 FAIL, 5 could not start | | `--test <selection>` | run one self-test and exit: 0 PASS, 4 FAIL, 5 could not start |
| `--tui` / `--no-tui` | the dashboard / the plain console (overrides `[UI] enable_tui`) | | `--tui` / `--no-tui` | the dashboard / the plain console (overrides `[UI] enable_tui`) |

View File

@ -1,10 +1,8 @@
# RGB camera profiles # RGB camera profiles
A profile is a **settings XML exported from the VimbaX Viewer**. A profile named A profile is a **settings XML exported from the VimbaX Viewer**. A profile named
`X` is `X.xml` in the profile directory, and `[Camera.rgb] profile = X` in `X` is `X.xml` in this directory, and `[Camera.rgb] profile = X` in `config.ini`
`config.ini` selects it. On a tower that directory is selects it.
`~/.config/fire_gimbal_control/camera_profiles/rgb/` (`[Camera.rgb] profile_dir`,
relative to `config.ini`), not this one: this README is all that lives here.
The program hands the XML straight to the Vimba X SDK (`LoadSettings`). Nothing The program hands the XML straight to the Vimba X SDK (`LoadSettings`). Nothing
else writes camera settings, and there is no fallback — a missing `X.xml` is a else writes camera settings, and there is no fallback — a missing `X.xml` is a
@ -18,12 +16,9 @@ combinations as you configure; that is the guardrail this design relies on.
## These files live on the device only ## These files live on the device only
The XMLs are exported on the LattePanda, because that is where the camera is, so The XMLs are exported on the LattePanda, because that is where the camera is, so
they exist nowhere in the repo. They live in the tower's own folder, they exist nowhere in the repo. `deploy.sh` therefore excludes `*.xml` and
`~/.config/fire_gimbal_control/`, which a deploy never reaches. A tower set up `*.txt` under this directory from its `rsync --delete` — without that, the next
before that folder existed still has them in this directory; `deploy.sh` deploy erases them. This README does sync; it is repo documentation.
excludes `*.xml` and `*.txt` here from its `rsync --delete` until
`scripts/tower-config move` has moved them. This README does sync; it is repo
documentation.
Keep a copy somewhere durable if a profile represents real tuning effort: a Keep a copy somewhere durable if a profile represents real tuning effort: a
reflash or a rebuilt device starts with an empty directory. reflash or a rebuilt device starts with an empty directory.
@ -32,7 +27,7 @@ reflash or a rebuilt device starts with an empty directory.
1. Open the VimbaX Viewer and connect to the camera. 1. Open the VimbaX Viewer and connect to the camera.
2. Configure it. The Viewer will not let you pick an invalid combination. 2. Configure it. The Viewer will not let you pick an invalid combination.
3. Save the settings XML over `<profile>.xml` in the profile directory. 3. Save the settings XML over `<profile>.xml` here.
4. Restart. The program notices the file changed, regenerates `<profile>.txt`, 4. Restart. The program notices the file changed, regenerates `<profile>.txt`,
and logs every setting that moved. and logs every setting that moved.

View File

@ -1,11 +1,7 @@
; Fire Gimbal Control - example configuration. ; Fire Gimbal Control - example configuration.
; ;
; A tower keeps its config.ini in its own folder, ~/.config/fire_gimbal_control/, ; Copy this file to "config.ini" and edit for your deployment:
; together with its scan grid and camera profiles. That folder is outside the ; cp config/config.example.ini config.ini
; project folder, so a deploy never overwrites or deletes anything in it.
; Create it from this file with:
; scripts/tower-config seed
; The paths below (grid_file, profile_dir) are relative to that folder.
; ;
; WARNING: do not edit config.ini while the program is running. `gimbal heading ; WARNING: do not edit config.ini while the program is running. `gimbal heading
; offset`, `gimbal heading setup` and `gimbal pitch setup` rewrite the [Motor] ; offset`, `gimbal heading setup` and `gimbal pitch setup` rewrite the [Motor]
@ -73,7 +69,7 @@ enabled = true
driver = vimba ; only vimba is implemented driver = vimba ; only vimba is implemented
id = DEV_0000000000 ; USB DEV_ id, or a GigE IP id = DEV_0000000000 ; USB DEV_ id, or a GigE IP
label = RGB ; output subdir + CamEvent "cam" label = RGB ; output subdir + CamEvent "cam"
profile_dir = camera_profiles/rgb profile_dir = config/camera_profiles/rgb
profile = standard ; <profile_dir>/standard.xml profile = standard ; <profile_dir>/standard.xml
; Clockwise rotation applied to saved images: 0, 90, 180 or 270. ; Clockwise rotation applied to saved images: 0, 90, 180 or 270.
rotation_deg = 0 rotation_deg = 0
@ -243,12 +239,12 @@ homing_timeout_s = 65
; leave it blank and auto-sweep stays off with a message saying so, rather than ; leave it blank and auto-sweep stays off with a message saying so, rather than
; the tower inventing an arc nobody chose. ; the tower inventing an arc nobody chose.
; A relative path resolves against the directory of the config file it is ; A relative path resolves against the directory of the config file it is
; written in (on a tower ~/.config/fire_gimbal_control/, so: scan.csv there). ; written in (config.ini sits at the repo root, so: config/scan.csv).
; The sweep always runs front to back, then returns to the first waypoint and ; The sweep always runs front to back, then returns to the first waypoint and
; repeats, so every pass captures the waypoints in the same order. The return ; repeats, so every pass captures the waypoints in the same order. The return
; leg is one long slew (the yaw arc cannot wrap), so order the grid with that ; leg is one long slew (the yaw arc cannot wrap), so order the grid with that
; in mind. ; in mind.
grid_file = scan.csv grid_file = config/scan.csv
[Paths] [Paths]
; Directory for saved .jxl images. Supports leading ~ and $ENV expansion. ; Directory for saved .jxl images. Supports leading ~ and $ENV expansion.

104
deploy.sh
View File

@ -14,22 +14,18 @@
# ./deploy.sh --stop shut the running app down cleanly # ./deploy.sh --stop shut the running app down cleanly
# ./deploy.sh --clean wipe the remote build dir first (fresh configure) # ./deploy.sh --clean wipe the remote build dir first (fresh configure)
# ./deploy.sh --push-config ALSO overwrite the device's config.ini with the local one # ./deploy.sh --push-config ALSO overwrite the device's config.ini with the local one
#
# NOTE: camera profile XMLs under config/camera_profiles/ are device-local and
# are never synced or deleted - export them on the device with the VimbaX Viewer.
# ./deploy.sh --check-deps only check the remote build dependencies # ./deploy.sh --check-deps only check the remote build dependencies
# ./deploy.sh --run --force skip the placeholder-config guard before running # ./deploy.sh --run --force skip the placeholder-config guard before running
# ./deploy.sh --run --foreground run attached to this terminal instead (debugging) # ./deploy.sh --run --foreground run attached to this terminal instead (debugging)
# #
# What belongs to one tower - its config.ini (calibration, camera IDs), its scan # Everything under config/ (the example, the camera-mode files, scan grids) is
# grid, the camera profile XMLs exported on it - lives in the tower's own folder, # mirrored on every deploy. The device's own config.ini at the repo root is NOT —
# ~/.config/fire_gimbal_control/, outside the project folder this script mirrors # it holds the live calibration, camera IDs and credentials for that rig, so a
# (scripts/tower-config). A deploy never reaches it. A tower set up before that # routine deploy must never clobber it. --push-config opts into overwriting it,
# folder existed still has them in the project folder; the excludes below keep # backing the device copy up to config.ini.bak-<timestamp> first.
# them safe there until `scripts/tower-config move` is run on it, and every
# deploy says so.
#
# Everything under config/ in this repo (the example config, the example scan
# grids) is mirrored on every deploy. --push-config opts into overwriting the
# config.ini the tower reads with the local one, backing the tower's copy up to
# config.ini.bak-<timestamp> beside it first.
# #
# --run starts the app inside a tmux session on the device (scripts/fgc), so it # --run starts the app inside a tmux session on the device (scripts/fgc), so it
# survives this ssh connection closing — reattach later with --attach. Only one # survives this ssh connection closing — reattach later with --attach. Only one
@ -98,7 +94,7 @@ if [[ -n "$SESSION_CMD" && "$SESSION_CMD" != "attach" && "$RUN" == "1" ]]; then
fi fi
if [[ "$PUSH_CONFIG" == "1" && ! -f config.ini ]]; then if [[ "$PUSH_CONFIG" == "1" && ! -f config.ini ]]; then
echo "error: --push-config given but there is no local config.ini to push" >&2 echo "error: --push-config given but there is no local config.ini to push" >&2
echo " (config.ini is gitignored; the device keeps its own in ~/.config/fire_gimbal_control/)" >&2 echo " (config.ini is gitignored; the device keeps its own at \$REMOTE_DIR/config.ini)" >&2
exit 1 exit 1
fi fi
if [[ "$PUSH_CONFIG" == "1" && -n "$SESSION_CMD" && "$SESSION_CMD" != "attach" ]]; then if [[ "$PUSH_CONFIG" == "1" && -n "$SESSION_CMD" && "$SESSION_CMD" != "attach" ]]; then
@ -195,12 +191,9 @@ echo ">> build id: $GIT_REV"
echo ">> ensuring remote dir exists" echo ">> ensuring remote dir exists"
ssh "$REMOTE_HOST" "mkdir -p '$REMOTE_DIR'" ssh "$REMOTE_HOST" "mkdir -p '$REMOTE_DIR'"
# A tower's own files live in ~/.config/fire_gimbal_control/, out of reach of # config.ini, the camera profile files and images/ are device-local (runtime
# this rsync. The config.ini and camera profile excludes below are for a tower # config, operator-exported camera settings, captured output) and are
# not moved there yet, which still keeps them in the project folder: they are # deliberately NOT mirrored, so a deploy never clobbers them.
# device-local (runtime config, operator-exported camera settings) and must not
# be clobbered before `scripts/tower-config move` runs. config.ini.moved-* is
# what that move leaves behind. images/ is captured output.
# #
# hardware.ini is excluded for the same reason but one step further: it normally # hardware.ini is excluded for the same reason but one step further: it normally
# lives in /etc/fire_gimbal_control/, out of reach of this rsync entirely, and # lives in /etc/fire_gimbal_control/, out of reach of this rsync entirely, and
@ -219,6 +212,20 @@ ssh "$REMOTE_HOST" "mkdir -p '$REMOTE_DIR'"
# build*/ covers every local build tree, not just build/. They are machine-specific # build*/ covers every local build tree, not just build/. They are machine-specific
# and the device regenerates its own build/ remotely, so shipping build-tui/ etc. # and the device regenerates its own build/ remotely, so shipping build-tui/ etc.
# just pushed tens of MB of x86 object files the device can never use. # just pushed tens of MB of x86 object files the device can never use.
CONFIG_EXCLUDES=(--exclude 'config.ini')
if [[ "$PUSH_CONFIG" == "1" ]]; then
# The device config holds that rig's calibration, camera IDs and credentials -
# things that exist nowhere else. Keep a copy before overwriting it.
echo ">> backing up the device config.ini before overwriting it"
ssh "$REMOTE_HOST" "cd '$REMOTE_DIR' && \
if [ -f config.ini ]; then \
cp -p config.ini \"config.ini.bak-\$(date +%Y%m%d-%H%M%S)\" && \
echo ' saved as config.ini.bak-<timestamp>'; \
else echo ' no existing device config.ini to back up'; fi"
CONFIG_EXCLUDES=()
echo ">> pushing local config.ini (overwrites the device's)"
fi
echo ">> syncing to $REMOTE_HOST:$REMOTE_DIR" echo ">> syncing to $REMOTE_HOST:$REMOTE_DIR"
rsync -avz --delete \ rsync -avz --delete \
--exclude 'build*/' \ --exclude 'build*/' \
@ -229,9 +236,8 @@ rsync -avz --delete \
--exclude '__pycache__' \ --exclude '__pycache__' \
--exclude '.deploy.env' \ --exclude '.deploy.env' \
--exclude 'hardware.ini' \ --exclude 'hardware.ini' \
--exclude 'config.ini' \ "${CONFIG_EXCLUDES[@]}" \
--exclude 'config.ini.bak-*' \ --exclude 'config.ini.bak-*' \
--exclude 'config.ini.moved-*' \
--exclude 'config/camera_profiles/**/*.xml' \ --exclude 'config/camera_profiles/**/*.xml' \
--exclude 'config/camera_profiles/**/*.txt' \ --exclude 'config/camera_profiles/**/*.txt' \
--exclude 'images/' \ --exclude 'images/' \
@ -242,37 +248,19 @@ rsync -avz --delete \
# shellcheck disable=SC2029 # shellcheck disable=SC2029
ssh "$REMOTE_HOST" "printf '%s\\n' '$GIT_REV' > '$REMOTE_DIR/.revision'" ssh "$REMOTE_HOST" "printf '%s\\n' '$GIT_REV' > '$REMOTE_DIR/.revision'"
# The tower's folder, and the config.ini the program there will load - asked of # Seed a config.ini on the device from the example if it doesn't have one yet,
# the tower, which searches the same places the program does. # so the first run finds a config. Never overwrites an existing device config.
# shellcheck disable=SC2029 # Skipped entirely under --push-config: the rsync above just placed one, and the
tower_config() { ssh "$REMOTE_HOST" "cd '$REMOTE_DIR' && scripts/tower-config $*"; } # "keeping existing device config.ini" line would be actively misleading there.
TOWER_DIR="$(tower_config dir)"
if [[ "$PUSH_CONFIG" == "1" ]]; then if [[ "$PUSH_CONFIG" == "1" ]]; then
# The tower's config holds that rig's calibration, camera IDs and credentials - echo ">> device config.ini is the one just pushed (previous copy: config.ini.bak-*)"
# things that exist nowhere else. Keep a copy before overwriting it.
TOWER_CFG="$(tower_config where || echo "$TOWER_DIR/config.ini")"
echo ">> pushing local config.ini to $TOWER_CFG (overwrites the tower's)"
# shellcheck disable=SC2029
ssh "$REMOTE_HOST" "mkdir -p \"\$(dirname '$TOWER_CFG')\" && \
if [ -f '$TOWER_CFG' ]; then \
cp -p '$TOWER_CFG' '$TOWER_CFG.bak-'\$(date +%Y%m%d-%H%M%S) && \
echo ' previous copy saved as config.ini.bak-<timestamp> beside it'; \
else echo ' the tower had no config.ini to back up'; fi"
scp -q config.ini "$REMOTE_HOST:$TOWER_CFG"
echo " relative paths in it (grid_file, profile_dir) now resolve from $(dirname "$TOWER_CFG")"
else else
# A new tower gets its folder, with a config.ini from the example. Never echo ">> ensuring remote config.ini exists"
# overwrites: an existing config.ini is kept wherever it is. ssh "$REMOTE_HOST" "cd '$REMOTE_DIR' && \
echo ">> ensuring the tower has a config.ini" if [ ! -f config.ini ]; then \
tower_config seed | sed 's/^>> / /' cp config/config.example.ini config.ini && \
TOWER_CFG="$(tower_config where)" echo ' seeded config.ini from config/config.example.ini — edit it on the device'; \
fi else echo ' keeping existing device config.ini'; fi"
if [[ "$(dirname "$TOWER_CFG")" != "$TOWER_DIR" ]]; then
echo "!! this tower's config.ini is still in the project folder ($TOWER_CFG)." >&2
echo "!! Move it, with its scan grid and camera profiles, out of a deploy's reach:" >&2
echo "!! ssh $REMOTE_HOST 'cd $REMOTE_DIR && scripts/fgc stop && scripts/tower-config move'" >&2
fi fi
if [[ "$CLEAN" == "1" ]]; then if [[ "$CLEAN" == "1" ]]; then
@ -289,20 +277,6 @@ ssh "$REMOTE_HOST" "$RESOLVE_CMAKE $CHECK_DEPS_CMD cd '$REMOTE_DIR' && \
\"\$CMAKE\" --build build $JOBS_FLAG" \"\$CMAKE\" --build build $JOBS_FLAG"
echo ">> build complete: $REMOTE_DIR/build/fire_gimbal_control" echo ">> build complete: $REMOTE_DIR/build/fire_gimbal_control"
# A pushed config.ini was written on another machine: check that it loads on
# this one, with this tower's hardware.ini layered over it, before anyone runs it.
if [[ "$PUSH_CONFIG" == "1" ]]; then
echo ">> checking the pushed config.ini on the tower"
# shellcheck disable=SC2029
if ! ssh "$REMOTE_HOST" "cd '$REMOTE_DIR' && hw=/etc/fire_gimbal_control/hardware.ini && \
if [ -f \$hw ]; then build/fgc-config check --config '$TOWER_CFG' --hardware \$hw; \
else build/fgc-config check --config '$TOWER_CFG'; fi"; then
echo "!! the pushed config.ini does not load on the tower; the previous one is" >&2
echo "!! $TOWER_CFG.bak-<timestamp>" >&2
exit 1
fi
fi
# A boot service installed before [Startup] existed still starts the program # A boot service installed before [Startup] existed still starts the program
# with --init/--start, which this build refuses: the tower would not come up at # with --init/--start, which this build refuses: the tower would not come up at
# its next boot. Only the installer can fix that (it moves them into config.ini # its next boot. Only the installer can fix that (it moves them into config.ini
@ -332,7 +306,7 @@ if [[ "$RUN" == "1" ]]; then
echo ">> checking device config.ini for unedited placeholders" echo ">> checking device config.ini for unedited placeholders"
# shellcheck disable=SC2029 # shellcheck disable=SC2029
if ! ssh "$REMOTE_HOST" " if ! ssh "$REMOTE_HOST" "
cfg='$TOWER_CFG' cfg='$REMOTE_DIR/config.ini'
[ -f \"\$cfg\" ] || { echo \"error: \$cfg missing on device\" >&2; exit 2; } [ -f \"\$cfg\" ] || { echo \"error: \$cfg missing on device\" >&2; exit 2; }
# Read an INI value (last assignment wins), strip inline ';' comment + ws, lowercase. # Read an INI value (last assignment wins), strip inline ';' comment + ws, lowercase.
ini_val() { grep -iE \"^[[:space:]]*\$1[[:space:]]*=\" \"\$cfg\" | tail -1 \ ini_val() { grep -iE \"^[[:space:]]*\$1[[:space:]]*=\" \"\$cfg\" | tail -1 \

View File

@ -112,5 +112,5 @@ homed this power cycle.
build/ CMake build output (gitignored) build/ CMake build output (gitignored)
include/fgc/, src/ headers and implementations (see modules-reference.md) include/fgc/, src/ headers and implementations (see modules-reference.md)
tests/ unit tests tests/ unit tests
config/config.example.ini template (scripts/tower-config seed copies it to ~/.config/fire_gimbal_control/) config/config.example.ini template (copy to config.ini)
``` ```

View File

@ -9,42 +9,16 @@ There are no hardcoded paths. The file is resolved in this order ([src/core/Path
1. `--config <path>` CLI flag 1. `--config <path>` CLI flag
2. `$FGC_CONFIG` environment variable 2. `$FGC_CONFIG` environment variable
3. `$XDG_CONFIG_HOME/fire_gimbal_control/config.ini` (else `~/.config/...`) - **the tower folder** 3. `./config.ini` (current directory)
4. `./config.ini` (current directory) 4. `<executable dir>/config.ini`
5. `<executable dir>/config.ini` 5. `$XDG_CONFIG_HOME/fire_gimbal_control/config.ini` (else `~/.config/...`)
If none exist, the program prints every location it searched and exits. If none exist, the program prints every location it searched and exits. Start from the template:
### The tower folder
Everything that belongs to one tower and is changed on it lives in one folder,
`~/.config/fire_gimbal_control/`:
| File | What |
|---|---|
| `config.ini` | what the tower does, and its calibration |
| `scan.csv` (or whatever `[Scan] grid_file` names) | the scan grid chosen for this tower's view |
| `camera_profiles/rgb/*.xml`, `*.txt` | the camera profiles exported on this tower |
The folder is outside the project folder, which is what a deploy (`rsync --delete`) and an update
replace, so neither can overwrite or delete anything in it. `hardware.ini`, which describes the
assembly and is not changed, stays in `/etc/fire_gimbal_control/`. Relative paths in `config.ini`
(`grid_file`, `profile_dir`) resolve from the folder `config.ini` is in.
[scripts/tower-config](../scripts/tower-config) manages it:
```bash ```bash
scripts/tower-config seed # a new tower: config.ini, scan.csv and camera_profiles/rgb from the examples cp config/config.example.ini config.ini
scripts/tower-config where # the config.ini the program will load
scripts/tower-config move --dry-run # a tower set up before the folder existed: what would move
scripts/tower-config move # ... and move it (stop the program first)
``` ```
Places 4 and 5 are the old ones: a tower not moved yet still has `config.ini` in the project
folder, and the program loads it from there and warns at every start. A development checkout may
keep its own `./config.ini` there too; the program only warns on a machine with
`/etc/fire_gimbal_control/hardware.ini`.
## `config.ini` keys ## `config.ini` keys
Parsed and validated by `ConfigLoader` ([src/core/Config.cpp](../src/core/Config.cpp)) into a typed Parsed and validated by `ConfigLoader` ([src/core/Config.cpp](../src/core/Config.cpp)) into a typed
@ -242,8 +216,7 @@ the tower can measure it. Yaw is untouched by it.
The capture **scan grid** is the ordered `(yaw,pitch)` waypoints auto-sweep visits, always front to The capture **scan grid** is the ordered `(yaw,pitch)` waypoints auto-sweep visits, always front to
back, then round to the first again — never in reverse. `[Scan] grid_file` points at an editable CSV back, then round to the first again — never in reverse. `[Scan] grid_file` points at an editable CSV
in the tower folder (`scan.csv`, first copied from the example [config/scan.csv](../config/scan.csv)) ([config/scan.csv](../config/scan.csv)) and is the **only** source of waypoints.
and is the **only** source of waypoints.
**Nothing is generated.** Leave `grid_file` blank (or name a file that is missing, malformed or **Nothing is generated.** Leave `grid_file` blank (or name a file that is missing, malformed or
empty) and the program logs why and leaves auto-sweep off, showing the reason in the TUI's scan-grid empty) and the program logs why and leaves auto-sweep off, showing the reason in the TUI's scan-grid
@ -749,8 +722,7 @@ It is modular (`test gimbal`, `test imu drift`, `test host`, …) and driven by
**profiles** configured under `[Test]` / `[TestProfile.<name>]` in `config.ini` **profiles** configured under `[Test]` / `[TestProfile.<name>]` in `config.ini`
(repetition counts, sampling windows, pass thresholds; default `standard`). While a (repetition counts, sampling windows, pass thresholds; default `standard`). While a
test (or calibration, or homing) runs, interfering commands are ignored and **Esc** test (or calibration, or homing) runs, interfering commands are ignored and **Esc**
cancels. A stop is not ignored: `exit`, Ctrl-C or SIGTERM (`scripts/fgc stop`) during homing cancels. Full reference: [test-command.md](test-command.md). Replaces the former
cancels the homing, as Esc does, and shuts the program down. Full reference: [test-command.md](test-command.md). Replaces the former
`gimbal diag` (now folded into `test gimbal encoder`). `gimbal diag` (now folded into `test gimbal encoder`).
## Terminal dashboard (TUI) ## Terminal dashboard (TUI)

View File

@ -16,42 +16,16 @@ between them; what differs lives in each tower's `hardware.ini`.
|---|---|---| |---|---|---|
| `~ggs/projects/fwt_2a/software/` | this repository, and the program's build in `build/` | `deploy.sh` (sync + build), or the installer | | `~ggs/projects/fwt_2a/software/` | this repository, and the program's build in `build/` | `deploy.sh` (sync + build), or the installer |
| `~ggs/projects/fwt_2a/firmware/` | the firmware repository, built on the tower | `firmware/deploy.sh` | | `~ggs/projects/fwt_2a/firmware/` | the firmware repository, built on the tower | `firmware/deploy.sh` |
| `~ggs/.config/fire_gimbal_control/config.ini` | what the tower DOES: scan, capture, logging, calibration | the installer (seeded once), `gimbal ... setup` (calibration), a person | | `.../software/config.ini` | what the tower DOES: scan, capture, logging, calibration | the installer (seeded once), `gimbal ... setup` (calibration), a person |
| `~ggs/.config/fire_gimbal_control/scan.csv` | the scan grid chosen for this tower (`[Scan] grid_file`) | the installer (the example grid), a person |
| `/etc/fire_gimbal_control/hardware.ini` | what the tower IS: board, axes, sensors, pins, drivetrain | the installer; a person when the hardware changes | | `/etc/fire_gimbal_control/hardware.ini` | what the tower IS: board, axes, sensors, pins, drivetrain | the installer; a person when the hardware changes |
| `/etc/fire_gimbal_control/credentials.env` | `FGC_MQTT_USER=` / `FGC_MQTT_PW=`, mode 0640 root:ggs | the installer | | `/etc/fire_gimbal_control/credentials.env` | `FGC_MQTT_USER=` / `FGC_MQTT_PW=`, mode 0640 root:ggs | the installer |
| `/etc/fire_gimbal_control/install.log` | one line per installer run | the installer | | `/etc/fire_gimbal_control/install.log` | one line per installer run | the installer |
| `~ggs/.local/share/fire_gimbal_control/logs/` | the program's log, test reports | the program | | `~ggs/.local/share/fire_gimbal_control/logs/` | the program's log, test reports | the program |
| `~ggs/.config/fire_gimbal_control/camera_profiles/rgb/*.xml` | the camera settings, exported ON the tower from its camera | the installer, or `scripts/export_camera_profile.py` | | `.../software/config/camera_profiles/rgb/*.xml` | the camera settings, exported ON the tower from its camera | the installer, or `scripts/export_camera_profile.py` |
The rule: what a tower changes lives in `~/.config/fire_gimbal_control/`, what `deploy.sh` never touches `config.ini`, `hardware.ini`, the camera profiles or
it never changes in `/etc/fire_gimbal_control/`, and the project folder holds the images: they exist only on the tower, and `rsync --delete` once erased a
only what a deploy or an update may replace. `deploy.sh` mirrors the project freshly exported profile before they were excluded.
folder with `rsync --delete`, so nothing of the tower's own may live there -
`rsync --delete` once erased a freshly exported profile that did.
### A tower set up before the tower folder existed
Its `config.ini`, scan grid and camera profiles are still in the project folder.
The program loads them from there and warns at every start, and every deploy
prints how to move them. Until then `deploy.sh`'s excludes keep them safe. Move
them once, on the tower:
```bash
ssh ggs@<tower>
cd ~/projects/fwt_2a/software
scripts/tower-config move --dry-run # what moves where, and what config.ini will say
scripts/fgc stop # the program writes config.ini itself
scripts/tower-config move
scripts/fgc start # or: sudo systemctl start fire-gimbal-control
```
It copies `config.ini`, the scan grid `[Scan] grid_file` names and every
camera's profiles into the tower folder, and rewrites those two paths
(`config/scan.csv` becomes `scan.csv`). `fgc-config` checks the result loads
before it is written. The old `config.ini` is renamed
`config.ini.moved-<timestamp>`, and the old grid and profiles stay where they
were. `scripts/install-tower` does the same move as one of its steps.
## 1. Updating the program from the dev box ## 1. Updating the program from the dev box
@ -172,11 +146,8 @@ it is shown dimmed.
`scripts/fgc stop` sends SIGTERM and waits up to 120 s; the program handles it `scripts/fgc stop` sends SIGTERM and waits up to 120 s; the program handles it
with the same shutdown path as the `exit` command (stop capture, close the with the same shutdown path as the `exit` command (stop capture, close the
camera, flush the image queue). A stop while the gimbal is homing cancels the camera, flush the image queue). Do **not** `kill -9` - an unclean exit leaves the
homing rather than waiting for it - a full home takes minutes on some towers, camera wedged for the next run. If it will not stop, attach and look at why.
longer than this wait - so the gimbal is left un-homed, and the next start homes
it again. Do **not** `kill -9` - an unclean exit leaves the camera wedged for the
next run. If it will not stop, attach and look at why.
`scripts/fgc start` refuses if a run is already up, and prints the previous `scripts/fgc start` refuses if a run is already up, and prints the previous
run's last output if it died rather than silently discarding it. run's last output if it died rather than silently discarding it.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 733 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 436 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 61 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 114 KiB

View File

@ -1,512 +0,0 @@
{
"frame": {
"width": 5328,
"height": 3040,
"green_range_12bit": [
128,
2792
],
"share_of_frame": {
"dark (< 300 DN)": 33.2,
"mid (300-1000 DN)": 38.5,
"bright (>= 1000 DN)": 28.3
}
},
"noise_calibration": 1.089,
"noise": {
"12-bit (this frame)": [
[
156,
5.4
],
[
184,
6.2
],
[
230,
7.9
],
[
274,
10.4
],
[
336,
14.8
],
[
454,
11.2
],
[
568,
12.4
],
[
738,
13.9
],
[
1299,
16.2
],
[
1596,
18.3
],
[
2259,
20.5
],
[
2551,
21.6
]
],
"8-bit, same frame (12-bit >> 4)": [
[
156,
6.4
],
[
184,
7.8
],
[
230,
9.3
],
[
274,
11.4
],
[
336,
15.6
],
[
454,
12.3
],
[
568,
13.4
],
[
738,
14.8
],
[
1300,
16.9
],
[
1597,
18.9
],
[
2259,
21.0
],
[
2551,
22.2
]
],
"camera RGB8 (other capture)": [
[
155,
9.6
],
[
182,
10.5
],
[
228,
12.2
],
[
272,
14.6
],
[
351,
15.1
],
[
428,
15.8
],
[
602,
18.3
],
[
748,
23.1
],
[
1272,
24.2
],
[
1681,
28.3
],
[
2285,
32.0
],
[
2442,
34.5
]
],
"4 Oct frame, 0 dB": [
[
661,
17.0
],
[
893,
21.3
],
[
1189,
27.8
],
[
1666,
31.6
],
[
2177,
36.6
],
[
2587,
37.3
],
[
3024,
41.4
]
],
"2 Oct frame, \u2248 15 dB": [
[
671,
32.9
],
[
860,
37.8
],
[
1208,
46.5
],
[
1603,
55.1
],
[
2180,
66.0
],
[
2601,
72.1
],
[
3054,
77.1
]
]
},
"dark_crop": {
"x": 40,
"y": 360,
"size": 400,
"mean_green_12bit": 194.1,
"green_range_12bit": [
129,
557
],
"distinct_green_levels": {
"12-bit": 272,
"8-bit same frame": 25,
"camera RGB8": 22
}
},
"mid_crop": {
"x": 3800,
"y": 1300,
"size": 400,
"mean_green_12bit": 1606.3
},
"sizes_mb": {
"12-bit d0": 38.58,
"12-bit d0.3": 3.4,
"12-bit d0.5": 2.13,
"12-bit d1.0": 0.92,
"8-bit d0": 15.53,
"8-bit d0.3": 2.51,
"8-bit d0.5": 1.54,
"8-bit d1.0": 0.63,
"8-bit values, 12-bit file d0": 39.04,
"8-bit values, 12-bit file d0.3": 3.5,
"8-bit values, 12-bit file d0.5": 2.19,
"8-bit values, 12-bit file d1.0": 0.94,
"camera RGB8 d0": 13.54,
"camera RGB8 d0.3": 2.64,
"camera RGB8 d0.5": 1.65,
"camera RGB8 d1.0": 0.66,
"camera RGB8 today e4 d0.8": 0.92
},
"errors": [
{
"source": "12-bit",
"distance": 0,
"size_mb": 38.58,
"dark (< 300 DN)": {
"rms_px": 0.0,
"rms_avg": 0.0
},
"mid (300-1000 DN)": {
"rms_px": 0.0,
"rms_avg": 0.0
},
"bright (>= 1000 DN)": {
"rms_px": 0.0,
"rms_avg": 0.0
}
},
{
"source": "12-bit",
"distance": 0.3,
"size_mb": 3.4,
"dark (< 300 DN)": {
"rms_px": 12.28,
"rms_avg": 0.61
},
"mid (300-1000 DN)": {
"rms_px": 18.0,
"rms_avg": 0.94
},
"bright (>= 1000 DN)": {
"rms_px": 35.8,
"rms_avg": 2.53
}
},
{
"source": "12-bit",
"distance": 0.5,
"size_mb": 2.13,
"dark (< 300 DN)": {
"rms_px": 14.29,
"rms_avg": 0.92
},
"mid (300-1000 DN)": {
"rms_px": 20.81,
"rms_avg": 1.42
},
"bright (>= 1000 DN)": {
"rms_px": 39.92,
"rms_avg": 3.79
}
},
{
"source": "12-bit",
"distance": 1.0,
"size_mb": 0.92,
"dark (< 300 DN)": {
"rms_px": 16.77,
"rms_avg": 1.64
},
"mid (300-1000 DN)": {
"rms_px": 24.29,
"rms_avg": 2.56
},
"bright (>= 1000 DN)": {
"rms_px": 43.02,
"rms_avg": 5.98
}
},
{
"source": "8-bit",
"distance": 0,
"size_mb": 15.53,
"dark (< 300 DN)": {
"rms_px": 4.6,
"rms_avg": 0.64
},
"mid (300-1000 DN)": {
"rms_px": 4.61,
"rms_avg": 0.63
},
"bright (>= 1000 DN)": {
"rms_px": 4.61,
"rms_avg": 0.63
}
},
{
"source": "8-bit",
"distance": 0.3,
"size_mb": 2.51,
"dark (< 300 DN)": {
"rms_px": 17.35,
"rms_avg": 1.6
},
"mid (300-1000 DN)": {
"rms_px": 21.66,
"rms_avg": 1.53
},
"bright (>= 1000 DN)": {
"rms_px": 29.53,
"rms_avg": 1.59
}
},
{
"source": "8-bit",
"distance": 0.5,
"size_mb": 1.54,
"dark (< 300 DN)": {
"rms_px": 18.59,
"rms_avg": 2.18
},
"mid (300-1000 DN)": {
"rms_px": 23.9,
"rms_avg": 2.13
},
"bright (>= 1000 DN)": {
"rms_px": 34.86,
"rms_avg": 2.29
}
},
{
"source": "8-bit",
"distance": 1.0,
"size_mb": 0.63,
"dark (< 300 DN)": {
"rms_px": 20.45,
"rms_avg": 3.7
},
"mid (300-1000 DN)": {
"rms_px": 26.71,
"rms_avg": 3.44
},
"bright (>= 1000 DN)": {
"rms_px": 41.05,
"rms_avg": 3.99
}
},
{
"source": "8-bit values, 12-bit file",
"distance": 0,
"size_mb": 39.04,
"dark (< 300 DN)": {
"rms_px": 4.6,
"rms_avg": 0.64
},
"mid (300-1000 DN)": {
"rms_px": 4.61,
"rms_avg": 0.63
},
"bright (>= 1000 DN)": {
"rms_px": 4.61,
"rms_avg": 0.63
}
},
{
"source": "8-bit values, 12-bit file",
"distance": 0.3,
"size_mb": 3.5,
"dark (< 300 DN)": {
"rms_px": 12.72,
"rms_avg": 0.88
},
"mid (300-1000 DN)": {
"rms_px": 18.29,
"rms_avg": 1.14
},
"bright (>= 1000 DN)": {
"rms_px": 35.88,
"rms_avg": 2.61
}
},
{
"source": "8-bit values, 12-bit file",
"distance": 0.5,
"size_mb": 2.19,
"dark (< 300 DN)": {
"rms_px": 14.55,
"rms_avg": 1.12
},
"mid (300-1000 DN)": {
"rms_px": 20.99,
"rms_avg": 1.57
},
"bright (>= 1000 DN)": {
"rms_px": 39.96,
"rms_avg": 3.83
}
},
{
"source": "8-bit values, 12-bit file",
"distance": 1.0,
"size_mb": 0.94,
"dark (< 300 DN)": {
"rms_px": 16.87,
"rms_avg": 1.76
},
"mid (300-1000 DN)": {
"rms_px": 24.36,
"rms_avg": 2.62
},
"bright (>= 1000 DN)": {
"rms_px": 43.03,
"rms_avg": 6.01
}
}
],
"same_setting": [
{
"source": "camera RGB8",
"distance": 0.8,
"effort": 4,
"size_mb": 0.92,
"dark (< 300 DN)": {
"rms_px": 18.01,
"rms_avg": 2.18
},
"mid (300-1000 DN)": {
"rms_px": 21.4,
"rms_avg": 2.25
},
"bright (>= 1000 DN)": {
"rms_px": 34.3,
"rms_avg": 2.82
}
},
{
"source": "BayerRG12p",
"distance": 0.8,
"effort": 4,
"size_mb": 1.22,
"dark (< 300 DN)": {
"rms_px": 16.26,
"rms_avg": 1.22
},
"mid (300-1000 DN)": {
"rms_px": 23.57,
"rms_avg": 1.9
},
"bright (>= 1000 DN)": {
"rms_px": 42.42,
"rms_avg": 4.35
}
}
]
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 152 KiB

View File

@ -25,16 +25,12 @@ tests; waiting for exposure and gain to stop changing never did.
The extra bits themselves add less than expected. In the daylight scene measured, the sensor's own The extra bits themselves add less than expected. In the daylight scene measured, the sensor's own
noise was 6–17 times larger than the error from rounding to 8 bits, which raised the total noise by noise was 6–17 times larger than the error from rounding to 8 bits, which raised the total noise by
at most 1.5%; and JPEG XL compression at distance 1.0 adds about 13 times more error than that at most 1.5%; and JPEG XL compression at distance 1.0 adds about 13 times more error than that
rounding. A dark, high-contrast scene from the Stäffelsberg tower (shaded forest under a hazy sky) rounding. 12 bits are expected to make a meaningful difference only where part of the image is dark
was measured as well: there the noise in the dark third of the frame is below one 8-bit step, and while the gain is low, as in high-contrast scenes (a bright sky over shaded forest or a backlit ridge
rounding to 8 bits raises it by 10–26%, but averaged over an area the 8-bit values still keep the at dawn or dusk), where faint smoke over dark terrain is a small change in a dark value; and only if
local brightness to 0.6 DN (12-bit scale). The larger difference comes from the encoder: saved at those values are saved with little or no loss. These conditions were not measured. The 12-bit format
distance 1.0, an 8-bit file breaks the shaded forest into flat blocks and keeps its local brightness costs little against 8-bit raw (same file size, about 10% more processing time), but the camera
2.3 times less accurately than a 12-bit file, and today's RGB8 file shows the same blocks at the same runs at 9.86 instead of 14.75 fps.
size as the 12-bit file. The same 8-bit values saved as a 12-bit file behave almost like 12-bit data,
so the gain comes mostly from the file's declared bit depth, not from the four extra bits. The
12-bit format costs little against 8-bit raw (about 10% more processing time; the same file size in
daylight, 46% larger in the dark scene), but the camera runs at 9.86 instead of 14.75 fps.
## Units and abbreviations ## Units and abbreviations
@ -344,7 +340,6 @@ session.*
| 4 Oct 16:20–16:41 | 111 | camera tests | 21 | tower 111's own camera connected but idle | | 4 Oct 16:20–16:41 | 111 | camera tests | 21 | tower 111's own camera connected but idle |
| 4 Oct 16:41–18:19 | 111 | processing, pile-up, heat | 98 | | | 4 Oct 16:41–18:19 | 111 | processing, pile-up, heat | 98 | |
| 4 Oct 23:03–23:32 | 121 and 111 | processing on the shared frame, both towers simultaneously | 29 | no camera needed | | 4 Oct 23:03–23:32 | 121 and 111 | processing on the shared frame, both towers simultaneously | 29 | no camera needed |
| 7 Oct, morning | Stäffelsberg | one `BayerRG12p` and one RGB8 frame of a dark, high-contrast view (Section 3.9.2) | – | hazy; gain and exposure not recorded |
--- ---
@ -622,13 +617,6 @@ Full-size `.jxl` files are kept outside the repository in `tools/camera-bench/ex
### 3.9 Information content of the 12-bit data ### 3.9 Information content of the 12-bit data
The extra four bits were examined in two scenes: the daylight scene of the main campaign (2 and
4 October, Section 3.9.1), where noise is large, and a dark, high-contrast scene photographed from
the Stäffelsberg tower on 7 October (Section 3.9.2), where large parts of the frame are dark at low
noise. Section 3.9.3 combines the two.
#### 3.9.1 Daylight scene (2 and 4 October)
![Noise vs brightness](images/raw12/noise_vs_signal.png) ![Noise vs brightness](images/raw12/noise_vs_signal.png)
***Figure 12.** Sensor noise σ (DN, 12-bit scale) against signal level (DN, 12-bit scale), measured ***Figure 12.** Sensor noise σ (DN, 12-bit scale) against signal level (DN, 12-bit scale), measured
@ -668,238 +656,45 @@ and the dropped bits hold real scene detail.
has a PSNR of 36.6 dB (Table 16), an RMS error of about 61 DN, which is 13 times the rounding error; has a PSNR of 36.6 dB (Table 16), an RMS error of about 61 DN, which is 13 times the rounding error;
even at distance 0.3 (40.3 dB) it is about 40 DN, and raw planes at distance 0.5 (47.2 dB) about even at distance 0.3 (40.3 dB) it is about 40 DN, and raw planes at distance 0.5 (47.2 dB) about
18 DN. Much of this error is noise the encoder discards, and JPEG XL spends relatively more 18 DN. Much of this error is noise the encoder discards, and JPEG XL spends relatively more
precision on dark tones, so the error in shadows is smaller than these averages (measured on the precision on dark tones, so the error in shadows is smaller than these averages; it was not measured
dark scene in Table 17d). Still, with lossy files the compression, not the bit depth, decides how separately. Still, with lossy files the compression, not the bit depth, decides how much fine tonal
much fine tonal detail survives. Correspondingly, 12-bit and 8-bit raw give the same file size at detail survives. Correspondingly, 12-bit and 8-bit raw give the same file size at distance 1.0
distance 1.0 on this scene (2.65 and 2.66 MB, Table 14). (2.65 and 2.66 MB, Table 14).
**Where 12 bits are expected to matter.** The noise falls with the signal, while the 8-bit step
stays at 16 DN. Extrapolating Figure 12, the noise drops below one 8-bit step at about 300 DN at
0 dB gain, or 100 DN at ≈ 15 dB. The 8-bit data is linear (no gamma curve), so this region, the
darkest 2–7% of the range, holds only 6–19 of the 256 8-bit levels. Gain multiplies the noise, so at
high gain the noise exceeds one 8-bit step again almost everywhere. Table 17b combines these
observations.
***Table 17b.** Situations in which 12-bit data is expected to differ from 8-bit data. Signal in DN
on the 12-bit scale (0–4095). "Measured" refers to this campaign.*
| Situation | Signal (DN), gain (dB) | Noise against one 8-bit step | Expected effect of 12 bits | Measured |
|---|---|---|---|---|
| Daylight, mid-tones and highlights (lit terrain, sky) | 700–4095, 0–15 | 1.6–5 steps | none measurable: total noise +0.2–1.5% | yes |
| Shadows in a high-contrast scene at low gain: shaded forest under a bright sky, backlit ridges, valleys at dawn and dusk | below ≈ 300 at 0 dB, ≈ 100 at 15 dB | below 1 step | 8 bits merge neighbouring levels; 12 bits keep small changes in dark areas, such as thin smoke over dark terrain | no (extrapolated) |
| Uniformly dark scene at high gain: night, gain up to 48 dB | any, high (up to 48) | above 1 step (gain amplifies noise) | little, except in the very darkest values | no |
| Strong processing of dark areas: brightening by 3 stops (×8) or more, contrast stretching | dark areas, any | – | 12 bits delay visible banding | 2 stops (×4) only: no difference (Figure 13) |
| Saving lossy at distance 1.0 | any | compression error ≈ 61 DN ≫ rounding 4.6 DN | largely lost to compression; kept in lossless files | yes, overall PSNR only |
The case most relevant to fire detection is the second: in a scene exposed for a bright sky, smoke
rising in front of shaded forest is a small brightness increase in an already dark area, at low gain.
Whether 12 bits help there, and whether that benefit survives distance 1.0, requires a test on such a
scene (a dawn or dusk frame with dark terrain under a bright sky, compared as in Figures 12 and 13).
![8 vs 12 bit at 15 dB](images/raw12/examples/121/shadow_8_vs_12.png) ![8 vs 12 bit at 15 dB](images/raw12/examples/121/shadow_8_vs_12.png)
![8 vs 12 bit at 0 dB](images/raw12/examples/lowgain/shadow_8_vs_12.png) ![8 vs 12 bit at 0 dB](images/raw12/examples/lowgain/shadow_8_vs_12.png)
***Figure 13a.** The darkest textured crop of the daylight scene (mean ≈ 810 DN) brightened by ***Figure 13.** The same darker crop brightened by 2 stops (pixel values multiplied by 4), saved as
2 stops (pixel values multiplied by 4), saved as 8-bit (left; the 12-bit value shifted down by 8-bit (left; the 12-bit value shifted down by 4 bits, i.e. divided by 16) and as 12-bit (right): (a,
4 bits, i.e. divided by 16) and as 12-bit (right): top, ≈ 15 dB gain, 227 distinct green levels in top) ≈ 15 dB gain, 227 distinct green levels in 8-bit against 3138 in 12-bit; (b, bottom) 0 dB gain.
8-bit against 3138 in 12-bit; bottom, 0 dB gain. No visible difference: the noise is larger than the No visible difference: the noise is larger than the extra levels.*
extra levels.*
At 0 dB gain the same settings produce 25–30% smaller files and about 3 dB higher PSNR (less noise At 0 dB gain the same settings produce 25–30% smaller files and about 3 dB higher PSNR (less noise
to encode): 12-bit colour at d1.0 is 1.9 MB, PSNR 40.0 dB, SSIM 0.987. to encode): 12-bit colour at d1.0 is 1.9 MB, PSNR 40.0 dB, SSIM 0.987.
#### 3.9.2 Dark, high-contrast scene (Stäffelsberg, 7 October)
**Scene and method.** On the morning of 7 October 2026, an Alvium 1800 U-1620c (5328 × 3040) on the
Stäffelsberg tower took two frames of the same view, one after the other: hazy sky and a distant valley above a
slope of shaded forest (Figure 13b). One was taken as `BayerRG12p`, converted to colour by the camera's
viewer program and saved as a 16-bit TIFF (the 12-bit values in the top 12 bits); the other as the camera's
RGB8 (today's format), saved as PNG. Gain and exposure were not recorded with the files. The 12-bit
frame spans 128–2792 DN (green), with nothing clipped: a third of it is darker than 300 DN, the
region where Section 3.9.1 expected 12 bits to matter.
The camera's RGB8 frame is a separate capture (trees move between frames) made by the camera's own
colour processing, so it is compared by eye and by statistics only. For an exact comparison of bit
depth, an **8-bit version of the same frame** was made by keeping the top 8 of the 12 bits (the
12-bit value divided by 16, rounded down, as `BayerRG8` delivers), and compared pixel by pixel with
the 12-bit original. The 12-bit frame was converted to colour by the viewer, not by the tower's
edge-aware conversion, so its noise is not directly comparable with Figure 12 (measured on the raw
green pixels); for comparison, the October frames' lossless 12-bit colour files were measured in
the same way. Compression used libjxl 0.12.0 on the laptop with the tower's settings (effort 3;
12-bit and 8-bit input as declared by their bit depth), single runs. Script:
`tools/camera-bench/compare_8_vs_12.py`; results in
`docs/images/raw12/examples/staeffelsberg/results.json`.
![Stäffelsberg overview](images/raw12/examples/staeffelsberg/overview.jpg)
***Figure 13b.** The Stäffelsberg 12-bit frame, shown upright (the frame is rotated by 90° so that
the sky is at the top), at a quarter of full size and with a display brightness curve (sRGB) so that
the forest is visible. Blue: the dark crop of Figures 13d and 13f (shaded forest, mean 194 DN).
Red: the mid-tone crop of Figure 13e (distant valley in haze, mean 1606 DN).*
**Noise.** The noise in each colour image was measured as the standard deviation of each pixel
around its local average (a Gaussian average over about 2 pixels), in 24 × 24-pixel patches grouped
by brightness, taking the quietest 10% of each group as in Section 2.11; the result was calibrated
on generated noise of known σ.
![Noise vs brightness, Stäffelsberg](images/raw12/examples/staeffelsberg/noise_vs_signal.png)
***Figure 13c.** Noise σ (DN, 12-bit scale) against signal (DN, 12-bit scale, green) in the colour
images: the Stäffelsberg 12-bit frame (blue), its 8-bit version (orange), the camera's RGB8 frame of
the same view (purple, another capture, multiplied by 16 to the 12-bit scale), and the October 0 dB
and ≈ 15 dB frames measured the same way (dashed). Dashed line: one 8-bit step (16 DN); dotted line:
the RMS error of rounding to 8 bits (4.6 DN). The bump at ≈ 340 DN is scene texture that remains
even in the quietest patches.*
***Table 17c.** Noise σ (DN, 12-bit scale) in the Stäffelsberg colour images at selected signal
levels (DN, 12-bit scale, green). In 8-bit steps: the 12-bit noise divided by 16; below 1, an 8-bit
value cannot record the variation the sensor delivers. Increase: how much rounding to 8 bits raises
the noise. Camera RGB8: another capture, the camera's own colour processing.*
| Signal (DN) | 12-bit noise (DN) | In 8-bit steps | 8-bit version, same frame (DN) | Increase from rounding | Camera RGB8 (DN) |
|---|---|---|---|---|---|
| ≈ 160 | 5.4 | 0.34 | 6.4 | +19% | 9.6 |
| ≈ 185 | 6.2 | 0.39 | 7.8 | +26% | 10.5 |
| ≈ 230 | 7.9 | 0.49 | 9.3 | +18% | 12.2 |
| ≈ 275 | 10.4 | 0.65 | 11.4 | +10% | 14.6 |
| ≈ 450 | 11.2 | 0.70 | 12.3 | +10% | 15.8 |
| ≈ 740 | 13.9 | 0.87 | 14.8 | +6% | 23.1 |
| ≈ 1300 | 16.2 | 1.01 | 16.9 | +4% | 24.2 |
| ≈ 2250 | 20.5 | 1.28 | 21.0 | +2% | 32.0 |
In this frame the noise stays below one 8-bit step up to about 1300 DN, which covers about 70% of
the frame. At ≈ 700 DN the October frames, measured the same way, have 17 DN (0 dB) and 33 DN
(≈ 15 dB) against 14 DN here, and they contain nothing darker. Rounding to 8 bits raises the noise
in the forest by 10–26%, against at most 1.5% in the daylight scene. The camera's RGB8 frame is
1.4–1.8 times noisier than the 12-bit frame at every brightness, probably from the camera's own
colour processing; this is a difference of the camera's conversion, not of bit depth.
**Levels and visibility.** The 400 × 400-pixel dark crop (green 129–557 DN) uses 272 distinct green
levels in 12-bit, 25 in its 8-bit version and 22 in the camera's RGB8.
![Dark crop, 8 vs 12 bit](images/raw12/examples/staeffelsberg/dark_crop.png)
***Figure 13d.** The dark crop (shaded forest, mean 194 DN) at 100%, brightened by 2 stops (pixel
values ×4, top) and 4 stops (×16, bottom). Left: the camera's RGB8 frame (another capture, taken
separately; branches moved). Middle: the 8-bit version of the 12-bit frame (top 8 bits). Right:
the 12-bit frame. Not compressed.*
![Mid-tone crop, 8 vs 12 bit](images/raw12/examples/staeffelsberg/mid_crop.png)
***Figure 13e.** Control: the mid-tone crop (distant valley in haze, mean 1606 DN, noise about one
8-bit step) with its contrast stretched 4 times around the crop's mean, 8-bit version (left) and
12-bit (right). Not compressed. No visible difference.*
Uncompressed, the 8-bit forest is slightly grainier than the 12-bit one at +4 stops, with no banding
(Figure 13d): even at 0.3–0.5 of a step the noise still dithers the rounding. Averaged over about
200 pixels (a Gaussian of 4 pixels), the 8-bit version differs from the 12-bit frame by 0.6 DN RMS
in every brightness range (Table 17d): the local brightness, which is what a faint, even haze such as
thin smoke changes, survives rounding to within less than a tenth of an 8-bit step.
**Compression.** Each version was saved as JPEG XL and compared with the 12-bit frame in three
brightness regions: dark (below 300 DN, 33% of the frame), mid (300–1000 DN, 39%) and bright
(1000 DN and above, 28%). Two errors are given: per pixel, which includes noise the encoder
discards; and of the local average, after averaging both images over about 200 pixels, which
measures lost scene brightness rather than lost noise. A third version tests the encoder rather than
the data: the same 8-bit values, stored as a 12-bit file (each value × 16 + 8).
***Table 17d.** File size (MB) and RMS error (DN, 12-bit scale) against the 12-bit frame, per
brightness region, for the 12-bit frame, its 8-bit version, and its 8-bit values stored in a 12-bit
file, at JPEG XL distance 0 (lossless), 0.3, 0.5 and 1.0, effort 3. Per pixel: the error at each
pixel. Local average: the error after averaging both images over about 200 pixels (Gaussian,
σ = 4 pixels). Today's file (camera RGB8, d0.8, effort 4) is 0.92 MB; the camera RGB8 frame at d1.0
is 0.66 MB (another capture, so no error can be computed).*
| File | Distance | Size (MB) | Per pixel, dark / mid / bright (DN) | Local average, dark / mid / bright (DN) |
|---|---|---|---|---|
| 12-bit | lossless | 38.6 | 0 / 0 / 0 | 0 / 0 / 0 |
| 12-bit | 0.3 | 3.40 | 12.3 / 18.0 / 35.8 | 0.6 / 0.9 / 2.5 |
| 12-bit | 0.5 | 2.13 | 14.3 / 20.8 / 39.9 | 0.9 / 1.4 / 3.8 |
| **12-bit** | **1.0** | **0.92** | **16.8 / 24.3 / 43.0** | **1.6 / 2.6 / 6.0** |
| 8-bit | lossless | 15.5 | 4.6 / 4.6 / 4.6 | 0.6 / 0.6 / 0.6 |
| 8-bit | 0.3 | 2.51 | 17.4 / 21.7 / 29.5 | 1.6 / 1.5 / 1.6 |
| 8-bit | 0.5 | 1.54 | 18.6 / 23.9 / 34.9 | 2.2 / 2.1 / 2.3 |
| **8-bit** | **1.0** | **0.63** | **20.4 / 26.7 / 41.0** | **3.7 / 3.4 / 4.0** |
| 8-bit values, 12-bit file | lossless | 39.0 | 4.6 / 4.6 / 4.6 | 0.6 / 0.6 / 0.6 |
| 8-bit values, 12-bit file | 0.3 | 3.50 | 12.7 / 18.3 / 35.9 | 0.9 / 1.1 / 2.6 |
| 8-bit values, 12-bit file | 0.5 | 2.19 | 14.6 / 21.0 / 40.0 | 1.1 / 1.6 / 3.8 |
| **8-bit values, 12-bit file** | **1.0** | **0.94** | **16.9 / 24.4 / 43.0** | **1.8 / 2.6 / 6.0** |
![Dark crop after compression](images/raw12/examples/staeffelsberg/dark_compressed.png)
***Figure 13f.** The dark crop of Figure 13d after saving, brightened by 4 stops (×16): (a) the
camera's RGB8 frame saved as today (distance 0.8, effort 4; another capture); (b) the 8-bit version
saved as an 8-bit file at distance 1.0; (c) the same 8-bit values saved as a 12-bit file at distance
1.0; (d) the 12-bit frame at distance 1.0 (the proposed setting). Effort 3 unless stated.*
At distance 1.0, the 8-bit file loses the shadow detail: the forest breaks into flat blocks
(Figure 13f b), and the local brightness in the dark region is off by 3.7 DN against 1.6 DN for the
12-bit file. Today's RGB8 file shows the same blocks (Figure 13f a), although it is as large as the
12-bit file (0.92 MB). In the bright region the order reverses (4.0 against 6.0 DN): an 8-bit file
gets about the same error at every brightness, a 12-bit file a smaller error in the dark and a larger
one in the bright.
This is a property of the encoder, not of the four extra bits: the same 8-bit values saved as a
12-bit file behave almost exactly like the 12-bit frame (1.8 against 1.6 DN in the dark, 0.94 against
0.92 MB, Figure 13f c). How the bits are spent depends on the bit depth the file declares. As a
consequence, on this scene the 12-bit file at distance 1.0 is 46% larger than the 8-bit file
(0.92 against 0.63 MB), whereas in the daylight scene the two were the same size (Table 14); dark
frames are in any case far smaller than the 2.65 MB of the daylight frame used for the storage
estimate (Section 3.12).
**Today's format against `BayerRG12p` at the same setting.** To answer directly whether a reader
would see a difference between today's files and 12-bit files, both frames were saved with the
same JPEG XL setting, today's distance 0.8 and effort 4. The two frames are separate captures, so
each file is compared with its own uncompressed original rather than with the other (Table 17e,
Figures 13g and 13h).
***Table 17e.** The camera's RGB8 frame and the `BayerRG12p` frame, both saved as JPEG XL at
distance 0.8, effort 4: file size (MB) and RMS error (DN, 12-bit scale; RGB8 values × 16) against
each file's own uncompressed frame, per brightness region as in Table 17d. Per pixel: the error at
each pixel. Local average: the error after averaging over about 200 pixels.*
| File (d0.8, effort 4) | Size (MB) | Per pixel, dark / mid / bright (DN) | Local average, dark / mid / bright (DN) |
|---|---|---|---|
| Camera RGB8 (today) | 0.92 | 18.0 / 21.4 / 34.3 | 2.2 / 2.3 / 2.8 |
| `BayerRG12p`, 12-bit file | 1.22 | 16.3 / 23.6 / 42.4 | 1.2 / 1.9 / 4.4 |
![Same setting, RGB8 and BayerRG12p](images/raw12/examples/staeffelsberg/same_setting.png)
***Figure 13g.** The dark crop of Figure 13d from the camera's RGB8 frame and from the `BayerRG12p`
frame, each uncompressed and saved at distance 0.8, effort 4 (the same setting for both). Rows: as
stored (×1, as a viewer shows the linear values), brightened by 2 stops (×4) and by 4 stops (×16).
The two frames are separate captures; branches moved between them.*
![Same setting, zoomed](images/raw12/examples/staeffelsberg/same_setting_zoom.png)
***Figure 13h.** The middle 200 × 200 pixels of the two saved crops of Figure 13g at +4 stops,
enlarged 2 times (each pixel shown as 2 × 2 screen pixels): camera RGB8 (left) and `BayerRG12p`
(right), both at distance 0.8, effort 4.*
As stored, the shaded forest is almost black and the two files cannot be told apart; at +2 stops the
RGB8 file is slightly flatter. At +4 stops the difference is clear: the RGB8 file breaks the forest
into flat blocks and loses the thin branches, while the 12-bit file stays close to its uncompressed
frame (Figure 13h). In the dark region the 12-bit file keeps the local brightness 1.8 times more
accurately (1.2 against 2.2 DN) and in the mid-tones 1.2 times; in the bright region the RGB8 file
is 1.5 times more accurate (2.8 against 4.4 DN), the same pattern as in Table 17d. At the same
setting the 12-bit file is 33% larger (1.22 against 0.92 MB); at distance 1.0 it is the same size as
today's file and still free of blocks (Figure 13f d). The difference therefore shows wherever the
shadows are brightened: a viewer applying a display curve, an operator raising the brightness, or a
detection step stretching the contrast of dark terrain.
In summary, for this dark scene: (1) the four extra bits themselves change little after saving:
1.6 against 1.8 DN of local-brightness error in the dark at distance 1.0, and 0.6 DN when 8-bit
values are kept losslessly; (2) saving as a 12-bit file keeps about 2.3 times more accurate shadow
brightness than saving as an 8-bit file at the same distance, and visibly more shadow detail than
today's RGB8 file at the same size; (3) per pixel, rounding to 8 bits raises the noise in the dark
by 10–26%, which matters only for lossless or near-lossless files; (4) with the same JPEG XL setting
(distance 0.8, effort 4), today's RGB8 file and the 12-bit file look the same as stored, but at
+4 stops the RGB8 shadows break into blocks while the 12-bit file keeps the forest detail and 1.8
times more accurate shadow brightness, in a 33% larger file.
#### 3.9.3 Where 12 bits matter
The noise falls with the signal, while the 8-bit step stays at 16 DN. In the daylight scene the
noise was above one 8-bit step everywhere (Figure 12); in the Stäffelsberg scene it was below one
step up to about 1300 DN, over 70% of the frame (Figure 13c). Gain multiplies the noise, so at high
gain the noise exceeds one 8-bit step again almost everywhere. Table 17b combines these
observations.
***Table 17b.** Situations in which 12-bit data differs from 8-bit data. Signal in DN on the 12-bit
scale (0–4095). "Measured" refers to this campaign.*
| Situation | Signal (DN), gain (dB) | Noise against one 8-bit step | Effect of 12 bits | Measured |
|---|---|---|---|---|
| Daylight, mid-tones and highlights (lit terrain, sky) | 700–4095, 0–15 | 1.6–5 steps | none measurable: total noise +0.2–1.5% | yes (3.9.1) |
| Shadows in a high-contrast scene at low gain: shaded forest under a bright sky, backlit ridges, valleys at dawn and dusk | below ≈ 1300 in the Stäffelsberg frame | 0.3–1 step | 25 instead of 272 levels in the dark crop, +10–26% noise per pixel; local brightness kept by 8-bit values to 0.6 DN; no visible banding at +4 stops | yes (3.9.2) |
| Uniformly dark scene at high gain: night, gain up to 48 dB | any, high (up to 48) | above 1 step (gain amplifies noise) | little, except in the very darkest values | no |
| Strong processing of dark areas: brightening by 3 stops (×8) or more, contrast stretching | dark areas, any | – | 12 bits delay visible banding | +2 stops in daylight, +4 stops at Stäffelsberg: no banding, 8-bit slightly grainier (Figures 13a, 13d) |
| Saving lossy at distance 1.0 | any | compression error ≈ 17–61 DN per pixel ≫ rounding 4.6 DN | an 8-bit file loses shadow detail (blocks) and 2.3 times more local brightness in the dark than a 12-bit file; caused by the declared file bit depth, not by the missing bits | yes (3.9.2, Table 17d) |
The case most relevant to fire detection is the second: smoke rising in front of shaded forest is a
small brightness increase in an already dark area, at low gain. The Stäffelsberg frame is such a
scene. There, the proposed 12-bit file at distance 1.0 keeps the local brightness of the shaded
forest to 1.6 DN RMS and its texture visibly better than today's RGB8 file of the same size; most of
that gain comes from saving a 12-bit file, and would also be available to 8-bit data saved as a
12-bit file. Whether these differences change smoke detection was not measured.
### 3.10 Saving throughput and pile-up ### 3.10 Saving throughput and pile-up
![Saves per second, same frame](images/raw12/pace_common.png) ![Saves per second, same frame](images/raw12/pace_common.png)
@ -1002,7 +797,7 @@ The towers have 11–22 GB of free storage, so images must leave the tower conti
| Processing | Unpack + colour conversion 38 ms (IOTA) / 50 ms (Delta); a 12-bit save 518 / 820 ms, slightly faster than today's RGB8 save (567 / 872 ms). | 3.5 | | Processing | Unpack + colour conversion 38 ms (IOTA) / 50 ms (Delta); a 12-bit save 518 / 820 ms, slightly faster than today's RGB8 save (567 / 872 ms). | 3.5 |
| Computers | The Delta takes 1.6 times as long on the identical frame. | 3.5 | | Computers | The Delta takes 1.6 times as long on the identical frame. | 3.5 |
| Quality | d1.0: 2.65 MB, SSIM 0.973. Raw planes keep sensor values much closer (47 vs 39 dB at 6 MB). | 3.7 | | Quality | d1.0: 2.65 MB, SSIM 0.973. Raw planes keep sensor values much closer (47 vs 39 dB at 6 MB). | 3.7 |
| 12 bits | In daylight, rounding to 8 bits adds at most 1.5% to the noise, and compression at d1.0 adds 13 times more error than rounding. In a dark, high-contrast scene (Stäffelsberg) rounding adds 10–26% noise in the shadows; at d1.0 an 8-bit file loses shadow detail and 2.3 times more shadow brightness than a 12-bit file, mostly because of the file's declared bit depth, not the extra bits. Today's RGB8 file shows the same loss: saved with the same setting (d0.8 e4) as a 12-bit file, it looks the same as stored but is blocky at +4 stops, with 1.8 times the shadow-brightness error, while the 12-bit file is 33% larger. | 3.9 | | 12 bits | In daylight, rounding to 8 bits adds at most 1.5% to the noise, and compression at d1.0 adds 13 times more error than rounding. 12 bits are expected to matter only in dark areas at low gain (high-contrast scenes), and only in lightly compressed files; not measured. | 3.9 |
| Throughput | IOTA keeps up with a frame every 0.7 s with one worker; the Delta needs two. | 3.10 | | Throughput | IOTA keeps up with a frame every 0.7 s with one worker; the Delta needs two. | 3.10 |
| Heat | One frame every 2 s: IOTA 53 °C / 5.1 W, Delta 66 °C / 5.7 W. | 3.11 | | Heat | One frame every 2 s: IOTA 53 °C / 5.1 W, Delta 66 °C / 5.7 W. | 3.11 |
| Storage | 12-bit colour d1.0: 2.7 MB per frame, below today's 3.3–4.0 MB. | 3.12 | | Storage | 12-bit colour d1.0: 2.7 MB per frame, below today's 3.3–4.0 MB. | 3.12 |
@ -1036,10 +831,6 @@ between neighbouring waypoints, which decides the real pace, remains to be measu
- One scene on two afternoons, at ≈ 15 dB gain in the main runs, plus one 0 dB frame. Night - One scene on two afternoons, at ≈ 15 dB gain in the main runs, plus one 0 dB frame. Night
conditions were not measured. conditions were not measured.
- The dark-scene comparison (Section 3.9.2) rests on one pair of frames, without recorded gain and
exposure; its 12-bit frame was converted to colour by the camera's viewer, not by the tower, and
compressed on a laptop (libjxl 0.12.0) rather than on a tower (0.11). Its 8-bit version is the
12-bit colour image cut to 8 bits, not a `BayerRG8` frame converted to colour.
- Auto-exposure steps were forced on a still camera, not caused by real slews. Real waypoint-to- - Auto-exposure steps were forced on a still camera, not caused by real slews. Real waypoint-to-
waypoint changes are usually much smaller than ×16, so real settling should be shorter. waypoint changes are usually much smaller than ×16, so real settling should be shorter.
- The tower program was stopped, so its own load (camera stream, control loop) is not included; the - The tower program was stopped, so its own load (camera stream, control loop) is not included; the
@ -1063,11 +854,9 @@ between neighbouring waypoints, which decides the real pace, remains to be measu
4. **Keep a waypoint's frame when brightness is in the camera's target band and steady for 3 4. **Keep a waypoint's frame when brightness is in the camera's target band and steady for 3
frames**, with a time limit. frames**, with a time limit.
5. **12-bit versus raw 8-bit is a choice, not a cost.** `BayerRG8` (14.75 fps) settles about 1 s 5. **12-bit versus raw 8-bit is a choice, not a cost.** `BayerRG8` (14.75 fps) settles about 1 s
faster in daylight and loses nothing measurable there. In the dark areas of a high-contrast scene faster in daylight and loses nothing measurable there; `BayerRG12p` is expected to help in dark areas
(Section 3.9.2), a 12-bit file at distance 1.0 keeps clearly more shadow detail than an 8-bit of high-contrast scenes at low gain, such as shaded terrain under a bright sky at dawn or dusk
file, but mainly because of how the file is encoded: if `BayerRG8` is used, its colour image (Section 3.9), which remains to be tested. The pixel format is part of the camera profile, so the program should accept both.
should be saved as a 12-bit (or 16-bit) JPEG XL file, which keeps nearly the same shadow detail.
The pixel format is part of the camera profile, so the program should accept both.
6. **Store raw 12-bit planes losslessly (17–18 MB per frame) only if the detection pipeline needs 6. **Store raw 12-bit planes losslessly (17–18 MB per frame) only if the detection pipeline needs
exact sensor values**; it requires about 6 times the storage. exact sensor values**; it requires about 6 times the storage.
@ -1078,8 +867,7 @@ between neighbouring waypoints, which decides the real pace, remains to be measu
1. ISO 19567-2:2019. *Photography — Digital cameras — Texture reproduction measurements — Part 2: 1. ISO 19567-2:2019. *Photography — Digital cameras — Texture reproduction measurements — Part 2:
Measurements using dead leaves pattern.* International Organization for Standardization. Measurements using dead leaves pattern.* International Organization for Standardization.
2. ISO/IEC 18181. *Information technology — JPEG XL image coding system*; reference implementation 2. ISO/IEC 18181. *Information technology — JPEG XL image coding system*; reference implementation
libjxl, versions 0.11.0 and 0.11.1 (towers) and 0.12.0 (dark-scene comparison, Section 3.9.2), libjxl, versions 0.11.0 and 0.11.1, https://github.com/libjxl/libjxl.
https://github.com/libjxl/libjxl.
3. OpenCV, versions 4.5.4 and 4.10: `cvtColor` colour conversions, Bayer demosaicing (bilinear and 3. OpenCV, versions 4.5.4 and 4.10: `cvtColor` colour conversions, Bayer demosaicing (bilinear and
edge-aware), https://docs.opencv.org. edge-aware), https://docs.opencv.org.
4. EMVA. *GenICam Pixel Format Naming Convention (PFNC)* and *GenICam Standard Features Naming 4. EMVA. *GenICam Pixel Format Naming Convention (PFNC)* and *GenICam Standard Features Naming
@ -1104,10 +892,6 @@ between neighbouring waypoints, which decides the real pace, remains to be measu
`tools/camera-bench/examples-*`. `tools/camera-bench/examples-*`.
- Figures: `plot_results.py`, `plot_common.py` (charts), `make_figures.py` (example crops); - Figures: `plot_results.py`, `plot_common.py` (charts), `make_figures.py` (example crops);
quality scores: `compare_examples.py`. quality scores: `compare_examples.py`.
- Dark-scene comparison (Section 3.9.2): `compare_8_vs_12.py`; figures and
[results.json](images/raw12/examples/staeffelsberg/results.json) in
`docs/images/raw12/examples/staeffelsberg/`. The two source frames (`BayerRG12p.tif`, 97 MB;
`RGB8.png`, 17 MB) are kept outside the repository, in `8bit_vs_12bit_test/` next to it.
## Appendix B. Processing times on each tower's own frame ## Appendix B. Processing times on each tower's own frame

View File

@ -4,7 +4,7 @@ This file gives an AI assistant the background needed to discuss the report
[raw12-pipeline-measurements.md](raw12-pipeline-measurements.md): what was asked, how the tests were [raw12-pipeline-measurements.md](raw12-pipeline-measurements.md): what was asked, how the tests were
run, what the numbers mean, what was decided, which points have been questioned, and what is still run, what the numbers mean, what was decided, which points have been questioned, and what is still
open. Read the report itself for the full tables and figures; this file explains around them. open. Read the report itself for the full tables and figures; this file explains around them.
State as of 2026-10-07, branch `raw12-capture` (dark-scene comparison added 2026-10-07). State as of 2026-10-05, branch `raw12-capture`.
## 1. The system in one paragraph ## 1. The system in one paragraph
@ -151,31 +151,9 @@ have 11–22 GB free, so images must leave the tower continuously.
backlit ridges under a bright sky at dawn or dusk; thin smoke over dark terrain is the relevant backlit ridges under a bright sky at dawn or dusk; thin smoke over dark terrain is the relevant
fire-detection case. At night with high gain, noise is amplified above the step again, so little fire-detection case. At night with high gain, noise is amplified above the step again, so little
benefit. Strong shadow lifting (≥ 3 stops) untested; 2 stops showed no difference. benefit. Strong shadow lifting (≥ 3 stops) untested; 2 stops showed no difference.
- The October scene had nothing darker than ≈ 700 DN (a dusk attempt on 121 needed 24 dB and was - None of the dark cases were measured (the darkest area in the scene was ≈ 700 DN; a dusk attempt
discarded). On 7 Oct the user took a dark, high-contrast pair at the Stäffelsberg tower (shaded on 121 needed 24 dB and was discarded). Proposed test: a dawn/dusk frame with dark terrain under a
forest under hazy sky, same 1620c model): `BayerRG12p` converted to colour by the camera viewer and bright sky, compared as in Figures 12 and 13.
saved as 16-bit TIFF (12-bit values << 4), and a separate RGB8 capture as PNG; files in
`../8bit_vs_12bit_test/` (outside the repo). Gain/exposure not recorded. Report Section 3.9.2,
script `tools/camera-bench/compare_8_vs_12.py`, results.json in
`docs/images/raw12/examples/staeffelsberg/`.
- Stäffelsberg findings: 33% of the frame < 300 DN; noise in the colour image (std around a local
mean, quietest 10% of 24 × 24 patches, calibrated) 5–15 DN below ≈ 1300 DN, i.e. below one 8-bit
step over ~70% of the frame. Rounding to 8 bits: +10–26% noise in the forest; dark crop 272 vs 25
green levels (camera RGB8 22); local-average error (Gaussian σ 4 px) only 0.6 DN; no visible
banding at +4 stops, 8-bit slightly grainier. Camera RGB8 1.4–1.8× noisier than the 12-bit colour
image (camera's own processing).
- Compression is where it shows: at d1.0 the 8-bit file has 3.7 DN local-average error in the dark
vs 1.6 DN for 12-bit (bright: 4.0 vs 6.0), visibly blocky shadows; today's RGB8 d0.8 e4 file
(0.92 MB) is equally blocky. The same 8-bit values stored in a 12-bit file give 1.8 DN and
0.94 MB: the effect is libjxl spending bits by declared bit depth, not the four extra bits. Sizes
on this scene at d1.0: 12-bit 0.92 MB, 8-bit 0.63 MB (+46%; equal in daylight). Implication: if
BayerRG8 were ever used, save its colour as a 12/16-bit JPEG XL.
- Same setting for both (user request, Table 17e, Figures 13g/13h): camera RGB8 and BayerRG12p
both at d0.8 e4, each scored against its own uncompressed frame (separate captures). As stored
(linear, near black) no visible difference; at +4 stops RGB8 is blocky, 12-bit keeps detail.
Local-average error dark/mid/bright: RGB8 2.2/2.3/2.8, 12-bit 1.2/1.9/4.4 DN; sizes 0.92 vs
1.22 MB (+33%).
- Not yet tested: real smoke; night at high gain; the tower's own de-Bayer on a dark frame.
- 12-bit vs 8-bit raw costs little in saving (same size, ~10% more time) but the camera runs at - 12-bit vs 8-bit raw costs little in saving (same size, ~10% more time) but the camera runs at
9.86 vs 14.75 fps, ~1 s slower settling in daylight. The main gain over today is frame rate, not 9.86 vs 14.75 fps, ~1 s slower settling in daylight. The main gain over today is frame rate, not
bit depth. bit depth.
@ -200,7 +178,7 @@ N workers, bounded queue; scheduler save gate pauses the scan when the queue is
Not done yet: exporting the new camera profiles on a tower (the 1620c is currently on tower 111 Not done yet: exporting the new camera profiles on a tower (the 1620c is currently on tower 111
and must be moved back to 121 first); field verification with real slews; checking that downstream and must be moved back to 121 first); field verification with real slews; checking that downstream
consumers decode 12-bit files. The branch is deployed on tower 111, consumers decode 12-bit files; the dawn/dusk 12-bit test. The branch is deployed on tower 111,
whose boot service still passes old `--init/--start` flags and needs `scripts/install-tower`. whose boot service still passes old `--init/--start` flags and needs `scripts/install-tower`.
## 10. Report structure (for orientation) ## 10. Report structure (for orientation)
@ -209,8 +187,7 @@ Abstract; Units and abbreviations; 1 Introduction (1.1 question, 1.2 pixel forma
mosaic vs raw planes, Figures 1a/1b, Table 1; 1.3 earlier work); 2 Materials and methods (2.1–2.12, mosaic vs raw planes, Figures 1a/1b, Table 1; 1.3 earlier work); 2 Materials and methods (2.1–2.12,
Tables 2–8, Figure 2); 3 Results (3.1 frame rate, 3.2 auto-exposure, 3.3 keep rules, 3.4 white Tables 2–8, Figure 2); 3 Results (3.1 frame rate, 3.2 auto-exposure, 3.3 keep rules, 3.4 white
balance, 3.5 processing per step, 3.6 effort and threads, 3.7 quality setting, 3.8 example images, balance, 3.5 processing per step, 3.6 effort and threads, 3.7 quality setting, 3.8 example images,
3.9 information content of 12-bit data (3.9.1 daylight, 3.9.2 Stäffelsberg dark scene, 3.9.3 3.9 information content of 12-bit data with Tables 17a/17b, 3.10 throughput, 3.11 heat,
summary; Figures 12, 13a–13h, Tables 17a–17e), 3.10 throughput, 3.11 heat,
3.12 storage; Figures 3–17, Tables 9–20); 4 Discussion (summary Table 21, pace Table 22, 3.12 storage; Figures 3–17, Tables 9–20); 4 Discussion (summary Table 21, pace Table 22,
limitations); 5 Conclusions and six recommendations; References [1]–[7]; Appendix A raw data and limitations); 5 Conclusions and six recommendations; References [1]–[7]; Appendix A raw data and
tools; Appendix B per-tower processing times. Figures live in `docs/images/raw12/`. tools; Appendix B per-tower processing times. Figures live in `docs/images/raw12/`.

View File

@ -214,7 +214,7 @@ config at all, because **none of those values trip the placeholder guard in
`DEV_0000000000`. `DEV_0000000000`.
```bash ```bash
ssh ggs@<tower-ip> 'grep -E "tower_name|zkms_server_ip|^id" ~/.config/fire_gimbal_control/config.ini' ssh ggs@<tower-ip> 'grep -E "tower_name|zkms_server_ip|^id" ~/projects/fwt_2a/software/config.ini'
``` ```
## 3. Host preconditions and build dependencies `[verified]` ## 3. Host preconditions and build dependencies `[verified]`
@ -671,7 +671,7 @@ a 32-byte buffer).
## 8. Camera profiles `[verified]` ## 8. Camera profiles `[verified]`
**Without a profile the program does not start** - `no camera profiles (*.xml) **Without a profile the program does not start** - `no camera profiles (*.xml)
in ~/.config/fire_gimbal_control/camera_profiles/rgb` is a startup error, by design. A fresh tower has in config/camera_profiles/rgb` is a startup error, by design. A fresh tower has
an empty profile directory, so this step is not optional. an empty profile directory, so this step is not optional.
The documented way to make one is the VimbaX Viewer, which is a GUI - awkward on The documented way to make one is the VimbaX Viewer, which is a GUI - awkward on

View File

@ -14,28 +14,10 @@ std::string expandUser(const std::string& path);
// Returns empty string if it cannot be determined. // Returns empty string if it cannot be determined.
std::string executableDir(); std::string executableDir();
// The folder that holds what belongs to ONE tower and is changed on it:
// config.ini, the scan grid, the camera profile XMLs
// ($XDG_CONFIG_HOME/fire_gimbal_control, else ~/.config/fire_gimbal_control).
// It is outside the deployed project folder, so a deploy or an update never
// overwrites or deletes anything in it. Empty if it cannot be determined.
std::string towerConfigDir();
// The ordered list of locations searched for a config file, given an optional // The ordered list of locations searched for a config file, given an optional
// explicit --config argument. Exposed so callers can report what was tried. // explicit --config argument. Exposed so callers can report what was tried.
//
// --config <path> explicit
// $FGC_CONFIG the environment
// <towerConfigDir()>/config.ini the normal answer on a tower
// ./config.ini the old place, and a development checkout
// <dir of the executable>/config.ini
std::vector<std::string> configSearchPaths(const std::string& cliArg = ""); std::vector<std::string> configSearchPaths(const std::string& cliArg = "");
// True when this is a tower (it has /etc/fire_gimbal_control/hardware.ini) and
// the config was found by searching, but outside towerConfigDir(): a tower whose
// files have not been moved yet (scripts/tower-config move).
bool configInOldPlace(const std::string& cliArg, const std::string& configPath);
// Resolve the config file path using configSearchPaths(): the first existing, // Resolve the config file path using configSearchPaths(): the first existing,
// readable file wins. Returns nullopt if none is found. // readable file wins. Returns nullopt if none is found.
std::optional<std::string> resolveConfigPath(const std::string& cliArg = ""); std::optional<std::string> resolveConfigPath(const std::string& cliArg = "");

View File

@ -120,8 +120,8 @@ int main(int argc, char* argv[]) {
if (!cfg_path) { if (!cfg_path) {
std::cerr << "No config.ini found. Searched:\n"; std::cerr << "No config.ini found. Searched:\n";
for (const auto& p : paths::configSearchPaths(cli_cfg)) std::cerr << " - " << p << "\n"; for (const auto& p : paths::configSearchPaths(cli_cfg)) std::cerr << " - " << p << "\n";
std::cerr << "Run scripts/tower-config seed to create one in " std::cerr << "Copy config/config.example.ini to one of these, pass --config, "
<< paths::towerConfigDir() << ", pass --config, or set $FGC_CONFIG.\n"; "or set $FGC_CONFIG.\n";
return 1; return 1;
} }
@ -198,10 +198,6 @@ int main(int argc, char* argv[]) {
} }
LOG_INFO << "Loaded config: " << *cfg_path; LOG_INFO << "Loaded config: " << *cfg_path;
if (paths::configInOldPlace(cli_cfg, *cfg_path))
LOG_WARN << "this tower's config.ini is still in the project folder, where a deploy "
"can reach it: run scripts/tower-config move to put it in "
<< paths::towerConfigDir();
// Only one instance may run: a second would fight the first over the motor's // Only one instance may run: a second would fight the first over the motor's
// serial port, the camera, and the exposure store. The lock is held for the // serial port, the camera, and the exposure store. The lock is held for the

View File

@ -20,10 +20,7 @@ anything the camera refused is reported rather than silently kept.
Usage, on the tower: Usage, on the tower:
GENICAM_GENTL64_PATH=/opt/VimbaX/cti ~/.venvs/vmbpy/bin/python \ GENICAM_GENTL64_PATH=/opt/VimbaX/cti ~/.venvs/vmbpy/bin/python \
scripts/export_camera_profile.py --profile standard scripts/export_camera_profile.py --profile standard --out config/camera_profiles/rgb
It writes into the tower's own folder, ~/.config/fire_gimbal_control/
camera_profiles/rgb, where a deploy cannot reach it; --out picks another.
... --dry-run report what would change, write nothing ... --dry-run report what would change, write nothing
... --camera DEV_xxx pick a camera explicitly ... --camera DEV_xxx pick a camera explicitly
@ -56,11 +53,6 @@ except ImportError:
# about half the time, with no lost frames measured on either computer. # about half the time, with no lost frames measured on either computer.
# Binning goes to 1 vertical-first (the horizontal minimum depends on it), and # Binning goes to 1 vertical-first (the horizontal minimum depends on it), and
# Width/Height ask for more than any sensor so the clamp lands on the maximum. # Width/Height ask for more than any sensor so the clamp lands on the maximum.
# The exposure the camera already holds caps its frame rate too, so a short
# fixed exposure is written before the rate, and auto-exposure is turned back
# on afterwards. Without it, tower 111's camera, left at 3.5 s by the old
# profile's 9 s auto limit, got a 'standard' profile clamped to 0.28 fps. A dry
# run writes nothing, so its frame rate still reflects the old exposure.
# The per-frame chunk data is what the keep rule reads; each item must be # The per-frame chunk data is what the keep rule reads; each item must be
# enabled before chunk mode itself, which locks them. # enabled before chunk mode itself, which locks them.
_RAW_COMMON = [ _RAW_COMMON = [
@ -74,8 +66,6 @@ _RAW_COMMON = [
("Height", 100000), ("Height", 100000),
("PixelFormat", "BayerRG12p"), ("PixelFormat", "BayerRG12p"),
("DeviceLinkThroughputLimit", 250000000), ("DeviceLinkThroughputLimit", 250000000),
("ExposureAuto", "Off"), # so the short exposure below sticks
("ExposureTime", 1000.0), # before the rate: see the note above
("AcquisitionFrameRateEnable", True), ("AcquisitionFrameRateEnable", True),
("AcquisitionFrameRate", 1000.0), # clamped to the camera's maximum ("AcquisitionFrameRate", 1000.0), # clamped to the camera's maximum
("ChunkModeActive", False), ("ChunkModeActive", False),
@ -163,9 +153,7 @@ def apply_feature(cam, name, value, dry_run):
def main(): def main():
ap = argparse.ArgumentParser() ap = argparse.ArgumentParser()
ap.add_argument("--profile", default="standard", choices=sorted(RECIPES)) ap.add_argument("--profile", default="standard", choices=sorted(RECIPES))
ap.add_argument("--out", default=os.path.join( ap.add_argument("--out", default="config/camera_profiles/rgb",
os.environ.get("XDG_CONFIG_HOME") or os.path.expanduser("~/.config"),
"fire_gimbal_control", "camera_profiles", "rgb"),
help="directory to write <profile>.xml into") help="directory to write <profile>.xml into")
ap.add_argument("--camera", default=None, help="camera id (default: the first real one)") ap.add_argument("--camera", default=None, help="camera id (default: the first real one)")
ap.add_argument("--dry-run", action="store_true") ap.add_argument("--dry-run", action="store_true")

View File

@ -152,8 +152,7 @@ cmd_start() {
# It is a *window* option, and a pane that dies immediately takes the window # It is a *window* option, and a pane that dies immediately takes the window
# with it before the option can be set — so create the session on a # with it before the option can be set — so create the session on a
# placeholder, set the option, then respawn the pane with the real command. # placeholder, set the option, then respawn the pane with the real command.
# -c: a tower not moved to ~/.config/fire_gimbal_control yet still has its # -c: the program resolves ./config.ini and the output dir from the cwd. The
# config.ini here, which the program finds as ./config.ini. The
# pane runs this script's hidden `_run`, so the Vimba environment is resolved # pane runs this script's hidden `_run`, so the Vimba environment is resolved
# inside the pane; run.sh finds the binary, one place only. # inside the pane; run.sh finds the binary, one place only.
tmux new-session -d -s "$session" -c "$repo" 'sleep 86400' tmux new-session -d -s "$session" -c "$repo" 'sleep 86400'

View File

@ -42,12 +42,7 @@ ETC="${INSTALL_TOWER_ETC:-/etc/fire_gimbal_control}"
HW="$ETC/hardware.ini" HW="$ETC/hardware.ini"
CRED="$ETC/credentials.env" CRED="$ETC/credentials.env"
INSTALL_LOG="$ETC/install.log" INSTALL_LOG="$ETC/install.log"
# config.ini lives in the tower's own folder, outside the project folder a deploy CFG="${INSTALL_TOWER_CONFIG:-$REPO/config.ini}"
# replaces (scripts/tower-config). A tower set up before that folder existed
# still has it in the project folder: found there, and moved by the plan below.
TOWER_CONFIG="$REPO/scripts/tower-config"
TOWER_DIR="$("$TOWER_CONFIG" dir)"
CFG="${INSTALL_TOWER_CONFIG:-$("$TOWER_CONFIG" where || echo "$TOWER_DIR/config.ini")}"
UNIT="${INSTALL_TOWER_UNIT:-/etc/systemd/system/fire-gimbal-control.service}" UNIT="${INSTALL_TOWER_UNIT:-/etc/systemd/system/fire-gimbal-control.service}"
LOGDIR="${XDG_DATA_HOME:-$HOME/.local/share}/fire_gimbal_control/logs" LOGDIR="${XDG_DATA_HOME:-$HOME/.local/share}/fire_gimbal_control/logs"
BIN="$REPO/build/fire_gimbal_control" BIN="$REPO/build/fire_gimbal_control"
@ -380,7 +375,7 @@ running firmware : ${S_FW_NOW:-unknown}
camera : ${S_CAM:-not found}${S_LPM:+ (LPM $(cat "$S_LPM") at $S_LPM)} camera : ${S_CAM:-not found}${S_LPM:+ (LPM $(cat "$S_LPM") at $S_LPM)}
usbfs_memory_mb : $S_USBFS usbfs_memory_mb : $S_USBFS
hardware.ini : $([[ -r $HW ]] && echo "present, $(hw_todos | grep -c . || true) TODO(s)" || echo absent) hardware.ini : $([[ -r $HW ]] && echo "present, $(hw_todos | grep -c . || true) TODO(s)" || echo absent)
config.ini : $([[ -r $CFG ]] && echo "$CFG, tower_name $(ini_get "$CFG" General tower_name 2>/dev/null || echo '?'), [Startup] home $(ini_get "$CFG" Startup home 2>/dev/null || echo unset), mode $(ini_get "$CFG" Startup mode 2>/dev/null || echo unset)" || echo absent) config.ini : $([[ -r $CFG ]] && echo "present, tower_name $(ini_get "$CFG" General tower_name 2>/dev/null || echo '?'), [Startup] home $(ini_get "$CFG" Startup home 2>/dev/null || echo unset), mode $(ini_get "$CFG" Startup mode 2>/dev/null || echo unset)" || echo absent)
app : $([[ $S_RUNNING -eq 1 ]] && echo "running (${S_ARGS:-no arguments})" || echo "not running") app : $([[ $S_RUNNING -eq 1 ]] && echo "running (${S_ARGS:-no arguments})" || echo "not running")
service : ${S_SVC_ENABLED:-absent} / ${S_SVC_ACTIVE:-inactive}, linger ${S_LINGER} service : ${S_SVC_ENABLED:-absent} / ${S_SVC_ACTIVE:-inactive}, linger ${S_LINGER}
PlatformIO : ${S_PIO:-not installed} PlatformIO : ${S_PIO:-not installed}
@ -456,9 +451,6 @@ verify() {
else else
check "config files load" 1 "fgc-config or config.ini missing" check "config files load" 1 "fgc-config or config.ini missing"
fi fi
check "tower files" "$([[ $(dirname "$CFG") == "$TOWER_DIR" ]] && echo 0 || echo 1)" \
"$([[ $(dirname "$CFG") == "$TOWER_DIR" ]] && echo "in $TOWER_DIR" ||
echo "config.ini is in the project folder, where a deploy reaches: scripts/tower-config move")"
local todo; todo="$(hw_todos | grep -c . || true)" local todo; todo="$(hw_todos | grep -c . || true)"
check "hardware.ini complete" "$([[ $todo -eq 0 ]] && echo 0 || echo 1)" "$todo TODO(s)" check "hardware.ini complete" "$([[ $todo -eq 0 ]] && echo 0 || echo 1)" "$todo TODO(s)"
local now; now="$(firmware_report)" local now; now="$(firmware_report)"
@ -700,16 +692,6 @@ for f in "${HW_FILLS[@]}"; do # a TODO answered now counts too
done done
declared_axes="$(ini_get "$HW_VIEW" Hardware axes 2>/dev/null | tr -d ' ' || echo yaw)" declared_axes="$(ini_get "$HW_VIEW" Hardware axes 2>/dev/null | tr -d ' ' || echo yaw)"
# The tower's own files: a tower set up before ~/.config/fire_gimbal_control
# existed keeps config.ini, its scan grid and its camera profiles in the project
# folder, where a deploy can overwrite or delete them. Moved first, so every step
# after works on the moved copy. Not when --config-like overrides point elsewhere.
if [[ -z ${INSTALL_TOWER_CONFIG:-} && -r $CFG && $(dirname "$CFG") != "$TOWER_DIR" ]]; then
plan "tower files" change "move config.ini, its scan grid and camera profiles to $TOWER_DIR"
else
plan "tower files" skip "$([[ -r $CFG ]] && echo "in $(dirname "$CFG")" || echo "new tower: created with config.ini")"
fi
# config.ini: what this tower DOES. # config.ini: what this tower DOES.
declare -a CFG_SETS=() declare -a CFG_SETS=()
CFG_SEED=0 CRED_WRITE=0 CFG_SEED=0 CRED_WRITE=0
@ -826,18 +808,7 @@ if [[ -n $S_PIO ]] && "$S_PIO" --version >/dev/null 2>&1; then plan platformio s
else plan platformio change "install PlatformIO Core into ~/.platformio"; fi else plan platformio change "install PlatformIO Core into ~/.platformio"; fi
# Camera profiles: the program will not start a real camera without them. # Camera profiles: the program will not start a real camera without them.
# Where the program looks for them: [Camera.rgb] profile_dir, relative to the PROFDIR="$REPO/config/camera_profiles/rgb"
# folder config.ini is in - the tower folder for a new tower.
prof_dir() {
local pd; pd="$(ini_get "$CFG" Camera.rgb profile_dir 2>/dev/null || true)"
case "$pd" in
"") echo "$TOWER_DIR/camera_profiles/rgb" ;;
/*) echo "$pd" ;;
\~*) echo "$HOME${pd#\~}" ;;
*) echo "$(dirname "$CFG")/$pd" ;;
esac
}
PROFDIR="$(prof_dir)"
cams="$(ini_get "$HW_VIEW" Hardware cameras 2>/dev/null || echo none)" cams="$(ini_get "$HW_VIEW" Hardware cameras 2>/dev/null || echo none)"
if [[ $cams == none || -z $cams ]]; then plan profiles skip "no camera declared" if [[ $cams == none || -z $cams ]]; then plan profiles skip "no camera declared"
else else
@ -1036,21 +1007,10 @@ if [[ ${DO[credentials]:-} == create ]]; then
say "wrote $CRED"; done_step credentials say "wrote $CRED"; done_step credentials
fi fi
if [[ ${DO[tower files]} == change ]]; then
hr "tower files"
stop_app
"$TOWER_CONFIG" move
CFG="$TOWER_DIR/config.ini"
done_step "tower files"
fi
if [[ ${DO[config.ini]} != skip ]]; then if [[ ${DO[config.ini]} != skip ]]; then
hr config.ini hr config.ini
args=(); for s in "${CFG_SETS[@]}"; do args+=(--set "$s"); done args=(); for s in "${CFG_SETS[@]}"; do args+=(--set "$s"); done
if [[ $CFG_SEED -eq 1 ]]; then if [[ $CFG_SEED -eq 1 ]]; then
# The folder, the example scan grid and an empty camera_profiles/rgb;
# the config.ini itself is written next, from the answers.
[[ -n ${INSTALL_TOWER_CONFIG:-} ]] || "$TOWER_CONFIG" seed --no-config
"$FGC_CONFIG" seed --example "$REPO/config/config.example.ini" --out "$CFG" --hardware "$HW" \ "$FGC_CONFIG" seed --example "$REPO/config/config.example.ini" --out "$CFG" --hardware "$HW" \
"${args[@]}" \ "${args[@]}" \
--drop "Network.mqtt_user=comes from $CRED (FGC_MQTT_USER)" \ --drop "Network.mqtt_user=comes from $CRED (FGC_MQTT_USER)" \
@ -1081,11 +1041,9 @@ if [[ ${DO[profiles]} == change ]]; then
if { [[ -x $HOME/.venvs/vmbpy/bin/python ]] || { if { [[ -x $HOME/.venvs/vmbpy/bin/python ]] || {
python3 -m venv "$HOME/.venvs/vmbpy" && python3 -m venv "$HOME/.venvs/vmbpy" &&
"$HOME/.venvs/vmbpy/bin/pip" install -q /opt/VimbaX/api/python/vmbpy-*.whl; }; }; then "$HOME/.venvs/vmbpy/bin/pip" install -q /opt/VimbaX/api/python/vmbpy-*.whl; }; }; then
PROFDIR="$(prof_dir)" # the config may have moved since the plan was made
mkdir -p "$PROFDIR"
for prof in "${PROF_MISSING[@]}"; do for prof in "${PROF_MISSING[@]}"; do
( cd "$REPO" && GENICAM_GENTL64_PATH=/opt/VimbaX/cti "$HOME/.venvs/vmbpy/bin/python" \ ( cd "$REPO" && GENICAM_GENTL64_PATH=/opt/VimbaX/cti "$HOME/.venvs/vmbpy/bin/python" \
scripts/export_camera_profile.py --profile "$prof" --out "$PROFDIR" ) || scripts/export_camera_profile.py --profile "$prof" ) ||
warn "could not export the $prof profile; export it later (docs/tower-installation.md section 8)" warn "could not export the $prof profile; export it later (docs/tower-installation.md section 8)"
done done
done_step profiles done_step profiles

View File

@ -1,176 +0,0 @@
#!/usr/bin/env bash
#
# tower-config - the folder that holds what belongs to ONE tower: its config.ini,
# its scan grid and its camera profile XMLs. Run it ON the tower, as the user the
# program runs as (ggs).
#
# scripts/tower-config dir print the folder
# (~/.config/fire_gimbal_control)
# scripts/tower-config where print the config.ini the program will
# load; exit 1 if there is none
# scripts/tower-config seed [--no-config]
# a new tower: create the folder with a
# config.ini from the example, the example
# scan grid and an empty camera_profiles/rgb.
# Never overwrites; --no-config leaves the
# config.ini out (install-tower writes its own)
# scripts/tower-config move [--dry-run] one time, on a tower set up before this
# folder existed: move config.ini, the scan
# grid and the camera profiles out of the
# project folder and into it
#
# Why a folder of its own: the project folder is what a deploy (rsync --delete)
# and an update replace. Anything the tower changes - its calibration in
# config.ini, the scan grid chosen for its view, the profiles exported from its
# camera - must live where neither can reach it. hardware.ini, which describes
# the assembly and is never changed, stays in /etc/fire_gimbal_control/.
#
# `move` copies config.ini into the folder and rewrites the paths in it that
# pointed into the project folder (grid_file, each camera's profile_dir), with
# fgc-config, so the result is checked to load before it is written. The old
# config.ini is then renamed config.ini.moved-<timestamp>, so nobody edits a copy
# the program no longer reads. The old scan grid and profiles stay where they
# were. It refuses while the program is running: the program writes config.ini
# itself, and would write to the old copy.
#
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO="$(cd "$here/.." && pwd)"
DIR="${XDG_CONFIG_HOME:-$HOME/.config}/fire_gimbal_control"
OLD="$REPO/config.ini"
HW=/etc/fire_gimbal_control/hardware.ini
FGC_CONFIG_BIN="$REPO/build/fgc-config"
say() { printf '>> %s\n' "$*"; }
die() { printf 'tower-config: %s\n' "$*" >&2; exit 1; }
usage() { awk 'NR>2 && /^#/ {sub(/^# ?/, ""); print; next} NR>2 {exit}' "${BASH_SOURCE[0]}"; }
# The same order the program searches, minus --config (src/core/Paths.cpp).
cmd_where() {
local c
for c in "${FGC_CONFIG:-}" "$DIR/config.ini" "$OLD"; do
[[ -n $c && -f $c ]] && { echo "$c"; return 0; }
done
return 1
}
cmd_seed() {
local with_config=1
[[ ${1:-} == --no-config ]] && with_config=0
if [[ $with_config -eq 1 ]]; then
local have
# An existing tower, wherever its config.ini is, is left alone: one not
# moved yet must not get a second scan grid that `move` would trip over.
if have="$(cmd_where)"; then
say "keeping the existing $have"
return 0
fi
mkdir -p "$DIR"
cp "$REPO/config/config.example.ini" "$DIR/config.ini"
say "created $DIR/config.ini from the example - edit it for this tower"
fi
mkdir -p "$DIR/camera_profiles/rgb"
if [[ -e $DIR/scan.csv ]]; then
say "keeping the existing $DIR/scan.csv"
else
cp "$REPO/config/scan.csv" "$DIR/scan.csv"
say "created $DIR/scan.csv from the example grid - replace it with this tower's"
fi
}
# A path from config.ini that pointed into the project folder, as it reads from
# the tower folder: the same path without its leading "config/".
moved_path() { local p="$1"; echo "${p#config/}"; }
is_relative() { [[ -n $1 && $1 != /* && $1 != \~* && $1 != \$* ]]; }
cmd_move() {
local dry=0
[[ ${1:-} == --dry-run ]] && dry=1
if [[ -f $DIR/config.ini ]]; then
if [[ -f $OLD ]]; then
die "$DIR/config.ini already exists, and so does $OLD.
The program reads $DIR/config.ini; the copy in the project folder is unused.
Compare the two, keep what is right in $DIR/config.ini, then rename the other."
fi
say "already moved: the program reads $DIR/config.ini"
return 0
fi
[[ -f $OLD ]] || die "there is no $OLD to move (a new tower: scripts/tower-config seed)"
[[ -x $FGC_CONFIG_BIN ]] || die "no $FGC_CONFIG_BIN - build first (./deploy.sh does)"
if [[ -f $REPO/hardware.ini && ! -f $HW ]]; then
die "this tower's hardware.ini is in the project folder ($REPO/hardware.ini).
Put it in /etc first, where it belongs:
sudo install -D -m 0644 $REPO/hardware.ini $HW"
fi
if [[ $dry -eq 0 ]] && "$here/fgc" status >/dev/null 2>&1; then
die "the program is running and writes config.ini itself. Stop it first:
scripts/fgc stop"
fi
# What has to come along, and what config.ini will say afterwards.
local -a sets=() copies=()
local grid; grid="$("$FGC_CONFIG_BIN" get "$OLD" Scan.grid_file 2>/dev/null || true)"
if is_relative "$grid"; then
[[ -f $REPO/$grid ]] || die "[Scan] grid_file = $grid, but $REPO/$grid does not exist. Fix config.ini first."
copies+=("$REPO/$grid|$DIR/$(moved_path "$grid")")
[[ $(moved_path "$grid") != "$grid" ]] && sets+=("Scan.grid_file=$(moved_path "$grid")")
fi
local cam
while read -r cam; do
[[ -n $cam ]] || continue
local pd; pd="$("$FGC_CONFIG_BIN" get "$OLD" "Camera.$cam.profile_dir" 2>/dev/null || true)"
is_relative "$pd" || continue
local f
for f in "$REPO/$pd"/*.xml "$REPO/$pd"/*.txt; do
[[ -f $f ]] && copies+=("$f|$DIR/$(moved_path "$pd")/$(basename "$f")")
done
[[ $(moved_path "$pd") != "$pd" ]] && sets+=("Camera.$cam.profile_dir=$(moved_path "$pd")")
done < <(sed -nE 's/^[[:space:]]*\[Camera\.([^]]+)\][[:space:]]*$/\1/p' "$OLD")
say "moving this tower's files to $DIR"
echo " $OLD -> $DIR/config.ini"
local c
for c in ${copies[@]+"${copies[@]}"}; do echo " ${c%%|*} -> ${c#*|}"; done
for c in ${sets[@]+"${sets[@]}"}; do echo " config.ini: set ${c%%=*} = ${c#*=}"; done
echo " $OLD renamed to config.ini.moved-<timestamp>"
[[ $dry -eq 1 ]] && { say "dry run: nothing changed"; return 0; }
for c in ${copies[@]+"${copies[@]}"}; do
local src="${c%%|*}" dst="${c#*|}"
if [[ -e $dst ]] && ! cmp -s "$src" "$dst"; then
die "$dst already exists and differs from $src. Nothing was moved; sort that out first."
fi
done
mkdir -p "$DIR"
for c in ${copies[@]+"${copies[@]}"}; do
mkdir -p "$(dirname "${c#*|}")"
cp -p "${c%%|*}" "${c#*|}"
done
cp -p "$OLD" "$DIR/config.ini"
if (( ${#sets[@]} )); then
local -a args=() hw_args=()
for c in "${sets[@]}"; do args+=(--set "$c"); done
[[ -f $HW ]] && hw_args=(--hardware "$HW")
if ! "$FGC_CONFIG_BIN" set "$DIR/config.ini" --config "$DIR/config.ini" ${hw_args[@]+"${hw_args[@]}"} \
"${args[@]}" --provenance "$(date -u +%Y-%m-%dT%H:%MZ), tower-config move"; then
rm -f "$DIR/config.ini"
die "the moved config.ini would not load; nothing was changed in $REPO"
fi
fi
local moved; moved="config.ini.moved-$(date +%Y%m%d-%H%M%S)"
mv "$OLD" "$REPO/$moved"
say "done. The program now reads $DIR/config.ini"
echo " the old copy is $REPO/$moved; the old scan grid and profiles are left where they were"
}
sub="${1:-}"; shift || true
case "$sub" in
dir) echo "$DIR" ;;
where) cmd_where ;;
seed) cmd_seed "$@" ;;
move) cmd_move "$@" ;;
""|-h|--help|help) usage; [[ -z $sub ]] && exit 1 || exit 0 ;;
*) die "unknown command: $sub (try 'tower-config --help')" ;;
esac

View File

@ -84,18 +84,8 @@ constexpr long kEncoderQuantumCounts = 13;
// (StopContinuousImageAcquisition + Close). Without this, a Ctrl-C / kill / systemd // (StopContinuousImageAcquisition + Close). Without this, a Ctrl-C / kill / systemd
// stop skips teardown and leaves the camera acquiring - which wedges it so the NEXT // stop skips teardown and leaves the camera acquiring - which wedges it so the NEXT
// run gets no frames. Only an atomic flag is set here (async-signal-safe). // run gets no frames. Only an atomic flag is set here (async-signal-safe).
//
// Installed as the very first thing run() does, so a stop that arrives while the
// camera opens or the gimbal homes is not the default action - which kills the
// process on the spot and wedges the camera. A second signal of the same kind
// goes back to that default: Ctrl-C twice still gets out of a start that hangs.
std::atomic<bool> g_shutdown_requested{false}; std::atomic<bool> g_shutdown_requested{false};
void handleShutdownSignal(int sig) { void handleShutdownSignal(int) { g_shutdown_requested.store(true); }
if (g_shutdown_requested.exchange(true)) {
std::signal(sig, SIG_DFL);
std::raise(sig);
}
}
long long nowEpochMs() { long long nowEpochMs() {
return std::chrono::duration_cast<std::chrono::milliseconds>( return std::chrono::duration_cast<std::chrono::milliseconds>(
@ -1205,12 +1195,6 @@ struct Application::Impl {
} }
} }
// A stop that came in while the program was starting up: do not begin a
// home that would only be cancelled.
if (shutdownRequested()) {
LOG_INFO << "stop requested: not homing";
return;
}
motor->sendCommand("HOME"); // homes all axes; firmware runs it non-blocking motor->sendCommand("HOME"); // homes all axes; firmware runs it non-blocking
auto allReady = [](const MotorTelemetry& t) { auto allReady = [](const MotorTelemetry& t) {
@ -2510,28 +2494,12 @@ struct Application::Impl {
Command c = parseCommand(line); Command c = parseCommand(line);
if (c.verb == "cancel" || (c.verb == "mode" && c.device == "manual")) if (c.verb == "cancel" || (c.verb == "mode" && c.device == "manual"))
cancel_procedure_ = true; cancel_procedure_ = true;
else if (c.verb == "exit")
running = false; // not an interfering command: a stop
else if (!c.empty()) else if (!c.empty())
LOG_WARN << "ignored '" << c.verb << "': homing in progress (Esc to cancel)"; LOG_WARN << "ignored '" << c.verb << "': homing in progress (Esc to cancel)";
local.pop(); local.pop();
} }
// A stop cancels homing, as Esc does, rather than waiting for it to finish:
// a full home takes minutes on some towers - longer than `fgc stop` (120 s)
// and systemd (150 s) wait before giving up or killing the program outright,
// which wedges the camera. Dropping the stop instead, as this once did, left
// the program running with its dashboard already closed.
if (shutdownRequested() && !cancel_procedure_) {
LOG_INFO << "stop requested: cancelling homing to shut down";
cancel_procedure_ = true;
}
} }
// A stop asked for in any of the ways there are: the `exit` command - which is
// also what the dashboard sends when it closes, on Ctrl-C and SIGTERM too - or
// SIGINT/SIGTERM caught by handleShutdownSignal() when there is no dashboard.
bool shutdownRequested() const { return !running || g_shutdown_requested.load(); }
// ---- `test` command ---- // ---- `test` command ----
static std::string lowerStr(std::string s) { static std::string lowerStr(std::string s) {
for (char& ch : s) ch = static_cast<char>(std::tolower((unsigned char)ch)); for (char& ch : s) ch = static_cast<char>(std::tolower((unsigned char)ch));
@ -3009,14 +2977,6 @@ struct Application::Impl {
int run() { int run() {
start_time = std::chrono::steady_clock::now(); start_time = std::chrono::steady_clock::now();
// Clean up on Ctrl-C / kill / systemd stop, not just on "exit"/Ctrl-D - so the
// camera is always stopped and closed (an unclean exit wedges it for next time).
// First, before anything slow: homing alone takes minutes on some towers.
// The TUI's own handler takes over while the dashboard runs (it turns the
// signal into "exit") and puts this one back when the dashboard closes.
std::signal(SIGINT, handleShutdownSignal);
std::signal(SIGTERM, handleShutdownSignal);
// Log level: config first, CLI overrides. // Log level: config first, CLI overrides.
if (!cfg.logging.level.empty() && !Logger::setLevelFromString(cfg.logging.level)) if (!cfg.logging.level.empty() && !Logger::setLevelFromString(cfg.logging.level))
LOG_WARN << "unknown Logging.level '" << cfg.logging.level << "', keeping default"; LOG_WARN << "unknown Logging.level '" << cfg.logging.level << "', keeping default";
@ -3202,8 +3162,12 @@ struct Application::Impl {
sendAnchorConfig(); sendAnchorConfig();
applyHomedTravel(); applyHomedTravel();
} }
// A stop that came in during homing: do not start sweeping on the way out. if (start_in != Mode::Manual) setMode(start_in);
if (start_in != Mode::Manual && !shutdownRequested()) setMode(start_in);
// Clean up on Ctrl-C / kill / systemd stop, not just on "exit"/Ctrl-D - so the
// camera is always stopped and closed (an unclean exit wedges it for next time).
std::signal(SIGINT, handleShutdownSignal);
std::signal(SIGTERM, handleShutdownSignal);
LOG_INFO << "Entering control loop (type 'exit' to quit, or Ctrl-C)"; LOG_INFO << "Entering control loop (type 'exit' to quit, or Ctrl-C)";
oneshot_since_ = std::chrono::steady_clock::now(); oneshot_since_ = std::chrono::steady_clock::now();

View File

@ -68,12 +68,6 @@ std::string executableDir() {
return self.parent_path().string(); return self.parent_path().string();
} }
std::string towerConfigDir() {
std::string xdg = envOr("XDG_CONFIG_HOME", expandUser("~/.config"));
if (xdg.empty()) return {};
return (fs::path(xdg) / "fire_gimbal_control").string();
}
std::vector<std::string> configSearchPaths(const std::string& cliArg) { std::vector<std::string> configSearchPaths(const std::string& cliArg) {
std::vector<std::string> paths; std::vector<std::string> paths;
if (!cliArg.empty()) paths.push_back(expandUser(cliArg)); if (!cliArg.empty()) paths.push_back(expandUser(cliArg));
@ -81,28 +75,16 @@ std::vector<std::string> configSearchPaths(const std::string& cliArg) {
if (const char* env = std::getenv("FGC_CONFIG"); env && *env) if (const char* env = std::getenv("FGC_CONFIG"); env && *env)
paths.push_back(expandUser(env)); paths.push_back(expandUser(env));
if (std::string dir = towerConfigDir(); !dir.empty())
paths.push_back((fs::path(dir) / "config.ini").string());
// The old places: a tower whose files have not been moved yet, and a
// development checkout with its own config.ini.
paths.push_back("config.ini"); // current working directory paths.push_back("config.ini"); // current working directory
if (std::string dir = executableDir(); !dir.empty()) if (std::string dir = executableDir(); !dir.empty())
paths.push_back((fs::path(dir) / "config.ini").string()); paths.push_back((fs::path(dir) / "config.ini").string());
return paths; std::string xdg = envOr("XDG_CONFIG_HOME", expandUser("~/.config"));
} if (!xdg.empty())
paths.push_back((fs::path(xdg) / "fire_gimbal_control" / "config.ini").string());
bool configInOldPlace(const std::string& cliArg, const std::string& configPath) { return paths;
if (!cliArg.empty()) return false;
if (const char* env = std::getenv("FGC_CONFIG"); env && *env) return false;
std::error_code ec;
if (!fs::is_regular_file("/etc/fire_gimbal_control/hardware.ini", ec)) return false;
const std::string dir = towerConfigDir();
if (dir.empty()) return false;
return fs::absolute(fs::path(configPath), ec).parent_path().lexically_normal() !=
fs::path(dir).lexically_normal();
} }
std::optional<std::string> resolveConfigPath(const std::string& cliArg) { std::optional<std::string> resolveConfigPath(const std::string& cliArg) {

View File

@ -29,29 +29,6 @@ TEST_CASE("configSearchPaths honours the CLI argument first") {
CHECK(p.front() == "/explicit/config.ini"); CHECK(p.front() == "/explicit/config.ini");
} }
TEST_CASE("the tower folder is searched before the old places") {
unsetenv("FGC_CONFIG");
setenv("XDG_CONFIG_HOME", "/home/tester/.config", 1);
auto p = paths::configSearchPaths("");
unsetenv("XDG_CONFIG_HOME");
// A copy left in the project folder - where a deploy can reach it - must
// never win over the tower's own.
REQUIRE(p.size() >= 2);
CHECK(p[0] == "/home/tester/.config/fire_gimbal_control/config.ini");
CHECK(p[1] == "config.ini");
}
TEST_CASE("towerConfigDir falls back to ~/.config") {
unsetenv("XDG_CONFIG_HOME");
setenv("HOME", "/home/tester", 1);
CHECK(paths::towerConfigDir() == "/home/tester/.config/fire_gimbal_control");
}
TEST_CASE("an explicit config is never reported as being in the old place") {
CHECK_FALSE(paths::configInOldPlace("/somewhere/config.ini", "/somewhere/config.ini"));
}
TEST_CASE("hardwareSearchPaths looks in the documented order") { TEST_CASE("hardwareSearchPaths looks in the documented order") {
unsetenv("FGC_HARDWARE"); unsetenv("FGC_HARDWARE");
auto p = paths::hardwareSearchPaths("", "/opt/fgc/config.ini"); auto p = paths::hardwareSearchPaths("", "/opt/fgc/config.ini");

View File

@ -22,7 +22,6 @@ Not part of the build: CMake only compiles the tools it names.
| `run_common_frame.sh` | the processing tests on one shared frame (`common-frame/`, git-ignored, rebuilt from 121's lossless 12-bit example), so towers compare exactly (~25–40 min) | | `run_common_frame.sh` | the processing tests on one shared frame (`common-frame/`, git-ignored, rebuilt from 121's lossless 12-bit example), so towers compare exactly (~25–40 min) |
| `plot_common.py` | the shared-frame tables and charts (per-step time, size per quality setting, saves per second); laptop venv | | `plot_common.py` | the shared-frame tables and charts (per-step time, size per quality setting, saves per second); laptop venv |
| `compare_examples.py` | scores every 12-bit example against the lossless 12-bit colour image (PSNR, SSIM); laptop venv | | `compare_examples.py` | scores every 12-bit example against the lossless 12-bit colour image (PSNR, SSIM); laptop venv |
| `compare_8_vs_12.py` | 8-bit against 12-bit on a dark scene: noise, levels, JPEG XL error per brightness region, crop figures (2026-10-07); laptop venv |
| `make_figures.py` | example-image figures (crops, difference maps, 8 vs 12-bit shadow) from the example `.jxl` files; laptop venv | | `make_figures.py` | example-image figures (crops, difference maps, 8 vs 12-bit shadow) from the example `.jxl` files; laptop venv |
| `plot_results.py` | every chart and table of `docs/raw12-pipeline-measurements.md` from the results folders; laptop venv | | `plot_results.py` | every chart and table of `docs/raw12-pipeline-measurements.md` from the results folders; laptop venv |
| `results-*/` | raw results of each run, one folder per tower and day | | `results-*/` | raw results of each run, one folder per tower and day |

View File

@ -1,264 +0,0 @@
#!/usr/bin/env python3
"""compare_8_vs_12.py - 8-bit against 12-bit on a dark, high-contrast scene.
~/.venvs/fgc-plots/bin/python compare_8_vs_12.py <BayerRG12p.tif> <RGB8.png> <out_dir> \
[<october lossless c12 .jxl> <label> ...]
Inputs: the camera's BayerRG12p frame converted to colour by the viewer and saved as 16-bit TIFF
(12-bit values in the top bits), and the camera's own RGB8 frame of the same view taken seconds
later (another capture: trees move, so no pixel-by-pixel comparison with it). The 8-bit version of
the same frame is the 12-bit value >> 4 (what BayerRG8 would deliver). Optional extra lossless
12-bit colour files from earlier campaigns get the same noise measurement, for comparison.
Writes to <out_dir>:
overview.jpg the frame upright, display gamma, crop boxes
noise_vs_signal.png noise against brightness, every image measured the same way
dark_crop.png dark crop, camera RGB8 / 8-bit / 12-bit, +2 and +4 stops
dark_compressed.png dark crop after JPEG XL, +4 stops: today's camera RGB8 file (d0.8 e4),
and at d1.0 the 8-bit file, the 8-bit values in a 12-bit file, 12-bit
mid_crop.png mid-tone crop, 8-bit / 12-bit, contrast x4 around its mean (control)
same_setting.png dark crop, camera RGB8 and 12-bit, uncompressed and both at d0.8 e4,
shown x1, +2 and +4 stops
same_setting_zoom.png the middle 200 x 200 pixels of the two d0.8 e4 crops at +4 stops, 2x
results.json noise, levels, sizes and errors per brightness region
"""
import json
import os
import sys
import cv2
import imagecodecs
import matplotlib
import matplotlib.ticker
import numpy as np
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
tif, png, out = sys.argv[1:4]
extra = list(zip(sys.argv[4::2], sys.argv[5::2]))
os.makedirs(out, exist_ok=True)
c12 = cv2.imread(tif, cv2.IMREAD_UNCHANGED)[:, :, ::-1] >> 4 # 0-4095, uint16
cam8 = cv2.imread(png, cv2.IMREAD_UNCHANGED)[:, :, ::-1].copy() # 0-255, uint8
c8 = (c12 >> 4).astype(np.uint8) # same frame, top 8 bits
c8in12 = c8.astype(np.uint16) * 16 + 8 # the same 8-bit values, declared as a 12-bit file
# (integers only: + 8 rather than + 7.5; the 0.5 is taken off again after decoding)
H, W = c12.shape[:2]
def to12(a):
"""Any decoded image on the 12-bit scale; 8-bit v stands for the middle of 16v ... 16v + 15."""
return a.astype(np.float32) * 16 + 7.5 if a.dtype == np.uint8 else a.astype(np.float32)
ref = c12.astype(np.float32)
lum_s = cv2.GaussianBlur(ref[..., 1].astype(np.float32), (0, 0), 8)
REGIONS = {"dark (< 300 DN)": lum_s < 300,
"mid (300-1000 DN)": (lum_s >= 300) & (lum_s < 1000),
"bright (>= 1000 DN)": lum_s >= 1000}
# --- noise against brightness -------------------------------------------------------------
P = 24
EDGES = np.array([100, 130, 160, 200, 250, 300, 400, 500, 700, 1000, 1400, 2000, 2400, 2800, 3300])
def noise_floor(g):
"""Per-pixel noise of a colour image's green channel, on the 12-bit scale.
Standard deviation of the residual after a Gaussian (sigma 2 px) local mean, per 24 x 24 patch;
patches grouped by brightness and the quietest 10% of each group averaged (texture only adds).
Plain standard deviation, not a median-based one, so it also works on 8-bit steps. Neighbouring
pixels of a de-Bayered image are correlated, which this measure tolerates better than
neighbour differences.
"""
g = g.astype(np.float32)
r = g - cv2.GaussianBlur(g, (0, 0), 2.0)
h, w = g.shape[0] // P * P, g.shape[1] // P * P
sd = r[:h, :w].reshape(h // P, P, w // P, P).std(axis=(1, 3)).ravel()
mn = g[:h, :w].reshape(h // P, P, w // P, P).mean(axis=(1, 3)).ravel()
rows = []
for a, b in zip(EDGES[:-1], EDGES[1:]):
m = (mn >= a) & (mn < b)
if m.sum() < 50:
continue
rows.append((float(np.median(mn[m])), float(np.sort(sd[m])[:max(1, m.sum() // 10)].mean())))
return rows
# The residual of white noise under this high-pass is a little below its true sigma: calibrate.
wn = np.random.default_rng(0).normal(0, 10, (1200, 1200)) + 1000
CAL = 10 / noise_floor(wn)[0][1]
curves = {"12-bit (this frame)": [(s, n * CAL) for s, n in noise_floor(ref[..., 1])],
"8-bit, same frame (12-bit >> 4)": [(s, n * CAL) for s, n in noise_floor(to12(c8)[..., 1])],
"camera RGB8 (other capture)": [(s, n * CAL) for s, n in noise_floor(to12(cam8)[..., 1])]}
for path, label in extra:
a = imagecodecs.jpegxl_decode(open(path, "rb").read()).astype(np.float32)
a = a / 16 if a.max() > 4095.5 else a
curves[label] = [(s, n * CAL) for s, n in noise_floor(a[..., 1])]
fig, ax = plt.subplots(figsize=(6.6, 3.6))
style = {"12-bit (this frame)": ("tab:blue", "-", "o"),
"8-bit, same frame (12-bit >> 4)": ("tab:orange", "-", "s"),
"camera RGB8 (other capture)": ("tab:purple", "-", "^")}
other = iter([("tab:green", "--", "."), ("tab:red", "--", ".")])
for label, rows in curves.items():
c, ls, mk = style.get(label) or next(other)
ax.plot([r[0] for r in rows], [r[1] for r in rows], color=c, ls=ls, marker=mk, ms=4, label=label)
ax.axhline(16, color="k", ls="--", lw=1, label="one 8-bit step (16)")
ax.axhline(16 / np.sqrt(12), color="gray", ls=":", lw=1, label="8-bit rounding error (4.6)")
ax.set_xscale("log"); ax.set_yscale("log")
ticks = matplotlib.ticker
ax.xaxis.set_major_locator(ticks.FixedLocator([150, 200, 300, 500, 700, 1000, 1500, 2000, 3000]))
ax.yaxis.set_major_locator(ticks.FixedLocator([5, 7, 10, 15, 20, 30, 50, 70]))
for axis in (ax.xaxis, ax.yaxis):
axis.set_major_formatter(ticks.ScalarFormatter()); axis.set_minor_formatter(ticks.NullFormatter())
ax.set_xlabel("signal, 12-bit units (green)"); ax.set_ylabel("noise σ, 12-bit units")
ax.set_title("Noise vs brightness in the colour image (quietest 10% of 24×24 patches)", fontsize=10)
ax.grid(True, which="both", alpha=0.3)
ax.legend(fontsize=7, loc="upper left")
fig.tight_layout(); fig.savefig(os.path.join(out, "noise_vs_signal.png"), dpi=100); plt.close(fig)
# --- JPEG XL at the tower's settings (effort 3), 12-bit and 8-bit of the same frame ----------
DIST = [0, 0.3, 0.5, 1.0]
CS = 400
dark_xy = (40, 360) # shaded forest, about 200 DN
mid_xy = (3800, 1300) # distant valley in haze, about 1400 DN
def crop(a, xy):
x, y = xy
return a[y:y + CS, x:x + CS]
# Errors against the 12-bit frame, per region. Per pixel: everything, including discarded noise.
# Averaged: after a Gaussian average (sigma 4 px, about 200 pixels) of both images - the error in
# local brightness that remains once noise is averaged away, i.e. lost scene information such as a
# faint, even brightening.
ref_avg = cv2.GaussianBlur(ref, (0, 0), 4)
size, dark_dec, errors = {}, {}, []
SOURCES = (("12-bit", c12, 12), ("8-bit", c8, 8), ("8-bit values, 12-bit file", c8in12, 12),
("camera RGB8", cam8, 8))
for name, img, bps in SOURCES:
for d in DIST:
kw = dict(lossless=True) if d == 0 else dict(distance=d)
data = imagecodecs.jpegxl_encode(np.ascontiguousarray(img), effort=3, bitspersample=bps, **kw)
size[(name, d)] = len(data) / 1e6
print(f"{name} d{d}: {len(data) / 1e6:.2f} MB", flush=True)
if name == "camera RGB8": # another capture: sizes only
continue
a = to12(imagecodecs.jpegxl_decode(data)).astype(np.float32)
if img is c8in12:
a -= 0.5
if name == "12-bit" and d == 0:
assert np.array_equal(a, ref), "12-bit lossless must be exact"
dark_dec[(name, d)] = crop(a, dark_xy).copy()
e_px = a - ref
e_av = cv2.GaussianBlur(a, (0, 0), 4) - ref_avg
row = dict(source=name, distance=d, size_mb=round(size[(name, d)], 2))
for rn, m in REGIONS.items():
row[rn] = dict(rms_px=round(float(np.sqrt((e_px[m] ** 2).mean())), 2),
rms_avg=round(float(np.sqrt((e_av[m] ** 2).mean())), 2))
errors.append(row)
del a, e_px, e_av
# Both formats with today's setting (distance 0.8, effort 4). The two frames are separate captures,
# so each is scored against its own uncompressed original.
same_setting, same_crops = [], {}
for name, img, bps in (("camera RGB8", cam8, 8), ("BayerRG12p", c12, 12)):
data = imagecodecs.jpegxl_encode(np.ascontiguousarray(img), effort=4, distance=0.8, bitspersample=bps)
own = to12(img)
a = to12(imagecodecs.jpegxl_decode(data))
same_crops[name] = (crop(own, dark_xy).copy(), crop(a, dark_xy).copy())
e_px = a - own
e_av = cv2.GaussianBlur(a, (0, 0), 4) - cv2.GaussianBlur(own, (0, 0), 4)
row = dict(source=name, distance=0.8, effort=4, size_mb=round(len(data) / 1e6, 2))
for rn, m in REGIONS.items():
row[rn] = dict(rms_px=round(float(np.sqrt((e_px[m] ** 2).mean())), 2),
rms_avg=round(float(np.sqrt((e_av[m] ** 2).mean())), 2))
same_setting.append(row)
print(json.dumps(row), flush=True)
del own, a, e_px, e_av
size[("camera RGB8 today e4", 0.8)] = same_setting[0]["size_mb"]
dark_dec[("camera RGB8 today", 0.8)] = same_crops["camera RGB8"][1]
# --- crops ---------------------------------------------------------------------------------
def upright(a):
return np.ascontiguousarray(np.rot90(a, 1)) # camera on its side: sky was on the right
def view(a12, stops):
return np.clip(a12 * 2.0 ** stops / 16, 0, 255).astype(np.uint8)
def grid(tiles, col_labels, row_labels, path, titles=None):
"""Tiles in rows; labels per column and row, or a title per tile."""
nr, nc = len(tiles), len(tiles[0])
fig, axs = plt.subplots(nr, nc, figsize=(nc * 3.0, nr * 3.25), squeeze=False)
for i in range(nr):
for j in range(nc):
axs[i][j].imshow(tiles[i][j], interpolation="nearest")
axs[i][j].set_xticks([]); axs[i][j].set_yticks([])
if titles:
axs[i][j].set_title(titles[i][j], fontsize=8)
elif i == 0:
axs[i][j].set_title(col_labels[j], fontsize=8)
if j == 0 and row_labels:
axs[i][j].set_ylabel(row_labels[i], fontsize=8)
fig.tight_layout(pad=0.4); fig.savefig(path, dpi=150); plt.close(fig)
d12, d8, dc = crop(ref, dark_xy), crop(to12(c8), dark_xy), crop(to12(cam8), dark_xy)
grid([[upright(view(a, s)) for a in (dc, d8, d12)] for s in (2, 4)],
["camera RGB8 (today, other capture)", "8-bit (same frame, 12-bit >> 4)", "12-bit"],
["+2 stops (×4)", "+4 stops (×16)"], os.path.join(out, "dark_crop.png"))
keys = [[("camera RGB8 today", 0.8), ("8-bit", 1.0)], [("8-bit values, 12-bit file", 1.0), ("12-bit", 1.0)]]
grid([[upright(view(dark_dec[k], 4)) for k in row] for row in keys], None, None,
os.path.join(out, "dark_compressed.png"),
titles=[["(a) camera RGB8 file, d0.8 e4 (today)", "(b) 8-bit file, d1.0"],
["(c) 8-bit values in a 12-bit file, d1.0", "(d) 12-bit file, d1.0"]])
tiles = [same_crops["camera RGB8"][0], same_crops["camera RGB8"][1],
same_crops["BayerRG12p"][0], same_crops["BayerRG12p"][1]]
grid([[upright(view(a, s)) for a in tiles] for s in (0, 2, 4)],
["camera RGB8, uncompressed", "camera RGB8, d0.8 e4", "BayerRG12p, uncompressed", "BayerRG12p, d0.8 e4"],
["as stored (×1)", "+2 stops (×4)", "+4 stops (×16)"], os.path.join(out, "same_setting.png"))
z = slice(100, 300)
grid([[np.kron(upright(view(same_crops[n][1], 4))[z, z], np.ones((2, 2, 1), np.uint8))
for n in ("camera RGB8", "BayerRG12p")]],
["camera RGB8, d0.8 e4", "BayerRG12p, d0.8 e4"], ["+4 stops (×16), 2× zoom"],
os.path.join(out, "same_setting_zoom.png"))
mid_mean = crop(ref, mid_xy).mean(axis=(0, 1)) # per channel
grid([[upright(np.clip((crop(a, mid_xy) - mid_mean) * 4 / 16 + 128, 0, 255).astype(np.uint8))
for a in (to12(c8), ref)]],
["8-bit (same frame)", "12-bit"], ["contrast ×4 around the mean"], os.path.join(out, "mid_crop.png"))
# Overview: upright, quarter size, display gamma (sRGB) so the forest is visible.
small = cv2.resize(ref, (W // 4, H // 4), interpolation=cv2.INTER_AREA) / 4095
small = np.where(small <= 0.0031308, 12.92 * small, 1.055 * small ** (1 / 2.4) - 0.055)
ov = cv2.cvtColor((np.clip(small, 0, 1) * 255).astype(np.uint8), cv2.COLOR_RGB2BGR)
for (x, y), c in ((dark_xy, (255, 160, 0)), (mid_xy, (0, 0, 255))):
cv2.rectangle(ov, (x // 4, y // 4), ((x + CS) // 4, (y + CS) // 4), c, 2)
cv2.imwrite(os.path.join(out, "overview.jpg"), upright(ov), [cv2.IMWRITE_JPEG_QUALITY, 88])
def levels(a):
g = np.round(a[..., 1]).astype(int)
return int(len(np.unique(g)))
res = dict(
frame=dict(width=W, height=H, green_range_12bit=[int(c12[..., 1].min()), int(c12[..., 1].max())],
share_of_frame={k: round(float(m.mean()) * 100, 1) for k, m in REGIONS.items()}),
noise_calibration=round(CAL, 3),
noise={k: [[round(s), round(n, 1)] for s, n in v] for k, v in curves.items()},
dark_crop=dict(x=dark_xy[0], y=dark_xy[1], size=CS, mean_green_12bit=round(float(d12[..., 1].mean()), 1),
green_range_12bit=[int(d12[..., 1].min()), int(d12[..., 1].max())],
distinct_green_levels={"12-bit": levels(d12), "8-bit same frame": levels(d8),
"camera RGB8": levels(dc)}),
mid_crop=dict(x=mid_xy[0], y=mid_xy[1], size=CS,
mean_green_12bit=round(float(crop(ref, mid_xy)[..., 1].mean()), 1)),
sizes_mb={f"{n} d{d}": round(v, 2) for (n, d), v in size.items()},
errors=errors,
same_setting=same_setting,
)
json.dump(res, open(os.path.join(out, "results.json"), "w"), indent=1)
print(json.dumps(res, indent=1))