fwt_software/docs/raw12-pipeline-measurements.md

# Raw 12-bit capture with on-tower colour conversion: measurements on the LattePanda IOTA and 3 Delta

*Fire-watch tower camera system — technical report, October 2026*

## Abstract

The towers' camera currently delivers 8-bit colour images (RGB8), converted to colour inside the
camera. This study tested whether the camera should instead deliver its raw 12-bit sensor data
(`BayerRG12p`), with the tower's computer converting it to colour and saving 12-bit JPEG XL. The
same camera (Allied Vision Alvium 1800 U-1620c, 16.2 MP) was measured on both tower computers, the
LattePanda IOTA (tower 121) and the LattePanda 3 Delta (tower 111). Frame rate, auto-exposure and
auto-white-balance settling, the duration of every processing step, file sizes, CPU temperature and
power were recorded, and the processing was repeated on one identical image on both computers.

Raw 12-bit doubles the camera's frame rate over the same USB link (9.86 fps against 4.95 fps), so
auto-exposure settles in about 60% of the time, with no lost frames on either tower. Converting to
colour on the tower costs 38–50 ms; the JPEG XL encode dominates, and the complete 12-bit save takes
slightly less time than today's RGB8 save (0.52 s on the IOTA, 0.82 s on the Delta, which is 1.6×
slower). At a realistic scan pace both computers keep up and stay cool (53 °C and 66 °C); the Delta
needs two save workers to match the IOTA's single one. A 12-bit colour file at quality setting
(distance) 1.0 is 2.7 MB, smaller than today's 3.3–4.0 MB. In daylight, however, sensor noise
exceeds one 8-bit quantisation step, so the extra bits mainly carry information in deep shadow, at
dusk and at night. A rule for when to keep a frame (brightness in the camera's target band and
steady for 3 frames) selected a good frame in 21 of 24 tests; waiting for exposure and gain to stop
changing never did.

## Units and abbreviations

| Symbol | Meaning |
|---|---|
| MB, GB, TB | 10⁶, 10⁹, 10¹² bytes |
| MB/s | 10⁶ bytes per second |
| fps | frames per second |
| ms, s | milliseconds, seconds |
| dB | decibel (camera gain; image quality PSNR) |
| DN | digital number: a pixel value, on a 0–255 scale (8-bit) or 0–4095 (12-bit) as stated |
| °C, W, GHz | CPU package temperature, package power, average core clock |
| MP | megapixels (10⁶ pixels) |
| d, e | JPEG XL distance (quality: 0 = lossless, higher = smaller and lossier) and effort (1–9) |
| PSNR | peak signal-to-noise ratio, in dB |
| SSIM | structural similarity index, 0–1 (1 = identical) [6] |
| IOTA, Delta | LattePanda IOTA (tower 121) and LattePanda 3 Delta (tower 111) |

---

## 1. Introduction

### 1.1 Background and question

Each tower scans a set of waypoints and keeps one picture per waypoint. After the gimbal stops, the
camera's auto-exposure needs a number of frames to adapt to the new view, so the scan waits; and
each kept picture is compressed to JPEG XL [2] on the tower. Today the camera converts its sensor
data to 8-bit colour (RGB8) itself. That format is three times larger than the raw sensor data, which
limits the frame rate over the camera's USB link and thereby lengthens the exposure settling.

This report answers: should the towers take raw 12-bit data from the camera instead, and convert it
to colour on the tower? Specifically: (1) how much faster is the camera, and does exposure settle
faster; (2) how long does each processing step take on each tower computer; (3) how large are the
files; (4) can both computers keep up without piling up images or overheating; and (5) what do the
extra bits add.

### 1.2 The camera's pixel formats

The sensor sees through a colour-filter mosaic: each pixel measures only red, green or blue, in the
pattern of Figure 1 ("BayerRG": a red pixel first). Producing a colour image means estimating the
two missing colours at every pixel from its neighbours, called **de-Bayering** (demosaicing). The
camera can either send the raw mosaic or de-Bayer it itself.

```
R G R G R G ...
G B G B G B ...
R G R G R G ...
```

***Figure 1.** The BayerRG colour-filter pattern of the sensor: each pixel records one colour.*

***Table 1.** Pixel formats considered, at the 1620c's full resolution (5328 × 3040 pixels) and a USB
link cap of 250 MB/s [4].*

| Format | What the camera sends | Bits per value | Bytes per pixel | Frame size (MB) | Maximum rate (fps) | Colour made by |
|---|---|---|---|---|---|---|
| `RGB8` (today) | three values per pixel, two of them estimated | 8 | 3 | 48.6 | 4.95 | the camera |
| `BayerRG8` | the raw mosaic, top 8 of the sensor's 12 bits | 8 | 1 | 16.2 | 14.75 | the tower |
| `BayerRG10p` | the raw mosaic, 10 bits, packed (4 pixels in 5 bytes) | 10 | 1.25 | 20.2 | 11.81 | the tower |
| **`BayerRG12p`** (proposed) | the raw mosaic, all 12 bits, packed (2 pixels in 3 bytes) | 12 | 1.5 | 24.3 | **9.86** | the tower |
| `BayerRG12` | the same 12 bits, each in a 16-bit word (4 bits of padding) | 12 | 2 | 32.4 | 7.41 | the tower |

RGB8 is the largest because two of its three values per pixel are estimates; it carries no more
measured information than `BayerRG8`. `BayerRG12p` and `BayerRG12` hold identical data; packing saves
25% of the bytes and so allows 33% more frames. The camera applies its white balance to the raw
formats too, so the tower does not need to. The camera also offers grey (`Mono8/10/12`, which
discards colour), `BGR8` (RGB8 in reverse order) and video formats (`YCbCr…`, which reduce colour
detail); none of these was considered.

### 1.3 Earlier work

An earlier campaign ([camera-capture-measurements.md](camera-capture-measurements.md)) used blurred,
dim indoor scenes and found that the camera link is stable at full resolution once the USB settings
of known issue #19 are applied; its compression figures were not representative of real scenes.
The implementation built on this report is described in
[raw12-pipeline-implementation-plan.md](raw12-pipeline-implementation-plan.md).

---

## 2. Materials and methods

### 2.1 Hardware and software

***Table 2.** Test systems. The same camera was moved from tower 121 to tower 111.*

| | Tower 121 | Tower 111 |
|---|---|---|
| Computer | LattePanda IOTA: Intel N150, 4 cores, ≤ 3.6 GHz, 7 GB RAM | LattePanda 3 Delta: Intel Celeron N5105, 4 cores, ≤ 2.9 GHz, 7.6 GB RAM |
| Storage write speed (MB/s) | 218 (eMMC) | 76 (eMMC) |
| Libraries | libjxl 0.11.1 [2], OpenCV 4.10 [3] | libjxl 0.11.0, OpenCV 4.5.4 |
| Camera | Alvium 1800 U-1620c `DEV_1AB22C0B7D4E` [5], 5328 × 3040 pixels | same camera |
| Link cap (MB/s) | 250 | 250 |
| Test dates and light | 2 Oct, 15:01–16:50, sun | 4 Oct, 16:19–18:19, sun, then evening |

The tower control program was stopped during every test (the camera accepts one program at a time,
and the CPU figures should have a single cause). Afterwards both cameras were returned to their
`standard` profile.

### 2.2 Scene

![The scene](images/raw12/examples/121/thumbs/c12_ea_lossless.jpg)

***Figure 2.** The test scene from tower 121's camera (2 Oct, 15:07; exposure 20 ms, gain ≈ 15 dB):
spruce, birch trunks, leaves and a building; about 13% of the pixels are clipped sky.*

### 2.3 Camera control

A Python program (`camera_bench.py`, Allied Vision vmbpy) drives the camera through the same GenICam
interface [4] the tower program uses. Each setting change is a GenICam feature write sent over the
USB control channel **while the camera keeps streaming**. `ExposureAuto`, `GainAuto` and
`BalanceWhiteAuto` take the values `Off`, `Once` or `Continuous`; setting one to `Off` freezes the
value the automatic control had reached, after which `ExposureTime`, `Gain` and `BalanceRatio` can be
written directly and take effect from the next frame exposed. No profile, user set or file on the
camera is changed.

***Table 3.** Camera settings common to all camera tests.*

| Setting | Value | Reason |
|---|---|---|
| Resolution | 5328 × 3040 pixels, binning 1 | full resolution, as decided |
| `DeviceLinkThroughputLimit` | 250 MB/s | agreed conservative link cap |
| `AcquisitionFrameRate` | the camera's maximum for the format (fps) | 9.86 (12-bit), 14.75 (8-bit raw), 4.95 (RGB8) |
| `ExposureAutoMax` | 20 ms | as in the `standard` profile |
| `IntensityAutoPrecedence` | `MinimizeNoise` | favours exposure over gain; with exposure at its 20 ms cap, the camera then uses gain |
| `IntensityControllerTarget` / `Tolerance` / `Rate` | 50% / 5% / 100 | camera defaults; rate 100 is the fastest |
| `GainAutoMax` | ≥ 43.5 dB (as left in the camera) | the bright steps reached 43.5 dB |
| Chunk data (per-frame metadata) | `ExposureTime`, `Gain`, `BalanceRatioRed`, `BalanceRatioBlue`, `Timestamp` | each frame carries the settings it was exposed with |
| Buffers | 5 | as in production |

For every frame the program records the host arrival time, the camera's frame number, completeness,
the chunk data, and the frame's **brightness** (DN, 0–255) and **clipped share** (% of samples
≥ 250 DN), sampled from the image buffer: every 997th byte for RGB8 and 8-bit raw, and the high byte
of every 331st pixel pair for `BayerRG12p`, scaled to 0–255 DN.

### 2.4 Frame rate and streaming

For each format, the camera's maximum frame rate was read, then the camera streamed for 60 s at that
rate. Counted: frames received, incomplete frames (SDK frame status) and lost frames (gaps in the
camera's frame counter). `BayerRG12p` was streamed once more while the tower saved 12-bit images
continuously in parallel.

### 2.5 Auto-exposure step test

***Table 4.** One auto-exposure step run.*

| Phase | Duration (s) | Procedure |
|---|---|---|
| 1. Settle | 8 | `ExposureAuto` and `GainAuto` = `Continuous`. "Before" = median exposure and gain of the last 5 frames. The brightness the camera steers is E₀ = exposure × 10^(gain/20). |
| 2. Force | 2 | `ExposureAuto` = `GainAuto` = `Off`. **Dark:** `ExposureTime` = max(`ExposureAutoMin`, E₀/16), `Gain` = 0 dB. **Bright:** `ExposureTime` = min(20 ms, 16·E₀); the remainder as `Gain` = 20·log₁₀(16·E₀ / exposure) dB, up to `GainAutoMax`. The camera delivers 10–30 frames at the forced settings. |
| 3. Release | 15 | Time zero is taken, then `ExposureAuto` = `GainAuto` = `Continuous`; every frame is recorded. |

***Table 5.** Starting conditions of the auto-exposure steps: the first frame after release, over all
24 runs.*

| | Before (settled) | Start of "dark" step | Start of "bright" step |
|---|---|---|---|
| Exposure (ms) | 20 (at its cap) | 7.4–11.5 | 20 |
| Gain (dB) | 15.4–19.5 | 0 | 39.6–43.5 (before + 24.1) |
| Brightness (DN, 0–255) | 117–122 | **9–13 (under-exposed)** | **254–255 (over-exposed)** |
| Clipped share (%) | 10–15 (sky) | 0 | **97–100** |

Settling was therefore measured from both under- and over-exposure. The bright start is fully
saturated, so the camera cannot measure how far over it is and can only step down until detail
reappears. Each format and direction was run twice: 12 runs per tower, 24 in total.

**Settling time** is judged on the image: the time from release after which brightness stays within
±10% of its final level (median over the run's last 2 s), and the number of frames this took. The
first 1–5 frames after release still carry the forced values (frames in transit); they count
towards the settling time.

### 2.6 Evaluation of keep rules

A keep rule decides which frame a waypoint keeps. Each candidate rule was applied offline to the 24
per-frame logs; for each, the first frame at which it would keep a picture was found, and counted as
**correct** if that frame's brightness was within 10% of the final level, otherwise **early**. The
camera's target band is 50 ± 5% of full scale, i.e. 114.8–140.3 DN on the 0–255 scale.

### 2.7 Auto white balance step test

***Table 6.** One auto-white-balance step run.*

| Phase | Duration (s) | Procedure |
|---|---|---|
| 1. Exposure | 6 | Auto-exposure settles; then `ExposureAuto` = `GainAuto` = `Off` with `ExposureTime` and `Gain` fixed at the median of the last 5 frames, so only white balance can change. `BalanceWhiteAutoRate` = 100 (fastest). |
| 2. Reference | 8 | `BalanceWhiteAuto` = `Continuous`. Reference = median red and blue ratios over the last 3 s (red 2.34–2.46, blue 1.86–1.97). |
| 3. Force | 2 | `BalanceWhiteAuto` = `Off`; red and blue `BalanceRatio` = reference × factor (green is the fixed reference). ×0.5: red ≈ 1.2, blue ≈ 0.95 (image too green); ×2: red ≈ 4.8, blue ≈ 3.8 (too magenta). |
| 4. Release | 15 | Time zero, then `BalanceWhiteAuto` = `Continuous`; every frame's ratios recorded. |

Settled = both ratios within ±5% of the reference and remaining there. Each case was run twice.

### 2.8 Processing tests

A C++ program (`raw12_bench.cpp`), built on each tower against its own libraries, performs the steps
the tower program would perform for one saved frame and times each with a monotonic clock.

***Table 7.** Processing steps timed.*

| Step | Operation |
|---|---|
| Unpack | `BayerRG12p` (two 12-bit pixels in three bytes) → one 16-bit word per pixel |
| Colour conversion | OpenCV `cvtColor`, bilinear (`COLOR_BayerBG2RGB`) or edge-aware (`…_EA`), 8- or 16-bit [3]; the camera's `BayerRG` corresponds to OpenCV's `BayerBG` (verified against the camera's RGB8) |
| JPEG XL encode | libjxl [2], into memory; effort and distance as stated; 12-bit input as 16-bit words with the bit depth taken from the file header; lossless with the original colour profile |
| File write | `fopen`/`fwrite`/`fflush`/`fclose` to the tower's eMMC |

Each time is the median of 3 runs (2 in the quality sweep). Before every run, the program verified
that its 8-bit encoder produces byte-for-byte the file of the production encoder, that 12-bit
lossless decodes exactly to the input, and that its colour order matches the camera's.

**Input frames.**
- **Shared frame** (identical on both towers; used for the tower comparison): tower 121's frame of
  2 Oct (15:07, 20 ms, ≈ 15 dB), rebuilt bit-exactly from its lossless 12-bit file. Its 8-bit raw
  version is the 12-bit value shifted down by 4 bits; its "RGB8" version is the edge-aware colour
  conversion of the 12-bit frame, scaled to 8 bits (the camera's own RGB8 of that instant does not
  exist). Identical MD5 checksums on both towers.
- **Synthetic frame** (identical on both towers): a "dead leaves" texture [1] with added sensor
  noise; sharper than any real scene and therefore the worst case for the encoder.
- Each tower's own camera capture from its test day.

### 2.9 Image quality metrics

Every 12-bit result was decoded and compared with the exact lossless 12-bit colour image of the same
frame (mosaic results first converted to colour in the same way): PSNR (dB) over all three channels on
the 12-bit scale, and SSIM [6] on luma.

### 2.10 Throughput, pile-up and heat

*Back to back:* save workers process the same frame from memory continuously for 60 s.
*Paced:* one frame is queued every 2.0, 1.5, 1.0, 0.7 or 0.5 s for 120 s, as a scan keeps one frame
per waypoint; recorded are the time from queueing to file written, the frames still queued at the end
("behind") and the peak memory. *Heat:* 10 minutes back to back and 10 minutes at one frame every 2 s,
each after a 5-minute cool-down; recorded once per second: CPU package temperature, average core
clock, CPU busy share and package power (Intel RAPL energy counter [7]).

### 2.11 Noise measurement

Sensor noise was measured on the raw 12-bit values: the green sites were divided into 24 × 24-pixel
patches; in each, noise σ = median absolute deviation of differences between horizontal neighbours,
÷ √2; patches were grouped by brightness, and the quietest 10% of each group taken as the noise floor
(scene texture can only add to it). This was done for the ≈ 15 dB frame and for a frame taken at
**0 dB gain** (exposure 124 ms, 4 Oct 16:19, exposure cap temporarily raised to 200 ms).

### 2.12 Campaign log

***Table 8.** Campaign log.*

| Date and time | Tower | Activity | Duration (min) | Notes |
|---|---|---|---|---|
| 2 Oct 15:01–15:22 | 121 | camera tests | 21 | sunny |
| 2 Oct 15:22–16:50 | 121 | processing, pile-up, heat (own and synthetic frames) | 88 | |
| 2 Oct 19:01 | 121 | attempted 0 dB frame | – | too dark (24 dB needed); discarded |
| 4 Oct 16:19 | 111 | 0 dB frame (camera moved from 121) | 2 | |
| 4 Oct 16:20–16:41 | 111 | camera tests | 21 | tower 111's own camera connected but idle |
| 4 Oct 16:41–18:19 | 111 | processing, pile-up, heat | 98 | |
| 4 Oct 23:03–23:32 | 121 and 111 | processing on the shared frame, both towers simultaneously | 29 | no camera needed |

---

## 3. Results

### 3.1 Frame rate and streaming

![Frame rate per format](images/raw12/fps_per_format.png)

***Figure 3.** Maximum full-resolution frame rate (fps) per pixel format at the 250 MB/s link cap.
The same camera was used on both towers.*

***Table 9.** 60 s streams at each format's maximum rate.*

| Format | Frame size (MB) | Maximum rate (fps) | 121: complete / incomplete / lost (frames) | 111: complete / incomplete / lost (frames) |
|---|---|---|---|---|
| BayerRG12p | 24.3 | 9.86 | 593 / 0 / 0 | 592 / 0 / 0 |
| BayerRG12 (unpacked) | 32.4 | 7.41 | 445 / 0 / 0 | 445 / 0 / 0 |
| BayerRG10p | 20.2 | 11.81 | 711 / 0 / 0 | 709 / 0 / 0 |
| BayerRG8 | 16.2 | 14.75 | 887 / 0 / 0 | 887 / 0 / 0 |
| RGB8 (today) | 48.6 | 4.95 | 298 / 0 / 0 | 297 / 0 / 0 |
| BayerRG12p while saving 12-bit continuously | 24.3 | 9.86 | 591 / 0 / 0 | 592 / 0 / 0 |

No frame was lost or incomplete in any format on either tower. On tower 111 the camera shares the
USB 3 controller with a 5G modem, a USB network adapter and a second (idle) camera.

### 3.2 Auto-exposure settling

![Brightness after a step, 121](images/raw12/ae_brightness_121.png)
![Brightness after a step, 111](images/raw12/ae_brightness_111.png)

***Figure 4.** Frame brightness (DN, 0–255) after auto-exposure is released, for each format: (a,
top) tower 121; (b, bottom) tower 111. Left: from under-exposure (×1/16); right: from over-exposure
(×16). Dashed line: settled level.*

With exposure at its 20 ms cap, auto-exposure adjusts through gain, by 1–2 dB per frame, and
sometimes pauses mid-way (flat segments in Figure 4, up to 2.8 s while still far too bright).

***Table 10.** Time (s) and frames until brightness stays within ±10% of its final level; two runs
each.*

| Format | Rate (fps) | Step | Tower 121: time (s) (frames) | Tower 111: time (s) (frames) |
|---|---|---|---|---|
| BayerRG8 | 14.75 | dark ×1/16 | 1.94 (28), 2.21 (32) | 1.79 (26), 1.87 (27) |
| BayerRG8 | 14.75 | bright ×16 | 2.69 (39), 2.55 (37) | 2.00 (29), 1.86 (27) |
| **BayerRG12p** | 9.86 | dark ×1/16 | 2.90 (28), 3.00 (29) | 6.64 (65), 4.51 (44) |
| **BayerRG12p** | 9.86 | bright ×16 | 6.45 (63), 2.69 (26) | 2.18 (21), 5.73 (56) |
| RGB8 (today) | 4.95 | dark ×1/16 | 4.56 (22), 5.17 (25) | 5.97 (29), 5.77 (28) |
| RGB8 (today) | 4.95 | bright ×16 | 5.17 (25), 5.17 (25) | 6.57 (32), 4.96 (24) |

Settling typically takes 25–35 frames, so **settling time scales with 1/frame rate**. The longer
12-bit runs (44–65 frames) contain mid-way pauses, which occurred at every rate but more often in the
12-bit runs here.

### 3.3 Keep rules

***Table 11.** Candidate keep rules scored on the 24 steps (12 per tower). Correct: the kept frame's
brightness is within 10% of the final level.*

| Rule | 121 correct / early (runs) | 111 correct / early (runs) | Median time when correct, 121 / 111 (s) |
|---|---|---|---|
| Exposure and gain unchanged for 3 frames | 0 / 12 | 0 / 12 | – |
| Brightness in the target band (50 ± 5%), 2 frames | 6 / 6 | 6 / 6 | 3.30 / 6.10 |
| Target band 2 frames and exposure/gain unchanged 2 frames | 6 / 6 | 7 / 5 | 3.35 / 5.02 |
| Target band 3 frames | 8 / 4 | 8 / 4 | 3.40 / 4.84 |
| **Target band and brightness steady (≤ 2% change) over 3 frames** | **11 / 1** | **10 / 2** | 3.61 / 5.45 |

Descending from over-exposure, brightness enters the band about 15% above its final level, so "in
band" alone fires early; also requiring the brightness to have stopped changing corrects most of
these cases.

### 3.4 Auto white balance

The profiles currently fix white balance (red 2.87, blue 2.0). With `BalanceWhiteAuto` =
`Continuous` the camera settled this scene at red ≈ 2.36–2.46, blue ≈ 1.87–1.97.

![Auto white balance, 121](images/raw12/awb_121.png)
![Auto white balance, 111](images/raw12/awb_111.png)

***Figure 5.** White-balance ratios relative to their settled values after release, for each
format: (a, top) tower 121; (b, bottom) tower 111. Left: ratios forced to ×0.5; right: ×2. Shaded:
±5% band.*

***Table 12.** Time until both ratios are within ±5% of the reference and remain there.*

| Format | Rate (fps) | Tower 121: time (s) (frames) | Tower 111: time (s) (frames) |
|---|---|---|---|
| BayerRG12p | 9.86 | 0.21–0.32 (2–3); once 1.33 | 0.22–0.41 (2–4) |
| RGB8 | 4.95 | 0.57–0.77 (2–3) | 0.55–0.76 (2–3) |
| BayerRG8 | 14.75 | 0.26–0.27 (3) | 0.19–0.26 (2–3) |

On a still scene the automatic ratios held within ±0.8–3.1%. White balance settles much faster than
exposure.

### 3.5 Processing time per step

![Same frame, time per step](images/raw12/stages_common.png)

***Figure 6.** Time per saved frame (ms) by step on the identical shared frame, for both computers;
4 threads; colour paths at d1.0 e3, today's RGB8 path at d0.8 e4.*

***Table 13.** Duration of each step on the shared frame (4 threads).*

| Step | IOTA (ms) | Delta (ms) | Delta / IOTA |
|---|---|---|---|
| Unpack 12-bit packed → 16-bit | 10 | 17 | 1.7 |
| Colour conversion, 8-bit raw, bilinear / edge-aware | 8 / 9 | 11 / 12 | 1.3 |
| Colour conversion, 12-bit raw, bilinear / edge-aware | 17 / 28 | 22 / 33 | 1.2 |
| JPEG XL encode, 12-bit colour, d1.0 e3 | 480 | 765 | 1.6 |
| JPEG XL encode, today's RGB8, d0.8 e4 | 566 | 867 | 1.5 |
| File write, 2.7 MB colour file | 1 | 4 | |
| File write, 17–21 MB lossless raw file | 7–8 | 162–173 | (storage 218 vs 76 MB/s) |

***Table 14.** Total time per saved frame and resulting file size, shared frame (file sizes are
identical on both computers).*

| Path | IOTA (ms) | Delta (ms) | File size (MB) |
|---|---|---|---|
| Today: RGB8 → JPEG XL, d0.8 e4 | 567 | 872 | 3.27 |
| Raw 8-bit → colour → JPEG XL, d1.0 e3 | 468 | 741 | 2.66 |
| Raw 12-bit → colour (bilinear) → JPEG XL, d1.0 e3 | 476 | 778 | 2.53 |
| **Raw 12-bit → colour (edge-aware) → JPEG XL, d1.0 e3** | **518** | **820** | **2.65** |
| Raw 12-bit planes, lossless, e1 | 91 | 297 | 17.65 |
| Raw 12-bit mosaic, lossless, e1 | 87 | 303 | 20.94 |

Unpacking and colour conversion together cost 38 ms on the IOTA and 50 ms on the Delta (7% of the
save); the JPEG XL encode accounts for 90%. Here the "RGB8" input is the tower's own colour
conversion of the frame, so its file equals the raw 8-bit path's; the camera's real RGB8 files were
somewhat larger (3.7–4.0 MB, Appendix B).

### 3.6 Effort and threads

![Effort × threads, 121](images/raw12/effort_threads_121.png)
![Effort × threads, 111](images/raw12/effort_threads_111.png)

***Figure 7.** Time per frame (ms) by JPEG XL effort (rows) and thread count (columns), each tower's
real frame, distance 1.0: (a, top) tower 121; (b, bottom) tower 111.*

Effort 1–4 lie within about 15% of each other, and effort 3 gives the smallest files. On the IOTA,
3 threads is slower than 2 at every effort (761 / 582 / 672 / 512 ms for 1–4 threads at effort 1);
the Delta scales normally (1440 / 985 / 886 / 802 ms). Four threads is fastest on both computers.

### 3.7 Quality setting: size, time and quality

![Same frame, size and time by quality](images/raw12/sizes_common.png)

***Figure 8.** File size (MB, left) and time per frame (ms, right) versus JPEG XL distance on the
shared frame, effort 3, 4 threads, both towers.*

***Table 15.** Size and time per distance on the shared frame (effort 3, 4 threads, file write
included; 2 runs per point).*

| Distance | 12-bit colour size (MB) | IOTA time (ms) | Delta time (ms) | Raw 12-bit planes size (MB) |
|---|---|---|---|---|
| 0 (lossless) | 44.8 | 1225 | 2906 | 16.8 (IOTA 455 ms, Delta 860 ms) |
| 0.3 | 7.19 | 742 | 1197 | 7.31 |
| 0.5 | 5.15 | 678 | 1033 | 5.99 |
| 0.8 | 3.16 | 570 | 848 | 4.32 |
| **1.0** | **2.65** | **558** | **847** | 3.78 |
| 1.5 | 1.63 | 515 | 751 | 2.44 |
| 2.0 | 1.32 | 558 | 738 | 1.96 |

![Size and quality, real frame](images/raw12/distance_real.png)

***Figure 9.** File size (MB), PSNR (dB) and fine-detail retention against each method's own
uncompressed image, versus distance; each tower's real frame.*

***Table 16.** Quality against the exact lossless 12-bit colour image of the same frame (real frame,
≈ 15 dB gain; the towers agree within 0.3 dB).*

| Saved as | File size, 121 / 111 (MB) | PSNR (dB) | SSIM |
|---|---|---|---|
| 12-bit colour, d0.3 | 7.2 / 7.4 | 40.3 | 0.994 |
| 12-bit colour, d0.5 | 5.1 / 5.3 | 38.8 | 0.989 |
| 12-bit colour, d0.8 | 3.2 / 3.3 | 37.1 | 0.978 |
| **12-bit colour, d1.0** | **2.6 / 2.7** | 36.6 | 0.973 |
| 12-bit colour, d1.5 | 1.6 / 1.7 | 35.4 | 0.960 |
| 12-bit colour, d2.0 | 1.3 / 1.3 | 34.8 | 0.953 |
| 12-bit colour, bilinear, d1.0 | 2.5 / 2.6 | 36.6 | 0.968 |
| 12-bit raw planes, d0.5 | 6.0 / 6.0 | **47.2** | 0.990 |
| 12-bit raw planes, lossless | 17.6 / 17.7 | exact | 1 |
| 12-bit raw mosaic, lossless | 20.9 / 19.0 | exact | 1 |
| 12-bit colour, lossless | 47.4 / 46.3 | exact | 1 |

Colour JPEG XL is tuned to human vision: it discards sensor noise and fine colour, so PSNR is
moderate while SSIM stays high. Raw planes preserve the sensor values much more closely at the same
size (47 vs 39 dB, equal SSIM); lossless, they take a third of the space of lossless colour.

### 3.8 Example images

![crops](images/raw12/examples/121/crops.jpg)

***Figure 10.** Positions of the detail crop (red) and shadow crop (blue) used in Figure 11.*

***Figure 11.** One example per compression method, tower 121's frame (480 × 320-pixel crops at
100%): detail crop of spruce needles over bright leaves; its absolute difference from the lossless
image (amplified ×8; black = identical); and a darker crop brightened by 2 stops. File size and save
time are single runs at 4 threads. The two 8-bit rows are separate captures seconds apart (branches
move), so they have no difference map.*

| Method | File size (MB), time (s) | Detail (100%) | Difference (×8) | Shadow (+2 stops) |
|---|---|---|---|---|
| Camera RGB8, d0.8 e4 (today) | 3.8, 0.62 | ![](images/raw12/examples/121/detail/rgb8_d0.8_e4_today.png) | – | ![](images/raw12/examples/121/shadow/rgb8_d0.8_e4_today.png) |
| Raw 8-bit → colour, d1.0 | 2.7, 0.45 | ![](images/raw12/examples/121/detail/c8_ea_d1.0.png) | – | ![](images/raw12/examples/121/shadow/c8_ea_d1.0.png) |
| 12-bit colour, lossless (reference) | 47.4, 0.23 | ![](images/raw12/examples/121/detail/c12_ea_lossless.png) | ![](images/raw12/examples/121/diff/c12_ea_lossless.png) | ![](images/raw12/examples/121/shadow/c12_ea_lossless.png) |
| 12-bit colour, d0.3 | 7.2, 0.77 | ![](images/raw12/examples/121/detail/c12_ea_d0.3.png) | ![](images/raw12/examples/121/diff/c12_ea_d0.3.png) | ![](images/raw12/examples/121/shadow/c12_ea_d0.3.png) |
| 12-bit colour, d0.5 | 5.1, 0.69 | ![](images/raw12/examples/121/detail/c12_ea_d0.5.png) | ![](images/raw12/examples/121/diff/c12_ea_d0.5.png) | ![](images/raw12/examples/121/shadow/c12_ea_d0.5.png) |
| 12-bit colour, d0.8 | 3.2, 0.58 | ![](images/raw12/examples/121/detail/c12_ea_d0.8.png) | ![](images/raw12/examples/121/diff/c12_ea_d0.8.png) | ![](images/raw12/examples/121/shadow/c12_ea_d0.8.png) |
| **12-bit colour, d1.0** | 2.6, 0.53 | ![](images/raw12/examples/121/detail/c12_ea_d1.0.png) | ![](images/raw12/examples/121/diff/c12_ea_d1.0.png) | ![](images/raw12/examples/121/shadow/c12_ea_d1.0.png) |
| 12-bit colour, d1.5 | 1.6, 0.46 | ![](images/raw12/examples/121/detail/c12_ea_d1.5.png) | ![](images/raw12/examples/121/diff/c12_ea_d1.5.png) | ![](images/raw12/examples/121/shadow/c12_ea_d1.5.png) |
| 12-bit colour, d2.0 | 1.3, 0.42 | ![](images/raw12/examples/121/detail/c12_ea_d2.0.png) | ![](images/raw12/examples/121/diff/c12_ea_d2.0.png) | ![](images/raw12/examples/121/shadow/c12_ea_d2.0.png) |
| 12-bit colour, bilinear, d1.0 | 2.5, 0.48 | ![](images/raw12/examples/121/detail/c12_bil_d1.0.png) | ![](images/raw12/examples/121/diff/c12_bil_d1.0.png) | ![](images/raw12/examples/121/shadow/c12_bil_d1.0.png) |
| 12-bit raw planes, d0.5 | 6.0, 0.54 | ![](images/raw12/examples/121/detail/raw12_planes_d0.5.png) | ![](images/raw12/examples/121/diff/raw12_planes_d0.5.png) | ![](images/raw12/examples/121/shadow/raw12_planes_d0.5.png) |
| 12-bit raw planes, lossless | 17.6, 0.09 | ![](images/raw12/examples/121/detail/raw12_planes_lossless.png) | ![](images/raw12/examples/121/diff/raw12_planes_lossless.png) | ![](images/raw12/examples/121/shadow/raw12_planes_lossless.png) |
| 12-bit raw mosaic, lossless | 20.9, 0.09 | ![](images/raw12/examples/121/detail/raw12_lossless.png) | ![](images/raw12/examples/121/diff/raw12_lossless.png) | ![](images/raw12/examples/121/shadow/raw12_lossless.png) |

Full-size `.jxl` files are kept outside the repository in `tools/camera-bench/examples-*`
(about 115 MB per set), including tower 111's and the 0 dB examples.

### 3.9 Information content of the 12-bit data

![Noise vs brightness](images/raw12/noise_vs_signal.png)

***Figure 12.** Sensor noise σ (DN, 12-bit) versus signal (DN, 12-bit), green sites, at 0 dB gain
and at ≈ 15 dB gain. Dashed: one 8-bit quantisation step (16 DN); dotted: the RMS error added by
rounding to 8 bits (4.6 DN).*

***Table 17.** Noise floor at three signal levels.*

| Signal (DN, 12-bit) | Noise at 0 dB (DN) | Noise at ≈ 15 dB (DN) | Noise in 8-bit steps, 0 dB / 15 dB |
|---|---|---|---|
| ≈ 700 (darkest in this scene) | 26 | 42 | 1.6 / 2.6 |
| ≈ 1500 | 35 | 58 | 2.2 / 3.6 |
| ≈ 3000 | 37 | 80 | 2.3 / 5.0 |

Rounding to 8 bits adds an RMS error of 4.6 DN (12-bit scale), small compared with this noise. In
this daylight scene, 8-bit data therefore already holds the information the sensor delivers. By
extrapolation, 12-bit data carries additional information below about 300 DN (0 dB) or 100 DN
(15 dB), i.e. in deep shadow: at dusk, at night, or on the dark side of a ridge.

![8 vs 12 bit at 15 dB](images/raw12/examples/121/shadow_8_vs_12.png)
![8 vs 12 bit at 0 dB](images/raw12/examples/lowgain/shadow_8_vs_12.png)

***Figure 13.** The same darker crop brightened by 2 stops, 8-bit (left; the 12-bit value shifted
down by 4 bits) versus 12-bit (right): (a, top) ≈ 15 dB gain, 3138 vs 227 distinct green levels;
(b, bottom) 0 dB gain. No visible difference.*

At 0 dB gain the same settings produce 25–30% smaller files and about 3 dB higher PSNR (less noise
to encode): 12-bit colour at d1.0 is 1.9 MB, PSNR 40.0 dB, SSIM 0.987.

### 3.10 Saving throughput and pile-up

![Saves per second, same frame](images/raw12/pace_common.png)

***Figure 14.** Frames saved per second, back to back, on the shared frame, for one worker × 4
threads and two workers × 2 threads.*

![Pile-up](images/raw12/pileup.png)

***Figure 15.** Worst time from queueing to file written (s, logarithmic) versus the interval
between queued frames (s), synthetic frame. A flat line means saving keeps up; a rising line means a
backlog builds.*

***Table 18.** Throughput and pile-up on the synthetic frame (worst case). "ok" = no backlog after
120 s; otherwise frames behind at the end.*

| Path | Workers × threads | Tower | Back to back (frames/s) | One frame every 2.0 / 1.5 / 1.0 / 0.7 / 0.5 s | Peak memory (GB) |
|---|---|---|---|---|---|
| Today: camera RGB8, d0.8 e4 | 1 × 4 | 121 | 1.33 | ok / – / ok / **3 behind** / – | 0.5 |
| Today: camera RGB8, d0.8 e4 | 1 × 4 | 111 | 0.88 | ok / – / **12 behind** / **65 behind** / – | 0.5 |
| **12-bit colour, d1.0 e3** | 1 × 4 | 121 | 1.44 | ok / ok / ok / **ok** / – | 0.7 |
| **12-bit colour, d1.0 e3** | 1 × 4 | 111 | 0.93 | ok / **ok** / **8 behind** / 60 behind / – | 0.7 |
| **12-bit colour, d1.0 e3** | 2 × 2 | 121 | 1.74 | – / – / ok / ok / **34 behind** | 1.1 |
| **12-bit colour, d1.0 e3** | 2 × 2 | 111 | 1.18 | – / – / **ok** / **28 behind** / 97 behind | 1.1 |
| 12-bit colour, d1.0 e3 | 4 × 1 | 121 / 111 | 1.99 / 1.37 | (back to back only; each save waits 2–3 s) | 2.0 |
| 12-bit colour, d0.5 e3 | 1 × 4 | 121 | 1.19 | – / – / ok / 24 behind / – | 0.8 |
| 12-bit colour, d0.5 e3 | 1 × 4 | 111 | 0.77 | – / – / 27 behind / 79 behind / – | 0.7 |
| 12-bit raw mosaic, lossless | 1 × 4 | 121 / 111 | 10.2 / 1.8 | ok at 1.0 and 0.7 on both | 0.3 |

On the shared real frame, back to back, a 12-bit save took 0.56 s (IOTA, 1 × 4) and 0.65 s (Delta,
2 × 2); the IOTA kept up with a frame every 0.7 s with one worker, the Delta with two (with one it
fell 27 frames behind in 120 s).

### 3.11 Heat and power

![Flat out](images/raw12/heat_max.png)
![Paced](images/raw12/heat_paced.png)

***Figure 16.** CPU package temperature (°C) and package power (W) over 10 minutes of 12-bit saving:
(a, top) back to back with 2 × 2 workers; (b, bottom) one frame every 2 s with 1 × 4. Solid: tower
121; dashed: tower 111.*

***Table 19.** Temperature, clock, load and power, 10 minutes per condition.*

| Load | Tower | Temperature mean / max (°C) | Clock (GHz) | CPU busy (%) | Power (W) | Saves per minute, first → last minute |
|---|---|---|---|---|---|---|
| Idle | 121 | 38 / 40 | 0.79 | 2 | 2.2 | – |
| Idle | 111 | 52 / 54 | 0.96 | 4 | 2.9 | – |
| 12-bit, one frame every 2 s (1 × 4) | 121 | **53 / 84** | 1.71 | 21 | **5.1** | 30 → 29 |
| 12-bit, one frame every 2 s (1 × 4) | 111 | **66 / 86** | 1.84 | 33 | **5.7** | 30 → 30 |
| 12-bit, back to back (2 × 2) | 121 | 80 / 87 | 2.88 | 81 | 12.5 | 122 → 103 (−16%) |
| 12-bit, back to back (2 × 2) | 111 | 78 / 83 | 2.44 | 81 | 8.1 | 74 → 73 |

At scan pace both computers stay far from their limits; the Delta runs about 13 °C warmer. Back to
back, the Delta sits at an 8 W power limit and does not slow further, whereas the IOTA starts at
12.5 W and loses 16% of its throughput as it heats.

### 3.12 Storage

![Storage per day](images/raw12/storage.png)

***Figure 17.** Storage per day (GB) at one frame every 2 s, per saving method, each tower's real
frame.*

***Table 20.** Storage for 24 hours of scanning at three frame intervals.*

| Saved as | Per frame, 121 / 111 (MB) | Every 2 s (GB/day) | Every 5 s (GB/day) | Every 10 s (GB/day) |
|---|---|---|---|---|
| Camera RGB8, d0.8 (today's quality) | 3.7 / 4.0 | 160–172 | 64–69 | 32–34 |
| **12-bit colour, d1.0** | **2.65 / 2.75** | **114–119** | **46–48** | **23–24** |
| 12-bit colour, d0.5 | 5.2 / 5.3 | 222–230 | 89–92 | 44–46 |
| 12-bit raw planes, lossless | 16.8 / 17.0 | 727–736 | 291–294 | 145–147 |
| 12-bit colour, lossless | 44.8 / 44.1 | ≈ 1900 | ≈ 770 | ≈ 385 |

The towers have 11–22 GB of free storage, so images must leave the tower continuously.

---

## 4. Discussion

### 4.1 Summary of findings

***Table 21.** Summary of findings.*

| Topic | Finding | Section |
|---|---|---|
| Frame rate | `BayerRG12p`: 9.86 fps, twice RGB8's 4.95 fps at 250 MB/s; `BayerRG8`: 14.75 fps. Unpacked `BayerRG12`: 7.41 fps. | 3.1 |
| Link | No lost or incomplete frames in any format on either tower, also while saving continuously. | 3.1 |
| Auto-exposure | From under- or over-exposure, settling takes about 25–35 frames: raw 8-bit 1.8–2.7 s, 12-bit 2.2–6.6 s (typically ≈ 3 s), RGB8 4.6–6.6 s. | 3.2 |
| Keep rule | Target band and steady for 3 frames: correct in 21 of 24 steps; "exposure and gain unchanged": 0 of 24. | 3.3 |
| White balance | Recovers in 2–4 frames (0.2–0.8 s), stable within ±1–3%. Exposure, not white balance, determines the wait. | 3.4 |
| Processing | Unpack + colour conversion 38 ms (IOTA) / 50 ms (Delta); a 12-bit save 518 / 820 ms, slightly faster than today's RGB8 save (567 / 872 ms). | 3.5 |
| Computers | The Delta takes 1.6 times as long on the identical frame. | 3.5 |
| Quality | d1.0: 2.65 MB, SSIM 0.973. Raw planes keep sensor values much closer (47 vs 39 dB at 6 MB). | 3.7 |
| 12 bits | In daylight, noise exceeds one 8-bit step even at 0 dB; extra information only below ≈ 300 DN (deep shadow). | 3.9 |
| Throughput | IOTA keeps up with a frame every 0.7 s with one worker; the Delta needs two. | 3.10 |
| Heat | One frame every 2 s: IOTA 53 °C / 5.1 W, Delta 66 °C / 5.7 W. | 3.11 |
| Storage | 12-bit colour d1.0: 2.7 MB per frame, below today's 3.3–4.0 MB. | 3.12 |

### 4.2 Overall pace

Saving runs in the background and overlaps the next movement, so the scan pace is set by the
slowest of the camera-side steps and the save rate.

***Table 22.** Time budget per waypoint.*

| Step | 12-bit raw (proposed) | Raw 8-bit | RGB8 (today) | Source |
|---|---|---|---|---|
| Camera frame interval (s) | 0.10 (9.86 fps) | 0.07 (14.75 fps) | 0.20 (4.95 fps) | measured |
| Slew to the next waypoint (s) | ≈ 1–1.5 (6° at 5–8 °/s) | same | same | estimate, not measured |
| Mechanical settling after standstill (s) | ≈ 0.2 | same | same | estimate, to be measured |
| Exposure settling after a large (×16) change (s) | ≈ 3 (2.2–6.6) | ≈ 2 (1.8–2.7) | ≈ 5 (4.6–6.6) | measured, 24 steps per tower |
| Exposure settling after a small change (s) | below the above | | | not measured |
| Save, 1 worker × 4 threads, IOTA / Delta (s) | **0.56 / 0.84** | 0.52 / 0.85 | 0.59 / 0.88 | measured, shared frame |
| Save, 2 workers × 2 threads, IOTA / Delta (s) | **0.47 / 0.65** | 0.42 / 0.60 | 0.43 / 0.64 | measured, shared frame |

A waypoint takes roughly 1.5–5 s, whereas a 12-bit save takes 0.56 s on the IOTA and 0.65 s on the
Delta with two workers. Saving only becomes the limit below about one waypoint per 0.6–0.85 s, which
a real scan does not reach. The camera side therefore sets the pace; settling after small changes
between neighbouring waypoints, which decides the real pace, remains to be measured.

### 4.3 Limitations

- One scene on two afternoons, at ≈ 15 dB gain in the main runs, plus one 0 dB frame. Night
  conditions were not measured.
- Auto-exposure steps were forced on a still camera, not caused by real slews. Real waypoint-to-
  waypoint changes are usually much smaller than ×16, so real settling should be shorter.
- The tower program was stopped, so its own load (camera stream, control loop) is not included; the
  stream-while-saving test shows the camera link is unaffected by saving.
- Detection quality was not measured; PSNR and SSIM stand in for what the algorithms receive.
- Tower 111's own camera (1800 U-1240c, 12 MP) was not tested; its frames are 25% smaller, so the
  Delta's limits above are conservative for it.
- Slew time and mechanical settling in Table 22 are estimates.

---

## 5. Conclusions and recommendations

1. **Take raw data from the camera** (`BayerRG12p`, 250 MB/s) and convert to colour on the tower,
   edge-aware. This doubles the frame rate and roughly halves the settling time, at no additional
   CPU, heat or storage cost compared with today, on either computer.
2. **Save 12-bit colour JPEG XL at distance 1.0, effort 3, 4 threads**: 2.7 MB per frame.
3. **Use one save worker on the IOTA and two workers × 2 threads on the Delta.** With the measured
   settling times (≈ 2–3 s per waypoint) the scan stays below either limit; the save queue still
   needs a bound.
4. **Keep a waypoint's frame when brightness is in the camera's target band and steady for 3
   frames**, with a time limit.
5. **12-bit versus raw 8-bit is a choice, not a cost.** `BayerRG8` (14.75 fps) settles about 1 s
   faster in daylight and loses nothing measurable there; `BayerRG12p` preserves deep shadows at dusk
   and night. The pixel format is part of the camera profile, so the program should accept both.
6. **Store raw 12-bit planes losslessly (17–18 MB per frame) only if the detection pipeline needs
   exact sensor values**; it requires about 6 times the storage.

---

## References

1. ISO 19567-2:2019. *Photography — Digital cameras — Texture reproduction measurements — Part 2:
   Measurements using dead leaves pattern.* International Organization for Standardization.
2. ISO/IEC 18181. *Information technology — JPEG XL image coding system*; reference implementation
   libjxl, versions 0.11.0 and 0.11.1, https://github.com/libjxl/libjxl.
3. OpenCV, versions 4.5.4 and 4.10: `cvtColor` colour conversions, Bayer demosaicing (bilinear and
   edge-aware), https://docs.opencv.org.
4. EMVA. *GenICam Pixel Format Naming Convention (PFNC)* and *GenICam Standard Features Naming
   Convention (SFNC)*. European Machine Vision Association.
5. Allied Vision. *Alvium 1800 U series technical manual* and *Vimba X SDK* (VmbCPP, vmbpy 1.0.5 and
   1.2.0).
6. Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli (2004). Image quality assessment: from error
   visibility to structural similarity. *IEEE Transactions on Image Processing* 13(4), 600–612.
7. Intel Running Average Power Limit (RAPL) energy counters, Linux `powercap` interface.

---

## Appendix A. Raw data and tools

- Programs and instructions to rerun every test: [tools/camera-bench](../tools/camera-bench/README.md).
- Camera tests, processing on each tower's own frame, heat:
  [tower 121](../tools/camera-bench/results-2026-10-02-tower121/),
  [tower 111](../tools/camera-bench/results-2026-10-04-tower111/).
- Shared-frame processing: [tower 121](../tools/camera-bench/results-common-tower121/),
  [tower 111](../tools/camera-bench/results-common-tower111/) (each with `frame_md5.txt`).
- Full-size example `.jxl` files (not in the repository, about 115 MB per set):
  `tools/camera-bench/examples-*`.
- Figures: `plot_results.py`, `plot_common.py` (charts), `make_figures.py` (example crops);
  quality scores: `compare_examples.py`.

## Appendix B. Processing times on each tower's own frame

![Stage times, real frame](images/raw12/stages_real.png)
![Stage times, synthetic frame](images/raw12/stages_synth.png)

***Figure B1.** Time per saved frame (ms) by step: (a, top) each tower's own real frame; (b,
bottom) the synthetic frame. Effort 3 (raw lossless: effort 1), 4 threads, distance 1.0.*

***Table B1.** Total time per saved frame (ms), effort 3, 4 threads, distance 1.0 (raw lossless:
effort 1).*

| Path | IOTA, own frame (ms) | IOTA, synthetic (ms) | Delta, own frame (ms) | Delta, synthetic (ms) |
|---|---|---|---|---|
| Today: camera RGB8 | 463 | 652 | 788 | 1071 |
| Raw 12-bit → colour (edge-aware) | 569 | 727 | 849 | 1088 |
| Raw 12-bit planes, lossless | 90 | 96 | 276 | 298 |

The camera's real RGB8 files on these frames were 3.7–4.0 MB. The results agree with the shared-frame
comparison (Section 3.5).