fwt_software/tools/camera-bench

camera-bench

Measures a tower's camera and CPU for the capture design: maximum and sustained frame rate per pixel format, losses while streaming, auto-exposure settle time, whether white balance applies to raw output, and the per-frame cost of copying, de-Bayering and JPEG XL encoding. The results of the first run, and what they mean, are in docs/camera-capture-measurements.md.

Not part of the build: CMake only compiles the tools it names.

File	What
camera_bench.py	drives the camera through vmbpy; one JSON line per measurement
bench.cpp	CPU cost per frame, using the production src/camera/JpegXlEncoder.cpp unchanged
lossless_check.cpp	encodes at jxl_distance = 0 through the production encoder, decodes, and compares pixels
encode_bench.cpp	JPEG XL cost per input format, effort, threads and quality; saving throughput and pile-up; CPU temperature, clock, load and power
make_synthetic.py	a sharp, detailed stand-in frame (dead-leaves texture with sensor noise) in the camera's formats
run_camera_tests.sh	the camera sequence: streaming per format and link limit, under load, auto-exposure, white balance (~25 min)
run_encode_tests.sh	the compression, pile-up and heat sequence (~90 min)
summarize_encode.py	Markdown tables from a run_encode_tests.sh result folder
raw12_bench.cpp	the 12-bit pipeline: unpack, de-Bayer, encode and write timed per stage; quality against the frame's own lossless image; pile-up, heat; example files (2026-10-02)
run_raw12_camera.sh / run_raw12_encode.sh	the 12-bit camera sequence (~25 min) and processing sequence (~1.5 h)
run_common_frame.sh	the processing tests on one shared frame (common-frame/, git-ignored, rebuilt from 121's lossless 12-bit example), so towers compare exactly (~25–40 min)
plot_common.py	the shared-frame tables and charts (per-step time, size per quality setting, saves per second); laptop venv
compare_examples.py	scores every 12-bit example against the lossless 12-bit colour image (PSNR, SSIM); laptop venv
compare_8_vs_12.py	8-bit against 12-bit on a dark scene: noise, levels, JPEG XL error per brightness region, crop figures (2026-10-07); laptop venv
make_figures.py	example-image figures (crops, difference maps, 8 vs 12-bit shadow) from the example .jxl files; laptop venv
plot_results.py	every chart and table of docs/raw12-pipeline-measurements.md from the results folders; laptop venv
results-*/	raw results of each run, one folder per tower and day
examples-*/	full-size example .jxl files (git-ignored, 100+ MB per tower)

Running it on a tower

The camera accepts one program at a time, so the tower program must be stopped, and the tower captures nothing until it is started again.

ssh ggs@<tower>
sudo systemctl stop fire-gimbal-control
cd ~/projects/fwt_2a/software/tools/camera-bench
PY="env GENICAM_GENTL64_PATH=/opt/VimbaX/cti $HOME/.venvs/vmbpy/bin/python camera_bench.py"

$PY info                                             # current settings and ranges
$PY maxfps BayerRG8 250000000                        # full res; also BayerRG12p, RGB8; also 450000000
$PY stream BayerRG8 250000000 14.75 60               # FPS = the max from maxfps; 60 s
$PY ae BayerRG8 250000000 14.75 dark 100             # dark|bright step, IntensityControllerRate
$PY wb BayerRG8 250000000                            # also RGB8, as the reference
$PY save BayerRG8 250000000 3 frames                 # also BayerRG12p and RGB8: bench.cpp needs all three

Then the CPU benchmark, built from the repo so it uses the production encoder:

cd ~/projects/fwt_2a/software
g++ -O2 -std=c++17 -Iinclude tools/camera-bench/bench.cpp src/camera/JpegXlEncoder.cpp \
    src/core/Logger.cpp -o /tmp/bench $(pkg-config --cflags --libs libjxl libjxl_threads opencv4) -pthread
cd tools/camera-bench
/tmp/bench all frames /tmp/bench-out        # every measurement
/tmp/bench raw8true frames /tmp/bench-out   # lossless raw 8-bit through a correct lossless path
/tmp/bench load frames /tmp/bench-out 62 &  # encode in a loop, while running `$PY stream ...`

lossless_check.cpp and encode_bench.cpp build the same way. lossless_check runs from the directory holding frames/.

The full sequences (2026-10-01)

Copy the programs into one working directory on the tower (for example ~/fgc-bench), with the binaries bench and encode_bench built there. Then, with the tower program stopped:

nohup sh run_camera_tests.sh results > camera.log 2>&1 &     # ~25 min; also saves frames/
python3 make_synthetic.py synth 5328 3040                     # the camera's resolution; needs numpy + OpenCV
nohup sh run_encode_tests.sh results-enc > enc.log 2>&1 &     # ~90 min
python3 summarize_encode.py results-enc

make_synthetic.py needs Python's numpy and OpenCV; where the tower lacks them (tower 111), generate the frame elsewhere and copy it. run_encode_tests.sh reads package power through a root loop (sudo -n), which it starts and stops itself. In run_camera_tests.sh, auto-exposure is capped at 20 ms (EXPOSURE_CAP_US), so long exposures cannot limit the frame rate being measured.

Finally, put the camera back on its profile, confirm frames flow, and restart the program:

$PY restore ~/projects/fwt_2a/software/config/camera_profiles/rgb/standard.xml
sudo systemctl start fire-gimbal-control
rm -rf frames /tmp/bench-out                # full-resolution frames are 16-49 MB each

The script never saves a camera user set or writes an XML. The program reloads its profile at start anyway, so nothing a run changes survives it.

The 12-bit pipeline run (2026-10-02)

On the tower, with raw12_bench built like bench and the tower program stopped:

python3 make_synthetic.py synth <width> <height>              # or copy one in
nohup sh run_raw12_camera.sh results-r12 > camera-r12.log 2>&1 &
nohup sh run_raw12_encode.sh results-r12 > encode-r12.log 2>&1 &    # after the first ends

Then on the laptop, in a venv with matplotlib, numpy, imagecodecs and opencv-python-headless (~/.venvs/fgc-plots), after copying results-r12/ and examples/jxl/ off the tower:

P=~/.venvs/fgc-plots/bin/python
$P compare_examples.py examples-<date>-<tower> results-<date>-<tower>/examples_compared.json
$P make_figures.py examples-<date>-<tower> ../../docs/images/raw12/examples/<tower> [dx,dy sx,sy]
$P plot_results.py ../../docs/images/raw12 121=results-2026-10-02-tower121 [111=...]

Things to know

• vmbpy 1.2.0 cannot stream under Python 3.14 as installed: its runtime type check raises NameError: name 'Camera' is not defined. camera_bench.py bypasses that check in its own process.
• Per-frame data: ChunkEnable is writable only while ChunkModeActive is off, so the script enables the items first and activates chunk mode last.
• Colour order: the camera's BayerRG8 is OpenCV's COLOR_BayerBG2RGB.
• The camera is the first real one found. Set CAM_ID=DEV_... to choose one.
• 12-bit JPEG XL from 16-bit words needs JxlEncoderSetFrameBitDepth(JXL_BIT_DEPTH_FROM_CODESTREAM), or libjxl rescales the values and "lossless" is not. Both C++ programs set it.
• The ae bright step is limited by the camera's limits: if gain is already at GainAutoMax and exposure near ExposureAutoMax, it cannot force a full ×16. The result records the factor actually reached (forced_factor).