From 4e8413bda6d99fbfc7b7a089fed33d0fdca94653 Mon Sep 17 00:00:00 2001 From: pgdalmeida Date: Mon, 5 Oct 2026 12:05:33 +0200 Subject: [PATCH] Explain in the 12-bit report where the extra bits make a real difference and where they do not --- docs/raw12-pipeline-measurements.md | 79 ++++++++++++++++++++++++----- 1 file changed, 66 insertions(+), 13 deletions(-) diff --git a/docs/raw12-pipeline-measurements.md b/docs/raw12-pipeline-measurements.md index 6584e41..a9ac66c 100644 --- a/docs/raw12-pipeline-measurements.md +++ b/docs/raw12-pipeline-measurements.md @@ -18,11 +18,19 @@ colour on the tower costs 38–50 ms; the JPEG XL encode dominates, and the comp slightly less time than today's RGB8 save (0.52 s on the IOTA, 0.82 s on the Delta, which is 1.6× slower). At a realistic scan pace both computers keep up and stay cool (53 °C and 66 °C); the Delta needs two save workers to match the IOTA's single one. A 12-bit colour file at quality setting -(distance) 1.0 is 2.7 MB, smaller than today's 3.3–4.0 MB. In daylight, however, sensor noise -exceeds one 8-bit quantisation step, so the extra bits mainly carry information in deep shadow, at -dusk and at night. A rule for when to keep a frame (brightness in the camera's target band and -steady for 3 frames) selected a good frame in 21 of 24 tests; waiting for exposure and gain to stop -changing never did. +(distance) 1.0 is 2.7 MB, smaller than today's 3.3–4.0 MB. A rule for when to keep a frame +(brightness in the camera's target band and steady for 3 frames) selected a good frame in 21 of 24 +tests; waiting for exposure and gain to stop changing never did. + +The extra bits themselves add less than expected. In the daylight scene measured, the sensor's own +noise was 6–17 times larger than the error from rounding to 8 bits, which raised the total noise by +at most 1.5%; and JPEG XL compression at distance 1.0 adds about 13 times more error than that +rounding. 12 bits are expected to make a meaningful difference only where part of the image is dark +while the gain is low, as in high-contrast scenes (a bright sky over shaded forest or a backlit ridge +at dawn or dusk), where faint smoke over dark terrain is a small change in a dark value; and only if +those values are saved with little or no loss. These conditions were not measured. The 12-bit format +costs little against 8-bit raw (same file size, about 10% more processing time), but the camera +runs at 9.86 instead of 14.75 fps. ## Units and abbreviations @@ -592,7 +600,7 @@ pixel-to-pixel variation. Dashed: one 8-bit quantisation step, the smallest chan can record (16 DN on the 12-bit scale). Dotted: the RMS error added by rounding to 8 bits (16/√12 = 4.6 DN). Noise above these lines means rounding to 8 bits loses nothing measurable.* -***Table 17.** Sensor noise floor σ (DN, 12-bit scale, 0–4095) at three signal levels, green pixels, +***Table 17a.** Sensor noise floor σ (DN, 12-bit scale, 0–4095) at three signal levels, green pixels, at 0 dB and ≈ 15 dB gain. Last column: the same noise in 8-bit steps (one step = 16 DN on the 12-bit scale); above 1, the 8-bit values already resolve all the sensor delivers.* @@ -602,10 +610,54 @@ scale); above 1, the 8-bit values already resolve all the sensor delivers.* | ≈ 1500 | 35 | 58 | 2.2 / 3.6 | | ≈ 3000 | 37 | 80 | 2.3 / 5.0 | -Rounding to 8 bits adds an RMS error of 4.6 DN (12-bit scale), small compared with this noise. In -this daylight scene, 8-bit data therefore already holds the information the sensor delivers. By -extrapolation, 12-bit data carries additional information below about 300 DN (0 dB) or 100 DN -(15 dB), i.e. in deep shadow: at dusk, at night, or on the dark side of a ridge. +**Rounding against noise.** Rounding to 8 bits adds an RMS error of 4.6 DN (12-bit scale). +Independent errors add as the square root of the sum of their squares, so in the worst case measured +(26 DN of noise at 0 dB) the total grows from 26 to 26.4 DN, i.e. by 1.5%; at ≈ 15 dB gain by 0.6%, +and in bright areas by less than 0.2%. Because the noise is larger than one 8-bit step, it also acts +as dither: averaging over an area recovers differences finer than one step even from 8-bit values. +For a single daylight frame, rounding to 8 bits therefore loses almost nothing measurable. + +**Averaging over many pixels.** Averaging an area of N pixels reduces the noise by √N, which might +seem to favour the 12-bit image. But while the noise exceeds one 8-bit step, the rounding error is +also random and independent from pixel to pixel, so it averages down at the same rate: an average of +8-bit values resolves differences much finer than one step, as precisely as the 12-bit average. A +12-bit image always holds at least as much information as its 8-bit version, but at a noise of +26 DN (≈ 2^4.7) the four bits removed by rounding mostly record the noise itself; the information +they carry about the scene is about 0.02 bits per pixel. This changes when the noise falls below one +8-bit step: the rounding error is then the same for neighbouring pixels, averaging cannot remove it, +and the dropped bits hold real scene detail. + +**Rounding against compression.** The saved file is lossy. At distance 1.0 the 12-bit colour file +has a PSNR of 36.6 dB (Table 16), an RMS error of about 61 DN, which is 13 times the rounding error; +even at distance 0.3 (40.3 dB) it is about 40 DN, and raw planes at distance 0.5 (47.2 dB) about +18 DN. Much of this error is noise the encoder discards, and JPEG XL spends relatively more +precision on dark tones, so the error in shadows is smaller than these averages; it was not measured +separately. Still, with lossy files the compression, not the bit depth, decides how much fine tonal +detail survives. Correspondingly, 12-bit and 8-bit raw give the same file size at distance 1.0 +(2.65 and 2.66 MB, Table 14). + +**Where 12 bits are expected to matter.** The noise falls with the signal, while the 8-bit step +stays at 16 DN. Extrapolating Figure 12, the noise drops below one 8-bit step at about 300 DN at +0 dB gain, or 100 DN at ≈ 15 dB. The 8-bit data is linear (no gamma curve), so this region, the +darkest 2–7% of the range, holds only 6–19 of the 256 8-bit levels. Gain multiplies the noise, so at +high gain the noise exceeds one 8-bit step again almost everywhere. Table 17b combines these +observations. + +***Table 17b.** Situations in which 12-bit data is expected to differ from 8-bit data. Signal in DN +on the 12-bit scale (0–4095). "Measured" refers to this campaign.* + +| Situation | Signal (DN), gain (dB) | Noise against one 8-bit step | Expected effect of 12 bits | Measured | +|---|---|---|---|---| +| Daylight, mid-tones and highlights (lit terrain, sky) | 700–4095, 0–15 | 1.6–5 steps | none measurable: total noise +0.2–1.5% | yes | +| Shadows in a high-contrast scene at low gain: shaded forest under a bright sky, backlit ridges, valleys at dawn and dusk | below ≈ 300 at 0 dB, ≈ 100 at 15 dB | below 1 step | 8 bits merge neighbouring levels; 12 bits keep small changes in dark areas, such as thin smoke over dark terrain | no (extrapolated) | +| Uniformly dark scene at high gain: night, gain up to 48 dB | any, high (up to 48) | above 1 step (gain amplifies noise) | little, except in the very darkest values | no | +| Strong processing of dark areas: brightening by 3 stops (×8) or more, contrast stretching | dark areas, any | – | 12 bits delay visible banding | 2 stops (×4) only: no difference (Figure 13) | +| Saving lossy at distance 1.0 | any | compression error ≈ 61 DN ≫ rounding 4.6 DN | largely lost to compression; kept in lossless files | yes, overall PSNR only | + +The case most relevant to fire detection is the second: in a scene exposed for a bright sky, smoke +rising in front of shaded forest is a small brightness increase in an already dark area, at low gain. +Whether 12 bits help there, and whether that benefit survives distance 1.0, requires a test on such a +scene (a dawn or dusk frame with dark terrain under a bright sky, compared as in Figures 12 and 13). ![8 vs 12 bit at 15 dB](images/raw12/examples/121/shadow_8_vs_12.png) ![8 vs 12 bit at 0 dB](images/raw12/examples/lowgain/shadow_8_vs_12.png) @@ -720,7 +772,7 @@ The towers have 11–22 GB of free storage, so images must leave the tower conti | Processing | Unpack + colour conversion 38 ms (IOTA) / 50 ms (Delta); a 12-bit save 518 / 820 ms, slightly faster than today's RGB8 save (567 / 872 ms). | 3.5 | | Computers | The Delta takes 1.6 times as long on the identical frame. | 3.5 | | Quality | d1.0: 2.65 MB, SSIM 0.973. Raw planes keep sensor values much closer (47 vs 39 dB at 6 MB). | 3.7 | -| 12 bits | In daylight, noise exceeds one 8-bit step even at 0 dB; extra information only below ≈ 300 DN (deep shadow). | 3.9 | +| 12 bits | In daylight, rounding to 8 bits adds at most 1.5% to the noise, and compression at d1.0 adds 13 times more error than rounding. 12 bits are expected to matter only in dark areas at low gain (high-contrast scenes), and only in lightly compressed files; not measured. | 3.9 | | Throughput | IOTA keeps up with a frame every 0.7 s with one worker; the Delta needs two. | 3.10 | | Heat | One frame every 2 s: IOTA 53 °C / 5.1 W, Delta 66 °C / 5.7 W. | 3.11 | | Storage | 12-bit colour d1.0: 2.7 MB per frame, below today's 3.3–4.0 MB. | 3.12 | @@ -777,8 +829,9 @@ between neighbouring waypoints, which decides the real pace, remains to be measu 4. **Keep a waypoint's frame when brightness is in the camera's target band and steady for 3 frames**, with a time limit. 5. **12-bit versus raw 8-bit is a choice, not a cost.** `BayerRG8` (14.75 fps) settles about 1 s - faster in daylight and loses nothing measurable there; `BayerRG12p` preserves deep shadows at dusk - and night. The pixel format is part of the camera profile, so the program should accept both. + faster in daylight and loses nothing measurable there; `BayerRG12p` is expected to help in dark areas + of high-contrast scenes at low gain, such as shaded terrain under a bright sky at dawn or dusk + (Section 3.9), which remains to be tested. The pixel format is part of the camera profile, so the program should accept both. 6. **Store raw 12-bit planes losslessly (17–18 MB per frame) only if the detection pipeline needs exact sensor values**; it requires about 6 times the storage.