Explain the raw mosaic and planes layouts in the 12-bit report, and spell out every term in its figure and table captions
This commit is contained in:
parent
1fc87bf1ee
commit
4abfd7cbb2
GPG Key ID:
D4A6C394DF13F1D7
|
|
@ -26,6 +26,8 @@ changing never did.
|
|||
|
||||
## Units and abbreviations
|
||||
|
||||
*Units, symbols and abbreviations used in this report.*
|
||||
|
||||
| Symbol | Meaning |
|
||||
|---|---|
|
||||
| MB, GB, TB | 10⁶, 10⁹, 10¹² bytes |
|
||||
|
|
@ -39,6 +41,7 @@ changing never did.
|
|||
| d, e | JPEG XL distance (quality: 0 = lossless, higher = smaller and lossier) and effort (1–9) |
|
||||
| PSNR | peak signal-to-noise ratio, in dB |
|
||||
| SSIM | structural similarity index, 0–1 (1 = identical) [6] |
|
||||
| stop | a factor of 2 in light or brightness: brightened by 2 stops = pixel values ×4; ×16 = 4 stops |
|
||||
| IOTA, Delta | LattePanda IOTA (tower 121) and LattePanda 3 Delta (tower 111) |
|
||||
|
||||
---
|
||||
|
|
@ -62,7 +65,7 @@ extra bits add.
|
|||
### 1.2 The camera's pixel formats
|
||||
|
||||
The sensor sees through a colour-filter mosaic: each pixel measures only red, green or blue, in the
|
||||
pattern of Figure 1 ("BayerRG": a red pixel first). Producing a colour image means estimating the
|
||||
pattern of Figure 1a ("BayerRG": a red pixel first). Producing a colour image means estimating the
|
||||
two missing colours at every pixel from its neighbours, called **de-Bayering** (demosaicing). The
|
||||
camera can either send the raw mosaic or de-Bayer it itself.
|
||||
|
||||
|
|
@ -72,10 +75,12 @@ G B G B G B ...
|
|||
R G R G R G ...
|
||||
```
|
||||
|
||||
***Figure 1.** The BayerRG colour-filter pattern of the sensor: each pixel records one colour.*
|
||||
***Figure 1a.** The BayerRG colour-filter pattern of the sensor: each pixel records one colour.*
|
||||
|
||||
***Table 1.** Pixel formats considered, at the 1620c's full resolution (5328 × 3040 pixels) and a USB
|
||||
link cap of 250 MB/s [4].*
|
||||
***Table 1.** Pixel formats considered, at the 1620c's full resolution (5328 × 3040 pixels) and a
|
||||
USB link cap of 250 MB/s [4]. Bits per value: the precision of each stored measurement (8 bits = 256
|
||||
levels, 10 bits = 1024, 12 bits = 4096). Packed: values stored back to back without padding bits.
|
||||
Maximum rate: the highest frame rate the camera accepts at this link cap.*
|
||||
|
||||
| Format | What the camera sends | Bits per value | Bytes per pixel | Frame size (MB) | Maximum rate (fps) | Colour made by |
|
||||
|---|---|---|---|---|---|---|
|
||||
|
|
@ -92,6 +97,40 @@ formats too, so the tower does not need to. The camera also offers grey (`Mono8/
|
|||
discards colour), `BGR8` (RGB8 in reverse order) and video formats (`YCbCr…`, which reduce colour
|
||||
detail); none of these was considered.
|
||||
|
||||
**Saving raw data: mosaic and planes.** Besides converting to colour, the tower can save the raw
|
||||
12-bit data as received, each pixel keeping the single value measured through its filter. Two
|
||||
layouts were tested; they hold identical values and differ only in their arrangement in the file:
|
||||
|
||||
- **Raw mosaic:** the sensor data as the camera sends it, a single-channel (grey) image of
|
||||
5328 × 3040 values still in the R G / G B pattern. Viewed directly it looks like a grey image with
|
||||
a fine checkerboard texture. Because colours alternate from pixel to pixel, the encoder sees the
|
||||
pattern as extremely fine detail: it compresses poorly (20.9 MB lossless), and lossy compression
|
||||
would mix values of different colours.
|
||||
- **Raw planes:** the same values reordered into four quarter-size images (2664 × 1520 each), one per
|
||||
filter position (red, first green, second green, blue), tiled 2 × 2 into one grey image of the
|
||||
same total size (Figure 1b). Within each tile neighbouring values belong to the same colour and are
|
||||
similar, so the image is smooth to the encoder: 17.6 MB lossless, and lossy compression works
|
||||
(distance 0.5: 6.0 MB at a PSNR of 47.2 dB, against 38.8 dB for 12-bit colour of similar size;
|
||||
Section 3.7). The reordering is exactly reversible, so the mosaic can always be rebuilt.
|
||||
|
||||
```
|
||||
Mosaic (5328 × 3040) Planes (2 × 2 tiles of 2664 × 1520)
|
||||
R G R G R G ... R R R ... | G G G ...
|
||||
G B G B G B ... → R R R ... | G G G ...
|
||||
R G R G R G ... ----------+----------
|
||||
G B G B G B ... G G G ... | B B B ...
|
||||
G G G ... | B B B ...
|
||||
```
|
||||
|
||||
***Figure 1b.** Rearranging the raw mosaic into four single-colour planes.*
|
||||
|
||||
Neither layout is a viewable colour picture: the reader must de-Bayer it, as the tower does before
|
||||
saving colour. Their value is that they preserve the measured sensor values, with nothing estimated
|
||||
and no colour processing applied, which matters only if the detection algorithms need exact sensor
|
||||
data. Lossless planes take about 6 times the storage of 12-bit colour at distance 1.0 (17.6 MB
|
||||
against 2.65 MB per frame), so they are an option (recommendation 6, Section 5) and are not part of
|
||||
the current implementation.
|
||||
|
||||
### 1.3 Earlier work
|
||||
|
||||
An earlier campaign ([camera-capture-measurements.md](camera-capture-measurements.md)) used blurred,
|
||||
|
|
@ -106,7 +145,9 @@ The implementation built on this report is described in
|
|||
|
||||
### 2.1 Hardware and software
|
||||
|
||||
***Table 2.** Test systems. The same camera was moved from tower 121 to tower 111.*
|
||||
***Table 2.** Test systems. The same camera was moved from tower 121 to tower 111, so only the
|
||||
computer differs between the towers. Storage write speed: sequential writing to the built-in eMMC
|
||||
flash storage.*
|
||||
|
||||
| | Tower 121 | Tower 111 |
|
||||
|---|---|---|
|
||||
|
|
@ -126,7 +167,8 @@ and the CPU figures should have a single cause). Afterwards both cameras were re
|
|||
|
||||
|
||||
***Figure 2.** The test scene from tower 121's camera (2 Oct, 15:07; exposure 20 ms, gain ≈ 15 dB):
|
||||
spruce, birch trunks, leaves and a building; about 13% of the pixels are clipped sky.*
|
||||
spruce, birch trunks, leaves and a building. About 13% of the pixels are clipped sky: saturated at
|
||||
the maximum value, so they show pure white without detail.*
|
||||
|
||||
### 2.3 Camera control
|
||||
|
||||
|
|
@ -138,7 +180,9 @@ value the automatic control had reached, after which `ExposureTime`, `Gain` and
|
|||
written directly and take effect from the next frame exposed. No profile, user set or file on the
|
||||
camera is changed.
|
||||
|
||||
***Table 3.** Camera settings common to all camera tests.*
|
||||
***Table 3.** Camera settings common to all camera tests. Names in code font are the camera's
|
||||
GenICam features [4]. Chunk data: values the camera attaches to each frame, describing the settings
|
||||
it was taken with.*
|
||||
|
||||
| Setting | Value | Reason |
|
||||
|---|---|---|
|
||||
|
|
@ -166,7 +210,10 @@ continuously in parallel.
|
|||
|
||||
### 2.5 Auto-exposure step test
|
||||
|
||||
***Table 4.** One auto-exposure step run.*
|
||||
***Table 4.** The three phases of one auto-exposure step run. Exposure is the time the sensor
|
||||
collects light (ms); gain amplifies the sensor signal (dB; +6 dB doubles the brightness, +20 dB
|
||||
multiplies it by 10). A step of ×16 makes the image 16 times brighter (×1/16: 16 times darker), i.e.
|
||||
4 stops, where one stop is a factor of 2.*
|
||||
|
||||
| Phase | Duration (s) | Procedure |
|
||||
|---|---|---|
|
||||
|
|
@ -174,8 +221,10 @@ continuously in parallel.
|
|||
| 2. Force | 2 | `ExposureAuto` = `GainAuto` = `Off`. **Dark:** `ExposureTime` = max(`ExposureAutoMin`, E₀/16), `Gain` = 0 dB. **Bright:** `ExposureTime` = min(20 ms, 16·E₀); the remainder as `Gain` = 20·log₁₀(16·E₀ / exposure) dB, up to `GainAutoMax`. The camera delivers 10–30 frames at the forced settings. |
|
||||
| 3. Release | 15 | Time zero is taken, then `ExposureAuto` = `GainAuto` = `Continuous`; every frame is recorded. |
|
||||
|
||||
***Table 5.** Starting conditions of the auto-exposure steps: the first frame after release, over all
|
||||
24 runs.*
|
||||
***Table 5.** Starting conditions of the auto-exposure steps: the first frame after release, ranges
|
||||
over all 24 runs. Before: the settled state before forcing. Dark / bright step: the settings forced
|
||||
16 times darker / brighter. Brightness: mean pixel value on the 0–255 scale (DN). Clipped share:
|
||||
percentage of sampled pixels at or above 250 DN, i.e. saturated white without detail.*
|
||||
|
||||
| | Before (settled) | Start of "dark" step | Start of "bright" step |
|
||||
|---|---|---|---|
|
||||
|
|
@ -202,7 +251,9 @@ camera's target band is 50 ± 5% of full scale, i.e. 114.8–140.3 DN on the 0
|
|||
|
||||
### 2.7 Auto white balance step test
|
||||
|
||||
***Table 6.** One auto-white-balance step run.*
|
||||
***Table 6.** The four phases of one auto-white-balance step run. White-balance ratios are the gains
|
||||
the camera applies to the red and blue values relative to green (no unit; 1 = unchanged); ×0.5
|
||||
halves them and ×2 doubles them.*
|
||||
|
||||
| Phase | Duration (s) | Procedure |
|
||||
|---|---|---|
|
||||
|
|
@ -218,7 +269,10 @@ Settled = both ratios within ±5% of the reference and remaining there. Each cas
|
|||
A C++ program (`raw12_bench.cpp`), built on each tower against its own libraries, performs the steps
|
||||
the tower program would perform for one saved frame and times each with a monotonic clock.
|
||||
|
||||
***Table 7.** Processing steps timed.*
|
||||
***Table 7.** Processing steps timed for one saved frame. Bilinear conversion estimates each missing
|
||||
colour as the average of the nearest pixels of that colour; edge-aware conversion interpolates along
|
||||
edges rather than across them, giving fewer false colour fringes at a slightly higher cost. Distance
|
||||
and effort are the JPEG XL quality and speed settings (Units and abbreviations).*
|
||||
|
||||
| Step | Operation |
|
||||
|---|---|
|
||||
|
|
@ -266,7 +320,8 @@ patches; in each, noise σ = median absolute deviation of differences between ho
|
|||
|
||||
### 2.12 Campaign log
|
||||
|
||||
***Table 8.** Campaign log.*
|
||||
***Table 8.** Campaign log: date and local time, tower, activity and duration (min) of each test
|
||||
session.*
|
||||
|
||||
| Date and time | Tower | Activity | Duration (min) | Notes |
|
||||
|---|---|---|---|---|
|
||||
|
|
@ -289,7 +344,10 @@ patches; in each, noise σ = median absolute deviation of differences between ho
|
|||
***Figure 3.** Maximum full-resolution frame rate (fps) per pixel format at the 250 MB/s link cap.
|
||||
The same camera was used on both towers.*
|
||||
|
||||
***Table 9.** 60 s streams at each format's maximum rate.*
|
||||
***Table 9.** Results of 60 s streams at each format's maximum rate. Complete: frames received
|
||||
intact. Incomplete: frames whose data arrived only partly (SDK frame status). Lost: frames missing
|
||||
from the camera's frame counter. Last row: the same stream while the tower saved 12-bit images
|
||||
continuously in parallel.*
|
||||
|
||||
| Format | Frame size (MB) | Maximum rate (fps) | 121: complete / incomplete / lost (frames) | 111: complete / incomplete / lost (frames) |
|
||||
|---|---|---|---|---|
|
||||
|
|
@ -308,15 +366,18 @@ USB 3 controller with a 5G modem, a USB network adapter and a second (idle) came
|
|||
|
||||
|
||||
|
||||
***Figure 4.** Frame brightness (DN, 0–255) after auto-exposure is released, for each format: (a,
|
||||
top) tower 121; (b, bottom) tower 111. Left: from under-exposure (×1/16); right: from over-exposure
|
||||
(×16). Dashed line: settled level.*
|
||||
***Figure 4.** Frame brightness (DN, 0–255) against time (s) after auto-exposure is switched back on
|
||||
at time 0, for each format: (a, top) tower 121; (b, bottom) tower 111. Left: starting from
|
||||
under-exposure, 16 times (4 stops) too dark; right: from over-exposure, 16 times too bright. Dashed
|
||||
line: settled level, the median brightness over the run's last 2 s.*
|
||||
|
||||
With exposure at its 20 ms cap, auto-exposure adjusts through gain, by 1–2 dB per frame, and
|
||||
sometimes pauses mid-way (flat segments in Figure 4, up to 2.8 s while still far too bright).
|
||||
|
||||
***Table 10.** Time (s) and frames until brightness stays within ±10% of its final level; two runs
|
||||
each.*
|
||||
***Table 10.** Settling time after a forced exposure step: time (s) from release, and in parentheses
|
||||
the number of frames, until brightness stays within ±10% of its final level (median of the run's
|
||||
last 2 s). Dark ×1/16: started 16 times (4 stops) too dark; bright ×16: 16 times too bright. Two
|
||||
runs per cell.*
|
||||
|
||||
| Format | Rate (fps) | Step | Tower 121: time (s) (frames) | Tower 111: time (s) (frames) |
|
||||
|---|---|---|---|---|
|
||||
|
|
@ -333,8 +394,12 @@ Settling typically takes 25–35 frames, so **settling time scales with 1/frame
|
|||
|
||||
### 3.3 Keep rules
|
||||
|
||||
***Table 11.** Candidate keep rules scored on the 24 steps (12 per tower). Correct: the kept frame's
|
||||
brightness is within 10% of the final level.*
|
||||
***Table 11.** Candidate keep rules scored on the 24 recorded steps (12 per tower). A keep rule
|
||||
decides which frame after a gimbal stop is saved. Correct: the kept frame's brightness is within
|
||||
±10% of the final level; early: the rule kept a frame before that. Target band: brightness between
|
||||
45% and 55% of full scale (114.8–140.3 DN), the range the camera's auto-exposure aims for. Steady:
|
||||
brightness changes by at most 2% from one frame to the next. Median time (s) from release to the
|
||||
kept frame, over the correct runs.*
|
||||
|
||||
| Rule | 121 correct / early (runs) | 111 correct / early (runs) | Median time when correct, 121 / 111 (s) |
|
||||
|---|---|---|---|
|
||||
|
|
@ -356,11 +421,14 @@ The profiles currently fix white balance (red 2.87, blue 2.0). With `BalanceWhit
|
|||
|
||||
|
||||
|
||||
***Figure 5.** White-balance ratios relative to their settled values after release, for each
|
||||
format: (a, top) tower 121; (b, bottom) tower 111. Left: ratios forced to ×0.5; right: ×2. Shaded:
|
||||
±5% band.*
|
||||
***Figure 5.** White-balance ratios (the red and blue gains relative to green) divided by their
|
||||
settled values, against time (s) after automatic white balance is switched back on at time 0, for
|
||||
each format: (a, top) tower 121; (b, bottom) tower 111. A value of 1.0 means settled. Left: ratios
|
||||
forced to half (×0.5); right: to double (×2). Shaded: the ±5% band counted as settled.*
|
||||
|
||||
***Table 12.** Time until both ratios are within ±5% of the reference and remain there.*
|
||||
***Table 12.** Settling time after a forced white-balance step: time (s), and in parentheses frames,
|
||||
until both the red and the blue ratio are within ±5% of the reference and remain there. Ranges over
|
||||
the ×0.5 and ×2 steps, two runs each.*
|
||||
|
||||
| Format | Rate (fps) | Tower 121: time (s) (frames) | Tower 111: time (s) (frames) |
|
||||
|---|---|---|---|
|
||||
|
|
@ -375,10 +443,14 @@ exposure.
|
|||
|
||||
|
||||
|
||||
***Figure 6.** Time per saved frame (ms) by step on the identical shared frame, for both computers;
|
||||
4 threads; colour paths at d1.0 e3, today's RGB8 path at d0.8 e4.*
|
||||
***Figure 6.** Time per saved frame (ms), split by step, on the shared frame (tower 121's frame
|
||||
copied bit-exactly to both towers, so only the computer differs). 4 encoder threads. Colour paths at
|
||||
d1.0 e3 (JPEG XL distance 1.0, effort 3); today's RGB8 path at its current setting d0.8 e4.*
|
||||
|
||||
***Table 13.** Duration of each step on the shared frame (4 threads).*
|
||||
***Table 13.** Duration (ms) of each processing step for one frame on the shared frame, 4 encoder
|
||||
threads, median of 3 runs. Delta / IOTA: ratio of the two times (above 1 = the Delta is slower).
|
||||
d1.0 e3: JPEG XL distance 1.0, effort 3. The last two rows differ mainly through the towers' storage
|
||||
write speeds.*
|
||||
|
||||
| Step | IOTA (ms) | Delta (ms) | Delta / IOTA |
|
||||
|---|---|---|---|
|
||||
|
|
@ -390,8 +462,9 @@ exposure.
|
|||
| File write, 2.7 MB colour file | 1 | 4 | |
|
||||
| File write, 17–21 MB lossless raw file | 7–8 | 162–173 | (storage 218 vs 76 MB/s) |
|
||||
|
||||
***Table 14.** Total time per saved frame and resulting file size, shared frame (file sizes are
|
||||
identical on both computers).*
|
||||
***Table 14.** Total time per saved frame (ms: unpack, colour conversion, encode and file write) and
|
||||
file size (MB), on the shared frame, 4 encoder threads. File sizes are identical on both computers.
|
||||
Raw planes and raw mosaic: raw data saved without colour conversion (Section 1.2).*
|
||||
|
||||
| Path | IOTA (ms) | Delta (ms) | File size (MB) |
|
||||
|---|---|---|---|
|
||||
|
|
@ -412,8 +485,10 @@ somewhat larger (3.7–4.0 MB, Appendix B).
|
|||
|
||||
|
||||
|
||||
***Figure 7.** Time per frame (ms) by JPEG XL effort (rows) and thread count (columns), each tower's
|
||||
real frame, distance 1.0: (a, top) tower 121; (b, bottom) tower 111.*
|
||||
***Figure 7.** Time per saved frame (ms) by JPEG XL effort (rows) and thread count (columns), each
|
||||
tower's own real frame, distance 1.0: (a, top) tower 121; (b, bottom) tower 111. Effort: how hard
|
||||
the encoder searches for a compact encoding (1 = fastest; higher = slower, usually smaller).
|
||||
Threads: processor threads one encode may use.*
|
||||
|
||||
Effort 1–4 lie within about 15% of each other, and effort 3 gives the smallest files. On the IOTA,
|
||||
3 threads is slower than 2 at every effort (761 / 582 / 672 / 512 ms for 1–4 threads at effort 1);
|
||||
|
|
@ -423,11 +498,14 @@ the Delta scales normally (1440 / 985 / 886 / 802 ms). Four threads is fastest o
|
|||
|
||||
|
||||
|
||||
***Figure 8.** File size (MB, left) and time per frame (ms, right) versus JPEG XL distance on the
|
||||
shared frame, effort 3, 4 threads, both towers.*
|
||||
***Figure 8.** File size (MB, left) and time per saved frame (ms, right) against JPEG XL distance on
|
||||
the shared frame, effort 3, 4 threads, both towers. Distance is the encoder's quality setting: 0 is
|
||||
lossless, 1.0 is intended to be visually indistinguishable from the original, and higher values give
|
||||
smaller files with more loss.*
|
||||
|
||||
***Table 15.** Size and time per distance on the shared frame (effort 3, 4 threads, file write
|
||||
included; 2 runs per point).*
|
||||
***Table 15.** File size (MB) and time per saved frame (ms) per JPEG XL distance on the shared frame
|
||||
(effort 3, 4 threads, file write included, median of 2 runs). Last column: raw planes (Section 1.2)
|
||||
at the same distance.*
|
||||
|
||||
| Distance | 12-bit colour size (MB) | IOTA time (ms) | Delta time (ms) | Raw 12-bit planes size (MB) |
|
||||
|---|---|---|---|---|
|
||||
|
|
@ -441,11 +519,16 @@ included; 2 runs per point).*
|
|||
|
||||
|
||||
|
||||
***Figure 9.** File size (MB), PSNR (dB) and fine-detail retention against each method's own
|
||||
uncompressed image, versus distance; each tower's real frame.*
|
||||
***Figure 9.** File size (MB), PSNR (dB) and fine-detail retention against JPEG XL distance, each
|
||||
tower's own real frame, each method compared with its own uncompressed image. Fine-detail retention:
|
||||
the amount of fine detail (Laplacian energy) in the decoded image relative to the uncompressed one;
|
||||
1 = all detail kept, below 1 = smoothed.*
|
||||
|
||||
***Table 16.** Quality against the exact lossless 12-bit colour image of the same frame (real frame,
|
||||
≈ 15 dB gain; the towers agree within 0.3 dB).*
|
||||
***Table 16.** Image quality against the exact lossless 12-bit colour image of the same frame (real
|
||||
frame, ≈ 15 dB gain; the towers agree within 0.3 dB). PSNR (dB): peak signal-to-noise ratio, higher
|
||||
means closer to the reference; +6 dB means half the error. SSIM: structural similarity, comparing
|
||||
local patterns of brightness and contrast, 0–1 with 1 = identical. Exact: decodes bit for bit to the
|
||||
original. Colour rows use edge-aware conversion unless stated.*
|
||||
|
||||
| Saved as | File size, 121 / 111 (MB) | PSNR (dB) | SSIM |
|
||||
|---|---|---|---|
|
||||
|
|
@ -469,12 +552,15 @@ size (47 vs 39 dB, equal SSIM); lossless, they take a third of the space of loss
|
|||
|
||||
|
||||
|
||||
***Figure 10.** Positions of the detail crop (red) and shadow crop (blue) used in Figure 11.*
|
||||
***Figure 10.** Tower 121's frame (downscaled), with the positions of the detail crop (red) and
|
||||
shadow crop (blue) used in Figure 11.*
|
||||
|
||||
***Figure 11.** One example per compression method, tower 121's frame (480 × 320-pixel crops at
|
||||
100%): detail crop of spruce needles over bright leaves; its absolute difference from the lossless
|
||||
image (amplified ×8; black = identical); and a darker crop brightened by 2 stops. File size and save
|
||||
time are single runs at 4 threads. The two 8-bit rows are separate captures seconds apart (branches
|
||||
***Figure 11.** One example per compression method, tower 121's frame, as 480 × 320-pixel crops at
|
||||
100% (one image pixel per screen pixel). Detail: spruce needles over bright leaves. Difference: the
|
||||
absolute difference from the lossless image, multiplied by 8 so small errors become visible (black =
|
||||
identical). Shadow: a darker crop brightened by 2 stops, i.e. every pixel value multiplied by 4
|
||||
(each stop doubles), so detail in the dark parts becomes visible. File size (MB) and save time (s)
|
||||
are single runs at 4 threads. The two 8-bit rows are separate captures taken seconds apart (branches
|
||||
move), so they have no difference map.*
|
||||
|
||||
| Method | File size (MB), time (s) | Detail (100%) | Difference (×8) | Shadow (+2 stops) |
|
||||
|
|
@ -500,11 +586,15 @@ Full-size `.jxl` files are kept outside the repository in `tools/camera-bench/ex
|
|||
|
||||
|
||||
|
||||
***Figure 12.** Sensor noise σ (DN, 12-bit) versus signal (DN, 12-bit), green sites, at 0 dB gain
|
||||
and at ≈ 15 dB gain. Dashed: one 8-bit quantisation step (16 DN); dotted: the RMS error added by
|
||||
rounding to 8 bits (4.6 DN).*
|
||||
***Figure 12.** Sensor noise σ (DN, 12-bit scale) against signal level (DN, 12-bit scale), measured
|
||||
on the green pixels, at 0 dB gain and at ≈ 15 dB gain. σ is the standard deviation of the random
|
||||
pixel-to-pixel variation. Dashed: one 8-bit quantisation step, the smallest change an 8-bit value
|
||||
can record (16 DN on the 12-bit scale). Dotted: the RMS error added by rounding to 8 bits (16/√12 =
|
||||
4.6 DN). Noise above these lines means rounding to 8 bits loses nothing measurable.*
|
||||
|
||||
***Table 17.** Noise floor at three signal levels.*
|
||||
***Table 17.** Sensor noise floor σ (DN, 12-bit scale, 0–4095) at three signal levels, green pixels,
|
||||
at 0 dB and ≈ 15 dB gain. Last column: the same noise in 8-bit steps (one step = 16 DN on the 12-bit
|
||||
scale); above 1, the 8-bit values already resolve all the sensor delivers.*
|
||||
|
||||
| Signal (DN, 12-bit) | Noise at 0 dB (DN) | Noise at ≈ 15 dB (DN) | Noise in 8-bit steps, 0 dB / 15 dB |
|
||||
|---|---|---|---|
|
||||
|
|
@ -520,9 +610,10 @@ extrapolation, 12-bit data carries additional information below about 300 DN (0
|
|||
|
||||
|
||||
|
||||
***Figure 13.** The same darker crop brightened by 2 stops, 8-bit (left; the 12-bit value shifted
|
||||
down by 4 bits) versus 12-bit (right): (a, top) ≈ 15 dB gain, 3138 vs 227 distinct green levels;
|
||||
(b, bottom) 0 dB gain. No visible difference.*
|
||||
***Figure 13.** The same darker crop brightened by 2 stops (pixel values multiplied by 4), saved as
|
||||
8-bit (left; the 12-bit value shifted down by 4 bits, i.e. divided by 16) and as 12-bit (right): (a,
|
||||
top) ≈ 15 dB gain, 227 distinct green levels in 8-bit against 3138 in 12-bit; (b, bottom) 0 dB gain.
|
||||
No visible difference: the noise is larger than the extra levels.*
|
||||
|
||||
At 0 dB gain the same settings produce 25–30% smaller files and about 3 dB higher PSNR (less noise
|
||||
to encode): 12-bit colour at d1.0 is 1.9 MB, PSNR 40.0 dB, SSIM 0.987.
|
||||
|
|
@ -531,17 +622,21 @@ to encode): 12-bit colour at d1.0 is 1.9 MB, PSNR 40.0 dB, SSIM 0.987.
|
|||
|
||||
|
||||
|
||||
***Figure 14.** Frames saved per second, back to back, on the shared frame, for one worker × 4
|
||||
threads and two workers × 2 threads.*
|
||||
***Figure 14.** Frames saved per second, back to back (the next save starts as soon as one
|
||||
finishes), on the shared frame, for one save worker with 4 threads and two workers with 2 threads
|
||||
each. A worker saves one frame at a time; two workers save two frames in parallel.*
|
||||
|
||||
|
||||
|
||||
***Figure 15.** Worst time from queueing to file written (s, logarithmic) versus the interval
|
||||
between queued frames (s), synthetic frame. A flat line means saving keeps up; a rising line means a
|
||||
backlog builds.*
|
||||
***Figure 15.** Worst time (s, logarithmic axis) from queueing a frame to its file being written,
|
||||
against the interval (s) between queued frames, synthetic frame. A flat line means saving keeps up;
|
||||
a rising line means a backlog builds.*
|
||||
|
||||
***Table 18.** Throughput and pile-up on the synthetic frame (worst case). "ok" = no backlog after
|
||||
120 s; otherwise frames behind at the end.*
|
||||
***Table 18.** Throughput and pile-up on the synthetic frame, the worst case for the encoder.
|
||||
Workers × threads: frames saved in parallel × encoder threads each. Back to back: frames saved per
|
||||
second when saving continuously. Paced columns: one frame queued every given interval for 120 s;
|
||||
"ok" = no frames waiting at the end, "N behind" = N frames still waiting, "–" = not tested. Peak
|
||||
memory: the most memory (GB) used by the queue and encoders.*
|
||||
|
||||
| Path | Workers × threads | Tower | Back to back (frames/s) | One frame every 2.0 / 1.5 / 1.0 / 0.7 / 0.5 s | Peak memory (GB) |
|
||||
|---|---|---|---|---|---|
|
||||
|
|
@ -565,11 +660,14 @@ fell 27 frames behind in 120 s).
|
|||
|
||||
|
||||
|
||||
***Figure 16.** CPU package temperature (°C) and package power (W) over 10 minutes of 12-bit saving:
|
||||
(a, top) back to back with 2 × 2 workers; (b, bottom) one frame every 2 s with 1 × 4. Solid: tower
|
||||
121; dashed: tower 111.*
|
||||
***Figure 16.** CPU package temperature (°C) and package power (W) over 10 minutes of 12-bit saving;
|
||||
the package is the whole processor chip. (a, top) back to back with 2 workers × 2 threads; (b,
|
||||
bottom) one frame every 2 s with 1 worker × 4 threads. Solid: tower 121; dashed: tower 111.*
|
||||
|
||||
***Table 19.** Temperature, clock, load and power, 10 minutes per condition.*
|
||||
***Table 19.** Processor temperature (mean and maximum, °C), average core clock (GHz), CPU busy (%
|
||||
of time the cores were working), and package power (W, read from the Intel RAPL energy counter [7]),
|
||||
10 minutes per condition after a 5-minute cool-down. Saves per minute in the first and last minute
|
||||
show any slowdown as the processor heats up and lowers its clock.*
|
||||
|
||||
| Load | Tower | Temperature mean / max (°C) | Clock (GHz) | CPU busy (%) | Power (W) | Saves per minute, first → last minute |
|
||||
|---|---|---|---|---|---|---|
|
||||
|
|
@ -588,10 +686,11 @@ back, the Delta sits at an 8 W power limit and does not slow further, whereas th
|
|||
|
||||
|
||||
|
||||
***Figure 17.** Storage per day (GB) at one frame every 2 s, per saving method, each tower's real
|
||||
frame.*
|
||||
***Figure 17.** Storage per day (GB) at one saved frame every 2 s, per saving method, file sizes
|
||||
from each tower's own real frame.*
|
||||
|
||||
***Table 20.** Storage for 24 hours of scanning at three frame intervals.*
|
||||
***Table 20.** Storage (GB/day) for 24 hours of scanning at one saved frame every 2, 5 or 10 s.
|
||||
Per-frame sizes (MB) are from each tower's own real frame; ranges span the two towers.*
|
||||
|
||||
| Saved as | Per frame, 121 / 111 (MB) | Every 2 s (GB/day) | Every 5 s (GB/day) | Every 10 s (GB/day) |
|
||||
|---|---|---|---|---|
|
||||
|
|
@ -609,7 +708,7 @@ The towers have 11–22 GB of free storage, so images must leave the tower conti
|
|||
|
||||
### 4.1 Summary of findings
|
||||
|
||||
***Table 21.** Summary of findings.*
|
||||
***Table 21.** Summary of findings, with the section where each is shown.*
|
||||
|
||||
| Topic | Finding | Section |
|
||||
|---|---|---|
|
||||
|
|
@ -631,7 +730,10 @@ The towers have 11–22 GB of free storage, so images must leave the tower conti
|
|||
Saving runs in the background and overlaps the next movement, so the scan pace is set by the
|
||||
slowest of the camera-side steps and the save rate.
|
||||
|
||||
***Table 22.** Time budget per waypoint.*
|
||||
***Table 22.** Time budget per waypoint (s) for each pixel format. Slew: the gimbal turning to the
|
||||
next waypoint. Mechanical settling: vibration dying down after it stops. Exposure settling:
|
||||
auto-exposure adapting to the new view. Save: runs in the background, overlapping the next waypoint.
|
||||
Source: whether the value was measured or estimated.*
|
||||
|
||||
| Step | 12-bit raw (proposed) | Raw 8-bit | RGB8 (today) | Source |
|
||||
|---|---|---|---|---|
|
||||
|
|
@ -718,11 +820,14 @@ between neighbouring waypoints, which decides the real pace, remains to be measu
|
|||
|
||||
|
||||
|
||||
***Figure B1.** Time per saved frame (ms) by step: (a, top) each tower's own real frame; (b,
|
||||
bottom) the synthetic frame. Effort 3 (raw lossless: effort 1), 4 threads, distance 1.0.*
|
||||
***Figure B1.** Time per saved frame (ms) split by step: (a, top) each tower's own real frame; (b,
|
||||
bottom) the synthetic frame (a computer-generated "dead leaves" texture of overlapping discs with
|
||||
added noise [1], harder to compress than any real scene). Effort 3 (raw lossless: effort 1), 4
|
||||
threads, distance 1.0.*
|
||||
|
||||
***Table B1.** Total time per saved frame (ms), effort 3, 4 threads, distance 1.0 (raw lossless:
|
||||
effort 1).*
|
||||
***Table B1.** Total time per saved frame (ms) on each tower's own real frame and on the synthetic
|
||||
frame, effort 3, 4 threads, distance 1.0 (raw lossless: effort 1). The real frames differ between
|
||||
towers, so Table 14 is the fair comparison.*
|
||||
|
||||
| Path | IOTA, own frame (ms) | IOTA, synthetic (ms) | Delta, own frame (ms) | Delta, synthetic (ms) |
|
||||
|---|---|---|---|---|
|
||||
|
|
|
|||
Loading…
Reference in New Issue