From 4abfd7cbb27aed7b1fb6771604074f30c4fb1741 Mon Sep 17 00:00:00 2001 From: pgdalmeida Date: Mon, 5 Oct 2026 11:39:31 +0200 Subject: [PATCH] Explain the raw mosaic and planes layouts in the 12-bit report, and spell out every term in its figure and table captions --- docs/raw12-pipeline-measurements.md | 249 ++++++++++++++++++++-------- 1 file changed, 177 insertions(+), 72 deletions(-) diff --git a/docs/raw12-pipeline-measurements.md b/docs/raw12-pipeline-measurements.md index 6926b3c..6584e41 100644 --- a/docs/raw12-pipeline-measurements.md +++ b/docs/raw12-pipeline-measurements.md @@ -26,6 +26,8 @@ changing never did. ## Units and abbreviations +*Units, symbols and abbreviations used in this report.* + | Symbol | Meaning | |---|---| | MB, GB, TB | 10⁶, 10⁹, 10¹² bytes | @@ -39,6 +41,7 @@ changing never did. | d, e | JPEG XL distance (quality: 0 = lossless, higher = smaller and lossier) and effort (1–9) | | PSNR | peak signal-to-noise ratio, in dB | | SSIM | structural similarity index, 0–1 (1 = identical) [6] | +| stop | a factor of 2 in light or brightness: brightened by 2 stops = pixel values ×4; ×16 = 4 stops | | IOTA, Delta | LattePanda IOTA (tower 121) and LattePanda 3 Delta (tower 111) | --- @@ -62,7 +65,7 @@ extra bits add. ### 1.2 The camera's pixel formats The sensor sees through a colour-filter mosaic: each pixel measures only red, green or blue, in the -pattern of Figure 1 ("BayerRG": a red pixel first). Producing a colour image means estimating the +pattern of Figure 1a ("BayerRG": a red pixel first). Producing a colour image means estimating the two missing colours at every pixel from its neighbours, called **de-Bayering** (demosaicing). The camera can either send the raw mosaic or de-Bayer it itself. @@ -72,10 +75,12 @@ G B G B G B ... R G R G R G ... ``` -***Figure 1.** The BayerRG colour-filter pattern of the sensor: each pixel records one colour.* +***Figure 1a.** The BayerRG colour-filter pattern of the sensor: each pixel records one colour.* -***Table 1.** Pixel formats considered, at the 1620c's full resolution (5328 × 3040 pixels) and a USB -link cap of 250 MB/s [4].* +***Table 1.** Pixel formats considered, at the 1620c's full resolution (5328 × 3040 pixels) and a +USB link cap of 250 MB/s [4]. Bits per value: the precision of each stored measurement (8 bits = 256 +levels, 10 bits = 1024, 12 bits = 4096). Packed: values stored back to back without padding bits. +Maximum rate: the highest frame rate the camera accepts at this link cap.* | Format | What the camera sends | Bits per value | Bytes per pixel | Frame size (MB) | Maximum rate (fps) | Colour made by | |---|---|---|---|---|---|---| @@ -92,6 +97,40 @@ formats too, so the tower does not need to. The camera also offers grey (`Mono8/ discards colour), `BGR8` (RGB8 in reverse order) and video formats (`YCbCr…`, which reduce colour detail); none of these was considered. +**Saving raw data: mosaic and planes.** Besides converting to colour, the tower can save the raw +12-bit data as received, each pixel keeping the single value measured through its filter. Two +layouts were tested; they hold identical values and differ only in their arrangement in the file: + +- **Raw mosaic:** the sensor data as the camera sends it, a single-channel (grey) image of + 5328 × 3040 values still in the R G / G B pattern. Viewed directly it looks like a grey image with + a fine checkerboard texture. Because colours alternate from pixel to pixel, the encoder sees the + pattern as extremely fine detail: it compresses poorly (20.9 MB lossless), and lossy compression + would mix values of different colours. +- **Raw planes:** the same values reordered into four quarter-size images (2664 × 1520 each), one per + filter position (red, first green, second green, blue), tiled 2 × 2 into one grey image of the + same total size (Figure 1b). Within each tile neighbouring values belong to the same colour and are + similar, so the image is smooth to the encoder: 17.6 MB lossless, and lossy compression works + (distance 0.5: 6.0 MB at a PSNR of 47.2 dB, against 38.8 dB for 12-bit colour of similar size; + Section 3.7). The reordering is exactly reversible, so the mosaic can always be rebuilt. + +``` +Mosaic (5328 × 3040) Planes (2 × 2 tiles of 2664 × 1520) +R G R G R G ... R R R ... | G G G ... +G B G B G B ... → R R R ... | G G G ... +R G R G R G ... ----------+---------- +G B G B G B ... G G G ... | B B B ... + G G G ... | B B B ... +``` + +***Figure 1b.** Rearranging the raw mosaic into four single-colour planes.* + +Neither layout is a viewable colour picture: the reader must de-Bayer it, as the tower does before +saving colour. Their value is that they preserve the measured sensor values, with nothing estimated +and no colour processing applied, which matters only if the detection algorithms need exact sensor +data. Lossless planes take about 6 times the storage of 12-bit colour at distance 1.0 (17.6 MB +against 2.65 MB per frame), so they are an option (recommendation 6, Section 5) and are not part of +the current implementation. + ### 1.3 Earlier work An earlier campaign ([camera-capture-measurements.md](camera-capture-measurements.md)) used blurred, @@ -106,7 +145,9 @@ The implementation built on this report is described in ### 2.1 Hardware and software -***Table 2.** Test systems. The same camera was moved from tower 121 to tower 111.* +***Table 2.** Test systems. The same camera was moved from tower 121 to tower 111, so only the +computer differs between the towers. Storage write speed: sequential writing to the built-in eMMC +flash storage.* | | Tower 121 | Tower 111 | |---|---|---| @@ -126,7 +167,8 @@ and the CPU figures should have a single cause). Afterwards both cameras were re ![The scene](images/raw12/examples/121/thumbs/c12_ea_lossless.jpg) ***Figure 2.** The test scene from tower 121's camera (2 Oct, 15:07; exposure 20 ms, gain ≈ 15 dB): -spruce, birch trunks, leaves and a building; about 13% of the pixels are clipped sky.* +spruce, birch trunks, leaves and a building. About 13% of the pixels are clipped sky: saturated at +the maximum value, so they show pure white without detail.* ### 2.3 Camera control @@ -138,7 +180,9 @@ value the automatic control had reached, after which `ExposureTime`, `Gain` and written directly and take effect from the next frame exposed. No profile, user set or file on the camera is changed. -***Table 3.** Camera settings common to all camera tests.* +***Table 3.** Camera settings common to all camera tests. Names in code font are the camera's +GenICam features [4]. Chunk data: values the camera attaches to each frame, describing the settings +it was taken with.* | Setting | Value | Reason | |---|---|---| @@ -166,7 +210,10 @@ continuously in parallel. ### 2.5 Auto-exposure step test -***Table 4.** One auto-exposure step run.* +***Table 4.** The three phases of one auto-exposure step run. Exposure is the time the sensor +collects light (ms); gain amplifies the sensor signal (dB; +6 dB doubles the brightness, +20 dB +multiplies it by 10). A step of ×16 makes the image 16 times brighter (×1/16: 16 times darker), i.e. +4 stops, where one stop is a factor of 2.* | Phase | Duration (s) | Procedure | |---|---|---| @@ -174,8 +221,10 @@ continuously in parallel. | 2. Force | 2 | `ExposureAuto` = `GainAuto` = `Off`. **Dark:** `ExposureTime` = max(`ExposureAutoMin`, E₀/16), `Gain` = 0 dB. **Bright:** `ExposureTime` = min(20 ms, 16·E₀); the remainder as `Gain` = 20·log₁₀(16·E₀ / exposure) dB, up to `GainAutoMax`. The camera delivers 10–30 frames at the forced settings. | | 3. Release | 15 | Time zero is taken, then `ExposureAuto` = `GainAuto` = `Continuous`; every frame is recorded. | -***Table 5.** Starting conditions of the auto-exposure steps: the first frame after release, over all -24 runs.* +***Table 5.** Starting conditions of the auto-exposure steps: the first frame after release, ranges +over all 24 runs. Before: the settled state before forcing. Dark / bright step: the settings forced +16 times darker / brighter. Brightness: mean pixel value on the 0–255 scale (DN). Clipped share: +percentage of sampled pixels at or above 250 DN, i.e. saturated white without detail.* | | Before (settled) | Start of "dark" step | Start of "bright" step | |---|---|---|---| @@ -202,7 +251,9 @@ camera's target band is 50 ± 5% of full scale, i.e. 114.8–140.3 DN on the 0 ### 2.7 Auto white balance step test -***Table 6.** One auto-white-balance step run.* +***Table 6.** The four phases of one auto-white-balance step run. White-balance ratios are the gains +the camera applies to the red and blue values relative to green (no unit; 1 = unchanged); ×0.5 +halves them and ×2 doubles them.* | Phase | Duration (s) | Procedure | |---|---|---| @@ -218,7 +269,10 @@ Settled = both ratios within ±5% of the reference and remaining there. Each cas A C++ program (`raw12_bench.cpp`), built on each tower against its own libraries, performs the steps the tower program would perform for one saved frame and times each with a monotonic clock. -***Table 7.** Processing steps timed.* +***Table 7.** Processing steps timed for one saved frame. Bilinear conversion estimates each missing +colour as the average of the nearest pixels of that colour; edge-aware conversion interpolates along +edges rather than across them, giving fewer false colour fringes at a slightly higher cost. Distance +and effort are the JPEG XL quality and speed settings (Units and abbreviations).* | Step | Operation | |---|---| @@ -266,7 +320,8 @@ patches; in each, noise σ = median absolute deviation of differences between ho ### 2.12 Campaign log -***Table 8.** Campaign log.* +***Table 8.** Campaign log: date and local time, tower, activity and duration (min) of each test +session.* | Date and time | Tower | Activity | Duration (min) | Notes | |---|---|---|---|---| @@ -289,7 +344,10 @@ patches; in each, noise σ = median absolute deviation of differences between ho ***Figure 3.** Maximum full-resolution frame rate (fps) per pixel format at the 250 MB/s link cap. The same camera was used on both towers.* -***Table 9.** 60 s streams at each format's maximum rate.* +***Table 9.** Results of 60 s streams at each format's maximum rate. Complete: frames received +intact. Incomplete: frames whose data arrived only partly (SDK frame status). Lost: frames missing +from the camera's frame counter. Last row: the same stream while the tower saved 12-bit images +continuously in parallel.* | Format | Frame size (MB) | Maximum rate (fps) | 121: complete / incomplete / lost (frames) | 111: complete / incomplete / lost (frames) | |---|---|---|---|---| @@ -308,15 +366,18 @@ USB 3 controller with a 5G modem, a USB network adapter and a second (idle) came ![Brightness after a step, 121](images/raw12/ae_brightness_121.png) ![Brightness after a step, 111](images/raw12/ae_brightness_111.png) -***Figure 4.** Frame brightness (DN, 0–255) after auto-exposure is released, for each format: (a, -top) tower 121; (b, bottom) tower 111. Left: from under-exposure (×1/16); right: from over-exposure -(×16). Dashed line: settled level.* +***Figure 4.** Frame brightness (DN, 0–255) against time (s) after auto-exposure is switched back on +at time 0, for each format: (a, top) tower 121; (b, bottom) tower 111. Left: starting from +under-exposure, 16 times (4 stops) too dark; right: from over-exposure, 16 times too bright. Dashed +line: settled level, the median brightness over the run's last 2 s.* With exposure at its 20 ms cap, auto-exposure adjusts through gain, by 1–2 dB per frame, and sometimes pauses mid-way (flat segments in Figure 4, up to 2.8 s while still far too bright). -***Table 10.** Time (s) and frames until brightness stays within ±10% of its final level; two runs -each.* +***Table 10.** Settling time after a forced exposure step: time (s) from release, and in parentheses +the number of frames, until brightness stays within ±10% of its final level (median of the run's +last 2 s). Dark ×1/16: started 16 times (4 stops) too dark; bright ×16: 16 times too bright. Two +runs per cell.* | Format | Rate (fps) | Step | Tower 121: time (s) (frames) | Tower 111: time (s) (frames) | |---|---|---|---|---| @@ -333,8 +394,12 @@ Settling typically takes 25–35 frames, so **settling time scales with 1/frame ### 3.3 Keep rules -***Table 11.** Candidate keep rules scored on the 24 steps (12 per tower). Correct: the kept frame's -brightness is within 10% of the final level.* +***Table 11.** Candidate keep rules scored on the 24 recorded steps (12 per tower). A keep rule +decides which frame after a gimbal stop is saved. Correct: the kept frame's brightness is within +±10% of the final level; early: the rule kept a frame before that. Target band: brightness between +45% and 55% of full scale (114.8–140.3 DN), the range the camera's auto-exposure aims for. Steady: +brightness changes by at most 2% from one frame to the next. Median time (s) from release to the +kept frame, over the correct runs.* | Rule | 121 correct / early (runs) | 111 correct / early (runs) | Median time when correct, 121 / 111 (s) | |---|---|---|---| @@ -356,11 +421,14 @@ The profiles currently fix white balance (red 2.87, blue 2.0). With `BalanceWhit ![Auto white balance, 121](images/raw12/awb_121.png) ![Auto white balance, 111](images/raw12/awb_111.png) -***Figure 5.** White-balance ratios relative to their settled values after release, for each -format: (a, top) tower 121; (b, bottom) tower 111. Left: ratios forced to ×0.5; right: ×2. Shaded: -±5% band.* +***Figure 5.** White-balance ratios (the red and blue gains relative to green) divided by their +settled values, against time (s) after automatic white balance is switched back on at time 0, for +each format: (a, top) tower 121; (b, bottom) tower 111. A value of 1.0 means settled. Left: ratios +forced to half (×0.5); right: to double (×2). Shaded: the ±5% band counted as settled.* -***Table 12.** Time until both ratios are within ±5% of the reference and remain there.* +***Table 12.** Settling time after a forced white-balance step: time (s), and in parentheses frames, +until both the red and the blue ratio are within ±5% of the reference and remain there. Ranges over +the ×0.5 and ×2 steps, two runs each.* | Format | Rate (fps) | Tower 121: time (s) (frames) | Tower 111: time (s) (frames) | |---|---|---|---| @@ -375,10 +443,14 @@ exposure. ![Same frame, time per step](images/raw12/stages_common.png) -***Figure 6.** Time per saved frame (ms) by step on the identical shared frame, for both computers; -4 threads; colour paths at d1.0 e3, today's RGB8 path at d0.8 e4.* +***Figure 6.** Time per saved frame (ms), split by step, on the shared frame (tower 121's frame +copied bit-exactly to both towers, so only the computer differs). 4 encoder threads. Colour paths at +d1.0 e3 (JPEG XL distance 1.0, effort 3); today's RGB8 path at its current setting d0.8 e4.* -***Table 13.** Duration of each step on the shared frame (4 threads).* +***Table 13.** Duration (ms) of each processing step for one frame on the shared frame, 4 encoder +threads, median of 3 runs. Delta / IOTA: ratio of the two times (above 1 = the Delta is slower). +d1.0 e3: JPEG XL distance 1.0, effort 3. The last two rows differ mainly through the towers' storage +write speeds.* | Step | IOTA (ms) | Delta (ms) | Delta / IOTA | |---|---|---|---| @@ -390,8 +462,9 @@ exposure. | File write, 2.7 MB colour file | 1 | 4 | | | File write, 17–21 MB lossless raw file | 7–8 | 162–173 | (storage 218 vs 76 MB/s) | -***Table 14.** Total time per saved frame and resulting file size, shared frame (file sizes are -identical on both computers).* +***Table 14.** Total time per saved frame (ms: unpack, colour conversion, encode and file write) and +file size (MB), on the shared frame, 4 encoder threads. File sizes are identical on both computers. +Raw planes and raw mosaic: raw data saved without colour conversion (Section 1.2).* | Path | IOTA (ms) | Delta (ms) | File size (MB) | |---|---|---|---| @@ -412,8 +485,10 @@ somewhat larger (3.7–4.0 MB, Appendix B). ![Effort × threads, 121](images/raw12/effort_threads_121.png) ![Effort × threads, 111](images/raw12/effort_threads_111.png) -***Figure 7.** Time per frame (ms) by JPEG XL effort (rows) and thread count (columns), each tower's -real frame, distance 1.0: (a, top) tower 121; (b, bottom) tower 111.* +***Figure 7.** Time per saved frame (ms) by JPEG XL effort (rows) and thread count (columns), each +tower's own real frame, distance 1.0: (a, top) tower 121; (b, bottom) tower 111. Effort: how hard +the encoder searches for a compact encoding (1 = fastest; higher = slower, usually smaller). +Threads: processor threads one encode may use.* Effort 1–4 lie within about 15% of each other, and effort 3 gives the smallest files. On the IOTA, 3 threads is slower than 2 at every effort (761 / 582 / 672 / 512 ms for 1–4 threads at effort 1); @@ -423,11 +498,14 @@ the Delta scales normally (1440 / 985 / 886 / 802 ms). Four threads is fastest o ![Same frame, size and time by quality](images/raw12/sizes_common.png) -***Figure 8.** File size (MB, left) and time per frame (ms, right) versus JPEG XL distance on the -shared frame, effort 3, 4 threads, both towers.* +***Figure 8.** File size (MB, left) and time per saved frame (ms, right) against JPEG XL distance on +the shared frame, effort 3, 4 threads, both towers. Distance is the encoder's quality setting: 0 is +lossless, 1.0 is intended to be visually indistinguishable from the original, and higher values give +smaller files with more loss.* -***Table 15.** Size and time per distance on the shared frame (effort 3, 4 threads, file write -included; 2 runs per point).* +***Table 15.** File size (MB) and time per saved frame (ms) per JPEG XL distance on the shared frame +(effort 3, 4 threads, file write included, median of 2 runs). Last column: raw planes (Section 1.2) +at the same distance.* | Distance | 12-bit colour size (MB) | IOTA time (ms) | Delta time (ms) | Raw 12-bit planes size (MB) | |---|---|---|---|---| @@ -441,11 +519,16 @@ included; 2 runs per point).* ![Size and quality, real frame](images/raw12/distance_real.png) -***Figure 9.** File size (MB), PSNR (dB) and fine-detail retention against each method's own -uncompressed image, versus distance; each tower's real frame.* +***Figure 9.** File size (MB), PSNR (dB) and fine-detail retention against JPEG XL distance, each +tower's own real frame, each method compared with its own uncompressed image. Fine-detail retention: +the amount of fine detail (Laplacian energy) in the decoded image relative to the uncompressed one; +1 = all detail kept, below 1 = smoothed.* -***Table 16.** Quality against the exact lossless 12-bit colour image of the same frame (real frame, -≈ 15 dB gain; the towers agree within 0.3 dB).* +***Table 16.** Image quality against the exact lossless 12-bit colour image of the same frame (real +frame, ≈ 15 dB gain; the towers agree within 0.3 dB). PSNR (dB): peak signal-to-noise ratio, higher +means closer to the reference; +6 dB means half the error. SSIM: structural similarity, comparing +local patterns of brightness and contrast, 0–1 with 1 = identical. Exact: decodes bit for bit to the +original. Colour rows use edge-aware conversion unless stated.* | Saved as | File size, 121 / 111 (MB) | PSNR (dB) | SSIM | |---|---|---|---| @@ -469,12 +552,15 @@ size (47 vs 39 dB, equal SSIM); lossless, they take a third of the space of loss ![crops](images/raw12/examples/121/crops.jpg) -***Figure 10.** Positions of the detail crop (red) and shadow crop (blue) used in Figure 11.* +***Figure 10.** Tower 121's frame (downscaled), with the positions of the detail crop (red) and +shadow crop (blue) used in Figure 11.* -***Figure 11.** One example per compression method, tower 121's frame (480 × 320-pixel crops at -100%): detail crop of spruce needles over bright leaves; its absolute difference from the lossless -image (amplified ×8; black = identical); and a darker crop brightened by 2 stops. File size and save -time are single runs at 4 threads. The two 8-bit rows are separate captures seconds apart (branches +***Figure 11.** One example per compression method, tower 121's frame, as 480 × 320-pixel crops at +100% (one image pixel per screen pixel). Detail: spruce needles over bright leaves. Difference: the +absolute difference from the lossless image, multiplied by 8 so small errors become visible (black = +identical). Shadow: a darker crop brightened by 2 stops, i.e. every pixel value multiplied by 4 +(each stop doubles), so detail in the dark parts becomes visible. File size (MB) and save time (s) +are single runs at 4 threads. The two 8-bit rows are separate captures taken seconds apart (branches move), so they have no difference map.* | Method | File size (MB), time (s) | Detail (100%) | Difference (×8) | Shadow (+2 stops) | @@ -500,11 +586,15 @@ Full-size `.jxl` files are kept outside the repository in `tools/camera-bench/ex ![Noise vs brightness](images/raw12/noise_vs_signal.png) -***Figure 12.** Sensor noise σ (DN, 12-bit) versus signal (DN, 12-bit), green sites, at 0 dB gain -and at ≈ 15 dB gain. Dashed: one 8-bit quantisation step (16 DN); dotted: the RMS error added by -rounding to 8 bits (4.6 DN).* +***Figure 12.** Sensor noise σ (DN, 12-bit scale) against signal level (DN, 12-bit scale), measured +on the green pixels, at 0 dB gain and at ≈ 15 dB gain. σ is the standard deviation of the random +pixel-to-pixel variation. Dashed: one 8-bit quantisation step, the smallest change an 8-bit value +can record (16 DN on the 12-bit scale). Dotted: the RMS error added by rounding to 8 bits (16/√12 = +4.6 DN). Noise above these lines means rounding to 8 bits loses nothing measurable.* -***Table 17.** Noise floor at three signal levels.* +***Table 17.** Sensor noise floor σ (DN, 12-bit scale, 0–4095) at three signal levels, green pixels, +at 0 dB and ≈ 15 dB gain. Last column: the same noise in 8-bit steps (one step = 16 DN on the 12-bit +scale); above 1, the 8-bit values already resolve all the sensor delivers.* | Signal (DN, 12-bit) | Noise at 0 dB (DN) | Noise at ≈ 15 dB (DN) | Noise in 8-bit steps, 0 dB / 15 dB | |---|---|---|---| @@ -520,9 +610,10 @@ extrapolation, 12-bit data carries additional information below about 300 DN (0 ![8 vs 12 bit at 15 dB](images/raw12/examples/121/shadow_8_vs_12.png) ![8 vs 12 bit at 0 dB](images/raw12/examples/lowgain/shadow_8_vs_12.png) -***Figure 13.** The same darker crop brightened by 2 stops, 8-bit (left; the 12-bit value shifted -down by 4 bits) versus 12-bit (right): (a, top) ≈ 15 dB gain, 3138 vs 227 distinct green levels; -(b, bottom) 0 dB gain. No visible difference.* +***Figure 13.** The same darker crop brightened by 2 stops (pixel values multiplied by 4), saved as +8-bit (left; the 12-bit value shifted down by 4 bits, i.e. divided by 16) and as 12-bit (right): (a, +top) ≈ 15 dB gain, 227 distinct green levels in 8-bit against 3138 in 12-bit; (b, bottom) 0 dB gain. +No visible difference: the noise is larger than the extra levels.* At 0 dB gain the same settings produce 25–30% smaller files and about 3 dB higher PSNR (less noise to encode): 12-bit colour at d1.0 is 1.9 MB, PSNR 40.0 dB, SSIM 0.987. @@ -531,17 +622,21 @@ to encode): 12-bit colour at d1.0 is 1.9 MB, PSNR 40.0 dB, SSIM 0.987. ![Saves per second, same frame](images/raw12/pace_common.png) -***Figure 14.** Frames saved per second, back to back, on the shared frame, for one worker × 4 -threads and two workers × 2 threads.* +***Figure 14.** Frames saved per second, back to back (the next save starts as soon as one +finishes), on the shared frame, for one save worker with 4 threads and two workers with 2 threads +each. A worker saves one frame at a time; two workers save two frames in parallel.* ![Pile-up](images/raw12/pileup.png) -***Figure 15.** Worst time from queueing to file written (s, logarithmic) versus the interval -between queued frames (s), synthetic frame. A flat line means saving keeps up; a rising line means a -backlog builds.* +***Figure 15.** Worst time (s, logarithmic axis) from queueing a frame to its file being written, +against the interval (s) between queued frames, synthetic frame. A flat line means saving keeps up; +a rising line means a backlog builds.* -***Table 18.** Throughput and pile-up on the synthetic frame (worst case). "ok" = no backlog after -120 s; otherwise frames behind at the end.* +***Table 18.** Throughput and pile-up on the synthetic frame, the worst case for the encoder. +Workers × threads: frames saved in parallel × encoder threads each. Back to back: frames saved per +second when saving continuously. Paced columns: one frame queued every given interval for 120 s; +"ok" = no frames waiting at the end, "N behind" = N frames still waiting, "–" = not tested. Peak +memory: the most memory (GB) used by the queue and encoders.* | Path | Workers × threads | Tower | Back to back (frames/s) | One frame every 2.0 / 1.5 / 1.0 / 0.7 / 0.5 s | Peak memory (GB) | |---|---|---|---|---|---| @@ -565,11 +660,14 @@ fell 27 frames behind in 120 s). ![Flat out](images/raw12/heat_max.png) ![Paced](images/raw12/heat_paced.png) -***Figure 16.** CPU package temperature (°C) and package power (W) over 10 minutes of 12-bit saving: -(a, top) back to back with 2 × 2 workers; (b, bottom) one frame every 2 s with 1 × 4. Solid: tower -121; dashed: tower 111.* +***Figure 16.** CPU package temperature (°C) and package power (W) over 10 minutes of 12-bit saving; +the package is the whole processor chip. (a, top) back to back with 2 workers × 2 threads; (b, +bottom) one frame every 2 s with 1 worker × 4 threads. Solid: tower 121; dashed: tower 111.* -***Table 19.** Temperature, clock, load and power, 10 minutes per condition.* +***Table 19.** Processor temperature (mean and maximum, °C), average core clock (GHz), CPU busy (% +of time the cores were working), and package power (W, read from the Intel RAPL energy counter [7]), +10 minutes per condition after a 5-minute cool-down. Saves per minute in the first and last minute +show any slowdown as the processor heats up and lowers its clock.* | Load | Tower | Temperature mean / max (°C) | Clock (GHz) | CPU busy (%) | Power (W) | Saves per minute, first → last minute | |---|---|---|---|---|---|---| @@ -588,10 +686,11 @@ back, the Delta sits at an 8 W power limit and does not slow further, whereas th ![Storage per day](images/raw12/storage.png) -***Figure 17.** Storage per day (GB) at one frame every 2 s, per saving method, each tower's real -frame.* +***Figure 17.** Storage per day (GB) at one saved frame every 2 s, per saving method, file sizes +from each tower's own real frame.* -***Table 20.** Storage for 24 hours of scanning at three frame intervals.* +***Table 20.** Storage (GB/day) for 24 hours of scanning at one saved frame every 2, 5 or 10 s. +Per-frame sizes (MB) are from each tower's own real frame; ranges span the two towers.* | Saved as | Per frame, 121 / 111 (MB) | Every 2 s (GB/day) | Every 5 s (GB/day) | Every 10 s (GB/day) | |---|---|---|---|---| @@ -609,7 +708,7 @@ The towers have 11–22 GB of free storage, so images must leave the tower conti ### 4.1 Summary of findings -***Table 21.** Summary of findings.* +***Table 21.** Summary of findings, with the section where each is shown.* | Topic | Finding | Section | |---|---|---| @@ -631,7 +730,10 @@ The towers have 11–22 GB of free storage, so images must leave the tower conti Saving runs in the background and overlaps the next movement, so the scan pace is set by the slowest of the camera-side steps and the save rate. -***Table 22.** Time budget per waypoint.* +***Table 22.** Time budget per waypoint (s) for each pixel format. Slew: the gimbal turning to the +next waypoint. Mechanical settling: vibration dying down after it stops. Exposure settling: +auto-exposure adapting to the new view. Save: runs in the background, overlapping the next waypoint. +Source: whether the value was measured or estimated.* | Step | 12-bit raw (proposed) | Raw 8-bit | RGB8 (today) | Source | |---|---|---|---|---| @@ -718,11 +820,14 @@ between neighbouring waypoints, which decides the real pace, remains to be measu ![Stage times, real frame](images/raw12/stages_real.png) ![Stage times, synthetic frame](images/raw12/stages_synth.png) -***Figure B1.** Time per saved frame (ms) by step: (a, top) each tower's own real frame; (b, -bottom) the synthetic frame. Effort 3 (raw lossless: effort 1), 4 threads, distance 1.0.* +***Figure B1.** Time per saved frame (ms) split by step: (a, top) each tower's own real frame; (b, +bottom) the synthetic frame (a computer-generated "dead leaves" texture of overlapping discs with +added noise [1], harder to compress than any real scene). Effort 3 (raw lossless: effort 1), 4 +threads, distance 1.0.* -***Table B1.** Total time per saved frame (ms), effort 3, 4 threads, distance 1.0 (raw lossless: -effort 1).* +***Table B1.** Total time per saved frame (ms) on each tower's own real frame and on the synthetic +frame, effort 3, 4 threads, distance 1.0 (raw lossless: effort 1). The real frames differ between +towers, so Table 14 is the fair comparison.* | Path | IOTA, own frame (ms) | IOTA, synthetic (ms) | Delta, own frame (ms) | Delta, synthetic (ms) | |---|---|---|---|---|