From ff89643a4fe30cc81a347842c0eff569d6a14a7b Mon Sep 17 00:00:00 2001 From: Peter Bochmann Date: Thu, 15 Oct 2026 19:02:20 +0200 Subject: [PATCH] Run the thermal camera's NUC while the gimbal moves The camera recalibrated against its shutter every 30 s and sent no frame for about 0.7 s meanwhile: a picture falling into that had no thermal image (0.8 % on tower 111, the 30 s rhythm shown by the new gap logging). The program now switches the camera's timer off and runs the NUC itself (NUC_Perform, which returns when it is done) when a move starts and nuc_interval_s (30) has passed, or after twice that with no move. The scheduler tells the cameras of every move (ICameraSource::moving), and a picture waits up to 1 s for a camera between frames (frameReady) rather than go without it. The camera's own timer goes back on exit; nuc_interval_s = 0 leaves it alone. Measured on tower 111 before writing it: NUC_Interval = 0 gave 70 s without a gap; NUC_Perform ran three NUCs of 657-677 ms; NUC_INPROGRESS never changes. Co-Authored-By: Claude Opus 5.5 --- config/config.example.ini | 1 + docs/configuration.md | 9 ++ docs/known-issues.md | 4 +- docs/tower-installation.md | 8 +- include/fgc/CompositeCameraSource.h | 10 +- include/fgc/Config.h | 4 + include/fgc/ICameraSource.h | 10 ++ include/fgc/TarisCameraSource.h | 11 ++ src/camera/CompositeCameraSource.cpp | 33 +++++- src/camera/TarisCameraSource.cpp | 157 +++++++++++++++++++-------- src/core/CaptureScheduler.cpp | 2 + src/core/Config.cpp | 4 + tests/test_composite.cpp | 44 ++++++++ tests/test_config.cpp | 3 + tests/test_scheduler.cpp | 4 + tests/test_tariscamerasource.cpp | 95 +++++++++++++--- 16 files changed, 337 insertions(+), 62 deletions(-) diff --git a/config/config.example.ini b/config/config.example.ini index b4209f4..bae976f 100644 --- a/config/config.example.ini +++ b/config/config.example.ini @@ -112,6 +112,7 @@ frame_timeout_ms = 5000 ;calib_profile = 0 ; 0: -40..120 C (forest), 1: 0..600 C ;frame_rate_hz = 5 ; the camera free-runs at this, throttled ;max_frame_age_ms = 400 ; older is not kept; below the scan's settle wait +;nuc_interval_s = 30 ; NUC at the start of a move, not mid-picture; 0 = camera's timer ;rotation_deg = 0 ;jxl_effort = 3 ; diff --git a/docs/configuration.md b/docs/configuration.md index 5509f9b..82cade6 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -78,6 +78,7 @@ Parsed and validated by `ConfigLoader` ([src/core/Config.cpp](../src/core/Config | `Camera.` (`taris`) | `calib_profile` | int ≥ 0 | `0` | Calibration profile set on connect: 0 = -40..120 °C at 0.01 K/count, 1 = 0..600 °C at 0.02 K/count. Switch live with `camera calib ` | | `Camera.` (`taris`) | `frame_rate_hz` | 0 < x ≤ 50 | `5` | The rate the camera free-runs at | | `Camera.` (`taris`) | `max_frame_age_ms` | int > 0 | `400` | A frame older than this when the picture is taken is not kept: that picture has no thermal image. Keep it below the scan's settle wait, or the frame can predate standstill (warned at startup) | +| `Camera.` (`taris`) | `nuc_interval_s` | int 0..3600 | `30` | Seconds between NUCs, run by the program at the start of a move so none blanks a picture (after twice this with no move, one runs anyway). `0` leaves the camera's own timer, which now and then takes a picture's frame | | `Camera.` (`taris`) | `library` | string | `libirbgrab_x86_64-linux.so` | The InfraTec SDK, found through the ld cache | | `Camera.` (`taris`, hardware.ini) | `interface` | string | — | The network port the camera is cabled to; only named in the message when it is not found | | `Hardware` (hardware.ini) | `save_workers` | int 1..4 | `1` | Frames saved at once. 1 on the IOTA, 2 on the 3 Delta ([measurements](raw12-pipeline-measurements.md)) | @@ -1080,6 +1081,14 @@ their CamEvents, see [mqtt-api.md](mqtt-api.md#thermal-camera-cam-tir)). the colour camera's `settle_wait_ms`. - **Kept grabbing**: from start to exit, also in manual mode. A microbolometer that is restarted begins with zero frames and a fresh NUC. +- **NUC while the gimbal moves.** The camera recalibrates against its shutter + every 30 s and sends no frame for about 0.7 s meanwhile; a picture falling + into that had no thermal image (0.8 % of them on tower 111). The program + switches the camera's timer off and runs the NUC itself at the start of a + move once `nuc_interval_s` has passed; the move and the settle wait leave + seconds before the next picture. A picture that still finds the camera + between frames waits up to 1 s for the next one. On exit the camera's timer + is put back. - **Split between the files** like any camera: `driver`, `id` and `interface` in hardware.ini (with `tir` in `[Hardware] cameras`), the rest in config.ini. `save_queue_max` counts frames, so raise it by 2. diff --git a/docs/known-issues.md b/docs/known-issues.md index b0d534c..9160fdc 100644 --- a/docs/known-issues.md +++ b/docs/known-issues.md @@ -169,7 +169,9 @@ scripts for this are worth keeping; note that wrapping them in `timeout` SIGTERM - ~~Support a second camera backend.~~ Done for the thermal camera (`driver = taris`, `TarisCameraSource`, behind `CompositeCameraSource`), tested with the TarisIR mini on tower 111 on 2026-10-14 (docs/tower-installation.md, Phase 2b). Not yet tested: a pulled camera cable during a scan - (the reconnect is only unit-tested), and a long unattended run. A NIR unit would be a third driver + (the reconnect is only unit-tested). The 0.8 % of pictures without a thermal frame in the first long run + were the camera's own NUC every 30 s (a 0.7 s pause in its stream); since 2026-10-15 the program runs the + NUC at the start of a move instead. A NIR unit would be a third driver behind the same key. - The InfraTec SDK costs about a quarter of one core on the 3 Delta even between pictures: three threads it starts in `dll_init` and one per connection poll the raw socket in `nanosleep` loops (measured on diff --git a/docs/tower-installation.md b/docs/tower-installation.md index eebca2f..bc51c5b 100644 --- a/docs/tower-installation.md +++ b/docs/tower-installation.md @@ -950,7 +950,13 @@ Measured on tower 111 (2026-10-14, SDK 4.3.44, SN 2017744): - A profile switch takes about 3 s; frames keep the old scale until `Calib_ChangeInProgress` reaches 0, and the program takes none meanwhile. - The camera's frame counter (`ImageNum`) is always 0; `MissedPackets` was - always 0 on the direct link. + always 0 on the direct link. It delivers about 50 fps whatever `Framerate_Hz` + says; the program polls it at 10 Hz. +- NUC (2026-10-15): the camera runs one every 30 s (`NUC_Interval`) and sends no + frame for 650-760 ms meanwhile. `NUC_Interval = 0` stops that; `NUC_Perform` + (1-point) runs one and returns when it is done, after about 0.67 s. + `NUC_INPROGRESS` stays 0 throughout, also for those. The program now runs the + NUC itself at the start of a move (`nuc_interval_s`). - The SDK switches the process's C locale to `LC_NUMERIC=de_DE` when it is loaded. The program puts it back after every SDK call and writes its JSON without the C library's number formatting. diff --git a/include/fgc/CompositeCameraSource.h b/include/fgc/CompositeCameraSource.h index f015eae..2972cbb 100644 --- a/include/fgc/CompositeCameraSource.h +++ b/include/fgc/CompositeCameraSource.h @@ -22,6 +22,10 @@ namespace fgc { // Every frame of one trigger carries the leader's timestamp, so the pair is // RGB/.jxl and TIR/.jxl with the same in both CamEvents. // +// A follower that is briefly without a fresh frame (frameReady() false: the +// thermal camera's NUC) holds the picture back for up to `follower_wait_ms`, +// so the pair stays whole; past that the leader goes on alone. +// // Indices are global: child k's camera i is offset(k) + i. Calls about one // camera go to the child that has it. // @@ -33,13 +37,15 @@ namespace fgc { class CompositeCameraSource : public ICameraSource { public: explicit CompositeCameraSource(std::vector> children, - std::vector counts = {}); + std::vector counts = {}, int follower_wait_ms = 1000); void open() override; void close() override; void start() override; void stop() override; bool trigger() override; + void moving() override; + bool frameReady() override; bool setProfile(int camera_index, const CameraProfile& profile) override; void setFrameCallback(FrameCallback cb) override; uint64_t frameSeq() override; @@ -67,6 +73,8 @@ private: FrameCallback callback_; bool in_trigger_ = false; long long shot_ts_ = 0; // the leader's frame time, during a trigger + int follower_wait_ms_; + long long waiting_since_ = 0; // steady ms a picture began waiting for a follower }; } // namespace fgc diff --git a/include/fgc/Config.h b/include/fgc/Config.h index 3b49aff..3f106a8 100644 --- a/include/fgc/Config.h +++ b/include/fgc/Config.h @@ -223,6 +223,10 @@ struct CameraSpec { // settle wait, or the frame may have been taken while the gimbal still moved. int max_frame_age_ms = 400; std::string library = "libirbgrab_x86_64-linux.so"; // the SDK, found through the ld cache + // Seconds between NUCs the program runs itself, each at the start of a move + // so it never blanks a picture; 0 leaves the camera's own timer, which fires + // mid-picture now and then (TarisCameraSource.h). + int nuc_interval_s = 30; // Mounting (hardware.ini): the camera's network port, for diagnostics only. std::string interface; diff --git a/include/fgc/ICameraSource.h b/include/fgc/ICameraSource.h index e750c71..b7192a3 100644 --- a/include/fgc/ICameraSource.h +++ b/include/fgc/ICameraSource.h @@ -86,6 +86,16 @@ public: // yet, and the caller simply retries. virtual bool trigger() = 0; + // The gimbal has started moving: nothing will be captured until it stands + // still again and has settled. A camera with housekeeping that blanks its + // stream - the thermal camera's NUC - does it now. Default: nothing. + virtual void moving() {} + + // Would trigger() hand over a fresh frame right now? False while a camera is + // briefly between frames (a NUC), so a picture can wait a moment for it + // rather than go without. Default: always. + virtual bool frameReady() { return true; } + // Apply a profile to one camera, identified by its index in the order the // source opened them (the same index Frame::cam_id carries). The backend // loads the profile's settings XML through its SDK - see CameraProfile.h. diff --git a/include/fgc/TarisCameraSource.h b/include/fgc/TarisCameraSource.h index 6bd954c..d735f0e 100644 --- a/include/fgc/TarisCameraSource.h +++ b/include/fgc/TarisCameraSource.h @@ -25,6 +25,13 @@ namespace fgc { // A camera that is missing, unlicensed or unreachable is not a startup error. // The source reports why in offline(), keeps retrying, and the scan goes on // without it. +// +// NUC: the camera recalibrates itself against its shutter every 30 s and sends +// no frame for about 0.7 s while it does (measured on tower 111). A picture +// that falls into that gets no thermal frame. So the program switches the +// camera's timer off and runs the NUC itself, at the start of a move - the +// move and the settle wait leave seconds before the next picture - once +// nuc_interval has passed, or after twice that without any move (manual mode). class TarisCameraSource : public ICameraSource { public: struct Options { @@ -38,6 +45,7 @@ public: int retry_min_ms = 2000; // between connection attempts, doubling... int retry_max_ms = 30000;// ...up to this int calib_timeout_ms = 15000;// a profile switch takes about 3 s + int nuc_interval_ms = 30000;// 0: leave the camera's own NUC timer }; static Options fromSpec(const CameraSpec& spec); @@ -54,6 +62,9 @@ public: void start() override; void stop() override; bool trigger() override; + // Runs a due NUC now (on the worker), while the gimbal moves. + void moving() override; + bool frameReady() override; void setFrameCallback(FrameCallback cb) override; int cameraCount() const override { return 1; } std::vector deviceInfo() override; diff --git a/src/camera/CompositeCameraSource.cpp b/src/camera/CompositeCameraSource.cpp index 5dd52ba..2953370 100644 --- a/src/camera/CompositeCameraSource.cpp +++ b/src/camera/CompositeCameraSource.cpp @@ -2,13 +2,22 @@ #include "fgc/Logger.h" +#include #include namespace fgc { +namespace { +long long steadyMs() { + using namespace std::chrono; + return duration_cast(steady_clock::now().time_since_epoch()).count(); +} +} // namespace + CompositeCameraSource::CompositeCameraSource(std::vector> children, - std::vector counts) - : children_(std::move(children)), counts_(std::move(counts)) { + std::vector counts, int follower_wait_ms) + : children_(std::move(children)), counts_(std::move(counts)), + follower_wait_ms_(follower_wait_ms) { if (children_.empty()) throw std::invalid_argument("CompositeCameraSource needs a camera"); if (counts_.empty()) for (const auto& c : children_) counts_.push_back(c->cameraCount()); @@ -57,6 +66,16 @@ void CompositeCameraSource::stop() { } bool CompositeCameraSource::trigger() { + bool followers_ready = true; + for (size_t k = 1; k < children_.size(); ++k) followers_ready &= children_[k]->frameReady(); + if (!followers_ready) { + const long long now = steadyMs(); + if (waiting_since_ == 0) waiting_since_ = now; + if (now - waiting_since_ < follower_wait_ms_) return false; // asked again next tick + LOG_DEBUG << "camera: a follower had no fresh frame for " << now - waiting_since_ + << " ms; taking the picture without it"; + } + waiting_since_ = 0; in_trigger_ = true; shot_ts_ = 0; const bool taken = children_[0]->trigger(); @@ -66,6 +85,16 @@ bool CompositeCameraSource::trigger() { return taken; } +void CompositeCameraSource::moving() { + for (auto& c : children_) c->moving(); +} + +bool CompositeCameraSource::frameReady() { + for (auto& c : children_) + if (!c->frameReady()) return false; + return true; +} + void CompositeCameraSource::setFrameCallback(FrameCallback cb) { callback_ = std::move(cb); // The leader's frames pass through untouched (they can be 24 MB); its diff --git a/src/camera/TarisCameraSource.cpp b/src/camera/TarisCameraSource.cpp index 791c840..1e08e8f 100644 --- a/src/camera/TarisCameraSource.cpp +++ b/src/camera/TarisCameraSource.cpp @@ -87,6 +87,7 @@ TarisCameraSource::Options TarisCameraSource::fromSpec(const CameraSpec& spec) { o.max_frame_age_ms = spec.max_frame_age_ms; o.library = spec.library; o.interface = spec.interface; + o.nuc_interval_ms = spec.nuc_interval_s * 1000; return o; } @@ -137,10 +138,10 @@ struct TarisCameraSource::Impl { double last_cam_ts = -1.0; // its MemInfo time, ms long long polls_since_frame = 0; std::map codes_since_frame; // getdata answers other than a new frame - bool nuc_in_gap = false; // NUC_INPROGRESS seen set meanwhile - // NUC_INPROGRESS, read on every poll; -1 = the camera does not answer it. - int nuc_state = 0; - long long nuc_started_ms = 0; + // NUC run by the program (Options::nuc_interval_ms). + int nuc_interval_found = -1; // the camera's own timer, put back on close + bool nuc_ours = false; // the camera's timer is off and we run it + long long last_nuc_ms = 0; // steady // For the summary: long long sum_frames = 0, sum_gaps = 0, sum_nucs = 0, sum_max_gap = 0; double sum_cam_span = 0.0; // camera clock covered by sum_frames @@ -162,8 +163,9 @@ struct TarisCameraSource::Impl { bool cal_ok = false; bool reset_wanted = false; uint64_t connects = 0; - bool nuc_now = false; // NUC_INPROGRESS at the last poll - long long nuc_now_since = 0; // steady ms it went on + bool nuc_now = false; // a NUC of ours is running + long long nuc_now_since = 0; // steady ms it began + bool move_started = false; // moving() since the worker last looked long long missed_window = 0; // misses since the last summary std::atomic running{false}; @@ -196,7 +198,9 @@ struct TarisCameraSource::Impl { void pause(long long ms) { std::unique_lock l(m); cv.wait_for(l, std::chrono::milliseconds(ms), - [this] { return !running.load() || cal_wanted >= 0 || reset_wanted; }); + [this] { + return !running.load() || cal_wanted >= 0 || reset_wanted || move_started; + }); } // ---- parameter helpers ------------------------------------------------- @@ -427,13 +431,7 @@ struct TarisCameraSource::Impl { if (!model.empty()) info.model = model; ++connects; } - int32_t nuc_active = -1, nuc_interval = -1; - getInt32(param::kNucActive, nuc_active); - getInt32(param::kNucInterval, nuc_interval); - LOG_INFO << "thermal camera: NUC " << (nuc_active == 1 ? "on" : nuc_active == 0 ? "off" : "?") - << ", automatic every " - << (nuc_interval > 0 ? std::to_string(nuc_interval) + " s" - : nuc_interval == 0 ? std::string("- (off)") : "?"); + takeOverNuc(); kpc = 0.0; scale_tries = 0; @@ -441,7 +439,6 @@ struct TarisCameraSource::Impl { last_cam_ts = -1.0; polls_since_frame = 0; codes_since_frame.clear(); - nuc_in_gap = false; zeros_logged = false; last_frame_steady = steadyMs(); // grace for the first frame return true; @@ -451,7 +448,6 @@ struct TarisCameraSource::Impl { // Take the newest frame from the SDK, if there is one. void poll() { ++polls_since_frame; - watchNuc(); // As before every getdata in the vendor's examples: without it the SDK // now and then answers 0x80000001 instead of a frame. setInt32(param::kPollingEnable, 1); @@ -584,32 +580,86 @@ struct TarisCameraSource::Impl { } } - // NUC_INPROGRESS on every poll: when the camera corrects itself, and for how - // long - the prime suspect for a picture without a thermal frame. - void watchNuc() { - if (nuc_state < 0) return; - int32_t v = 0; - if (!getInt32(param::kNucInProgress, v)) { - LOG_INFO << "thermal camera: NUC_INPROGRESS not readable; NUC timing not logged"; - nuc_state = -1; + // ---- NUC --------------------------------------------------------------- + // Switch the camera's own NUC timer off: from now on the program runs the + // NUC, while the gimbal moves. The timer's value goes back on close. (Read + // back right after a set, the camera still reports the old value.) + void takeOverNuc() { + nuc_ours = false; + if (o.nuc_interval_ms <= 0) { + LOG_INFO << "thermal camera: NUC left to the camera's own timer"; return; } - const long long now = steadyMs(); - const int on = v != 0; - if (on) nuc_in_gap = true; - if (on && !nuc_state) { - nuc_started_ms = now; - ++sum_nucs; - LOG_INFO << "thermal camera: NUC started"; - } else if (!on && nuc_state) { - LOG_INFO << "thermal camera: NUC finished after " << now - nuc_started_ms << " ms"; + int32_t found = -1; + if (!getInt32(param::kNucInterval, found)) { + LOG_WARN << "thermal camera: cannot read NUC_Interval; NUC left to the camera"; + return; } - if (on != nuc_state) { + if (nuc_interval_found < 0) nuc_interval_found = found; + const Ret r = setInt32(param::kNucInterval, 0); + if (!ok(r)) { + LOG_WARN << "thermal camera: cannot switch the camera's NUC timer off (" << describe(r) + << "); a picture may now and then fall into a NUC"; + return; + } + nuc_ours = true; + last_nuc_ms = steadyMs() - o.nuc_interval_ms; // the first move runs one + LOG_INFO << "thermal camera: NUC every " << o.nuc_interval_ms / 1000 + << " s at the start of a move (camera's own timer: " << found << " s, now off)"; + } + + // Put the camera's timer back, so it corrects itself without us. + void handBackNuc() { + if (!nuc_ours || nuc_interval_found <= 0) return; + const Ret r = setInt32(param::kNucInterval, nuc_interval_found); + LOG_INFO << "thermal camera: NUC timer back to " << nuc_interval_found << " s (" + << describe(r) << ")"; + nuc_ours = false; + } + + // A NUC when one is due and the gimbal has just started moving, or when it + // is overdue (twice the interval) with no move to wait for. + void serviceNuc() { + bool moved; + { std::lock_guard l(m); - nuc_now = on; - nuc_now_since = now; + moved = move_started; + move_started = false; // always: pause() wakes on it } - nuc_state = on; + if (!nuc_ours) return; + const long long since = steadyMs() - last_nuc_ms; + if (moved ? since < o.nuc_interval_ms : since < 2LL * o.nuc_interval_ms) return; + runNuc(moved ? "move" : "no move for a while"); + } + + void runNuc(const char* why) { + const long long start = steadyMs(); + { + std::lock_guard l(m); + nuc_now = true; + nuc_now_since = start; + } + Int2 nuc{kNucMode1Point, 0}; + // Returns once the NUC is done, about 0.7 s. + const Ret r = api->dev_setparam(dev, param::kNucPerform, &nuc, kType2Int32); + const long long done = steadyMs(); + { + std::lock_guard l(m); + nuc_now = false; + // A frame from before the NUC is no longer fresh for a picture. + delivered_seq = latest_seq; + } + last_nuc_ms = done; + ++sum_nucs; + // A frame is due as soon as the NUC is over; do not count it as a stall. + last_frame_steady = std::max(last_frame_steady, done); + if (!ok(r)) { + LOG_WARN << "thermal camera: NUC failed (" << describe(r) + << "); handing NUC back to the camera's own timer"; + handBackNuc(); + return; + } + LOG_DEBUG << "thermal camera: NUC (" << why << ") took " << done - start << " ms"; } // A new frame: log the pause before it if it was long, and count it. @@ -620,7 +670,9 @@ struct TarisCameraSource::Impl { ++sum_frames; if (cam_gap > 0.0) sum_cam_span += cam_gap; sum_max_gap = std::max(sum_max_gap, host_gap); - if (host_gap > kGapLogMs || cam_gap > kGapLogMs) { + // A pause of our own making is no news. + const bool ours = now - last_nuc_ms < 2000; + if ((host_gap > kGapLogMs || cam_gap > kGapLogMs) && !ours) { ++sum_gaps; std::string codes; for (const auto& [code, n] : codes_since_frame) @@ -629,14 +681,12 @@ struct TarisCameraSource::Impl { " x" + std::to_string(n); LOG_INFO << "thermal camera: frame gap " << host_gap << " ms (camera clock " << static_cast(cam_gap) << " ms), " << polls_since_frame - << " polls: " << (codes.empty() ? "-" : codes) << "; NUC " - << (nuc_state < 0 ? "unknown" : nuc_in_gap ? "in progress" : "not seen"); + << " polls: " << (codes.empty() ? "-" : codes); } } last_cam_ts = cam_ts; polls_since_frame = 0; codes_since_frame.clear(); - nuc_in_gap = nuc_state > 0; if (now - sum_since_ms >= kSummaryEveryMs) summary(now); } @@ -652,7 +702,8 @@ struct TarisCameraSource::Impl { LOG_INFO << "thermal camera, last " << (now - sum_since_ms) / 60000 << " min: " << sum_frames << " frames (" << std::lround(fps * 10) / 10.0 << " fps by the camera's clock), longest gap " << sum_max_gap << " ms, " - << sum_gaps << " over " << kGapLogMs << " ms, " << sum_nucs << " NUC, " + << sum_gaps << " unexplained over " << kGapLogMs << " ms, " << sum_nucs + << " NUC" << (nuc_ours ? " (ours)" : "") << ", " << missed << " picture(s) without a thermal frame"; } sum_frames = sum_gaps = sum_nucs = sum_max_gap = 0; @@ -709,6 +760,7 @@ struct TarisCameraSource::Impl { // retry pause from pausing. std::lock_guard l(m); reset_wanted = false; + move_started = false; // nor may a move cut the retry pause short } std::string why; bool up; @@ -729,6 +781,7 @@ struct TarisCameraSource::Impl { } serviceRequests(); if (!connected) continue; + serviceNuc(); poll(); const long long now = steadyMs(); @@ -746,6 +799,7 @@ struct TarisCameraSource::Impl { pause(poll_ms); } LocaleGuard keep; + if (connected) handBackNuc(); disconnect(); if (sdk_init) api->dll_uninit(); sdk_init = false; @@ -815,6 +869,23 @@ void TarisCameraSource::stop() { impl_->gate = false; } void TarisCameraSource::setFrameCallback(FrameCallback cb) { impl_->callback = std::move(cb); } +void TarisCameraSource::moving() { + { + std::lock_guard l(impl_->m); + impl_->move_started = true; + } + impl_->cv.notify_all(); // the worker may be pausing between polls +} + +bool TarisCameraSource::frameReady() { + Impl& d = *impl_; + std::lock_guard l(d.m); + if (!d.running || !d.why_offline.empty()) return true; // nothing to wait for + if (d.nuc_now) return false; + return d.latest_seq > d.delivered_seq && + unixMs() - d.latest.timestamp_ms <= d.o.max_frame_age_ms; +} + bool TarisCameraSource::trigger() { Impl& d = *impl_; if (!d.gate || !d.callback) return false; @@ -838,7 +909,7 @@ bool TarisCameraSource::trigger() { (new_frame ? "" : ", none since the last picture") + (d.nuc_now ? ", NUC in progress for " + std::to_string(now - d.nuc_now_since) + " ms" - : ", no NUC running"); + : ""); LOG_WARN << "thermal camera: no thermal frame for this picture: " << why; return false; } diff --git a/src/core/CaptureScheduler.cpp b/src/core/CaptureScheduler.cpp index 41661bc..774cd06 100644 --- a/src/core/CaptureScheduler.cpp +++ b/src/core/CaptureScheduler.cpp @@ -110,6 +110,8 @@ void CaptureScheduler::sendMove() { // form ("ERR axis not present") and moves nothing, so address yaw alone. motor_.sendCommand(moveCommand(yaw_target_, pitch_present_ ? std::optional(pitch_target_) : std::nullopt)); + // The time the move and the settle wait take is free for the cameras. + camera_.moving(); } bool CaptureScheduler::settledAt(const MotorTelemetry& t) const { diff --git a/src/core/Config.cpp b/src/core/Config.cpp index 8dd4b62..96a12bd 100644 --- a/src/core/Config.cpp +++ b/src/core/Config.cpp @@ -393,6 +393,7 @@ AppConfig ConfigLoader::fromMap(const std::map& kv) { c.frame_rate_hz = getDouble(kv, pre + "frame_rate_hz", c.frame_rate_hz); c.max_frame_age_ms = getInt(kv, pre + "max_frame_age_ms", c.max_frame_age_ms); c.library = get(kv, pre + "library", c.library); + c.nuc_interval_s = getInt(kv, pre + "nuc_interval_s", c.nuc_interval_s); c.interface = get(kv, pre + "interface"); // Per-profile timing: Camera...settle_wait_ms @@ -434,6 +435,9 @@ AppConfig ConfigLoader::fromMap(const std::map& kv) { throw std::runtime_error("Camera." + name + ".frame_rate_hz must be in (0, 50]"); if (c.max_frame_age_ms <= 0) throw std::runtime_error("Camera." + name + ".max_frame_age_ms must be > 0"); + if (c.nuc_interval_s < 0 || c.nuc_interval_s > 3600) + throw std::runtime_error("Camera." + name + ".nuc_interval_s must be in 0..3600 " + "(0 = the camera's own timer)"); if (c.library.empty()) throw std::runtime_error("Camera." + name + ".library must not be empty"); } diff --git a/tests/test_composite.cpp b/tests/test_composite.cpp index e307430..be5c51b 100644 --- a/tests/test_composite.cpp +++ b/tests/test_composite.cpp @@ -2,7 +2,9 @@ #include +#include #include +#include #include using namespace fgc; @@ -20,6 +22,8 @@ struct FakeSource : ICameraSource { bool throws_on_open = false; int triggers = 0; bool started = false; + bool frame_ready = true; // frameReady() + int moves = 0; std::string why_offline; FrameCallback cb; @@ -32,6 +36,8 @@ struct FakeSource : ICameraSource { void close() override {} void start() override { started = true; } void stop() override { started = false; } + void moving() override { ++moves; } + bool frameReady() override { return frame_ready; } bool trigger() override { ++triggers; if (!ready) return false; @@ -172,6 +178,44 @@ TEST_CASE("the indices are the configured ones, also before the leader has opene CHECK_THROWS(CompositeCameraSource(std::vector>{}, {})); } +TEST_CASE("a picture waits a moment for a follower between frames, then goes without it") { + auto c = std::make_unique(1, 1000); + auto t = std::make_unique(1, 5000); + FakeSource* colour = c.get(); + FakeSource* thermal = t.get(); + std::vector> v; + v.push_back(std::move(c)); + v.push_back(std::move(t)); + CompositeCameraSource src(std::move(v), {1, 1}, 60); + std::vector frames; + src.setFrameCallback([&](const Frame& f) { frames.push_back(f); }); + + // The thermal camera is in its NUC: the picture is held, the leader not asked. + thermal->frame_ready = false; + CHECK_FALSE(src.trigger()); + CHECK(colour->triggers == 0); + CHECK_FALSE(src.frameReady()); + // It comes back within the wait: one whole pair. + thermal->frame_ready = true; + REQUIRE(src.trigger()); + CHECK(frames.size() == 2); + + // It does not come back: after the wait the colour picture goes alone. + frames.clear(); + thermal->frame_ready = false; + CHECK_FALSE(src.trigger()); + std::this_thread::sleep_for(std::chrono::milliseconds(80)); + REQUIRE(src.trigger()); + CHECK(colour->triggers == 2); +} + +TEST_CASE("every source hears that the gimbal moves") { + Rig r; + r.src->moving(); + CHECK(r.colour->moves == 1); + CHECK(r.thermal->moves == 1); +} + TEST_CASE("start and stop reach every source") { Rig r; r.src->start(); diff --git a/tests/test_config.cpp b/tests/test_config.cpp index 249d17d..d8b8b8a 100644 --- a/tests/test_config.cpp +++ b/tests/test_config.cpp @@ -312,6 +312,7 @@ TEST_CASE("A thermal camera needs no settings XML and is always lossless") { CHECK(tir.max_frame_age_ms == 400); CHECK(tir.library == "libirbgrab_x86_64-linux.so"); CHECK(tir.interface == "enp1s0"); + CHECK(tir.nuc_interval_s == 30); // [Camera] jxl_distance is for colour cameras; counts default to lossless. CHECK(tir.jxl_distance == 0.0); CHECK(c.camera.cameras[0].jxl_distance == doctest::Approx(0.8)); @@ -368,6 +369,8 @@ TEST_CASE("Thermal camera blocks reject what cannot work") { // An empty value means the default, as for every key. CHECK(ConfigLoader::fromMap(base({{"Camera.tir.library", ""}})).camera.cameras[1].library == "libirbgrab_x86_64-linux.so"); + CHECK_THROWS(ConfigLoader::fromMap(base({{"Camera.tir.nuc_interval_s", "-1"}}))); + CHECK_NOTHROW(ConfigLoader::fromMap(base({{"Camera.tir.nuc_interval_s", "0"}}))); // Two cameras, one output folder: their pictures would overwrite each other. CHECK_THROWS(ConfigLoader::fromMap(base({{"Camera.rgb.label", "X"}, {"Camera.tir.label", "X"}}))); // A disabled thermal camera needs no serial, like any other. diff --git a/tests/test_scheduler.cpp b/tests/test_scheduler.cpp index a18aea6..fa28d4c 100644 --- a/tests/test_scheduler.cpp +++ b/tests/test_scheduler.cpp @@ -26,6 +26,7 @@ struct FakeMotor : IMotorController { struct FakeCamera : ICameraSource { int triggers = 0; + int moves = 0; bool ok = true; // A stream of frames for the keep rule; empty = a camera with no frame data. std::vector stream; @@ -44,6 +45,7 @@ struct FakeCamera : ICameraSource { void start() override {} void stop() override {} bool trigger() override { ++triggers; return ok; } + void moving() override { ++moves; } void setFrameCallback(FrameCallback) override {} int cameraCount() const override { return 1; } }; @@ -85,6 +87,7 @@ TEST_CASE("CaptureScheduler sweeps the scan grid with MOVE + settle + trigger") CHECK(motor.cmds[0] == "MOVE 0,0"); CHECK(sch.moving()); CHECK(cam.triggers == 0); // not yet settled + CHECK(cam.moves == 1); // the cameras hear of every move (the thermal NUC) // At rest early: the frame waits for the interval, counted from the mode start. settleAt(motor, 0, 0); @@ -103,6 +106,7 @@ TEST_CASE("CaptureScheduler sweeps the scan grid with MOVE + settle + trigger") clock = 1010; sch.tick(); CHECK(motor.cmds.back() == "MOVE 900,0"); + CHECK(cam.moves == 2); CHECK(sch.overruns() == 0); } diff --git a/tests/test_tariscamerasource.cpp b/tests/test_tariscamerasource.cpp index f5491fe..bf9988a 100644 --- a/tests/test_tariscamerasource.cpp +++ b/tests/test_tariscamerasource.cpp @@ -35,7 +35,9 @@ struct FakeSdk { uint32_t w = 8, h = 4; Ret no_frame = kErrNotFound; // what getdata says before any frame std::atomic frozen{false}; // stop producing frames (stall) - std::atomic nuc{0}; // NUC_INPROGRESS + std::atomic nuc_interval{30}; // the camera's own NUC timer, s + std::atomic nucs{0}; // NUC_Perform calls + std::atomic nuc_mode{0}; void push(uint16_t v, uint16_t hot = 0) { std::lock_guard l(m); @@ -99,9 +101,7 @@ Ret fGetParam(Handle, uint32_t what, void* data, uint32_t type) { return kSuccess; } if (what == param::kCameraTempHousing && type == kTypeSingle) { *static_cast(data) = 308.15f; return kSuccess; } - if (what == param::kNucInProgress && type == kTypeInt32) { *static_cast(data) = g->nuc; return kSuccess; } - if (what == param::kNucActive && type == kTypeInt32) { *static_cast(data) = 1; return kSuccess; } - if (what == param::kNucInterval && type == kTypeInt32) { *static_cast(data) = 0; return kSuccess; } + if (what == param::kNucInterval && type == kTypeInt32) { *static_cast(data) = g->nuc_interval; return kSuccess; } return kErrNotSupported; } Ret fSetParam(Handle, uint32_t what, void* data, uint32_t type) { @@ -113,6 +113,17 @@ Ret fSetParam(Handle, uint32_t what, void* data, uint32_t type) { } if (what == param::kFramerateHz && type == kTypeSingle) { g->rate = *static_cast(data); return kSuccess; } if (what == param::kPollingEnable) return kSuccess; + if (what == param::kNucInterval && type == kTypeInt32) { + g->nuc_interval = *static_cast(data); + return kSuccess; + } + if (what == param::kNucPerform && type == kType2Int32) { + // As the camera: the call returns when the NUC is done. + g->nuc_mode = static_cast(data)->value1; + ++g->nucs; + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + return kSuccess; + } return kErrNotSupported; } // A memory object is a copy of the newest frame. @@ -372,23 +383,79 @@ TEST_CASE("a camera that goes quiet is reconnected") { CHECK(t.sdk.connects >= 2); } -TEST_CASE("a NUC pausing the stream costs that picture, and the stream goes on") { - auto o = fastOptions(); - o.max_frame_age_ms = 50; +TEST_CASE("the program takes over the camera's NUC timer and hands it back") { + { + Harness t; + t.cam.open(); + t.sdk.push(29315); + REQUIRE(t.online()); + CHECK(t.sdk.nuc_interval == 0); // the camera's own timer is off + t.cam.close(); + CHECK(t.sdk.nuc_interval == 30); // and back as found + } + { + auto o = fastOptions(); + o.nuc_interval_ms = 0; // leave it to the camera + Harness t(o); + t.cam.open(); + t.sdk.push(29315); + REQUIRE(t.online()); + t.cam.moving(); + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + CHECK(t.sdk.nuc_interval == 30); + CHECK(t.sdk.nucs == 0); + } +} + +TEST_CASE("a move runs a due NUC, and no more often than the interval") { + auto o = fastOptions(); + o.nuc_interval_ms = 400; Harness t(o); t.cam.open(); t.cam.start(); t.sdk.push(29315); - REQUIRE(t.shot()); - // The camera corrects itself: no new frame while it does. - t.sdk.nuc = 1; - t.sdk.frozen = true; - std::this_thread::sleep_for(std::chrono::milliseconds(120)); + REQUIRE(t.online()); + CHECK(t.sdk.nucs == 0); + t.cam.moving(); // the first move after connecting runs one + CHECK(waitFor([&] { return t.sdk.nucs == 1; }, 1000)); + CHECK(t.sdk.nuc_mode == kNucMode1Point); + t.cam.moving(); // not due yet + std::this_thread::sleep_for(std::chrono::milliseconds(150)); + CHECK(t.sdk.nucs == 1); + std::this_thread::sleep_for(std::chrono::milliseconds(300)); + t.cam.moving(); // due again + CHECK(waitFor([&] { return t.sdk.nucs == 2; }, 1000)); +} + +TEST_CASE("an overdue NUC runs without a move") { + auto o = fastOptions(); + o.nuc_interval_ms = 100; + Harness t(o); + t.cam.open(); + t.sdk.push(29315); + REQUIRE(t.online()); + // Twice the interval with no move at all (manual mode, say). + CHECK(waitFor([&] { return t.sdk.nucs >= 1; }, 1500)); +} + +TEST_CASE("after a NUC only a frame from after it is taken") { + auto o = fastOptions(); + o.nuc_interval_ms = 400; + Harness t(o); + t.cam.open(); + t.cam.start(); + t.sdk.push(29315); + REQUIRE(t.online()); + CHECK(t.cam.frameReady()); + t.sdk.frozen = true; // nothing new comes during and after the NUC + t.cam.moving(); + REQUIRE(waitFor([&] { return t.sdk.nucs == 1; }, 1000)); + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + CHECK_FALSE(t.cam.frameReady()); // the frame from before the NUC is stale CHECK_FALSE(t.cam.trigger()); - CHECK(t.cam.deviceInfo()[0].frames_dropped >= 1); - t.sdk.nuc = 0; t.sdk.frozen = false; t.sdk.push(29400); + CHECK(waitFor([&] { return t.cam.frameReady(); }, 1000)); CHECK(t.shot()); }