Run the thermal camera's NUC while the gimbal moves

The camera recalibrated against its shutter every 30 s and sent no frame for
about 0.7 s meanwhile: a picture falling into that had no thermal image (0.8 %
on tower 111, the 30 s rhythm shown by the new gap logging). The program now
switches the camera's timer off and runs the NUC itself (NUC_Perform, which
returns when it is done) when a move starts and nuc_interval_s (30) has passed,
or after twice that with no move. The scheduler tells the cameras of every move
(ICameraSource::moving), and a picture waits up to 1 s for a camera between
frames (frameReady) rather than go without it. The camera's own timer goes
back on exit; nuc_interval_s = 0 leaves it alone.

Measured on tower 111 before writing it: NUC_Interval = 0 gave 70 s without a
gap; NUC_Perform ran three NUCs of 657-677 ms; NUC_INPROGRESS never changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Peter Bochmann 2026-10-15 19:02:20 +02:00
parent 1c2f0a0506
commit ff89643a4f
Signed by: peter.bochmann
GPG Key ID: 94E8C523489B8AC1
16 changed files with 337 additions and 62 deletions

View File

@ -112,6 +112,7 @@ frame_timeout_ms = 5000
;calib_profile = 0 ; 0: -40..120 C (forest), 1: 0..600 C ;calib_profile = 0 ; 0: -40..120 C (forest), 1: 0..600 C
;frame_rate_hz = 5 ; the camera free-runs at this, throttled ;frame_rate_hz = 5 ; the camera free-runs at this, throttled
;max_frame_age_ms = 400 ; older is not kept; below the scan's settle wait ;max_frame_age_ms = 400 ; older is not kept; below the scan's settle wait
;nuc_interval_s = 30 ; NUC at the start of a move, not mid-picture; 0 = camera's timer
;rotation_deg = 0 ;rotation_deg = 0
;jxl_effort = 3 ;jxl_effort = 3
; ;

View File

@ -78,6 +78,7 @@ Parsed and validated by `ConfigLoader` ([src/core/Config.cpp](../src/core/Config
| `Camera.<name>` (`taris`) | `calib_profile` | int ≥ 0 | `0` | Calibration profile set on connect: 0 = -40..120 °C at 0.01 K/count, 1 = 0..600 °C at 0.02 K/count. Switch live with `camera calib <n>` | | `Camera.<name>` (`taris`) | `calib_profile` | int ≥ 0 | `0` | Calibration profile set on connect: 0 = -40..120 °C at 0.01 K/count, 1 = 0..600 °C at 0.02 K/count. Switch live with `camera calib <n>` |
| `Camera.<name>` (`taris`) | `frame_rate_hz` | 0 < x ≤ 50 | `5` | The rate the camera free-runs at | | `Camera.<name>` (`taris`) | `frame_rate_hz` | 0 < x ≤ 50 | `5` | The rate the camera free-runs at |
| `Camera.<name>` (`taris`) | `max_frame_age_ms` | int > 0 | `400` | A frame older than this when the picture is taken is not kept: that picture has no thermal image. Keep it below the scan's settle wait, or the frame can predate standstill (warned at startup) | | `Camera.<name>` (`taris`) | `max_frame_age_ms` | int > 0 | `400` | A frame older than this when the picture is taken is not kept: that picture has no thermal image. Keep it below the scan's settle wait, or the frame can predate standstill (warned at startup) |
| `Camera.<name>` (`taris`) | `nuc_interval_s` | int 0..3600 | `30` | Seconds between NUCs, run by the program at the start of a move so none blanks a picture (after twice this with no move, one runs anyway). `0` leaves the camera's own timer, which now and then takes a picture's frame |
| `Camera.<name>` (`taris`) | `library` | string | `libirbgrab_x86_64-linux.so` | The InfraTec SDK, found through the ld cache | | `Camera.<name>` (`taris`) | `library` | string | `libirbgrab_x86_64-linux.so` | The InfraTec SDK, found through the ld cache |
| `Camera.<name>` (`taris`, hardware.ini) | `interface` | string | — | The network port the camera is cabled to; only named in the message when it is not found | | `Camera.<name>` (`taris`, hardware.ini) | `interface` | string | — | The network port the camera is cabled to; only named in the message when it is not found |
| `Hardware` (hardware.ini) | `save_workers` | int 1..4 | `1` | Frames saved at once. 1 on the IOTA, 2 on the 3 Delta ([measurements](raw12-pipeline-measurements.md)) | | `Hardware` (hardware.ini) | `save_workers` | int 1..4 | `1` | Frames saved at once. 1 on the IOTA, 2 on the 3 Delta ([measurements](raw12-pipeline-measurements.md)) |
@ -1080,6 +1081,14 @@ their CamEvents, see [mqtt-api.md](mqtt-api.md#thermal-camera-cam-tir)).
the colour camera's `settle_wait_ms`. the colour camera's `settle_wait_ms`.
- **Kept grabbing**: from start to exit, also in manual mode. A microbolometer - **Kept grabbing**: from start to exit, also in manual mode. A microbolometer
that is restarted begins with zero frames and a fresh NUC. that is restarted begins with zero frames and a fresh NUC.
- **NUC while the gimbal moves.** The camera recalibrates against its shutter
every 30 s and sends no frame for about 0.7 s meanwhile; a picture falling
into that had no thermal image (0.8 % of them on tower 111). The program
switches the camera's timer off and runs the NUC itself at the start of a
move once `nuc_interval_s` has passed; the move and the settle wait leave
seconds before the next picture. A picture that still finds the camera
between frames waits up to 1 s for the next one. On exit the camera's timer
is put back.
- **Split between the files** like any camera: `driver`, `id` and `interface` - **Split between the files** like any camera: `driver`, `id` and `interface`
in hardware.ini (with `tir` in `[Hardware] cameras`), the rest in config.ini. in hardware.ini (with `tir` in `[Hardware] cameras`), the rest in config.ini.
`save_queue_max` counts frames, so raise it by 2. `save_queue_max` counts frames, so raise it by 2.

View File

@ -169,7 +169,9 @@ scripts for this are worth keeping; note that wrapping them in `timeout` SIGTERM
- ~~Support a second camera backend.~~ Done for the thermal camera (`driver = taris`, - ~~Support a second camera backend.~~ Done for the thermal camera (`driver = taris`,
`TarisCameraSource`, behind `CompositeCameraSource`), tested with the TarisIR mini on tower 111 on `TarisCameraSource`, behind `CompositeCameraSource`), tested with the TarisIR mini on tower 111 on
2026-10-14 (docs/tower-installation.md, Phase 2b). Not yet tested: a pulled camera cable during a scan 2026-10-14 (docs/tower-installation.md, Phase 2b). Not yet tested: a pulled camera cable during a scan
(the reconnect is only unit-tested), and a long unattended run. A NIR unit would be a third driver (the reconnect is only unit-tested). The 0.8 % of pictures without a thermal frame in the first long run
were the camera's own NUC every 30 s (a 0.7 s pause in its stream); since 2026-10-15 the program runs the
NUC at the start of a move instead. A NIR unit would be a third driver
behind the same key. behind the same key.
- The InfraTec SDK costs about a quarter of one core on the 3 Delta even between pictures: three threads - The InfraTec SDK costs about a quarter of one core on the 3 Delta even between pictures: three threads
it starts in `dll_init` and one per connection poll the raw socket in `nanosleep` loops (measured on it starts in `dll_init` and one per connection poll the raw socket in `nanosleep` loops (measured on

View File

@ -950,7 +950,13 @@ Measured on tower 111 (2026-10-14, SDK 4.3.44, SN 2017744):
- A profile switch takes about 3 s; frames keep the old scale until - A profile switch takes about 3 s; frames keep the old scale until
`Calib_ChangeInProgress` reaches 0, and the program takes none meanwhile. `Calib_ChangeInProgress` reaches 0, and the program takes none meanwhile.
- The camera's frame counter (`ImageNum`) is always 0; `MissedPackets` was - The camera's frame counter (`ImageNum`) is always 0; `MissedPackets` was
always 0 on the direct link. always 0 on the direct link. It delivers about 50 fps whatever `Framerate_Hz`
says; the program polls it at 10 Hz.
- NUC (2026-10-15): the camera runs one every 30 s (`NUC_Interval`) and sends no
frame for 650-760 ms meanwhile. `NUC_Interval = 0` stops that; `NUC_Perform`
(1-point) runs one and returns when it is done, after about 0.67 s.
`NUC_INPROGRESS` stays 0 throughout, also for those. The program now runs the
NUC itself at the start of a move (`nuc_interval_s`).
- The SDK switches the process's C locale to `LC_NUMERIC=de_DE` when it is - The SDK switches the process's C locale to `LC_NUMERIC=de_DE` when it is
loaded. The program puts it back after every SDK call and writes its JSON loaded. The program puts it back after every SDK call and writes its JSON
without the C library's number formatting. without the C library's number formatting.

View File

@ -22,6 +22,10 @@ namespace fgc {
// Every frame of one trigger carries the leader's timestamp, so the pair is // Every frame of one trigger carries the leader's timestamp, so the pair is
// RGB/<t>.jxl and TIR/<t>.jxl with the same <t> in both CamEvents. // RGB/<t>.jxl and TIR/<t>.jxl with the same <t> in both CamEvents.
// //
// A follower that is briefly without a fresh frame (frameReady() false: the
// thermal camera's NUC) holds the picture back for up to `follower_wait_ms`,
// so the pair stays whole; past that the leader goes on alone.
//
// Indices are global: child k's camera i is offset(k) + i. Calls about one // Indices are global: child k's camera i is offset(k) + i. Calls about one
// camera go to the child that has it. // camera go to the child that has it.
// //
@ -33,13 +37,15 @@ namespace fgc {
class CompositeCameraSource : public ICameraSource { class CompositeCameraSource : public ICameraSource {
public: public:
explicit CompositeCameraSource(std::vector<std::unique_ptr<ICameraSource>> children, explicit CompositeCameraSource(std::vector<std::unique_ptr<ICameraSource>> children,
std::vector<int> counts = {}); std::vector<int> counts = {}, int follower_wait_ms = 1000);
void open() override; void open() override;
void close() override; void close() override;
void start() override; void start() override;
void stop() override; void stop() override;
bool trigger() override; bool trigger() override;
void moving() override;
bool frameReady() override;
bool setProfile(int camera_index, const CameraProfile& profile) override; bool setProfile(int camera_index, const CameraProfile& profile) override;
void setFrameCallback(FrameCallback cb) override; void setFrameCallback(FrameCallback cb) override;
uint64_t frameSeq() override; uint64_t frameSeq() override;
@ -67,6 +73,8 @@ private:
FrameCallback callback_; FrameCallback callback_;
bool in_trigger_ = false; bool in_trigger_ = false;
long long shot_ts_ = 0; // the leader's frame time, during a trigger long long shot_ts_ = 0; // the leader's frame time, during a trigger
int follower_wait_ms_;
long long waiting_since_ = 0; // steady ms a picture began waiting for a follower
}; };
} // namespace fgc } // namespace fgc

View File

@ -223,6 +223,10 @@ struct CameraSpec {
// settle wait, or the frame may have been taken while the gimbal still moved. // settle wait, or the frame may have been taken while the gimbal still moved.
int max_frame_age_ms = 400; int max_frame_age_ms = 400;
std::string library = "libirbgrab_x86_64-linux.so"; // the SDK, found through the ld cache std::string library = "libirbgrab_x86_64-linux.so"; // the SDK, found through the ld cache
// Seconds between NUCs the program runs itself, each at the start of a move
// so it never blanks a picture; 0 leaves the camera's own timer, which fires
// mid-picture now and then (TarisCameraSource.h).
int nuc_interval_s = 30;
// Mounting (hardware.ini): the camera's network port, for diagnostics only. // Mounting (hardware.ini): the camera's network port, for diagnostics only.
std::string interface; std::string interface;

View File

@ -86,6 +86,16 @@ public:
// yet, and the caller simply retries. // yet, and the caller simply retries.
virtual bool trigger() = 0; virtual bool trigger() = 0;
// The gimbal has started moving: nothing will be captured until it stands
// still again and has settled. A camera with housekeeping that blanks its
// stream - the thermal camera's NUC - does it now. Default: nothing.
virtual void moving() {}
// Would trigger() hand over a fresh frame right now? False while a camera is
// briefly between frames (a NUC), so a picture can wait a moment for it
// rather than go without. Default: always.
virtual bool frameReady() { return true; }
// Apply a profile to one camera, identified by its index in the order the // Apply a profile to one camera, identified by its index in the order the
// source opened them (the same index Frame::cam_id carries). The backend // source opened them (the same index Frame::cam_id carries). The backend
// loads the profile's settings XML through its SDK - see CameraProfile.h. // loads the profile's settings XML through its SDK - see CameraProfile.h.

View File

@ -25,6 +25,13 @@ namespace fgc {
// A camera that is missing, unlicensed or unreachable is not a startup error. // A camera that is missing, unlicensed or unreachable is not a startup error.
// The source reports why in offline(), keeps retrying, and the scan goes on // The source reports why in offline(), keeps retrying, and the scan goes on
// without it. // without it.
//
// NUC: the camera recalibrates itself against its shutter every 30 s and sends
// no frame for about 0.7 s while it does (measured on tower 111). A picture
// that falls into that gets no thermal frame. So the program switches the
// camera's timer off and runs the NUC itself, at the start of a move - the
// move and the settle wait leave seconds before the next picture - once
// nuc_interval has passed, or after twice that without any move (manual mode).
class TarisCameraSource : public ICameraSource { class TarisCameraSource : public ICameraSource {
public: public:
struct Options { struct Options {
@ -38,6 +45,7 @@ public:
int retry_min_ms = 2000; // between connection attempts, doubling... int retry_min_ms = 2000; // between connection attempts, doubling...
int retry_max_ms = 30000;// ...up to this int retry_max_ms = 30000;// ...up to this
int calib_timeout_ms = 15000;// a profile switch takes about 3 s int calib_timeout_ms = 15000;// a profile switch takes about 3 s
int nuc_interval_ms = 30000;// 0: leave the camera's own NUC timer
}; };
static Options fromSpec(const CameraSpec& spec); static Options fromSpec(const CameraSpec& spec);
@ -54,6 +62,9 @@ public:
void start() override; void start() override;
void stop() override; void stop() override;
bool trigger() override; bool trigger() override;
// Runs a due NUC now (on the worker), while the gimbal moves.
void moving() override;
bool frameReady() override;
void setFrameCallback(FrameCallback cb) override; void setFrameCallback(FrameCallback cb) override;
int cameraCount() const override { return 1; } int cameraCount() const override { return 1; }
std::vector<CameraDeviceInfo> deviceInfo() override; std::vector<CameraDeviceInfo> deviceInfo() override;

View File

@ -2,13 +2,22 @@
#include "fgc/Logger.h" #include "fgc/Logger.h"
#include <chrono>
#include <stdexcept> #include <stdexcept>
namespace fgc { namespace fgc {
namespace {
long long steadyMs() {
using namespace std::chrono;
return duration_cast<milliseconds>(steady_clock::now().time_since_epoch()).count();
}
} // namespace
CompositeCameraSource::CompositeCameraSource(std::vector<std::unique_ptr<ICameraSource>> children, CompositeCameraSource::CompositeCameraSource(std::vector<std::unique_ptr<ICameraSource>> children,
std::vector<int> counts) std::vector<int> counts, int follower_wait_ms)
: children_(std::move(children)), counts_(std::move(counts)) { : children_(std::move(children)), counts_(std::move(counts)),
follower_wait_ms_(follower_wait_ms) {
if (children_.empty()) throw std::invalid_argument("CompositeCameraSource needs a camera"); if (children_.empty()) throw std::invalid_argument("CompositeCameraSource needs a camera");
if (counts_.empty()) if (counts_.empty())
for (const auto& c : children_) counts_.push_back(c->cameraCount()); for (const auto& c : children_) counts_.push_back(c->cameraCount());
@ -57,6 +66,16 @@ void CompositeCameraSource::stop() {
} }
bool CompositeCameraSource::trigger() { bool CompositeCameraSource::trigger() {
bool followers_ready = true;
for (size_t k = 1; k < children_.size(); ++k) followers_ready &= children_[k]->frameReady();
if (!followers_ready) {
const long long now = steadyMs();
if (waiting_since_ == 0) waiting_since_ = now;
if (now - waiting_since_ < follower_wait_ms_) return false; // asked again next tick
LOG_DEBUG << "camera: a follower had no fresh frame for " << now - waiting_since_
<< " ms; taking the picture without it";
}
waiting_since_ = 0;
in_trigger_ = true; in_trigger_ = true;
shot_ts_ = 0; shot_ts_ = 0;
const bool taken = children_[0]->trigger(); const bool taken = children_[0]->trigger();
@ -66,6 +85,16 @@ bool CompositeCameraSource::trigger() {
return taken; return taken;
} }
void CompositeCameraSource::moving() {
for (auto& c : children_) c->moving();
}
bool CompositeCameraSource::frameReady() {
for (auto& c : children_)
if (!c->frameReady()) return false;
return true;
}
void CompositeCameraSource::setFrameCallback(FrameCallback cb) { void CompositeCameraSource::setFrameCallback(FrameCallback cb) {
callback_ = std::move(cb); callback_ = std::move(cb);
// The leader's frames pass through untouched (they can be 24 MB); its // The leader's frames pass through untouched (they can be 24 MB); its

View File

@ -87,6 +87,7 @@ TarisCameraSource::Options TarisCameraSource::fromSpec(const CameraSpec& spec) {
o.max_frame_age_ms = spec.max_frame_age_ms; o.max_frame_age_ms = spec.max_frame_age_ms;
o.library = spec.library; o.library = spec.library;
o.interface = spec.interface; o.interface = spec.interface;
o.nuc_interval_ms = spec.nuc_interval_s * 1000;
return o; return o;
} }
@ -137,10 +138,10 @@ struct TarisCameraSource::Impl {
double last_cam_ts = -1.0; // its MemInfo time, ms double last_cam_ts = -1.0; // its MemInfo time, ms
long long polls_since_frame = 0; long long polls_since_frame = 0;
std::map<Ret, int> codes_since_frame; // getdata answers other than a new frame std::map<Ret, int> codes_since_frame; // getdata answers other than a new frame
bool nuc_in_gap = false; // NUC_INPROGRESS seen set meanwhile // NUC run by the program (Options::nuc_interval_ms).
// NUC_INPROGRESS, read on every poll; -1 = the camera does not answer it. int nuc_interval_found = -1; // the camera's own timer, put back on close
int nuc_state = 0; bool nuc_ours = false; // the camera's timer is off and we run it
long long nuc_started_ms = 0; long long last_nuc_ms = 0; // steady
// For the summary: // For the summary:
long long sum_frames = 0, sum_gaps = 0, sum_nucs = 0, sum_max_gap = 0; long long sum_frames = 0, sum_gaps = 0, sum_nucs = 0, sum_max_gap = 0;
double sum_cam_span = 0.0; // camera clock covered by sum_frames double sum_cam_span = 0.0; // camera clock covered by sum_frames
@ -162,8 +163,9 @@ struct TarisCameraSource::Impl {
bool cal_ok = false; bool cal_ok = false;
bool reset_wanted = false; bool reset_wanted = false;
uint64_t connects = 0; uint64_t connects = 0;
bool nuc_now = false; // NUC_INPROGRESS at the last poll bool nuc_now = false; // a NUC of ours is running
long long nuc_now_since = 0; // steady ms it went on long long nuc_now_since = 0; // steady ms it began
bool move_started = false; // moving() since the worker last looked
long long missed_window = 0; // misses since the last summary long long missed_window = 0; // misses since the last summary
std::atomic<bool> running{false}; std::atomic<bool> running{false};
@ -196,7 +198,9 @@ struct TarisCameraSource::Impl {
void pause(long long ms) { void pause(long long ms) {
std::unique_lock<std::mutex> l(m); std::unique_lock<std::mutex> l(m);
cv.wait_for(l, std::chrono::milliseconds(ms), cv.wait_for(l, std::chrono::milliseconds(ms),
[this] { return !running.load() || cal_wanted >= 0 || reset_wanted; }); [this] {
return !running.load() || cal_wanted >= 0 || reset_wanted || move_started;
});
} }
// ---- parameter helpers ------------------------------------------------- // ---- parameter helpers -------------------------------------------------
@ -427,13 +431,7 @@ struct TarisCameraSource::Impl {
if (!model.empty()) info.model = model; if (!model.empty()) info.model = model;
++connects; ++connects;
} }
int32_t nuc_active = -1, nuc_interval = -1; takeOverNuc();
getInt32(param::kNucActive, nuc_active);
getInt32(param::kNucInterval, nuc_interval);
LOG_INFO << "thermal camera: NUC " << (nuc_active == 1 ? "on" : nuc_active == 0 ? "off" : "?")
<< ", automatic every "
<< (nuc_interval > 0 ? std::to_string(nuc_interval) + " s"
: nuc_interval == 0 ? std::string("- (off)") : "?");
kpc = 0.0; kpc = 0.0;
scale_tries = 0; scale_tries = 0;
@ -441,7 +439,6 @@ struct TarisCameraSource::Impl {
last_cam_ts = -1.0; last_cam_ts = -1.0;
polls_since_frame = 0; polls_since_frame = 0;
codes_since_frame.clear(); codes_since_frame.clear();
nuc_in_gap = false;
zeros_logged = false; zeros_logged = false;
last_frame_steady = steadyMs(); // grace for the first frame last_frame_steady = steadyMs(); // grace for the first frame
return true; return true;
@ -451,7 +448,6 @@ struct TarisCameraSource::Impl {
// Take the newest frame from the SDK, if there is one. // Take the newest frame from the SDK, if there is one.
void poll() { void poll() {
++polls_since_frame; ++polls_since_frame;
watchNuc();
// As before every getdata in the vendor's examples: without it the SDK // As before every getdata in the vendor's examples: without it the SDK
// now and then answers 0x80000001 instead of a frame. // now and then answers 0x80000001 instead of a frame.
setInt32(param::kPollingEnable, 1); setInt32(param::kPollingEnable, 1);
@ -584,32 +580,86 @@ struct TarisCameraSource::Impl {
} }
} }
// NUC_INPROGRESS on every poll: when the camera corrects itself, and for how // ---- NUC ---------------------------------------------------------------
// long - the prime suspect for a picture without a thermal frame. // Switch the camera's own NUC timer off: from now on the program runs the
void watchNuc() { // NUC, while the gimbal moves. The timer's value goes back on close. (Read
if (nuc_state < 0) return; // back right after a set, the camera still reports the old value.)
int32_t v = 0; void takeOverNuc() {
if (!getInt32(param::kNucInProgress, v)) { nuc_ours = false;
LOG_INFO << "thermal camera: NUC_INPROGRESS not readable; NUC timing not logged"; if (o.nuc_interval_ms <= 0) {
nuc_state = -1; LOG_INFO << "thermal camera: NUC left to the camera's own timer";
return; return;
} }
const long long now = steadyMs(); int32_t found = -1;
const int on = v != 0; if (!getInt32(param::kNucInterval, found)) {
if (on) nuc_in_gap = true; LOG_WARN << "thermal camera: cannot read NUC_Interval; NUC left to the camera";
if (on && !nuc_state) { return;
nuc_started_ms = now;
++sum_nucs;
LOG_INFO << "thermal camera: NUC started";
} else if (!on && nuc_state) {
LOG_INFO << "thermal camera: NUC finished after " << now - nuc_started_ms << " ms";
} }
if (on != nuc_state) { if (nuc_interval_found < 0) nuc_interval_found = found;
const Ret r = setInt32(param::kNucInterval, 0);
if (!ok(r)) {
LOG_WARN << "thermal camera: cannot switch the camera's NUC timer off (" << describe(r)
<< "); a picture may now and then fall into a NUC";
return;
}
nuc_ours = true;
last_nuc_ms = steadyMs() - o.nuc_interval_ms; // the first move runs one
LOG_INFO << "thermal camera: NUC every " << o.nuc_interval_ms / 1000
<< " s at the start of a move (camera's own timer: " << found << " s, now off)";
}
// Put the camera's timer back, so it corrects itself without us.
void handBackNuc() {
if (!nuc_ours || nuc_interval_found <= 0) return;
const Ret r = setInt32(param::kNucInterval, nuc_interval_found);
LOG_INFO << "thermal camera: NUC timer back to " << nuc_interval_found << " s ("
<< describe(r) << ")";
nuc_ours = false;
}
// A NUC when one is due and the gimbal has just started moving, or when it
// is overdue (twice the interval) with no move to wait for.
void serviceNuc() {
bool moved;
{
std::lock_guard<std::mutex> l(m); std::lock_guard<std::mutex> l(m);
nuc_now = on; moved = move_started;
nuc_now_since = now; move_started = false; // always: pause() wakes on it
} }
nuc_state = on; if (!nuc_ours) return;
const long long since = steadyMs() - last_nuc_ms;
if (moved ? since < o.nuc_interval_ms : since < 2LL * o.nuc_interval_ms) return;
runNuc(moved ? "move" : "no move for a while");
}
void runNuc(const char* why) {
const long long start = steadyMs();
{
std::lock_guard<std::mutex> l(m);
nuc_now = true;
nuc_now_since = start;
}
Int2 nuc{kNucMode1Point, 0};
// Returns once the NUC is done, about 0.7 s.
const Ret r = api->dev_setparam(dev, param::kNucPerform, &nuc, kType2Int32);
const long long done = steadyMs();
{
std::lock_guard<std::mutex> l(m);
nuc_now = false;
// A frame from before the NUC is no longer fresh for a picture.
delivered_seq = latest_seq;
}
last_nuc_ms = done;
++sum_nucs;
// A frame is due as soon as the NUC is over; do not count it as a stall.
last_frame_steady = std::max(last_frame_steady, done);
if (!ok(r)) {
LOG_WARN << "thermal camera: NUC failed (" << describe(r)
<< "); handing NUC back to the camera's own timer";
handBackNuc();
return;
}
LOG_DEBUG << "thermal camera: NUC (" << why << ") took " << done - start << " ms";
} }
// A new frame: log the pause before it if it was long, and count it. // A new frame: log the pause before it if it was long, and count it.
@ -620,7 +670,9 @@ struct TarisCameraSource::Impl {
++sum_frames; ++sum_frames;
if (cam_gap > 0.0) sum_cam_span += cam_gap; if (cam_gap > 0.0) sum_cam_span += cam_gap;
sum_max_gap = std::max(sum_max_gap, host_gap); sum_max_gap = std::max(sum_max_gap, host_gap);
if (host_gap > kGapLogMs || cam_gap > kGapLogMs) { // A pause of our own making is no news.
const bool ours = now - last_nuc_ms < 2000;
if ((host_gap > kGapLogMs || cam_gap > kGapLogMs) && !ours) {
++sum_gaps; ++sum_gaps;
std::string codes; std::string codes;
for (const auto& [code, n] : codes_since_frame) for (const auto& [code, n] : codes_since_frame)
@ -629,14 +681,12 @@ struct TarisCameraSource::Impl {
" x" + std::to_string(n); " x" + std::to_string(n);
LOG_INFO << "thermal camera: frame gap " << host_gap << " ms (camera clock " LOG_INFO << "thermal camera: frame gap " << host_gap << " ms (camera clock "
<< static_cast<long long>(cam_gap) << " ms), " << polls_since_frame << static_cast<long long>(cam_gap) << " ms), " << polls_since_frame
<< " polls: " << (codes.empty() ? "-" : codes) << "; NUC " << " polls: " << (codes.empty() ? "-" : codes);
<< (nuc_state < 0 ? "unknown" : nuc_in_gap ? "in progress" : "not seen");
} }
} }
last_cam_ts = cam_ts; last_cam_ts = cam_ts;
polls_since_frame = 0; polls_since_frame = 0;
codes_since_frame.clear(); codes_since_frame.clear();
nuc_in_gap = nuc_state > 0;
if (now - sum_since_ms >= kSummaryEveryMs) summary(now); if (now - sum_since_ms >= kSummaryEveryMs) summary(now);
} }
@ -652,7 +702,8 @@ struct TarisCameraSource::Impl {
LOG_INFO << "thermal camera, last " << (now - sum_since_ms) / 60000 << " min: " LOG_INFO << "thermal camera, last " << (now - sum_since_ms) / 60000 << " min: "
<< sum_frames << " frames (" << std::lround(fps * 10) / 10.0 << sum_frames << " frames (" << std::lround(fps * 10) / 10.0
<< " fps by the camera's clock), longest gap " << sum_max_gap << " ms, " << " fps by the camera's clock), longest gap " << sum_max_gap << " ms, "
<< sum_gaps << " over " << kGapLogMs << " ms, " << sum_nucs << " NUC, " << sum_gaps << " unexplained over " << kGapLogMs << " ms, " << sum_nucs
<< " NUC" << (nuc_ours ? " (ours)" : "") << ", "
<< missed << " picture(s) without a thermal frame"; << missed << " picture(s) without a thermal frame";
} }
sum_frames = sum_gaps = sum_nucs = sum_max_gap = 0; sum_frames = sum_gaps = sum_nucs = sum_max_gap = 0;
@ -709,6 +760,7 @@ struct TarisCameraSource::Impl {
// retry pause from pausing. // retry pause from pausing.
std::lock_guard<std::mutex> l(m); std::lock_guard<std::mutex> l(m);
reset_wanted = false; reset_wanted = false;
move_started = false; // nor may a move cut the retry pause short
} }
std::string why; std::string why;
bool up; bool up;
@ -729,6 +781,7 @@ struct TarisCameraSource::Impl {
} }
serviceRequests(); serviceRequests();
if (!connected) continue; if (!connected) continue;
serviceNuc();
poll(); poll();
const long long now = steadyMs(); const long long now = steadyMs();
@ -746,6 +799,7 @@ struct TarisCameraSource::Impl {
pause(poll_ms); pause(poll_ms);
} }
LocaleGuard keep; LocaleGuard keep;
if (connected) handBackNuc();
disconnect(); disconnect();
if (sdk_init) api->dll_uninit(); if (sdk_init) api->dll_uninit();
sdk_init = false; sdk_init = false;
@ -815,6 +869,23 @@ void TarisCameraSource::stop() { impl_->gate = false; }
void TarisCameraSource::setFrameCallback(FrameCallback cb) { impl_->callback = std::move(cb); } void TarisCameraSource::setFrameCallback(FrameCallback cb) { impl_->callback = std::move(cb); }
void TarisCameraSource::moving() {
{
std::lock_guard<std::mutex> l(impl_->m);
impl_->move_started = true;
}
impl_->cv.notify_all(); // the worker may be pausing between polls
}
bool TarisCameraSource::frameReady() {
Impl& d = *impl_;
std::lock_guard<std::mutex> l(d.m);
if (!d.running || !d.why_offline.empty()) return true; // nothing to wait for
if (d.nuc_now) return false;
return d.latest_seq > d.delivered_seq &&
unixMs() - d.latest.timestamp_ms <= d.o.max_frame_age_ms;
}
bool TarisCameraSource::trigger() { bool TarisCameraSource::trigger() {
Impl& d = *impl_; Impl& d = *impl_;
if (!d.gate || !d.callback) return false; if (!d.gate || !d.callback) return false;
@ -838,7 +909,7 @@ bool TarisCameraSource::trigger() {
(new_frame ? "" : ", none since the last picture") + (new_frame ? "" : ", none since the last picture") +
(d.nuc_now ? ", NUC in progress for " + (d.nuc_now ? ", NUC in progress for " +
std::to_string(now - d.nuc_now_since) + " ms" std::to_string(now - d.nuc_now_since) + " ms"
: ", no NUC running"); : "");
LOG_WARN << "thermal camera: no thermal frame for this picture: " << why; LOG_WARN << "thermal camera: no thermal frame for this picture: " << why;
return false; return false;
} }

View File

@ -110,6 +110,8 @@ void CaptureScheduler::sendMove() {
// form ("ERR axis not present") and moves nothing, so address yaw alone. // form ("ERR axis not present") and moves nothing, so address yaw alone.
motor_.sendCommand(moveCommand(yaw_target_, pitch_present_ ? std::optional<long>(pitch_target_) motor_.sendCommand(moveCommand(yaw_target_, pitch_present_ ? std::optional<long>(pitch_target_)
: std::nullopt)); : std::nullopt));
// The time the move and the settle wait take is free for the cameras.
camera_.moving();
} }
bool CaptureScheduler::settledAt(const MotorTelemetry& t) const { bool CaptureScheduler::settledAt(const MotorTelemetry& t) const {

View File

@ -393,6 +393,7 @@ AppConfig ConfigLoader::fromMap(const std::map<std::string, std::string>& kv) {
c.frame_rate_hz = getDouble(kv, pre + "frame_rate_hz", c.frame_rate_hz); c.frame_rate_hz = getDouble(kv, pre + "frame_rate_hz", c.frame_rate_hz);
c.max_frame_age_ms = getInt(kv, pre + "max_frame_age_ms", c.max_frame_age_ms); c.max_frame_age_ms = getInt(kv, pre + "max_frame_age_ms", c.max_frame_age_ms);
c.library = get(kv, pre + "library", c.library); c.library = get(kv, pre + "library", c.library);
c.nuc_interval_s = getInt(kv, pre + "nuc_interval_s", c.nuc_interval_s);
c.interface = get(kv, pre + "interface"); c.interface = get(kv, pre + "interface");
// Per-profile timing: Camera.<name>.<profile>.settle_wait_ms // Per-profile timing: Camera.<name>.<profile>.settle_wait_ms
@ -434,6 +435,9 @@ AppConfig ConfigLoader::fromMap(const std::map<std::string, std::string>& kv) {
throw std::runtime_error("Camera." + name + ".frame_rate_hz must be in (0, 50]"); throw std::runtime_error("Camera." + name + ".frame_rate_hz must be in (0, 50]");
if (c.max_frame_age_ms <= 0) if (c.max_frame_age_ms <= 0)
throw std::runtime_error("Camera." + name + ".max_frame_age_ms must be > 0"); throw std::runtime_error("Camera." + name + ".max_frame_age_ms must be > 0");
if (c.nuc_interval_s < 0 || c.nuc_interval_s > 3600)
throw std::runtime_error("Camera." + name + ".nuc_interval_s must be in 0..3600 "
"(0 = the camera's own timer)");
if (c.library.empty()) if (c.library.empty())
throw std::runtime_error("Camera." + name + ".library must not be empty"); throw std::runtime_error("Camera." + name + ".library must not be empty");
} }

View File

@ -2,7 +2,9 @@
#include <doctest/doctest.h> #include <doctest/doctest.h>
#include <chrono>
#include <stdexcept> #include <stdexcept>
#include <thread>
#include <vector> #include <vector>
using namespace fgc; using namespace fgc;
@ -20,6 +22,8 @@ struct FakeSource : ICameraSource {
bool throws_on_open = false; bool throws_on_open = false;
int triggers = 0; int triggers = 0;
bool started = false; bool started = false;
bool frame_ready = true; // frameReady()
int moves = 0;
std::string why_offline; std::string why_offline;
FrameCallback cb; FrameCallback cb;
@ -32,6 +36,8 @@ struct FakeSource : ICameraSource {
void close() override {} void close() override {}
void start() override { started = true; } void start() override { started = true; }
void stop() override { started = false; } void stop() override { started = false; }
void moving() override { ++moves; }
bool frameReady() override { return frame_ready; }
bool trigger() override { bool trigger() override {
++triggers; ++triggers;
if (!ready) return false; if (!ready) return false;
@ -172,6 +178,44 @@ TEST_CASE("the indices are the configured ones, also before the leader has opene
CHECK_THROWS(CompositeCameraSource(std::vector<std::unique_ptr<ICameraSource>>{}, {})); CHECK_THROWS(CompositeCameraSource(std::vector<std::unique_ptr<ICameraSource>>{}, {}));
} }
TEST_CASE("a picture waits a moment for a follower between frames, then goes without it") {
auto c = std::make_unique<FakeSource>(1, 1000);
auto t = std::make_unique<FakeSource>(1, 5000);
FakeSource* colour = c.get();
FakeSource* thermal = t.get();
std::vector<std::unique_ptr<ICameraSource>> v;
v.push_back(std::move(c));
v.push_back(std::move(t));
CompositeCameraSource src(std::move(v), {1, 1}, 60);
std::vector<Frame> frames;
src.setFrameCallback([&](const Frame& f) { frames.push_back(f); });
// The thermal camera is in its NUC: the picture is held, the leader not asked.
thermal->frame_ready = false;
CHECK_FALSE(src.trigger());
CHECK(colour->triggers == 0);
CHECK_FALSE(src.frameReady());
// It comes back within the wait: one whole pair.
thermal->frame_ready = true;
REQUIRE(src.trigger());
CHECK(frames.size() == 2);
// It does not come back: after the wait the colour picture goes alone.
frames.clear();
thermal->frame_ready = false;
CHECK_FALSE(src.trigger());
std::this_thread::sleep_for(std::chrono::milliseconds(80));
REQUIRE(src.trigger());
CHECK(colour->triggers == 2);
}
TEST_CASE("every source hears that the gimbal moves") {
Rig r;
r.src->moving();
CHECK(r.colour->moves == 1);
CHECK(r.thermal->moves == 1);
}
TEST_CASE("start and stop reach every source") { TEST_CASE("start and stop reach every source") {
Rig r; Rig r;
r.src->start(); r.src->start();

View File

@ -312,6 +312,7 @@ TEST_CASE("A thermal camera needs no settings XML and is always lossless") {
CHECK(tir.max_frame_age_ms == 400); CHECK(tir.max_frame_age_ms == 400);
CHECK(tir.library == "libirbgrab_x86_64-linux.so"); CHECK(tir.library == "libirbgrab_x86_64-linux.so");
CHECK(tir.interface == "enp1s0"); CHECK(tir.interface == "enp1s0");
CHECK(tir.nuc_interval_s == 30);
// [Camera] jxl_distance is for colour cameras; counts default to lossless. // [Camera] jxl_distance is for colour cameras; counts default to lossless.
CHECK(tir.jxl_distance == 0.0); CHECK(tir.jxl_distance == 0.0);
CHECK(c.camera.cameras[0].jxl_distance == doctest::Approx(0.8)); CHECK(c.camera.cameras[0].jxl_distance == doctest::Approx(0.8));
@ -368,6 +369,8 @@ TEST_CASE("Thermal camera blocks reject what cannot work") {
// An empty value means the default, as for every key. // An empty value means the default, as for every key.
CHECK(ConfigLoader::fromMap(base({{"Camera.tir.library", ""}})).camera.cameras[1].library == CHECK(ConfigLoader::fromMap(base({{"Camera.tir.library", ""}})).camera.cameras[1].library ==
"libirbgrab_x86_64-linux.so"); "libirbgrab_x86_64-linux.so");
CHECK_THROWS(ConfigLoader::fromMap(base({{"Camera.tir.nuc_interval_s", "-1"}})));
CHECK_NOTHROW(ConfigLoader::fromMap(base({{"Camera.tir.nuc_interval_s", "0"}})));
// Two cameras, one output folder: their pictures would overwrite each other. // Two cameras, one output folder: their pictures would overwrite each other.
CHECK_THROWS(ConfigLoader::fromMap(base({{"Camera.rgb.label", "X"}, {"Camera.tir.label", "X"}}))); CHECK_THROWS(ConfigLoader::fromMap(base({{"Camera.rgb.label", "X"}, {"Camera.tir.label", "X"}})));
// A disabled thermal camera needs no serial, like any other. // A disabled thermal camera needs no serial, like any other.

View File

@ -26,6 +26,7 @@ struct FakeMotor : IMotorController {
struct FakeCamera : ICameraSource { struct FakeCamera : ICameraSource {
int triggers = 0; int triggers = 0;
int moves = 0;
bool ok = true; bool ok = true;
// A stream of frames for the keep rule; empty = a camera with no frame data. // A stream of frames for the keep rule; empty = a camera with no frame data.
std::vector<FrameStats> stream; std::vector<FrameStats> stream;
@ -44,6 +45,7 @@ struct FakeCamera : ICameraSource {
void start() override {} void start() override {}
void stop() override {} void stop() override {}
bool trigger() override { ++triggers; return ok; } bool trigger() override { ++triggers; return ok; }
void moving() override { ++moves; }
void setFrameCallback(FrameCallback) override {} void setFrameCallback(FrameCallback) override {}
int cameraCount() const override { return 1; } int cameraCount() const override { return 1; }
}; };
@ -85,6 +87,7 @@ TEST_CASE("CaptureScheduler sweeps the scan grid with MOVE + settle + trigger")
CHECK(motor.cmds[0] == "MOVE 0,0"); CHECK(motor.cmds[0] == "MOVE 0,0");
CHECK(sch.moving()); CHECK(sch.moving());
CHECK(cam.triggers == 0); // not yet settled CHECK(cam.triggers == 0); // not yet settled
CHECK(cam.moves == 1); // the cameras hear of every move (the thermal NUC)
// At rest early: the frame waits for the interval, counted from the mode start. // At rest early: the frame waits for the interval, counted from the mode start.
settleAt(motor, 0, 0); settleAt(motor, 0, 0);
@ -103,6 +106,7 @@ TEST_CASE("CaptureScheduler sweeps the scan grid with MOVE + settle + trigger")
clock = 1010; clock = 1010;
sch.tick(); sch.tick();
CHECK(motor.cmds.back() == "MOVE 900,0"); CHECK(motor.cmds.back() == "MOVE 900,0");
CHECK(cam.moves == 2);
CHECK(sch.overruns() == 0); CHECK(sch.overruns() == 0);
} }

View File

@ -35,7 +35,9 @@ struct FakeSdk {
uint32_t w = 8, h = 4; uint32_t w = 8, h = 4;
Ret no_frame = kErrNotFound; // what getdata says before any frame Ret no_frame = kErrNotFound; // what getdata says before any frame
std::atomic<bool> frozen{false}; // stop producing frames (stall) std::atomic<bool> frozen{false}; // stop producing frames (stall)
std::atomic<int> nuc{0}; // NUC_INPROGRESS std::atomic<int> nuc_interval{30}; // the camera's own NUC timer, s
std::atomic<int> nucs{0}; // NUC_Perform calls
std::atomic<int> nuc_mode{0};
void push(uint16_t v, uint16_t hot = 0) { void push(uint16_t v, uint16_t hot = 0) {
std::lock_guard<std::mutex> l(m); std::lock_guard<std::mutex> l(m);
@ -99,9 +101,7 @@ Ret fGetParam(Handle, uint32_t what, void* data, uint32_t type) {
return kSuccess; return kSuccess;
} }
if (what == param::kCameraTempHousing && type == kTypeSingle) { *static_cast<float*>(data) = 308.15f; return kSuccess; } if (what == param::kCameraTempHousing && type == kTypeSingle) { *static_cast<float*>(data) = 308.15f; return kSuccess; }
if (what == param::kNucInProgress && type == kTypeInt32) { *static_cast<int32_t*>(data) = g->nuc; return kSuccess; } if (what == param::kNucInterval && type == kTypeInt32) { *static_cast<int32_t*>(data) = g->nuc_interval; return kSuccess; }
if (what == param::kNucActive && type == kTypeInt32) { *static_cast<int32_t*>(data) = 1; return kSuccess; }
if (what == param::kNucInterval && type == kTypeInt32) { *static_cast<int32_t*>(data) = 0; return kSuccess; }
return kErrNotSupported; return kErrNotSupported;
} }
Ret fSetParam(Handle, uint32_t what, void* data, uint32_t type) { Ret fSetParam(Handle, uint32_t what, void* data, uint32_t type) {
@ -113,6 +113,17 @@ Ret fSetParam(Handle, uint32_t what, void* data, uint32_t type) {
} }
if (what == param::kFramerateHz && type == kTypeSingle) { g->rate = *static_cast<float*>(data); return kSuccess; } if (what == param::kFramerateHz && type == kTypeSingle) { g->rate = *static_cast<float*>(data); return kSuccess; }
if (what == param::kPollingEnable) return kSuccess; if (what == param::kPollingEnable) return kSuccess;
if (what == param::kNucInterval && type == kTypeInt32) {
g->nuc_interval = *static_cast<int32_t*>(data);
return kSuccess;
}
if (what == param::kNucPerform && type == kType2Int32) {
// As the camera: the call returns when the NUC is done.
g->nuc_mode = static_cast<Int2*>(data)->value1;
++g->nucs;
std::this_thread::sleep_for(std::chrono::milliseconds(20));
return kSuccess;
}
return kErrNotSupported; return kErrNotSupported;
} }
// A memory object is a copy of the newest frame. // A memory object is a copy of the newest frame.
@ -372,23 +383,79 @@ TEST_CASE("a camera that goes quiet is reconnected") {
CHECK(t.sdk.connects >= 2); CHECK(t.sdk.connects >= 2);
} }
TEST_CASE("a NUC pausing the stream costs that picture, and the stream goes on") { TEST_CASE("the program takes over the camera's NUC timer and hands it back") {
{
Harness t;
t.cam.open();
t.sdk.push(29315);
REQUIRE(t.online());
CHECK(t.sdk.nuc_interval == 0); // the camera's own timer is off
t.cam.close();
CHECK(t.sdk.nuc_interval == 30); // and back as found
}
{
auto o = fastOptions(); auto o = fastOptions();
o.max_frame_age_ms = 50; o.nuc_interval_ms = 0; // leave it to the camera
Harness t(o);
t.cam.open();
t.sdk.push(29315);
REQUIRE(t.online());
t.cam.moving();
std::this_thread::sleep_for(std::chrono::milliseconds(100));
CHECK(t.sdk.nuc_interval == 30);
CHECK(t.sdk.nucs == 0);
}
}
TEST_CASE("a move runs a due NUC, and no more often than the interval") {
auto o = fastOptions();
o.nuc_interval_ms = 400;
Harness t(o); Harness t(o);
t.cam.open(); t.cam.open();
t.cam.start(); t.cam.start();
t.sdk.push(29315); t.sdk.push(29315);
REQUIRE(t.shot()); REQUIRE(t.online());
// The camera corrects itself: no new frame while it does. CHECK(t.sdk.nucs == 0);
t.sdk.nuc = 1; t.cam.moving(); // the first move after connecting runs one
t.sdk.frozen = true; CHECK(waitFor([&] { return t.sdk.nucs == 1; }, 1000));
std::this_thread::sleep_for(std::chrono::milliseconds(120)); CHECK(t.sdk.nuc_mode == kNucMode1Point);
t.cam.moving(); // not due yet
std::this_thread::sleep_for(std::chrono::milliseconds(150));
CHECK(t.sdk.nucs == 1);
std::this_thread::sleep_for(std::chrono::milliseconds(300));
t.cam.moving(); // due again
CHECK(waitFor([&] { return t.sdk.nucs == 2; }, 1000));
}
TEST_CASE("an overdue NUC runs without a move") {
auto o = fastOptions();
o.nuc_interval_ms = 100;
Harness t(o);
t.cam.open();
t.sdk.push(29315);
REQUIRE(t.online());
// Twice the interval with no move at all (manual mode, say).
CHECK(waitFor([&] { return t.sdk.nucs >= 1; }, 1500));
}
TEST_CASE("after a NUC only a frame from after it is taken") {
auto o = fastOptions();
o.nuc_interval_ms = 400;
Harness t(o);
t.cam.open();
t.cam.start();
t.sdk.push(29315);
REQUIRE(t.online());
CHECK(t.cam.frameReady());
t.sdk.frozen = true; // nothing new comes during and after the NUC
t.cam.moving();
REQUIRE(waitFor([&] { return t.sdk.nucs == 1; }, 1000));
std::this_thread::sleep_for(std::chrono::milliseconds(50));
CHECK_FALSE(t.cam.frameReady()); // the frame from before the NUC is stale
CHECK_FALSE(t.cam.trigger()); CHECK_FALSE(t.cam.trigger());
CHECK(t.cam.deviceInfo()[0].frames_dropped >= 1);
t.sdk.nuc = 0;
t.sdk.frozen = false; t.sdk.frozen = false;
t.sdk.push(29400); t.sdk.push(29400);
CHECK(waitFor([&] { return t.cam.frameReady(); }, 1000));
CHECK(t.shot()); CHECK(t.shot());
} }