Compare commits

..

2 Commits

Author SHA1 Message Date
Peter Bochmann ff89643a4f
Run the thermal camera's NUC while the gimbal moves
The camera recalibrated against its shutter every 30 s and sent no frame for
about 0.7 s meanwhile: a picture falling into that had no thermal image (0.8 %
on tower 111, the 30 s rhythm shown by the new gap logging). The program now
switches the camera's timer off and runs the NUC itself (NUC_Perform, which
returns when it is done) when a move starts and nuc_interval_s (30) has passed,
or after twice that with no move. The scheduler tells the cameras of every move
(ICameraSource::moving), and a picture waits up to 1 s for a camera between
frames (frameReady) rather than go without it. The camera's own timer goes
back on exit; nuc_interval_s = 0 leaves it alone.

Measured on tower 111 before writing it: NUC_Interval = 0 gave 70 s without a
gap; NUC_Perform ran three NUCs of 657-677 ms; NUC_INPROGRESS never changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 19:02:20 +02:00
Peter Bochmann 1c2f0a0506
Log why a picture got no thermal frame
Every miss now says how old the newest frame was and whether a NUC was
running; every pause over 500 ms in the frame stream is logged with the
camera's own clock, the polls and getdata answers in it, and NUC_INPROGRESS,
which is watched on every poll; a summary of the stream goes out every ten
minutes, and the NUC configuration at connect. Adds the SDK's NUC parameters
(410-415), for running the NUC on purpose later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 17:59:49 +02:00
17 changed files with 452 additions and 19 deletions

View File

@ -112,6 +112,7 @@ frame_timeout_ms = 5000
;calib_profile = 0 ; 0: -40..120 C (forest), 1: 0..600 C
;frame_rate_hz = 5 ; the camera free-runs at this, throttled
;max_frame_age_ms = 400 ; older is not kept; below the scan's settle wait
;nuc_interval_s = 30 ; NUC at the start of a move, not mid-picture; 0 = camera's timer
;rotation_deg = 0
;jxl_effort = 3
;

View File

@ -78,6 +78,7 @@ Parsed and validated by `ConfigLoader` ([src/core/Config.cpp](../src/core/Config
| `Camera.<name>` (`taris`) | `calib_profile` | int ≥ 0 | `0` | Calibration profile set on connect: 0 = -40..120 °C at 0.01 K/count, 1 = 0..600 °C at 0.02 K/count. Switch live with `camera calib <n>` |
| `Camera.<name>` (`taris`) | `frame_rate_hz` | 0 < x ≤ 50 | `5` | The rate the camera free-runs at |
| `Camera.<name>` (`taris`) | `max_frame_age_ms` | int > 0 | `400` | A frame older than this when the picture is taken is not kept: that picture has no thermal image. Keep it below the scan's settle wait, or the frame can predate standstill (warned at startup) |
| `Camera.<name>` (`taris`) | `nuc_interval_s` | int 0..3600 | `30` | Seconds between NUCs, run by the program at the start of a move so none blanks a picture (after twice this with no move, one runs anyway). `0` leaves the camera's own timer, which now and then takes a picture's frame |
| `Camera.<name>` (`taris`) | `library` | string | `libirbgrab_x86_64-linux.so` | The InfraTec SDK, found through the ld cache |
| `Camera.<name>` (`taris`, hardware.ini) | `interface` | string | — | The network port the camera is cabled to; only named in the message when it is not found |
| `Hardware` (hardware.ini) | `save_workers` | int 1..4 | `1` | Frames saved at once. 1 on the IOTA, 2 on the 3 Delta ([measurements](raw12-pipeline-measurements.md)) |
@ -1080,6 +1081,14 @@ their CamEvents, see [mqtt-api.md](mqtt-api.md#thermal-camera-cam-tir)).
the colour camera's `settle_wait_ms`.
- **Kept grabbing**: from start to exit, also in manual mode. A microbolometer
that is restarted begins with zero frames and a fresh NUC.
- **NUC while the gimbal moves.** The camera recalibrates against its shutter
every 30 s and sends no frame for about 0.7 s meanwhile; a picture falling
into that had no thermal image (0.8 % of them on tower 111). The program
switches the camera's timer off and runs the NUC itself at the start of a
move once `nuc_interval_s` has passed; the move and the settle wait leave
seconds before the next picture. A picture that still finds the camera
between frames waits up to 1 s for the next one. On exit the camera's timer
is put back.
- **Split between the files** like any camera: `driver`, `id` and `interface`
in hardware.ini (with `tir` in `[Hardware] cameras`), the rest in config.ini.
`save_queue_max` counts frames, so raise it by 2.

View File

@ -169,7 +169,9 @@ scripts for this are worth keeping; note that wrapping them in `timeout` SIGTERM
- ~~Support a second camera backend.~~ Done for the thermal camera (`driver = taris`,
`TarisCameraSource`, behind `CompositeCameraSource`), tested with the TarisIR mini on tower 111 on
2026-10-14 (docs/tower-installation.md, Phase 2b). Not yet tested: a pulled camera cable during a scan
(the reconnect is only unit-tested), and a long unattended run. A NIR unit would be a third driver
(the reconnect is only unit-tested). The 0.8 % of pictures without a thermal frame in the first long run
were the camera's own NUC every 30 s (a 0.7 s pause in its stream); since 2026-10-15 the program runs the
NUC at the start of a move instead. A NIR unit would be a third driver
behind the same key.
- The InfraTec SDK costs about a quarter of one core on the 3 Delta even between pictures: three threads
it starts in `dll_init` and one per connection poll the raw socket in `nanosleep` loops (measured on

View File

@ -950,7 +950,13 @@ Measured on tower 111 (2026-10-14, SDK 4.3.44, SN 2017744):
- A profile switch takes about 3 s; frames keep the old scale until
`Calib_ChangeInProgress` reaches 0, and the program takes none meanwhile.
- The camera's frame counter (`ImageNum`) is always 0; `MissedPackets` was
always 0 on the direct link.
always 0 on the direct link. It delivers about 50 fps whatever `Framerate_Hz`
says; the program polls it at 10 Hz.
- NUC (2026-10-15): the camera runs one every 30 s (`NUC_Interval`) and sends no
frame for 650-760 ms meanwhile. `NUC_Interval = 0` stops that; `NUC_Perform`
(1-point) runs one and returns when it is done, after about 0.67 s.
`NUC_INPROGRESS` stays 0 throughout, also for those. The program now runs the
NUC itself at the start of a move (`nuc_interval_s`).
- The SDK switches the process's C locale to `LC_NUMERIC=de_DE` when it is
loaded. The program puts it back after every SDK call and writes its JSON
without the C library's number formatting.

View File

@ -22,6 +22,10 @@ namespace fgc {
// Every frame of one trigger carries the leader's timestamp, so the pair is
// RGB/<t>.jxl and TIR/<t>.jxl with the same <t> in both CamEvents.
//
// A follower that is briefly without a fresh frame (frameReady() false: the
// thermal camera's NUC) holds the picture back for up to `follower_wait_ms`,
// so the pair stays whole; past that the leader goes on alone.
//
// Indices are global: child k's camera i is offset(k) + i. Calls about one
// camera go to the child that has it.
//
@ -33,13 +37,15 @@ namespace fgc {
class CompositeCameraSource : public ICameraSource {
public:
explicit CompositeCameraSource(std::vector<std::unique_ptr<ICameraSource>> children,
std::vector<int> counts = {});
std::vector<int> counts = {}, int follower_wait_ms = 1000);
void open() override;
void close() override;
void start() override;
void stop() override;
bool trigger() override;
void moving() override;
bool frameReady() override;
bool setProfile(int camera_index, const CameraProfile& profile) override;
void setFrameCallback(FrameCallback cb) override;
uint64_t frameSeq() override;
@ -67,6 +73,8 @@ private:
FrameCallback callback_;
bool in_trigger_ = false;
long long shot_ts_ = 0; // the leader's frame time, during a trigger
int follower_wait_ms_;
long long waiting_since_ = 0; // steady ms a picture began waiting for a follower
};
} // namespace fgc

View File

@ -223,6 +223,10 @@ struct CameraSpec {
// settle wait, or the frame may have been taken while the gimbal still moved.
int max_frame_age_ms = 400;
std::string library = "libirbgrab_x86_64-linux.so"; // the SDK, found through the ld cache
// Seconds between NUCs the program runs itself, each at the start of a move
// so it never blanks a picture; 0 leaves the camera's own timer, which fires
// mid-picture now and then (TarisCameraSource.h).
int nuc_interval_s = 30;
// Mounting (hardware.ini): the camera's network port, for diagnostics only.
std::string interface;

View File

@ -86,6 +86,16 @@ public:
// yet, and the caller simply retries.
virtual bool trigger() = 0;
// The gimbal has started moving: nothing will be captured until it stands
// still again and has settled. A camera with housekeeping that blanks its
// stream - the thermal camera's NUC - does it now. Default: nothing.
virtual void moving() {}
// Would trigger() hand over a fresh frame right now? False while a camera is
// briefly between frames (a NUC), so a picture can wait a moment for it
// rather than go without. Default: always.
virtual bool frameReady() { return true; }
// Apply a profile to one camera, identified by its index in the order the
// source opened them (the same index Frame::cam_id carries). The backend
// loads the profile's settings XML through its SDK - see CameraProfile.h.

View File

@ -58,8 +58,16 @@ inline constexpr uint32_t kCameraTempHousing = 217; // Kelvin
inline constexpr uint32_t kFramerateHz = 240;
inline constexpr uint32_t kFrameWidth = 294;
inline constexpr uint32_t kFrameHeight = 295;
// Non-uniformity correction, done in the camera with its own shutter.
inline constexpr uint32_t kNucActive = 410; // R bool: correction applied
inline constexpr uint32_t kNucPerform = 411; // W Int2 {mode, flags}: run one now
inline constexpr uint32_t kNucInProgress = 412; // R bool
inline constexpr uint32_t kNucInterval = 415; // R/W s between automatic ones, 0 = off
} // namespace param
// kNucPerform's mode (Int2::value1); value2 takes flags, 0 for none.
inline constexpr int32_t kNucMode1Point = 2;
#pragma pack(push, 1)
struct String {
char* text;
@ -73,6 +81,10 @@ struct IdxInt32 {
int32_t index;
int32_t value;
};
struct Int2 {
int32_t value1;
int32_t value2;
};
// Per-frame information (mem_getinfo). struct_size must be set before the call.
struct MemInfo {
int32_t struct_size;
@ -89,12 +101,14 @@ inline constexpr uint32_t kBaseInt64 = 0x50000000;
inline constexpr uint32_t kBaseFloat32 = 0xA0000000;
inline constexpr uint32_t kBaseString = 0xF0000000;
inline constexpr uint32_t kExIndex = 0x01000000;
inline constexpr uint32_t kExArray2 = 0x02020000; // a pair
inline constexpr uint32_t kTypeInt32 = kBaseInt32 | sizeof(int32_t);
inline constexpr uint32_t kTypeInt64 = kBaseInt64 | sizeof(int64_t);
inline constexpr uint32_t kTypeSingle = kBaseFloat32 | sizeof(float);
inline constexpr uint32_t kTypeString = kBaseString | sizeof(String);
inline constexpr uint32_t kTypeIdxString = kExIndex | kBaseString | sizeof(IdxString);
inline constexpr uint32_t kTypeIdxInt32 = kExIndex | kBaseInt32 | sizeof(IdxInt32);
inline constexpr uint32_t kType2Int32 = kExArray2 | kBaseInt32 | sizeof(Int2);
// The entry points, as function pointers so a test can stand in for the SDK.
struct Api {

View File

@ -25,6 +25,13 @@ namespace fgc {
// A camera that is missing, unlicensed or unreachable is not a startup error.
// The source reports why in offline(), keeps retrying, and the scan goes on
// without it.
//
// NUC: the camera recalibrates itself against its shutter every 30 s and sends
// no frame for about 0.7 s while it does (measured on tower 111). A picture
// that falls into that gets no thermal frame. So the program switches the
// camera's timer off and runs the NUC itself, at the start of a move - the
// move and the settle wait leave seconds before the next picture - once
// nuc_interval has passed, or after twice that without any move (manual mode).
class TarisCameraSource : public ICameraSource {
public:
struct Options {
@ -38,6 +45,7 @@ public:
int retry_min_ms = 2000; // between connection attempts, doubling...
int retry_max_ms = 30000;// ...up to this
int calib_timeout_ms = 15000;// a profile switch takes about 3 s
int nuc_interval_ms = 30000;// 0: leave the camera's own NUC timer
};
static Options fromSpec(const CameraSpec& spec);
@ -54,6 +62,9 @@ public:
void start() override;
void stop() override;
bool trigger() override;
// Runs a due NUC now (on the worker), while the gimbal moves.
void moving() override;
bool frameReady() override;
void setFrameCallback(FrameCallback cb) override;
int cameraCount() const override { return 1; }
std::vector<CameraDeviceInfo> deviceInfo() override;

View File

@ -2,13 +2,22 @@
#include "fgc/Logger.h"
#include <chrono>
#include <stdexcept>
namespace fgc {
namespace {
long long steadyMs() {
using namespace std::chrono;
return duration_cast<milliseconds>(steady_clock::now().time_since_epoch()).count();
}
} // namespace
CompositeCameraSource::CompositeCameraSource(std::vector<std::unique_ptr<ICameraSource>> children,
std::vector<int> counts)
: children_(std::move(children)), counts_(std::move(counts)) {
std::vector<int> counts, int follower_wait_ms)
: children_(std::move(children)), counts_(std::move(counts)),
follower_wait_ms_(follower_wait_ms) {
if (children_.empty()) throw std::invalid_argument("CompositeCameraSource needs a camera");
if (counts_.empty())
for (const auto& c : children_) counts_.push_back(c->cameraCount());
@ -57,6 +66,16 @@ void CompositeCameraSource::stop() {
}
bool CompositeCameraSource::trigger() {
bool followers_ready = true;
for (size_t k = 1; k < children_.size(); ++k) followers_ready &= children_[k]->frameReady();
if (!followers_ready) {
const long long now = steadyMs();
if (waiting_since_ == 0) waiting_since_ = now;
if (now - waiting_since_ < follower_wait_ms_) return false; // asked again next tick
LOG_DEBUG << "camera: a follower had no fresh frame for " << now - waiting_since_
<< " ms; taking the picture without it";
}
waiting_since_ = 0;
in_trigger_ = true;
shot_ts_ = 0;
const bool taken = children_[0]->trigger();
@ -66,6 +85,16 @@ bool CompositeCameraSource::trigger() {
return taken;
}
void CompositeCameraSource::moving() {
for (auto& c : children_) c->moving();
}
bool CompositeCameraSource::frameReady() {
for (auto& c : children_)
if (!c->frameReady()) return false;
return true;
}
void CompositeCameraSource::setFrameCallback(FrameCallback cb) {
callback_ = std::move(cb);
// The leader's frames pass through untouched (they can be 24 MB); its

View File

@ -13,6 +13,7 @@
#include <deque>
#include <clocale>
#include <fstream>
#include <map>
#include <mutex>
#include <sstream>
#include <thread>
@ -70,6 +71,11 @@ private:
constexpr long long kLogEveryMs = 60000; // repeated trouble: one line a minute
constexpr long long kTelemetryEveryMs = 5000;
// Diagnostics for pictures that get no thermal frame: a pause in the frame
// stream this long is logged with what the camera said meanwhile, and a summary
// of the stream goes out this often.
constexpr long long kGapLogMs = 500;
constexpr long long kSummaryEveryMs = 600000;
} // namespace
@ -81,6 +87,7 @@ TarisCameraSource::Options TarisCameraSource::fromSpec(const CameraSpec& spec) {
o.max_frame_age_ms = spec.max_frame_age_ms;
o.library = spec.library;
o.interface = spec.interface;
o.nuc_interval_ms = spec.nuc_interval_s * 1000;
return o;
}
@ -127,6 +134,19 @@ struct TarisCameraSource::Impl {
bool zeros_logged = false;
std::string last_offline_logged;
// Stream diagnostics (see kGapLogMs). Since the last frame taken:
double last_cam_ts = -1.0; // its MemInfo time, ms
long long polls_since_frame = 0;
std::map<Ret, int> codes_since_frame; // getdata answers other than a new frame
// NUC run by the program (Options::nuc_interval_ms).
int nuc_interval_found = -1; // the camera's own timer, put back on close
bool nuc_ours = false; // the camera's timer is off and we run it
long long last_nuc_ms = 0; // steady
// For the summary:
long long sum_frames = 0, sum_gaps = 0, sum_nucs = 0, sum_max_gap = 0;
double sum_cam_span = 0.0; // camera clock covered by sum_frames
long long sum_since_ms = 0;
// ---- shared, under m -------------------------------------------------
std::mutex m;
std::condition_variable cv;
@ -137,14 +157,16 @@ struct TarisCameraSource::Impl {
CameraDeviceInfo info;
std::deque<long long> arrivals; // steady ms of recent frames, for fps
long long missed_shots = 0;
long long missed_logged_ms = 0;
long long missed_since_log = 0;
long long packets_lost = 0;
int cal_wanted = -1; // a pending setFeature("CalibProfile")
bool cal_done = false;
bool cal_ok = false;
bool reset_wanted = false;
uint64_t connects = 0;
bool nuc_now = false; // a NUC of ours is running
long long nuc_now_since = 0; // steady ms it began
bool move_started = false; // moving() since the worker last looked
long long missed_window = 0; // misses since the last summary
std::atomic<bool> running{false};
std::atomic<bool> gate{false};
@ -176,7 +198,9 @@ struct TarisCameraSource::Impl {
void pause(long long ms) {
std::unique_lock<std::mutex> l(m);
cv.wait_for(l, std::chrono::milliseconds(ms),
[this] { return !running.load() || cal_wanted >= 0 || reset_wanted; });
[this] {
return !running.load() || cal_wanted >= 0 || reset_wanted || move_started;
});
}
// ---- parameter helpers -------------------------------------------------
@ -407,9 +431,14 @@ struct TarisCameraSource::Impl {
if (!model.empty()) info.model = model;
++connects;
}
takeOverNuc();
kpc = 0.0;
scale_tries = 0;
last_sdk_ts = -1.0;
last_cam_ts = -1.0;
polls_since_frame = 0;
codes_since_frame.clear();
zeros_logged = false;
last_frame_steady = steadyMs(); // grace for the first frame
return true;
@ -418,12 +447,14 @@ struct TarisCameraSource::Impl {
// ---- frames ------------------------------------------------------------
// Take the newest frame from the SDK, if there is one.
void poll() {
++polls_since_frame;
// As before every getdata in the vendor's examples: without it the SDK
// now and then answers 0x80000001 instead of a frame.
setInt32(param::kPollingEnable, 1);
Handle mem = 0;
const Ret r = api->dev_getdata(dev, kMemIrDigitFrame, &mem);
if (!ok(r)) {
++codes_since_frame[r];
if (r == kErrNucOrLicence) {
++nuc_or_licence;
const long long now = steadyMs();
@ -461,7 +492,10 @@ struct TarisCameraSource::Impl {
mi.image_num = -1;
mi.timestamp_ms = -1.0;
}
if (mi.timestamp_ms >= 0.0 && mi.timestamp_ms == last_sdk_ts) return; // seen it
if (mi.timestamp_ms >= 0.0 && mi.timestamp_ms == last_sdk_ts) { // seen it
++codes_since_frame[kSuccess];
return;
}
last_sdk_ts = mi.timestamp_ms;
const auto* counts = reinterpret_cast<const uint16_t*>(ptr);
@ -493,6 +527,7 @@ struct TarisCameraSource::Impl {
f.thermal = t;
const long long now = steadyMs();
noteFrame(now, mi.timestamp_ms);
last_frame_steady = now;
{
std::lock_guard<std::mutex> l(m);
@ -545,6 +580,137 @@ struct TarisCameraSource::Impl {
}
}
// ---- NUC ---------------------------------------------------------------
// Switch the camera's own NUC timer off: from now on the program runs the
// NUC, while the gimbal moves. The timer's value goes back on close. (Read
// back right after a set, the camera still reports the old value.)
void takeOverNuc() {
nuc_ours = false;
if (o.nuc_interval_ms <= 0) {
LOG_INFO << "thermal camera: NUC left to the camera's own timer";
return;
}
int32_t found = -1;
if (!getInt32(param::kNucInterval, found)) {
LOG_WARN << "thermal camera: cannot read NUC_Interval; NUC left to the camera";
return;
}
if (nuc_interval_found < 0) nuc_interval_found = found;
const Ret r = setInt32(param::kNucInterval, 0);
if (!ok(r)) {
LOG_WARN << "thermal camera: cannot switch the camera's NUC timer off (" << describe(r)
<< "); a picture may now and then fall into a NUC";
return;
}
nuc_ours = true;
last_nuc_ms = steadyMs() - o.nuc_interval_ms; // the first move runs one
LOG_INFO << "thermal camera: NUC every " << o.nuc_interval_ms / 1000
<< " s at the start of a move (camera's own timer: " << found << " s, now off)";
}
// Put the camera's timer back, so it corrects itself without us.
void handBackNuc() {
if (!nuc_ours || nuc_interval_found <= 0) return;
const Ret r = setInt32(param::kNucInterval, nuc_interval_found);
LOG_INFO << "thermal camera: NUC timer back to " << nuc_interval_found << " s ("
<< describe(r) << ")";
nuc_ours = false;
}
// A NUC when one is due and the gimbal has just started moving, or when it
// is overdue (twice the interval) with no move to wait for.
void serviceNuc() {
bool moved;
{
std::lock_guard<std::mutex> l(m);
moved = move_started;
move_started = false; // always: pause() wakes on it
}
if (!nuc_ours) return;
const long long since = steadyMs() - last_nuc_ms;
if (moved ? since < o.nuc_interval_ms : since < 2LL * o.nuc_interval_ms) return;
runNuc(moved ? "move" : "no move for a while");
}
void runNuc(const char* why) {
const long long start = steadyMs();
{
std::lock_guard<std::mutex> l(m);
nuc_now = true;
nuc_now_since = start;
}
Int2 nuc{kNucMode1Point, 0};
// Returns once the NUC is done, about 0.7 s.
const Ret r = api->dev_setparam(dev, param::kNucPerform, &nuc, kType2Int32);
const long long done = steadyMs();
{
std::lock_guard<std::mutex> l(m);
nuc_now = false;
// A frame from before the NUC is no longer fresh for a picture.
delivered_seq = latest_seq;
}
last_nuc_ms = done;
++sum_nucs;
// A frame is due as soon as the NUC is over; do not count it as a stall.
last_frame_steady = std::max(last_frame_steady, done);
if (!ok(r)) {
LOG_WARN << "thermal camera: NUC failed (" << describe(r)
<< "); handing NUC back to the camera's own timer";
handBackNuc();
return;
}
LOG_DEBUG << "thermal camera: NUC (" << why << ") took " << done - start << " ms";
}
// A new frame: log the pause before it if it was long, and count it.
void noteFrame(long long now, double cam_ts) {
const long long host_gap = now - last_frame_steady;
const double cam_gap = last_cam_ts >= 0.0 && cam_ts >= 0.0 ? cam_ts - last_cam_ts : -1.0;
if (last_cam_ts >= 0.0) {
++sum_frames;
if (cam_gap > 0.0) sum_cam_span += cam_gap;
sum_max_gap = std::max(sum_max_gap, host_gap);
// A pause of our own making is no news.
const bool ours = now - last_nuc_ms < 2000;
if ((host_gap > kGapLogMs || cam_gap > kGapLogMs) && !ours) {
++sum_gaps;
std::string codes;
for (const auto& [code, n] : codes_since_frame)
codes += (codes.empty() ? "" : ", ") +
(code == kSuccess ? std::string("same frame") : describe(code)) +
" x" + std::to_string(n);
LOG_INFO << "thermal camera: frame gap " << host_gap << " ms (camera clock "
<< static_cast<long long>(cam_gap) << " ms), " << polls_since_frame
<< " polls: " << (codes.empty() ? "-" : codes);
}
}
last_cam_ts = cam_ts;
polls_since_frame = 0;
codes_since_frame.clear();
if (now - sum_since_ms >= kSummaryEveryMs) summary(now);
}
void summary(long long now) {
if (sum_since_ms != 0 && sum_frames > 0) {
long long missed;
{
std::lock_guard<std::mutex> l(m);
missed = missed_window;
missed_window = 0;
}
const double fps = sum_cam_span > 0.0 ? sum_frames * 1000.0 / sum_cam_span : 0.0;
LOG_INFO << "thermal camera, last " << (now - sum_since_ms) / 60000 << " min: "
<< sum_frames << " frames (" << std::lround(fps * 10) / 10.0
<< " fps by the camera's clock), longest gap " << sum_max_gap << " ms, "
<< sum_gaps << " unexplained over " << kGapLogMs << " ms, " << sum_nucs
<< " NUC" << (nuc_ours ? " (ours)" : "") << ", "
<< missed << " picture(s) without a thermal frame";
}
sum_frames = sum_gaps = sum_nucs = sum_max_gap = 0;
sum_cam_span = 0.0;
sum_since_ms = now;
}
void telemetry() {
float k = 0.f;
if (getSingle(param::kCameraTempHousing, k) && k > 0.f) {
@ -594,6 +760,7 @@ struct TarisCameraSource::Impl {
// retry pause from pausing.
std::lock_guard<std::mutex> l(m);
reset_wanted = false;
move_started = false; // nor may a move cut the retry pause short
}
std::string why;
bool up;
@ -614,6 +781,7 @@ struct TarisCameraSource::Impl {
}
serviceRequests();
if (!connected) continue;
serviceNuc();
poll();
const long long now = steadyMs();
@ -631,6 +799,7 @@ struct TarisCameraSource::Impl {
pause(poll_ms);
}
LocaleGuard keep;
if (connected) handBackNuc();
disconnect();
if (sdk_init) api->dll_uninit();
sdk_init = false;
@ -700,26 +869,48 @@ void TarisCameraSource::stop() { impl_->gate = false; }
void TarisCameraSource::setFrameCallback(FrameCallback cb) { impl_->callback = std::move(cb); }
void TarisCameraSource::moving() {
{
std::lock_guard<std::mutex> l(impl_->m);
impl_->move_started = true;
}
impl_->cv.notify_all(); // the worker may be pausing between polls
}
bool TarisCameraSource::frameReady() {
Impl& d = *impl_;
std::lock_guard<std::mutex> l(d.m);
if (!d.running || !d.why_offline.empty()) return true; // nothing to wait for
if (d.nuc_now) return false;
return d.latest_seq > d.delivered_seq &&
unixMs() - d.latest.timestamp_ms <= d.o.max_frame_age_ms;
}
bool TarisCameraSource::trigger() {
Impl& d = *impl_;
if (!d.gate || !d.callback) return false;
Frame f;
{
std::lock_guard<std::mutex> l(d.m);
const bool fresh = d.latest_seq > d.delivered_seq &&
unixMs() - d.latest.timestamp_ms <= d.o.max_frame_age_ms;
const bool new_frame = d.latest_seq > d.delivered_seq;
const long long age = d.latest_seq ? unixMs() - d.latest.timestamp_ms : -1;
const bool fresh = new_frame && age <= d.o.max_frame_age_ms;
if (!fresh) {
// Every miss, with what tells its cause apart: the stream paused (and
// whether a NUC was running), or nothing came since the last picture.
++d.missed_shots;
++d.missed_since_log;
++d.missed_window;
d.info.frames_dropped = d.missed_shots + d.packets_lost;
const long long now = steadyMs();
if (now - d.missed_logged_ms >= kLogEveryMs) {
LOG_WARN << "thermal camera: no fresh frame for " << d.missed_since_log
<< " picture(s) in the last minute"
<< (d.why_offline.empty() ? "" : " (" + d.why_offline + ")");
d.missed_logged_ms = now;
d.missed_since_log = 0;
}
std::string why = !d.why_offline.empty()
? d.why_offline
: "newest frame " + std::to_string(age) + " ms old (limit " +
std::to_string(d.o.max_frame_age_ms) + " ms)" +
(new_frame ? "" : ", none since the last picture") +
(d.nuc_now ? ", NUC in progress for " +
std::to_string(now - d.nuc_now_since) + " ms"
: "");
LOG_WARN << "thermal camera: no thermal frame for this picture: " << why;
return false;
}
f = d.latest;

View File

@ -110,6 +110,8 @@ void CaptureScheduler::sendMove() {
// form ("ERR axis not present") and moves nothing, so address yaw alone.
motor_.sendCommand(moveCommand(yaw_target_, pitch_present_ ? std::optional<long>(pitch_target_)
: std::nullopt));
// The time the move and the settle wait take is free for the cameras.
camera_.moving();
}
bool CaptureScheduler::settledAt(const MotorTelemetry& t) const {

View File

@ -393,6 +393,7 @@ AppConfig ConfigLoader::fromMap(const std::map<std::string, std::string>& kv) {
c.frame_rate_hz = getDouble(kv, pre + "frame_rate_hz", c.frame_rate_hz);
c.max_frame_age_ms = getInt(kv, pre + "max_frame_age_ms", c.max_frame_age_ms);
c.library = get(kv, pre + "library", c.library);
c.nuc_interval_s = getInt(kv, pre + "nuc_interval_s", c.nuc_interval_s);
c.interface = get(kv, pre + "interface");
// Per-profile timing: Camera.<name>.<profile>.settle_wait_ms
@ -434,6 +435,9 @@ AppConfig ConfigLoader::fromMap(const std::map<std::string, std::string>& kv) {
throw std::runtime_error("Camera." + name + ".frame_rate_hz must be in (0, 50]");
if (c.max_frame_age_ms <= 0)
throw std::runtime_error("Camera." + name + ".max_frame_age_ms must be > 0");
if (c.nuc_interval_s < 0 || c.nuc_interval_s > 3600)
throw std::runtime_error("Camera." + name + ".nuc_interval_s must be in 0..3600 "
"(0 = the camera's own timer)");
if (c.library.empty())
throw std::runtime_error("Camera." + name + ".library must not be empty");
}

View File

@ -2,7 +2,9 @@
#include <doctest/doctest.h>
#include <chrono>
#include <stdexcept>
#include <thread>
#include <vector>
using namespace fgc;
@ -20,6 +22,8 @@ struct FakeSource : ICameraSource {
bool throws_on_open = false;
int triggers = 0;
bool started = false;
bool frame_ready = true; // frameReady()
int moves = 0;
std::string why_offline;
FrameCallback cb;
@ -32,6 +36,8 @@ struct FakeSource : ICameraSource {
void close() override {}
void start() override { started = true; }
void stop() override { started = false; }
void moving() override { ++moves; }
bool frameReady() override { return frame_ready; }
bool trigger() override {
++triggers;
if (!ready) return false;
@ -172,6 +178,44 @@ TEST_CASE("the indices are the configured ones, also before the leader has opene
CHECK_THROWS(CompositeCameraSource(std::vector<std::unique_ptr<ICameraSource>>{}, {}));
}
TEST_CASE("a picture waits a moment for a follower between frames, then goes without it") {
auto c = std::make_unique<FakeSource>(1, 1000);
auto t = std::make_unique<FakeSource>(1, 5000);
FakeSource* colour = c.get();
FakeSource* thermal = t.get();
std::vector<std::unique_ptr<ICameraSource>> v;
v.push_back(std::move(c));
v.push_back(std::move(t));
CompositeCameraSource src(std::move(v), {1, 1}, 60);
std::vector<Frame> frames;
src.setFrameCallback([&](const Frame& f) { frames.push_back(f); });
// The thermal camera is in its NUC: the picture is held, the leader not asked.
thermal->frame_ready = false;
CHECK_FALSE(src.trigger());
CHECK(colour->triggers == 0);
CHECK_FALSE(src.frameReady());
// It comes back within the wait: one whole pair.
thermal->frame_ready = true;
REQUIRE(src.trigger());
CHECK(frames.size() == 2);
// It does not come back: after the wait the colour picture goes alone.
frames.clear();
thermal->frame_ready = false;
CHECK_FALSE(src.trigger());
std::this_thread::sleep_for(std::chrono::milliseconds(80));
REQUIRE(src.trigger());
CHECK(colour->triggers == 2);
}
TEST_CASE("every source hears that the gimbal moves") {
Rig r;
r.src->moving();
CHECK(r.colour->moves == 1);
CHECK(r.thermal->moves == 1);
}
TEST_CASE("start and stop reach every source") {
Rig r;
r.src->start();

View File

@ -312,6 +312,7 @@ TEST_CASE("A thermal camera needs no settings XML and is always lossless") {
CHECK(tir.max_frame_age_ms == 400);
CHECK(tir.library == "libirbgrab_x86_64-linux.so");
CHECK(tir.interface == "enp1s0");
CHECK(tir.nuc_interval_s == 30);
// [Camera] jxl_distance is for colour cameras; counts default to lossless.
CHECK(tir.jxl_distance == 0.0);
CHECK(c.camera.cameras[0].jxl_distance == doctest::Approx(0.8));
@ -368,6 +369,8 @@ TEST_CASE("Thermal camera blocks reject what cannot work") {
// An empty value means the default, as for every key.
CHECK(ConfigLoader::fromMap(base({{"Camera.tir.library", ""}})).camera.cameras[1].library ==
"libirbgrab_x86_64-linux.so");
CHECK_THROWS(ConfigLoader::fromMap(base({{"Camera.tir.nuc_interval_s", "-1"}})));
CHECK_NOTHROW(ConfigLoader::fromMap(base({{"Camera.tir.nuc_interval_s", "0"}})));
// Two cameras, one output folder: their pictures would overwrite each other.
CHECK_THROWS(ConfigLoader::fromMap(base({{"Camera.rgb.label", "X"}, {"Camera.tir.label", "X"}})));
// A disabled thermal camera needs no serial, like any other.

View File

@ -26,6 +26,7 @@ struct FakeMotor : IMotorController {
struct FakeCamera : ICameraSource {
int triggers = 0;
int moves = 0;
bool ok = true;
// A stream of frames for the keep rule; empty = a camera with no frame data.
std::vector<FrameStats> stream;
@ -44,6 +45,7 @@ struct FakeCamera : ICameraSource {
void start() override {}
void stop() override {}
bool trigger() override { ++triggers; return ok; }
void moving() override { ++moves; }
void setFrameCallback(FrameCallback) override {}
int cameraCount() const override { return 1; }
};
@ -85,6 +87,7 @@ TEST_CASE("CaptureScheduler sweeps the scan grid with MOVE + settle + trigger")
CHECK(motor.cmds[0] == "MOVE 0,0");
CHECK(sch.moving());
CHECK(cam.triggers == 0); // not yet settled
CHECK(cam.moves == 1); // the cameras hear of every move (the thermal NUC)
// At rest early: the frame waits for the interval, counted from the mode start.
settleAt(motor, 0, 0);
@ -103,6 +106,7 @@ TEST_CASE("CaptureScheduler sweeps the scan grid with MOVE + settle + trigger")
clock = 1010;
sch.tick();
CHECK(motor.cmds.back() == "MOVE 900,0");
CHECK(cam.moves == 2);
CHECK(sch.overruns() == 0);
}

View File

@ -35,6 +35,9 @@ struct FakeSdk {
uint32_t w = 8, h = 4;
Ret no_frame = kErrNotFound; // what getdata says before any frame
std::atomic<bool> frozen{false}; // stop producing frames (stall)
std::atomic<int> nuc_interval{30}; // the camera's own NUC timer, s
std::atomic<int> nucs{0}; // NUC_Perform calls
std::atomic<int> nuc_mode{0};
void push(uint16_t v, uint16_t hot = 0) {
std::lock_guard<std::mutex> l(m);
@ -98,6 +101,7 @@ Ret fGetParam(Handle, uint32_t what, void* data, uint32_t type) {
return kSuccess;
}
if (what == param::kCameraTempHousing && type == kTypeSingle) { *static_cast<float*>(data) = 308.15f; return kSuccess; }
if (what == param::kNucInterval && type == kTypeInt32) { *static_cast<int32_t*>(data) = g->nuc_interval; return kSuccess; }
return kErrNotSupported;
}
Ret fSetParam(Handle, uint32_t what, void* data, uint32_t type) {
@ -109,6 +113,17 @@ Ret fSetParam(Handle, uint32_t what, void* data, uint32_t type) {
}
if (what == param::kFramerateHz && type == kTypeSingle) { g->rate = *static_cast<float*>(data); return kSuccess; }
if (what == param::kPollingEnable) return kSuccess;
if (what == param::kNucInterval && type == kTypeInt32) {
g->nuc_interval = *static_cast<int32_t*>(data);
return kSuccess;
}
if (what == param::kNucPerform && type == kType2Int32) {
// As the camera: the call returns when the NUC is done.
g->nuc_mode = static_cast<Int2*>(data)->value1;
++g->nucs;
std::this_thread::sleep_for(std::chrono::milliseconds(20));
return kSuccess;
}
return kErrNotSupported;
}
// A memory object is a copy of the newest frame.
@ -368,6 +383,82 @@ TEST_CASE("a camera that goes quiet is reconnected") {
CHECK(t.sdk.connects >= 2);
}
TEST_CASE("the program takes over the camera's NUC timer and hands it back") {
{
Harness t;
t.cam.open();
t.sdk.push(29315);
REQUIRE(t.online());
CHECK(t.sdk.nuc_interval == 0); // the camera's own timer is off
t.cam.close();
CHECK(t.sdk.nuc_interval == 30); // and back as found
}
{
auto o = fastOptions();
o.nuc_interval_ms = 0; // leave it to the camera
Harness t(o);
t.cam.open();
t.sdk.push(29315);
REQUIRE(t.online());
t.cam.moving();
std::this_thread::sleep_for(std::chrono::milliseconds(100));
CHECK(t.sdk.nuc_interval == 30);
CHECK(t.sdk.nucs == 0);
}
}
TEST_CASE("a move runs a due NUC, and no more often than the interval") {
auto o = fastOptions();
o.nuc_interval_ms = 400;
Harness t(o);
t.cam.open();
t.cam.start();
t.sdk.push(29315);
REQUIRE(t.online());
CHECK(t.sdk.nucs == 0);
t.cam.moving(); // the first move after connecting runs one
CHECK(waitFor([&] { return t.sdk.nucs == 1; }, 1000));
CHECK(t.sdk.nuc_mode == kNucMode1Point);
t.cam.moving(); // not due yet
std::this_thread::sleep_for(std::chrono::milliseconds(150));
CHECK(t.sdk.nucs == 1);
std::this_thread::sleep_for(std::chrono::milliseconds(300));
t.cam.moving(); // due again
CHECK(waitFor([&] { return t.sdk.nucs == 2; }, 1000));
}
TEST_CASE("an overdue NUC runs without a move") {
auto o = fastOptions();
o.nuc_interval_ms = 100;
Harness t(o);
t.cam.open();
t.sdk.push(29315);
REQUIRE(t.online());
// Twice the interval with no move at all (manual mode, say).
CHECK(waitFor([&] { return t.sdk.nucs >= 1; }, 1500));
}
TEST_CASE("after a NUC only a frame from after it is taken") {
auto o = fastOptions();
o.nuc_interval_ms = 400;
Harness t(o);
t.cam.open();
t.cam.start();
t.sdk.push(29315);
REQUIRE(t.online());
CHECK(t.cam.frameReady());
t.sdk.frozen = true; // nothing new comes during and after the NUC
t.cam.moving();
REQUIRE(waitFor([&] { return t.sdk.nucs == 1; }, 1000));
std::this_thread::sleep_for(std::chrono::milliseconds(50));
CHECK_FALSE(t.cam.frameReady()); // the frame from before the NUC is stale
CHECK_FALSE(t.cam.trigger());
t.sdk.frozen = false;
t.sdk.push(29400);
CHECK(waitFor([&] { return t.cam.frameReady(); }, 1000));
CHECK(t.shot());
}
TEST_CASE("a missing SDK library leaves the camera offline, not the program down") {
auto o = fastOptions();
o.library = "libdoes_not_exist_irbgrab.so";