AgriChrono: A Multi-modal Dataset Capturing Crop Growth and Lighting Variability with a Field Robot

TL;DR

we propose AgriChrono, a comprehensive framework purpose-built for "in-the-wild" scene-level 3D reconstruction, featuring three core contributions: a custom robotic platform, an 18 TB multi-modal dataset, and a stress-test benchmark. First, we deploy a modular robotic platform equipped with a highly accurate, time-synchronized sensor suite (LiDAR-RGB-Depth-IMU) to continuously log the high-fidelity 6-DoF poses and structural data required for reliable 3D reconstruction. Second, utilizing this platform, we release a massive dataset that rigorously captures continuous morphological crop growth and severe daily illumination shifts over a month. Finally, we establish a 7-scenario 3D reconstruction benchmark to evaluate 16 state-of-the-art NeRF and 3DGS methods. While validating our platform's data-gathering efficacy, this benchmark explicitly exposes critical robustness issues in existing baselines, demonstrating their failure to maintain consistent performance under wind-induced crop motion and lighting variations. Ultimately, AgriChrono provides a crucial stress-test to catalyze the development of robust, next-generation in-the-wild 3D reconstruction models.

Key Contributions

• Modular Robotic Acquisition Platform — we develop a custom, remote-operable field robot equipped with a precise, time-synchronized multi-modal sensor suite (RGB, Depth, LiDAR, IMU, and Pose). This platform serves as a foundational tool, validating that high-fidelity 3D reconstruction is achievable in challenging, "in-the-wild" agricultural environments.
• Large-Scale In-the-Wild Dataset — we release the comprehensive 18 TB AgriChrono dataset, uniquely capturing continuous morphological crop growth and systematic daily illumination changes. Collected daily over a one-month period, this dataset documents complex, scene-level structural changes to significantly accelerate real-world precision agriculture research.
• 3D Reconstruction Stress-Test Benchmark — we introduce a rigorous benchmark featuring 7 distinct scenarios to evaluate 16 state-of-the-art NeRF and 3DGS methods. This benchmark illustrates the significant challenges of non-rigid, dynamic scenes, explicitly exposing the fundamental robustness limitations of existing baselines against wind-induced motion and lighting shifts.

Project Design

This project was conducted across three distinct crop sites (Canola, Canola Genotypes, and Flax) at the North Dakota State University (NDSU) experimental farm. To enable agricultural researchers to operate the robot effortlessly without terminal commands, we developed a custom WebRTC-based low-latency Web UI using Flask and Janus. Initially, we intended to use a single ZED X camera for both data acquisition and monitoring. However, this approach presented several constraints: the ZED SDK's lack of simultaneous processing support, high latency in ROS implementation, storage capacity issues due to ROSBAG's low compression rate, and a fixed camera field-of-view (FoV). To resolve this, ZED data saving was restricted exclusively to its highly compressed .svo2 format, and an independent Obsbot PTZ camera was installed specifically for monitoring, thereby reducing system load and enhancing teleoperation convenience. (For an overview, see GitHub.)

Built on a Scout UGV base, the platform integrates an NVIDIA Jetson AGX Orin, dual ZED X cameras, and Livox LiDAR. To meet varying sensor voltage needs from the robot's 24–27V output, we used DC-DC converters to supply 12V (LiDAR) and 18V (Jetson). This enabled over 4 hours of continuous operation per charge, supporting our rigorous daily schedule. Due to severe GPS interference from the nearby Hector International Airport, we omitted our planned RTK-GNSS module, pivoting to purely vision and IMU-based localization (AprilTags planned for future work).
(See Hardware GitHub.)

To manage the robot remotely nationwide, we used Tailscale VPN for static IP assignment, ensuring consistent connectivity. We also implemented a fail-safe mechanism that instantly halts the robot upon communication loss, prioritizing safe unattended operation. These optimizations enabled stable, long-range teleoperation from UCLA — 1,500 miles away — yielding a massive 18 TB dataset.
(See Software GitHub.)

To document continuous morphological crop growth and natural illumination variations, data were systematically collected four times daily over a three-week period. To counteract extreme field temperatures exceeding 100°F, we installed a sun-shielded top panel for thermal protection. However, this passive cooling was not completely sufficient during prolonged operations, occasionally resulting in FPS degradation due to thermal throttling (as future work, we plan to upgrade the thermal management system, including improved active ventilation). Collection was suspended during severe weather to prevent hardware failure. When post-rain muddy soil rendered the paths impassable, we manually placed wooden planks along the traversal routes to secure necessary traction, allowing us to maintain a consistent data collection frequency. (See Dataset GitHub.)

To provide a highly reliable benchmark, we established initial geometries via point clouds and utilized Visual-Inertial Odometry (VIO) fusing RGB and IMU data, validated to yield the highest pose accuracy through rigorous comparative experiments. Combining these poses with the RGB images, we constructed and publicly released a 3D reconstruction benchmark in COLMAP format, featuring seven distinct scenarios. Our analysis revealed that under favorable conditions — stable lighting and dense canopy structures — state-of-the-art models achieved high accuracy, effectively validating our sensor pipeline. Conversely, under harsher scenarios (e.g., intense illumination, sparse crops, wind-induced motion), baseline performance fluctuated significantly, demonstrating that existing AI models remain highly vulnerable to unconstrained, "in-the-wild" environments.
(Dataset download: Benchmark GitHub · Leaderboard: Benchmark Page)

Novel View Synthesis Benchmark

Qualitative comparison of novel view synthesis results across evaluated methods.

Conclusion

We present AgriChrono, a comprehensive field robotics framework whose benchmark results serve a dual purpose. First, they validate our robotic platform's design, proving it successfully captures the high-fidelity data required for reliable 3D reconstruction in complex agricultural fields. Second, by explicitly exposing the performance degradation of current baselines under dynamic "in-the-wild" stressors, our benchmark provides a vital stress-test for the AI community. This research acts as a critical stepping stone, catalyzing the development of more robust models capable of unconstrained scene understanding — eventually bridging the gap to extract concrete downstream phenotyping metrics.