Photo from Dr. Steven Mirsky’s LinkedIn
Teaching AI the Language of Plants
Above rows of research plants, BenchBot moves, stopping, imaging, labeling, and moving again. At first glance, it may look like a simple rail-mounted robot gliding over crops, but for USDA’s Agricultural Research Service and its university partners, BenchBot represents a way to teach artificial intelligence how plants grow, struggle, and respond to the world around them.
Developed as a lower-cost, open-access alternative to expensive proprietary phenotyping systems, BenchBot is designed to collect consistent, high-quality plant images at scale. The system can move in three dimensions over crops, trays, or experimental plots, positioning cameras precisely to capture standardized images over time. Those images can be paired with metadata such as time, treatment, genotype, and environmental conditions, making them useful not only for measuring traits like growth, canopy cover, stress, and disease symptoms, but also for training computer-vision models that can recognize those patterns automatically.
That matters because agriculture has a data problem. While industries such as autonomous driving have benefited from massive labeled image datasets, plant science still lacks enough public, high-quality, annotated imagery to train AI systems that work reliably across crops, regions, and field conditions. BenchBot helps address that gap by turning plant phenotyping into a repeatable, scalable data pipeline, one that can feed shared repositories, support USDA’s Digital Agricultural Systems Hub, and eventually help power field-ready tools for breeders, agronomists, and growers.
In that sense, BenchBot is a bridge between traditional agronomic experiments and the future of AI-enabled agriculture, where cameras on tractors, sprayers, drones, or irrigation systems could detect crop stress, disease, weeds, or performance differences earlier and more objectively than the human eye. As Dr. Steven Mirsky explains in the interview, even if BenchBot itself never becomes a field tool for farmers, the models and insights it helps create could shape the next generation of data-driven agricultural decision-making.
Q&A with Dr. Steven Mirsky, Director of Digital Agriculture, USDA’s Agricultural Research Service
Q: How did the idea for BenchBot originate, and what problem were you trying to solve in plant phenotyping or agricultural research? What gaps did you see in existing phenotyping platforms that motivated the development of a lower-cost, open-source system?
A: The idea for BenchBot originated several years ago as a way to help our team standardize and improve the efficiency of collecting images of plants grown in pots in a greenhouse for a breeding program. It quickly became clear that there were no high-throughput phenotyping tools that could integrate with our setup without significant investment. Available options were either extremely expensive proprietary systems, accessible only to well-funded groups, or creative one-off rigs that were difficult to replicate, maintain, or scale.
This led us to develop our own high-throughput phenotyping tool, with the goal of creating a platform that is affordable, replicable, and designed to generate high-quality AI-ready phenotyping data. We wanted a system that breeders, physiologists, pathologists, and agronomists could all use without requiring specialized robotics expertise or a substantial budget. BenchBot is an open-source, modular platform that we are now scaling across the country to support collaborations between ARS, university breeders, and plant scientists, enabling the development of accessible phenotyping solutions for a broad range of stakeholders.
Q: When you look at the long arc of agricultural research, where does BenchBot fit into the evolution of AI and automation in farming?
A: When looking at the long arc of agricultural research, we’ve progressed from clipboards and visual scoring to handheld sensors and, more recently, to large, high-throughput phenotyping systems. BenchBot fits within this transition from isolated, lab-specific instruments to networked platforms that can generate data at scale to support AI-driven insights.
I see BenchBot as a transitional technology that helps build the training data and models needed before fully autonomous systems become widespread in farmers’ fields. It serves as a bridge between traditional agronomic experiments and a future where AI can continuously, objectively, and accurately measure and predict plant performance.
Q: For producers and readers who may not be familiar, how would you describe BenchBot in simple terms?
A: In simple terms, BenchBot is an autonomous robotic imaging platform that helps researchers collect consistent, high-quality data on plants. It is designed to support both general phenotyping and the development of training datasets for computer vision models.
The system runs on a rail-based structure and can move in three dimensions (X, Y, and Z), which allows it to position cameras precisely over plants, trays, or experimental plots. Once set up, it automatically captures standardized images over time, making it possible to measure traits such as growth, stress, and overall plant performance in a consistent and objective way.
BenchBot currently uses RGB cameras, but it is designed to be sensor-agnostic, meaning different types of sensors can be easily integrated depending on the user’s research needs. While it is primarily used in outdoor environments, it can also be adapted for indoor setups.
Because the system is automated and requires minimal labor to operate, it significantly reduces the cost and effort associated with data collection compared to other high-throughput platforms. It also standardizes the entire data pipeline. Images are consistently captured, automatically labeled, and stored with their metadata in a unified format. Since BenchBot systems are connected to the internet, data is uploaded in real time to centralized servers, enabling collaboration and ensuring that datasets collected across different locations are directly comparable.
Q: What are the most important technical innovations behind BenchBot?
A: The most important technical innovations behind BenchBot start with its mechanical design, which is intentionally simple and modular. It is built using standard rails and off-the-shelf components, making it easy to replicate, maintain, and adapt across different environments, including indoor, greenhouse, and semi-field setups.
Second, the sensing system is designed to be flexible and “plug-and-play.” While we commonly use RGB cameras, the platform can accommodate a wide range of sensors, such as multispectral, depth, or thermal, without requiring a complete redesign. This allows the system to evolve alongside research needs.
Third, BenchBot is built around a standardized data and metadata pipeline. Every image is automatically tagged with key contextual information, such as time, treatment, genotype, and environmental conditions. This ensures that the data is immediately usable for AI applications and supports the development of robust, scalable models.
Finally, the system is designed for connectivity and scale. Because BenchBot platforms are networked and upload data in real time to centralized servers, they enable consistent data collection across locations and facilitate collaboration between research teams.
Q: How much of BenchBot is open-source, and why is open-access design important for your goals? What were some of the biggest engineering challenges in creating a robust but affordable imaging platform?
A: BenchBot is designed to be fully open-access. The design files, bill of materials, control software, and data standards are all shared with collaborators. The goal is to enable any lab to build a functionally equivalent system without starting from scratch or becoming dependent on a proprietary vendor.
One of the biggest engineering challenges was balancing ruggedness, precision, and cost. We needed a system that could operate reliably day after day with minimal maintenance, while still upholding the level of stability and accuracy required for precise imaging. At the same time, we aimed to keep the total cost accessible for a variety of stakeholders. Achieving that balance required careful decisions about where to invest in higher-quality components and where a simpler, more cost-effective approach would be sufficient.
Open-access design is central to our broader goals. Rather than creating a single, fixed instrument, we are building a network of interoperable platforms that can be deployed across USDA and partner institutions. These systems can be adapted to different regions, crops, and research needs, while still contributing standardized data into a shared ecosystem that supports collaboration and scalable AI development.
Q: BenchBot was designed as a sub-$20,000 system. What tradeoffs or breakthroughs made that price point possible?
A: We never wanted cost to be a barrier to entry for BenchBot, so we kept budget in mind at every step of the development. That pushed us to prioritize off-the-shelf components wherever possible, using standard linear motion systems, modular framing, and widely available industrial cameras and control electronics instead of custom-built parts.
One of the main challenges was designing a system capable of delivering standardized, high-quality data under highly variable environmental conditions, particularly in outdoor or non-climate-controlled settings. Unlike premium phenotyping platforms that rely on tightly controlled environments, BenchBot had to tolerate changing light, temperature, and other external factors. BenchBot is therefore intentionally compact, modular, fairly rugged, and customizable.
A key breakthrough was recognizing that it is still possible to achieve highly reproducible, high-quality data by carefully designing the interaction between optics, motion, and data capture protocols even within a smaller, lower-cost system.
Q: What kinds of plant traits or characteristics can BenchBot detect or measure?
A: BenchBot is currently used to measure a range of plant traits, including canopy cover, plant height and architecture, color changes, and indicators of plant health such as injury, senescence (plant deterioration), and disease symptoms like lesion development.
With depth sensing or multi-angle imaging, the system can also estimate structural traits such as plant volume, biomass, and overall architecture.
As additional sensing capabilities such as multispectral and depth sensors are integrated, BenchBot is increasingly able to detect early stress indicators, including chlorosis, water stress signals, and early-stage disease symptoms, often before they are visible to the human eye.
The ultimate goal is to translate images into reliable, quantitative traits that breeders and agronomists can use in their research and breeding programs.
Q: How many images can a BenchBot collect in a day, and what scale of dataset are you building? You’ve talked about the shortage of labeled agricultural images compared to other industries. Why is this such a bottleneck?
A: Depending on the configuration and data collection protocol, a single BenchBot can capture tens of thousands of images per day. When scaled across multiple deployments and growing seasons, this quickly translates into datasets of millions of images spanning different crops, growth stages, management practices, and stress conditions.
Agriculture still lags behind sectors like autonomous driving or facial recognition in terms of large, well-labeled image datasets. One of the main reasons is the inherent complexity of agricultural systems. Field trials vary widely, growing seasons are limited, and plants can appear very similar to one another, making consistent annotation challenging. In addition, accurate labeling often requires specialized agronomic and botanical expertise.
These factors make high-quality, curated datasets both difficult and expensive to produce. Without sufficient labeled data, AI models struggle to generalize across environments, varieties, and management conditions.
BenchBot is designed to address this bottleneck by enabling the large-scale, standardized collection of annotated plant images across diverse locations. By building a geographically distributed and consistent dataset, we can support the development of more robust and transferable AI models for agriculture.
Q: What types of AI models are being trained with BenchBot data (weed ID, biomass estimation, stress detection, etc.)?
A: We’re using BenchBot data to train a variety of AI models for different applications in plant research:
Classification and detection models: These identify disease symptoms, nutrient deficiencies, or specific plant structures.
Segmentation models: These separate plants from complex backgrounds and quantify traits like leaf area and canopy cover. Segmentation also enables the creation of synthetic plant and field images, which can accelerate AI training without waiting for natural imagery to appear in a plot or field.
Regression models: These predict biomass, yield components, or physiological traits based on image features captured at specific growth stages or seasonal timings.
Multi-task and multi-modal models: These combine imagery with environmental and management data to provide more comprehensive predictions and analyses.
The same datasets support a wide range of use cases, including weed identification, biomass estimation, stress detection, and trait scoring for breeding programs. By standardizing and annotating images at scale, BenchBot allows AI models to be more accurate and generalizable across crops, environments, and conditions.
Q: How important are 3D imaging and depth data to future agricultural AI?
A: 3-D imaging and depth data will be increasingly important as agriculture moves from static snapshots to understanding plant structure and dynamics over time. Many traits that breeders and agronomists care about, such as plant architecture, fruit set, or biomass distribution, are inherently three-dimensional.
Depth information also helps resolve situations where 2-D images can be misleading, such as overlapping leaves, mixed canopies, or complex intercropping systems. For autonomous systems like robots and drones, depth data is critical not only for accurate measurement but also for navigation and targeted interventions. BenchBot provides a semi-controlled environment to develop and test these capabilities before deploying them in more unpredictable field conditions.
Q: What is the USDA’s broader vision for automated phenotyping and national image repositories? How will BenchBot deployments across different ARS stations improve agricultural research nationwide?
A: At USDA, the broader vision is to build a national infrastructure for digital agriculture. This includes automated phenotyping platforms, standardized data pipelines, and shared image repositories that the research community can access, customize, and leverage.
Deploying BenchBots across different ARS locations creates a federated phenotyping network. While each site operates under the same core hardware and protocols, they can focus on diverse environments, germplasm, and management practices. This approach produces datasets that are far more comprehensive and representative than any single location could generate.
Such a network also fosters interdisciplinary collaboration. Breeders, pathologists, soil scientists, and agronomists across the country can contribute to and benefit from the same digital ecosystem, rather than developing isolated, incompatible systems. In this way, BenchBot deployments are helping to standardize and scale agricultural research nationwide.
Q: How does this project integrate with USDA’s SCINet high-performance computing system?
A: SCINet serves as the computational backbone for much of what we do. BenchBot generates large volumes of images and associated metadata, and SCINet provides the storage, high-performance computing, and secure access needed to train and evaluate AI models at scale.
Our vision is for a researcher to run an experiment locally, sync their BenchBot data into a centralized repository, and then use SCINet to perform training, model benchmarking, and cross-site analyses. This tight integration, from instrument to data pipeline to high-performance computing, transforms BenchBot from a clever imaging robot into a scalable engine for digital agriculture solutions nationwide.
Q: What does it mean for U.S. agriculture that this dataset will be open-access?
A: Open access changes the game for U.S. agriculture. It allows AI developers, breeders, agronomists, and startups to build on the same high-quality, curated image datasets without each having to reinvent the wheel or invest in their own large phenotyping facilities.
For the agricultural sector, this public access can accelerate innovation and level the playing field, enabling small companies and public institutions to compete based on model quality rather than the size of their private data silos. It also promotes transparency and reproducibility: models trained on publicly available BenchBot data can be validated, extended, or challenged by the broader research community, fostering more reliable and vetted solutions.
Q: How do academic partners, tech collaborators, and USDA each contribute to the BenchBot ecosystem?
A: USDA provides the core mission, infrastructure, and long-term commitment to the public good. We run experiments, host BenchBots at research stations, and ensure that the data we collect addresses agronomic questions that matter to American farmers.
Academic partners contribute regional expertise in plant science, genetics, and physiology, and often lead the development of new traits and AI model architectures.
Tech collaborators bring strengths in hardware engineering, cloud and edge computing, MLOps, and business strategy, helping make BenchBot and its surrounding software robust, user-friendly, and scalable.
This is very much an ecosystem approach: no single organization can do it all alone. Together, USDA, academic institutions, and technology partners create a collaborative network that advances phenotyping, AI development, and agricultural research at scale.
Q: What could a world with widespread AI-powered imaging look like for growers?
A: AI-powered imaging systems could continuously monitor every plant and field, capturing growth, stress signals, disease symptoms, and yield potential in real time. Problems are flagged early, before they become apparent to the human eye, and precise recommendations are delivered for irrigation, fertilization, pest control, or harvest decisions. Weed hotspots, nutrient deficiencies, and hybrid performance are tracked continuously across different conditions.
In this scenario, the grower’s role shifts from monitoring to decision-making. AI synthesizes millions of observations into actionable insights, turning complex plant data into a practical guide for every field operation. The result is a farm that is more resilient, efficient, and productive where technology works seamlessly with agronomic expertise to optimize yield, sustainability, and profitability.
Q: Do you see tools emerging from this research that will reach farmers in the next 5-10 years?
A: Absolutely. Some tools are already beginning to emerge. Over the next five to 10 years, I expect to see field-ready camera systems integrated into tractors, sprayers, and irrigation equipment that assess crop conditions in real time. Decision-support tools will use sensor-detected plant traits to guide recommendations for seeding, fertilization, weed control, and variety selection. For specialty and high-value crops, more targeted monitoring of disease, nutrient status, and water stress will become common.
Even if BenchBot itself isn’t in the field, the models and insights it generates will power many of these tools, enabling farmers to make faster, more precise, and data-driven decisions.
Q: What crops or regions do you think will benefit the most early on?
A: Early benefits are likely to appear in areas with strong research infrastructure, high-stakes management decisions, and clear potential return on investment. This includes major row crops such as corn, soybeans, wheat, and cotton in the Midwest, Great Plains, and parts of the South, as well as high-value specialty crops.
Regions with intensive management and variable conditions, such as areas facing water scarcity, complex rotations, or high pest and disease pressure, are particularly well-suited for AI-driven monitoring. Anywhere that small differences in timing or treatment can have significant economic or environmental consequences stands to gain the most from real-time image-based insights.
Q: Has the BenchBot revealed anything surprising about plant growth, stress responses, or traits so far? When you imagine U.S. agriculture 10-20 years from now, how central do you think AI-enabled imagery will be?
A: One of the most striking findings from BenchBot is how AI models can detect subtle changes in color, texture, or structure well before a human would recognize a plant as “stressed.” While these features may appear in the images, the real value lies in the ability of AI to interpret them and flag stress early. This reveals a wealth of plant signals that we’ve historically not measured quantitatively or even been aware of.
Looking 10 to 20 years ahead, I expect AI-enabled imagery to be as central to crop management as soil testing and yield monitors are today, and perhaps even more so. It will be embedded across drones, tractors, scouting tools, processing facilities, and satellites. BenchBot is helping us teach AI the language of plant signals, laying the foundation for future systems that are fluent in it, able to interpret and act on it rapidly and reliably.
Q: What originally drew you to agricultural robotics and phenotyping?
A: I’ve always been drawn to the intersection of ecology, agronomy, and technology. Early in my career, I worked on more traditional, isolated research projects, focusing on individual experiments or specific traits. Over time, I realized that working in isolation was limiting progress in precision agriculture. If we wanted to understand plants and fields at scale, we needed integrated research that spanned locations, crops, and processes across the U.S.
This realization naturally led to partnerships with other researchers, creating a networked approach to experiments. But to make large-scale, collaborative work effective, it became clear that standardized data collection and protocols were essential to ensure quality and integrity. Without that, you can’t reliably feed information into automated pipelines or develop AI models that generalize.
To address this, my team started developing tools for data collection to capture high-quality, consistent plant data. That work organically grew into what we do today: leading an AI and computer vision team focused on platforms like BenchBot that collect images, standardize data, and train models. It wasn’t a sudden leap; it was a natural evolution, driven by the practical needs of precision agriculture and the desire to make research at scale truly actionable.
