How Agricultural Robotics Changed My View of Engineering Research

Abstract

An agricultural robotics project changed how I understand engineering research. I once imagined research as the missing technical box in a linear pipeline: the problem, goal, and measure of success seemed to be defined before the work began. Real plants, physical systems, and agricultural expertise changed that view. The intended use changed what counted as success, the physical setting changed the system, and domain knowledge changed what the system needed to represent. I now think of research as a movement between a concrete setting and technical abstraction; each return to the setting can change both. The experience also showed me why I am drawn to robotics: ideas meet physical consequences.

Thirty-Eight Degrees Outside

One August day in Shanghai, the temperature outside was close to 38 °C. Then I entered a controlled indoor growing facility and felt the contrast immediately. Instead of open fields under the sun, I saw rows of plants growing inside a carefully maintained environment, surrounded by equipment and the quiet infrastructure required to keep that environment stable.

Until then, agriculture had remained a narrow image in my mind: soil, tools, outdoor labor, and farmers under the sun. The indoor facility showed me a technological system tied directly to food and daily life.

Later, I ate vegetables grown there. Plants I had approached through cameras, robots, and data had become food on my plate. The connection was immediate.

At first, the experience changed how I saw agriculture. Writing about it later made me realize that it had also changed how I saw engineering research.

Learning to See Agriculture as a System

The project took place during my internship at the Shanghai Artificial Intelligence Research Institute (SAIRI), in the summer after my third undergraduate year. Our team worked with the Shanghai Academy of Agricultural Sciences, bringing agricultural expertise into a project built around robotics and computer vision.

I worked on the upstream sensing and robotics part of a larger pipeline. I built the robotic-arm control, the mobile platform, the RGB-D image acquisition, and the three-dimensional reconstruction, and I led the leaf-area estimation and root measurement work with colleagues. The goal was to turn plant-shape and growth data into inputs for a model that could assess plant state and guide the growing environment.

The broader system had not reached that stage when I left. I completed my assigned components before my measurements entered the downstream model or guided adjustments to lighting, humidity, or nutrients.

At first, I understood my role in a simple way. The agricultural side had a need. My job was to implement the missing robotic capability. If existing techniques were insufficient, then research would help fill the gap. In this picture, the problem moved in one direction: from application, to engineering, to research, and eventually back to application.

When the Problem Pushed Back

The system was meant to measure the same plant over time without destroying it. Those measurements were useful only if they remained comparable and answered the biological question, which required us to consider proxy traits, calibration, workflow, and the research question together [1]. A visually impressive point cloud was not enough. We needed measurements that were repeatable and relevant to the task, while balancing precision against acquisition time, image count, processing cost, and biological variation.

Leaves made this difficult almost immediately. The plants were grown densely, so neighboring leaves blocked possible viewpoints. Leaves also reflected light, bent, crossed one another, and changed as the plants grew. Motion of the arm and small changes in the relative pose between camera and plant added further instability. Early reconstructions sometimes barely resembled plants. Even after the overall reconstructions improved, individual leaf-area estimates could still fluctuate in ways that did not make biological sense.

Plant phenotyping faces the same problems at a broader scale: changing illumination, growth, self-occlusion, and leaf crossover complicate the extraction of reliable traits [2]. In three-dimensional sensing, visibility, viewpoint selection, registration, and plant structure determine what can be measured [3].

We could continue improving the reconstruction algorithm, but the setting had exposed a more basic problem: from its original position, the camera could not see enough of the target plant.

What we did about it was not an algorithmic change. We added a second robotic arm whose job was to lift a plant out of the dense growing area so that the first arm could observe it on its own. Once a plant stands alone, the neighbors that were blocking the useful viewpoints are simply not there, and the camera can be moved to the positions the measurement actually needs rather than the positions the shelf allows. The reconstruction code did not get better. The scene did.

The difficulty was no longer only “How can the reconstruction be improved?” It had become “How should the plant, camera, and robots be arranged so that the required information becomes observable?” A physical constraint had changed the architecture of the solution, and it did so at a level no amount of work inside the reconstruction step could have reached.

Removing the plant also exposed its roots. Seeing them did not tell us whether measuring them mattered. I discussed their value with a collaborator in agriculture, and that conversation led us to add root measurements to the workflow.

These moments changed three parts of the problem: the task changed what counted as success, the physical setting changed the system architecture, and agricultural knowledge changed what the system needed to represent.

The project was no longer simply giving me a difficult problem to solve. It was changing the problem while I worked on it.

A Loop, Not a Pipeline

My earlier pipeline assumed that the need, problem boundary, and measure of success had already been defined correctly. The work showed me that research can fill a capability gap and also revise the problem itself.

Kline and Rosenberg describe innovation as feedback among design, production, use, and research rather than a one-way sequence [4]. My experience made that feedback concrete: it did not merely test an answer; it changed the question.

I now think of this as a loop with three movements: encounter the setting, translate what matters into a technical problem, and return with evidence. Each return can revise the original question.

Encounter begins with a problem in context. It gets close enough to a concrete setting to reveal its goals, labor, constraints, stakeholders, and possible consequences. It can come through firsthand work, observation, collaboration, real data, and evidence from prior deployments.

Literature provides a starting point. Firsthand investigation, sustained collaboration with people who know the setting, real data, and deployment evidence reveal what abstractions miss. Application claims should grow from that contact.

Translation turns the parts of a situation that matter into technical representations: state variables, models, questions, constraints, objectives, and metrics. It also makes disagreements over objectives, tradeoffs, and success visible. Every abstraction chooses what to preserve and what to discard.

Between Translation and Return, theory, controlled experiments, formal analysis, and new capabilities can decompose the problem, enable repetition, and make comparison possible. Claims about use must retain a route back to the setting that gave them meaning.

Return tests a result against its claim through domain consultation, real-data testing, scenario experiments, a pilot, or long-term use. The form of evidence depends on the claim.

Returning to the setting may expose a bad method, the wrong metric, a misunderstood goal, or a more important problem. Any of these can become the next research question.

In this project, tests with real plants changed the system architecture, and a conversation with an agricultural collaborator changed what we measured.

I call these three movements the ENTRE loop: ENcounter, Translation, REturn. The name is there so that I keep the order straight when a project starts pulling me out of it.

The loop also describes something narrower and more current than a greenhouse. Sim-to-real has the same shape. A simulator is Translation: it preserves what someone decided mattered and discards the rest. Hardware is Return, and it does not only score the result—it reports which discarded detail was load-bearing, and that report often changes what the model should have been representing in the first place.

Domain Knowledge Is Part of the Research Method

I once thought of interdisciplinary work mainly as a collection of skills. Agricultural robotics required mechanics, electronics, computer vision, control, and software. Agriculture seemed like one more body of knowledge to add to the list.

The project taught me a more important distinction. Technical disciplines help make the robot work. Domain knowledge helps determine why it should work, what information it must preserve, and what its failures mean.

The discussion about roots changed what the code and the system needed to represent. Agricultural knowledge entered as we formed the problem, not after we finished the technical work.

Claims and Evidence

Two papers I read at the time changed how I hear a result. A sweet-pepper harvesting robot performed very differently in commercial greenhouse tests depending on whether the crop had been adapted for it [5]; its capability was a property of the environment as much as of the robot. A long-term agricultural data-collection project met seasonal growth, changing light and wind, uneven terrain and human interference [6]—none of which a short demonstration would have shown.

Domain consultation, a scenario test, a limited pilot and a season of operation answer different questions. “Tested in the real world” is not one threshold, and which rung a result comes from is most of what it is worth.

After the Greenhouse

The internship was my first time inside an industry project, and my first time working across disciplines that did not share a vocabulary. I learned some agriculture. I learned what it looks like when a research question and a delivery schedule have to occupy the same room.

The ENTRE loop is what I worked out from that, and it is shaped by the industrial side of the project rather than by a paper: a way of moving between a setting and a technical problem that expects the setting to revise the problem. My hands got better at the tools over those months. What I value more is the part underneath them—a clearer sense of how research and an industrial project actually relate, and of where a question comes from.

References

1. H. Poorter, G. M. Hummel, K. A. Nagel, F. Fiorani, P. von Gillhaussen, O. Virnich, U. Schurr, J. A. Postma, R. van de Zedde, and A. Wiese-Klinkenberg, “Pitfalls and potential of high-throughput plant phenotyping platforms,” Front. Plant Sci., vol. 14, Aug. 2023, Art. no. 1233794.
2. S. Das Choudhury, A. Samal, and T. Awada, “Leveraging image analysis for high-throughput plant phenotyping,” Front. Plant Sci., vol. 10, Apr. 2019, Art. no. 508.
3. S. Paulus, “Measuring crops in 3D: Using geometry for plant phenotyping,” Plant Methods, vol. 15, Sep. 2019, Art. no. 103.
4. S. J. Kline and N. Rosenberg, “An overview of innovation,” in The Positive Sum Strategy: Harnessing Technology for Economic Growth, R. Landau and N. Rosenberg, Eds. Washington, DC, USA: National Academies Press, 1986, pp. 275–306.
5. B. Arad, J. Balendonck, R. Barth, O. Ben-Shahar, Y. Edan, T. Hellström, J. Hemming, P. Kurtser, O. Ringdahl, T. Tielen, and B. van Tuijl, “Development of a sweet pepper harvesting robot,” J. Field Robot., vol. 37, no. 6, pp. 1027–1039, Sep. 2020.
6. R. Polvara, S. Molina, I. Hroob, A. Papadimitriou, K. Tsiolis, D. Giakoumis, S. Likothanassis, D. Tzovaras, G. Cielniak, and M. Hanheide, “Bacchus long-term (BLT) data set: Acquisition of the agricultural multimodal BLT data set with automated robot deployment,” J. Field Robot., vol. 41, no. 7, pp. 2280–2298, Oct. 2024.