Localization and order recognition improvement for the Japanese beetle

PESSL INSTRUMENTS GMBH

Pessl Instruments GmbH has been producing reliable measuring instruments for 37 years and has developed various devices for the measurement and monitoring of different agricultural parameters, being one of the leading IoT providers for agriculture offering innovative and cost-effective solutions for more efficient farm management. Participating in the IPM-Popillia project gives PESSL a chance to put into the farmer's disposal all its expertise on sustainable agriculture and fulfill its mission to offer high added value and customized cost-effective solutions and contribute to global environmental protection.

iSCOUT®

With the inspiration of helping the farmer to grow his crop in the optimal and most healthy way, the PESSL team developed an optical high-resolution camera system embedded inside an insect trap, together with a computer vision software, which is able to recognize insects from the photo and remotely monitor insect pressure at fields. Different types of traps are available, to cover a broad spectrum of insect species. All of the photos and data from computer vision software are displayed online, on a web portal called FieldClimate. iSCOUT® can be hung wherever in the field needed, due to its low weight. In the field, the device is self-sufficient, as it is powered by a solar panel and a battery. The iSCOUT® can be used to monitor the first occurrence of pest species on the field site, to observe population dynamics and apply insecticides following IPM guidelines as well as give support at the time mass trapping should be started.

IPM-POPILLIA PROJECT

The PESSL team works on a monitoring device for P. japonica. The aim is the remote observation of the first occurrence in an area as well as following the seasonal dynamics of an already established population (on low to moderate population sites). The target gets attracted by a lure and enters the trap system. After entering into the trap insects will get fixed on a glue board and photographed. Those photos are the base for the development of an automatic detection tool specifically for P. japonica. The traps have also been equipped with sensors collecting climatic data, temperature, and relative humidity. So we are able to get more detailed information about those parameters correlating with the flight behavior of P. japonica. The first 8 prototype traps produced by PESSL were installed within infested zones in Italy and in Switzerland, to evaluate that innovative monitoring tool under field conditions in 2021.

ARTIFICIAL INTELLIGENCE (AI)

Artificial intelligence (AI) is a thriving field with many practical applications and active research topics. Intelligent software is used to automate routine labor, understand speech or images, make diagnoses in medicine and support basic scientific research. The true challenge to artificial intelligence proved to be solving the tasks that are easy for people to perform but hard for people to describe formally - problems that we solve intuitively, that feel automatic, like recognizing spoken words or faces in images.

MACHINE LEARNING (ML)

The difficulties faced by systems relying on hard-coded knowledge suggested that AI systems need the ability to acquire their own knowledge, by extracting patterns from raw data. This capability is known as machine learning (ML) and it is seen as a part of artificial intelligence.

LABELLING PROCESS

Using the labeling tool, each image is inspected, insects are identified and labeled accordingly. Labeled data in machine learning refer to data that has been annotated, to indicate the target, or the answer you want your machine learning model to predict.

Labeling Popillia japonica is quite a unique challenge because in some cases our trap devices were overflowing with beetles and we couldn't utilize those photographs. In others, with normal conditions, a bounding box is drawn around each insect, and existing boxes are adjusted so they fit their body perfectly.

On the left: trap literally overflowing with Popillia japonica beetles. On the right, an example of the picture we usually work with.

INSECT DETECTOR

Our traps usually capture multiple insects glued to a sticky plate on a single photo and each insect is automatically recognized in terms of localization and order classification. For that task, we have trained the insect detector on thousands of iSCOUT images, which were analyzed and annotated by an entomology expert. When P. japonica was first introduced to our traps, the system was not able to successfully recognize them, because it was unfamiliar with that species. We used annotated P. japonica images merged with our existing dataset for training a new version of the insect detector. As a result, localization became more precise and order classification of P. japonica as Coleoptera more stable.

 In the images above, a comparison between two insect detector variants are shown. The results of inference with the prior insect detector are on the left, while the results with the latest insect detector are on the right. It is clear that the localization became more precise and order classification of P. japonica as Coleoptera more stable (more insects were recognized and less mistakes on the order classification were performed).

ABOUT THE AUTHORS:

Dr. Christina Pilz, Product manager for camera solutions and team lead

I started working for PESSL in November 2014, while being responsible for the Decision Support Systems in Plant Health Management (Disease models and Camera Products). Previous to Pessl's career, I collected professional experience particularly in the biology and microbiological control of agricultural pest insects, having studied agriculture at the University of Agriculture and Applied Life Sciences (BOKU) in Vienna and working, at the Research Station Agroscope in Zürich as well as at the Plant Protection Service in Hungary and at the Agency for Health and Food Security, Austria. At PESSL I started by supporting the development of disease model implementations and since around four years my focus lay also on electronically monitoring devices of pest insects in viticulture, agriculture, and fruit production.

Damir Najvirt, Machine learning engineer

I studied physics, specializing in stochastic processes and their role in the development of financial markets at the Faculty of Natural Sciences and Mathematics in Maribor, Slovenia. I have been working for PESSL since 2016, where I developed image processing solutions for our camera products and automated systems for fruits and insects recognition. I am responsible for designing, evaluating, and enhancing our machine learning stack. I am especially passionate about the transformation of data into valuable information that, in turn, guides farmers all over the world. 

Eva Munda, Entomologist assistant

My entomology career started when I was about five years old and breeding mosquitoes in the living room. My parents were not amused, but I learned a lot about the life cycle of a mosquito. It was only a natural decision for me to study ecology and nature conservation at the Faculty of Natural Sciences and Mathematics in Maribor. I started to work for Pessl instruments in June 2021 as an Entomological assistant, with the main focus on insect identification. In my free time, I like to cook, paint, and spend some time in nature hiking, mushrooming, or birdwatching.

M.Sc. Junia Rojic, Project support officer

At PESSL, I am responsible for all the administrative tasks related to projects, including among other proposals, reports, deliverables, time schedules, meetings, minutes, and budget control. I studied Industrial Engineering at the Federal University of São Carlos, Brazil, recognized in Austria as a Masters' degree in Materials Science. With more than 12 years of experience in Project Management, including many years in Brazil working at R&D and IT companies, I am delighted to be part of a European Project, with many important stakeholders, and most importantly, supporting agriculture and food production in the 21st century. In my free time I do sports, like volleyball and indoor football, but also enjoy spending some time reading and in nature hiking.

#pesslinstruments, #iScout, #ipm-popillia