KAMI: AI-based respiratory rate monitoring for cows

As part of the project "Artificial Intelligence for Recording Breathing in Dairy Cows (KAMI)," researchers explored how the respiratory rate of cows can be measured using cameras. Respiratory rate is an important indicator of stress and disease in cows.

The project was carried out in collaboration with the Leibniz Institute for Agricultural Engineering and Bioeconomy (ATB), the Teaching and Research Institute for Animal Breeding and Husbandry (LVAT), and the University of Hildesheim.

Traditional methods for measuring respiratory rate are time-consuming and can cause stress to the animals. Cameras offer a contactless and continuous measurement approach.

The project tested infrared and depth cameras. These cameras were installed in the cows’ resting areas, where their position remains relatively constant.

The depth camera uses stereoscopic imaging to calculate distance information. These distance data were combined with RGB color information to estimate the cows' respiratory rate.

A key aspect of the project was the collection of precise ground truth data. For this purpose, a respiration sensor was used. This sensor, developed and validated by the Leibniz Institute for Agricultural Engineering and Bioeconomy (ATB) and the startup Gouna, measures the respiratory rate (RR) based on the pressure difference in the cows' nostrils.

For our experiment, we equipped the sensor with an additional LED that blinked at the beginning of each exhalation. This modification allowed us to extract the LED signals from our video recordings, creating a highly precise, time-resolved dataset of each cow’s breathing events. This type of detailed ground truth data, capturing each individual breath, is unique in research so far. It has enabled us to develop new, innovative end-to-end models for AI-supported respiratory rate detection.

The image on the left shows the averaged distance of the cow’s flank (marked in orange) to the camera over time. In this example, individual breaths are clearly visible.
The right image displays data extracted from point tracking (example point marked in green) in the RGB footage.

The biggest challenge was distinguishing respiratory movement from other cow movements and developing a robust method to extract the number of breaths from time series data, as shown in the image. Using artificial intelligence and machine learning, algorithms were developed that can reliably measure the respiratory rate.

As part of the KAMI project, dida used depth cameras to capture the respiratory rate of dairy cows, while our partners at the University of Hildesheim developed methods based on infrared cameras.

We developed and compared the accuracy of four different methods:

• M1: RGB Point Tracking Model – This method tracked distinctive points on the cow’s flank in the RGB video data and analyzed their movement patterns to determine the respiratory rate.

• M2: Geometric Features Using Depth Information – This method calculated geometric features, such as Dirichlet energy, based on depth information from the cow’s flank to derive the respiratory rate.

• M3: End-to-End Model Using Only RGB Data – A neural network was trained to directly predict the respiratory rate from RGB video data.

• M4: End-to-End Model with RGB and Depth Data as 4-Channel Input – Similar to M3, but with additional depth information as an extra input channel.

Results

The results showed that M1 and M4 were able to measure respiratory rate with similarly high accuracy. The deviation between the predicted respiratory rate and the actual rate, measured with a reference sensor, was less than three breaths per minute.

The use of depth information alone (M2) was not suitable for accurate respiratory rate detection. The combination of RGB and depth information in M4 led to an improvement compared to using RGB data alone (M3).

The results indicate that depth cameras, combined with RGB data and suitable algorithms (such as M1 and M4), provide a promising option for contactless and continuous respiratory rate measurement in dairy cows. This technology could help improve animal welfare and health in dairy farming.

If you would like to speak with us about this project, please reach out and we will schedule an introductory meeting right away.