Successful experiment to detect hand luggage using computer vision

Every airline faces one common challenge: Carry-on-baggage capacity is limited onboard and an additional reloading into the cargo compartment causes unnecessary delay minutes and disruptions. With the Go-Life of the Lufthansa Group Open Lab Network, end of April, the Lufthansa Group Innovation Management offers a matching platform for use case ideas and experiment capabilities. With the idea in mind to find a smart tech solution and run a fast experiment that can help assess the amount of passengers' hand luggage when boarding, Amin Riahi (Manager Ground Ops Systems at Lufthansa Group) contacted the Lufthansa Group Open Lab Network in May 2023 and was matched with lab capabilities and expertise to ideate, plan and run a computer vision lab use case.

The Inside Sessions (Season 3 Episode 1) | Carry-on Baggage detection with AI

In this particular use case, experts from zeroG, Lufthansa Industry Solutions & Lufthansa Systems took over and were responsible for testing a computer vision solution, which is programmed and trained so well that it reliably counts the exact number of pieces of different hand luggage that passenger carrying when boarding. After training the AI, the project team invited about 50 employees from the Lufthansa Group in June to test their AI in the boarding lab at Lufthansa Systems in Raunheim. During multiple simulated boarding processes, the detection of our test passengers' luggage was tested with 2 different camera settings (i.e. with a ceiling camera & front camera perspective) and with different hand luggage types.

Fabian Vogel (Senior Data Scientist, zeroG) summarizes this trial as follows: "We have gained some interesting insights after 7 weeks, e.g. that the camera should be positioned in the ceiling perspective after passing the gate to get more valid images of the luggage carried and that we need more images so that the AI can better recognize more difficult luggage (incl. different types of trolleys and backpacks)."

Following these successful experiments, Amin Riahi is now combining these findings with existing tools such as the "Carry On Baggage Prediction Tool", which predicts critical flights with hand luggage, and will test this use case under real conditions at the two German airports in FRA & HAM and collect further data to increase its maturity.

Christian Haude (Head of Open Lab Network) is very satisfied with the results of the first use cases and sums up: “This first use case shows: Experimentation works! With the right idea and skills, it is possible to test and validate ideas, take them to the next level and learn even faster. We are very much looking forward to testing more innovative ideas from colleagues!”