Successful experiment to detect hand luggage using computer vision

The first open lab use case successfully demonstrated the power of experimentation and the effectiveness of cross-company collaboration within the Lufthansa Group.

Every airline faces one common challenge: Carry-on-baggage capacity is limited onboard and an additional reloading into the cargo compartment causes unnecessary delay minutes and disruptions. With the Go-Life of the Lufthansa Group Open Lab Network, end of April, the Lufthansa Group Innovation Management offers a matching platform for use case ideas and experiment capabilities. With the idea in mind to find a smart tech solution and run a fast experiment that can help assess the amount of passengers' hand luggage when boarding, Amin Riahi (Manager Ground Ops Systems at Lufthansa Group) contacted the Lufthansa Group Open Lab Network in May 2023 and was matched with lab capabilities and expertise to ideate, plan and run a computer vision lab use case.

In this particular use case, experts from zeroG, Lufthansa Industry Solutions & Lufthansa Systems took over and were responsible for testing a computer vision solution, which is programmed and trained so well that it reliably counts the exact number of pieces of different hand luggage that passenger carrying when boarding. After training the AI, the project team invited about 50 employees from the Lufthansa Group in June to test their AI in the boarding lab at Lufthansa Systems in Raunheim. During multiple simulated boarding processes, the detection of our test passengers' luggage was tested with 2 different camera settings (i.e. with a ceiling camera & front camera perspective) and with different hand luggage types.

ZeroG's Computer Vision AI solution with the ceiling camera view that counts and analyses 4 different types of hand luggage such as backpack, suitcase, handbag and jackets.
Using test passengers and different camera positions, the team was able to test how reliably the AI records the number of hand luggage items per passenger in the Lufthansa Systems boarding lab. The colleagues demanded a lot from the AI in the process
Project team (from left to right): Fabian Vogel from zeroG (AI detection and counting module implementation), Philip Lohse from LHIND (front-end programming of the dashboard), Stephan Madlung from LHIND (project steering of the Open Lab Innovation Hand Luggage Uses Cases), Jan Burkhardt from LHIND (Connectivity), Manuel Siefker from the LSY CUTE Lab

Fabian Vogel (Senior Data Scientist, zeroG) summarizes this trial as follows: "We have gained some interesting insights after 7 weeks, e.g. that the camera should be positioned in the ceiling perspective after passing the gate to get more valid images of the luggage carried and that we need more images so that the AI can better recognize more difficult luggage (incl. different types of trolleys and backpacks)."

Following these successful experiments, Amin Riahi is now combining these findings with existing tools such as the "Carry On Baggage Prediction Tool", which predicts critical flights with hand luggage, and will test this use case under real conditions at the two German airports in FRA & HAM and collect further data to increase its maturity.

Christian Haude (Head of Open Lab Network) is very satisfied with the results of the first use cases and sums up: “This first use case shows: Experimentation works! With the right idea and skills, it is possible to test and validate ideas, take them to the next level and learn even faster. We are very much looking forward to testing more innovative ideas from colleagues!”