Yesterday I attended the Beyond Text: Multimodal LLM Integration event, organised by VAIA - Flanders AI Academy and partners.
It is inspiring to see the latest developments in multimodal AI, with a strong focus on Flemish research and applications. The afternoon offered a good overview of how Large Language Models (LLMs) increasingly understand more than just text: from speech and images to environment and gestures, but also how this brings its own set of challenges.
Below are some key points I took away from the sessions:
🔹 Flemish AI models (VLAM): Tanguy Coenen (imec) shared how the VLAM programme is working towards powerful, culturally grounded foundation models for Flanders. By enriching open models with unique Flemish data (text, audio, video) from our rich media archives, VLAM is building AI systems that reflect the language and cultural identity of our region. This strategic initiative forms the backbone of local AI innovation and shows how the Vlaams AI Onderzoeksprogramma / Flanders AI Research Program aims for models that better fit our context.
🔹 Evaluation of multimodal LLMs: Matthew Blaschko (KU Leuven) showed how difficult it is to measure multimodal models accurately. Traditional benchmarks often provide only one overall score, while what you really want to know is where the model fails: image, audio or the combination. Fine-grained tests, such as "What did the person in the video do after the siren was heard?", make it clear whether a model truly understands multimodality.
🔹 Social interaction with AI: Ruben Janssens (UGent, IDLab) demonstrated how multimodal LLMs can enable genuine social interactions with robots. By combining speech with visual perception, an AI system can respond to non-verbal signals and environmental context. Think of a robot that also interprets facial expressions and body language. This should enable more natural conversations in areas such as education or care. Ruben also emphasised the importance of local models for sensitive situations, so that such social robots can be deployed safely and with privacy in mind.
These VAIA sessions are personally very valuable to me. They are strong on content, less commercially driven, primarily focused on knowledge sharing, and they bridge academia and industry, which consistently brings new insights and inspiration.
Thanks to Kevin D'hooge and the VAIA - Flanders AI Academy team for the excellent organisation and the pleasant catch-up during the networking reception!
