Program

Dr. Dejan Ničković Website

AIT Austrian Institute of Technology, Austria

Robustness verification of Deep Neural Networks

Deep Neural Networks (DNNs) have become the dominant solution for perception, prediction, and decision-making tasks across a wide range of domains. As these systems are increasingly deployed in safety-critical applications, robustness has emerged as one of their most important properties, reflecting the ability of a model to maintain correct and predictable behavior under perturbations, uncertainties, and distributional variations. In this presentation, we will formally introduce the notion of robustness and discuss the challenges associated with assessing and guaranteeing robustness in DNNs. We will provide an overview of the main verification and analysis techniques that have been developed to reason about robustness, highlighting their underlying principles, strengths, and limitations. More specifically, we will examine approaches for robustness testing, verification of local robustness, and verification of global robustness, together with their representative application scenarios. Finally, we will look beyond traditional neural network architectures and discuss emerging challenges in reasoning about the robustness of modern DNN-based systems, such as large-scale foundation models.

Prof. Sebastian Elbaum Website

University of Virginia, USA

Sensors, Rules, Models: Rethinking V&V for Autonomous Vehicles

The promise of autonomous vehicles (AVs) hinges on a single, uncompromising prerequisite: rigorous driving assurances. Yet, specifying, testing, and verifying these safety architectures remains an incredible challenge. This difficulty is driven by a trio of distinct hurdles: first, the vast, unpredictable environment that must be mapped against intricate driving rules; second, the acute semantic gap dividing high-dimensional raw sensor data from high-level symbolic safety concepts; and third, the relentless pace of model evolution, where spiraling complexity causes late-stage development costs to escalate prohibitively. Reflecting on our work within this high-stakes landscape, I will walk you through a family of end-to-end analyses that overcome these hurdles by breaking the boundaries of traditional validation and verification. I will show you the core insights behind these techniques, lay out the empirical evidence of their efficacy and their impact in practice, and put hard questions and real-world problems to the room, challenging us to collectively map out the remaining frontiers in autonomous systems safety.

Dr. Son Tong Website

Siemens, Belgium

Engineering Workflow, Reimagined with Agentic AI

Modern engineering relies on a wide range of tools for requirements management, design, simulation, optimization, validation, data analytics, and AI. This talk explores a future vision in which these tools collaborate as autonomous AI agents within an Agentic AI framework, helping engineers automate workflows, accelerate development, and improve performance while maintaining engineer-centric supervision. The session concludes with an introduction to the European AI-BOOST Generative AI Challenge on ADAS scenario generation

Prof. Fabrizio Pastore Website

University of Luxembourg, Luxembourg

Supporting Automated Testing and Safety Analysis of Deep Neural Networks for Autonomous Systems

Deep neural networks are a building block of perception layers for safety-critical systems; therefore, engineers need solutions to cost-effectively identify and characterize the portions of the input space where a DNN may underperform to improve systems’ robustness.

This talk provides an overview of recent research approaches combining simulators, search algorithms, and diffusion models, which can cost-effectively determine worst-case scenarios if driven by appropriate fitness functions. Further, the talk will provide an overview of failure explanation approaches that enable the detection of the situations in which a DNN may fail and, consequently, enable the identification of countermeasures.

Adj. Prof. Sigrid Eldh Website

Ericsson AB, Sweden; Carleton University, Canada; Mälardalen University, Sweden

Testing Agents and Agent Testing in Industry

The future of developing and testing any software (even AI) is AI enabled. Through carefully crafting of Agents, in an orchestration framework, you can now build agents. With the right frameworks and set up you can interactively produce a relative trustworthy test environment – the limit will be your own knowledge and ingenuity. In Industry more of the testing is now handed over to a series of agents aiding in different tasks, with the goal to provide quality systems. We will address several layers of testing and verification approaches throughout the software life cycle.

Dr. Nicola Franco Website

The Italian Institute of Artificial Intelligence for Industry, Italy

Hands-on AI Red Teaming: Attacking and Defending LLM Agents with HackAgent

AI agents are changing the way applications interact with users, data, and external tools. At the same time, they introduce new attack surfaces that cannot be fully addressed with traditional software security techniques. Prompt injection, malicious instructions, unsafe tool use, and multimodal manipulation can all influence how these systems behave. In this seminar, participants will learn the fundamentals of adversarial attacks against LLMs and multimodal LLMs, with a strong focus on practical experimentation. After a brief overview of the main concepts and threat models, participants will engage in a hands-on session using HackAgent to explore realistic attack scenarios in a controlled environment.

Dr. Wissam Mallouli Website

Montimage EURL, France

Smart Network Fuzzer for Advanced Security Testing

This talk introduces a smart network fuzzer designed to enhance security testing of complex and dynamic networked systems. The approach leverages intelligent traffic generation and adaptive fuzzing strategies to uncover vulnerabilities that traditional testing methods may miss.

By combining protocol awareness, behavioral analysis, and AI-driven techniques, the solution enables efficient exploration of attack surfaces in modern environments such as cloud-native infrastructures and software-defined networks. The presentation will include a demonstration showcasing how the fuzzer can detect anomalies, trigger unexpected behaviors, and support proactive cybersecurity validation.

Dr. Gül Çalikli Website

University of Glasgow, Scotland

Context Matters: Building and Evaluating Trustworthy AI for Software Engineering

Large language models are rapidly transforming software engineering, enabling developers to generate code, refactor software, and automatically produce unit tests. Yet two fundamental questions remain: what information should AI systems use to assist developers effectively, and how should we evaluate their capabilities on realistic software projects? This lecture will present recent research on context-aware AI for software engineering. First, it will explain how augmenting AI coding assistants with developers' gaze data enables prompts to adapt to developers' cognitive states, thereby improving code comprehension and readability. This will be followed by a discussion of context-aware LLM-based unit test generation, showing how repository structure, dependency information, and retrieved examples influence the quality of generated tests. Finally, I will present a realistic evaluation framework that benchmarks AI-generated tests on post-cutoff open-source repositories using repository-native execution environments and quality measures such as mutation testing, code coverage, and maintainability, revealing substantial differences between benchmark performance and real-world behaviour.

Prof. Ezio Bartocci Website

Technical University of Wien (TUW), Austria

Agentic AI: Foundations and Failure Modes

Wired into a loop with tools, memory, and a goal, large language models have become agents that plan, act, and change the world on their own — and this breaks the classical testing contract. Behaviour is stochastic, the "specification" is an ambiguous natural-language goal, memory makes runs stateful and tests non-idempotent, and errors compound over ever-longer task horizons. The stakes rise sharply when such agents are embedded in cyber-physical systems — robot task planning, autonomous laboratories, industrial control, in-vehicle assistants — where a wrong action is not a wrong answer but a physical hazard, and is often irreversible. This lecture dissects what an agent actually is (the reason–act–observe loop, chain-of-thought, memory, tool use, multi-agent coordination), explains why testing and benchmarking under-determine the reliability of such systems, and presents a structured taxonomy of agent failures grounded in empirical studies of real execution traces (MAST) and the adversarial failure modes of the OWASP Top 10 for Agentic Applications. It closes by arguing that an LLM agent is, formally, a reactive system whose behaviour is a trace — making the specification and runtime monitoring machinery long established for cyber-physical systems the natural instrument for its assurance, and setting up the companion session on runtime assurance for agentic AI.

Ken Friedl Website

BMW Group, Germany

Benchmarking Generative AI for the In-Car Domain: Methods, Tools, and Hands-On Evaluation (Part 1/3)

Generative AI in the in-car domain, particularly in conversational speech-based assistants, is redefining in-vehicle interactions through natural dialogue, contextual awareness, and proactive assistance within driving contexts. This talk centers on AI-specific testing and benchmarking for voice-based systems. We will talk about BMW’s evolution from a hardware-driven manufacturer to an integrated software solutions provider. And how the advent of non-deterministic AI paved the way for flexible, scalabe testing solutions. As LLM-based voice assistants become embedded in modern cars, testing strategies must consider synthetic data generation, automated evaluation, and end-to-end benchmarking to ensure a reliable, safe and therefore trustworthy customer experience across navigation, infotainment and safety-relevant domains.

Lev Sorokin Website

BMW Group, Germany and TUM, Germany

Benchmarking Generative AI for the In-Car Domain: Methods, Tools, and Hands-On Evaluation (Part 2-3)

As generative AI becomes part of customer-facing and potentially safety-relevant workflows, systematic evaluation is essential to assess their capabilities, limitations and robustness. This talk presents scalable and automated testing approaches for evaluating Generative-AI based conversational assistants. Participants will gain insights into the design of benchmarking tasks, the construction of datasets, and the assessment of system performance using fully automated evaluation of metrics. In the accompanying hands-on session, participants will evaluate an AI-powered in-car assistant on realistic vehicle-related use cases. Through guided experiments, attendees will learn how to design benchmark scenarios, analyze system behavior, and interpret evaluation outcomes. The session concludes with a discussion of current research challenges and emerging directions for the development and evaluation of trustworthy conversational AI systems.