Arrive and connect with others.

Welcome to the dida conference.

Large Language Models impress with their adeptness in context-aware text generation, logic, as well as reasoning. Typically, downstream models fine tuned on chat data possess the remarkable ability to be directed towards solving tasks described in natural language without explicit further weight adaptation. In relevant applications, interesting use cases often relate multiple external data sources with each other and are characterized by a complex multistep decision process. In this talk, we discuss how predefining decision steps and integrating external data filtering can break down multifaceted problems into manageable, self-contained language processing tasks, which can readily be solved by LLMs.

Pretraining involves training an AI model on a large dataset to learn general features, which can then be finetuned on specific tasks with smaller datasets. This decreases the need for time intensive dataset acquisition and training efforts for each new use case, reducing the costs of application development. While pretrained models are widely used in computer vision and natural language processing, their adoption for satellite data and earth observation applications remains limited. Our investigation focuses on comparing the capabilities of transfer-learning and meta-learning approaches for the pretraining of AI models in earth observation tasks, particularly crop type classification, and their potential to generalize insights across different geographical regions.

Within the sector initiative “Digitale Schiene Deutschland”, our client Deutsche Bahn is developing an automated driving system for trains. As a part of the efforts towards such a system we developed, together with Deutsche Bahn, a machine learning solution to detect anomalous and hazardous objects on and around the tracks using onboard RGB cameras. It is intentionally required that this system does not simply detect objects within a given collection of classes (such as people, signals or vehicles), but rather has the ability to detect any object and rank them by how anomalous they are. This presentation explains the challenges encountered, presents several approaches explored, and provides an overview of the final solution: In order to detect objects of possibly unkown classes we developed a unique pipeline containing multiple machine learning components, including a monocular depth estimation model, a segmentation stage, image embedding models and an anomaly detection model. As dataset, Digitale Schiene Deutschland provides us with OSDAR23, an open dataset that contains 45 scenes. Each scene contains images taken by several RGB cameras and infrared cameras, together with radar and lidar data. This dataset contains annotations for twenty classes of objects, which we use both for finetuning our model and for evaluating the final results. Besides, we were also granted access to a larger amount of unannotated data, which were used for self-supervised learning.

Food and drinks will be available for all guests.

Diffusion models have emerged as a distinct class of generative models with an impressive ability to learn complex data distributions such as those of natural images, music, and human speech. In the context of speech enhancement, diffusion models can be used to learn the conditional distribution of clean speech given the noisy mixture. Following this idea, we have proposed the method “Score-based Generative Models for Speech Enhancement” (SGMSE), a continuous-time diffusion model based on an Ornstein-Uhlenbeck process. In our experiments, we show competitive speech enhancement performance compared to predictive baselines, while generalization is better when evaluated in a mismatched training scenario. Subjective listening tests show that, on average, the enhanced speech is preferred over the predictive baselines and is often perceived as natural-sounding. However, for very challenging input, the model tends to hallucinate and generates speech-like sounds without semantic meaning. To address this problem, we have combined predictive and generative approaches, and conditioned the model on visual input of the speaker’s lip movements. Moreover, to improve robustness and address the problem of slow sampling speed in diffusion models, we have used a Brownian bridge as a stochastic process, and proposed a two-step training for diffusion-based speech enhancement that enables single and few-step generation.

In recent years, the advent of Large Language Models (LLMs) has changed the landscape of data extraction. These LLMs boast unparalleled text processing capabilities and come pre-trained on vast amounts of data, rendering them effective for information retrieval tasks. However, traditional methods such as graph neural networks and extractive models have historically been favored for their efficiency in resource utilization. Despite this, the question persists: how do LLMs compare with those models in practical data extraction applications? This presentation aims to delve into this inquiry, providing a comprehensive examination of LLMs' advantages and disadvantages compared to extractive models. Drawing from our project experiences and internal research, we aim to elucidate the practical implications of utilizing LLMs for data extraction, offering insights into their efficacy, resource requirements, and overall performance in real-world scenarios. Through this exploration, attendees will gain a deeper understanding of the role of LLMs in modern data extraction workflows and the considerations involved in their implementation.

The relationship between human cognition and neural network interpretation of the world is one of the most intriguing questions in modern AI research. This talk explores a recent line of inquiry into this question using images and addresses three key issues: How can we derive a geometric object that encapsulates human understandings of images? How does the geometry of these image representations compare to that derived from neural network representations? Can we realign a neural network representation to achieve a more human-like understanding of the world, thereby improving neural network performance?

This presentation dives into the transformative potential of machine learning (ML) technologies in the public sector, highlighting opportunities for efficiency, transparency, and improved service delivery. Using a case example from practice, it illustrates how ML applications already have found their way into public processes.

Check out Karolina on SoundCloud.

• Jonas Golde (HU Berlin): fabricator - An Open Source Toolkit for Generating Labeled Training Data with Teacher LLMs

• Fabio Barth (HU Berlin): Occiglot: A research collective for open-source development of Large Language Models by and for Europe

• Jacek Wiland, Max Ploner (HU Berlin): BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Models

• Mark Budgen (dida): Defect detection and classification for unroasted coffee beans

• Maximilian Trescher (dida): Artificial intelligence for detecting respiration in dairy cows

• William Clemens (dida): LaserSKI - Defect Detection in Laser Diodes

• Konrad Mundinger (Zuse Institute Berlin): Neural Parameter Regression for Explicit Representations of PDE Solution Operators

• Julius Richter (Universität Hamburg): Diffusion Models for Audio-Visual Speech Enhancement

• Jakob Wagner (appliedAI): Neural Operators: Solutions with the Continuity Framework

• Faried Abu Zaid (appliedAI): Applications of Uniformly Scaling Flows

• Robert Müller (appliedAI): Imitation Actor Critic