Government & Public Sector
The ongoing digitalization of public administration in Germany not only simplifies access to public services, but also enables more efficient and modern internal workflows. By eliminating repetitive tasks, jobs in public administration become more attractive and the processing of citizens' affairs becomes faster. By automatically preparing and creating technically correct responses to ongoing procedures, employees in public administration can focus their work on the actual substantive processing of applications. For example, comments on formal errors in business registrations or explanations on individual points in tax assessments can be written automatically.
Motivation
One consequence of digitalization is that information on ongoing procedures, applications and processes is available digitally in public institutions. By using generative AI and LLMs, this information can be used to generate texts that, for example, summarize the current processing status of a business registration in a technically correct way or point out formal errors in the application and add suggestions for correction. The manual preparation of such reports by specialist staff is usually a repetitive and time-consuming task, in which information is transferred from one format to another, i.e. the information in the system must be presented in a way that citizens can understand. LLMs have demonstrated that they can support and relieve specialist staff here so that they can focus on the actual content processing. This not only makes internal workflows in public administrations more efficient, but also increases overall trust in the future viability of government and public processes as well as proximity to citizens.
Challenges
LLMs show great potential in the automatic creation of reports. However, the publicly available models are pre-trained for generic tasks and therefore only produce satisfactory results for generic tasks. In order to map the specific information on procedures in public institutions and linguistic peculiarities, LLMs must be adapted with training data appropriate to the individual case. However, the training of LLMs requires enormous computing resources and memory. Another critical aspect is that LLMs for public administration must be provided on their own servers or infrastructures to avoid sharing sensitive data with third parties.
Solution approaches
The adaptation of ML models to application-specific or domain-specific training data is called fine-tuning. Holistic fine-tuning, in which all model parameters are retrained, is impractical for LLMs with billions of parameters. However, there are also modern, more efficient fine-tuning approaches in which only some of the parameters are retrained. With the digitally prepared text modules and documents on public procedures, LLMs can be efficiently adapted in this way. In addition, Explainable AI (XAI) approaches can be used to provide reliable and transparent model decisions. This increases the public's trust in digital processes. This makes it possible to develop a generative AI system that, for example, translates the information on the status of a business registration into a text that is understandable for the applicant and appropriate to the individual case.
