Blog - Software Development

When building Python applications for production, it's good practice to pin all dependency versions, a process also known as “freezing the requirements”. This makes the deployments reproducible and predictable. (For libraries and user applications, the needs are quite different; in this case, one should support a large range of versions for each dependency, in order to reduce the potential for conflicts.) In this post, we explain how to manage a layered requirements setup without forgoing the improved conflict resolution algorithm introduced recently in pip. We provide a Makefile that you can use right away in any of your projects!

Many machine learning (ML) projects are doomed to fail. This can be due to various reasons and often they occur in combination. To avoid failure, all involved stakeholders need to understand the technical and organizational requirements of the project. Besides all preliminary discussions that define the project, it is important to summarize the project-relevant information in a comprehensive proposal. It should cover the technical and organizational requirements, possible problem areas and technical restrictions. In this article, I will describe the most important modules in machine learning project proposals. For a software provider like dida, the project proposal is the first step towards meeting the needs of the customer.

By 2021, there is little doubt that Machine Learning (ML) brings great potential to today’s world. In a study by Bitkom , 30% of companies in Germany state that they have planned or least discussed attempts to leverage the value of ML. But while the companies’ willingness to invest in ML is rising, Accenture estimates that 80% – 85% of these projects remain a proof of concept and are not brought into production. Therefore at dida, we made it our core mission to bridge that gap between proof of concept and production software, which we achieve by applying data-centric techniques, among other things. In this article, we will see why many ML Projects do not make it into production, introduce the concepts of model- and data-centric ML, and give examples how we at dida improve projects by applying data-centric techniques.

This article provides an overview of our tech stack at dida. Of course we adapt the tools we use to what is required by a given project, but the ones we have listed here are our go-to tools whenever we are free to choose. I will first describe the tools that shape our software development process, and then our favourite Python libraries and software tools for machine and deep learning.

Here I will a give a brief introduction to using Docker containers for software deployment. I assume that we want to deploy a Python application myapp that has a web API.

In our day-to-day work, we at dida found many parallels in the workflow of different projects. In order to speed up the development procedure in future projects and to facilitate adherence to best practices, we started the development of didatools . The package is meant as a collection of templates of code for training machine learning models, deployment and other frequently required patterns. It is easily extendable and the user should feel encouraged to modify the code to their liking. Here I guide you through the first steps of initializing and using didatools .

Anyone wanting to train a Machine Learning (ML) model these days has a plethora of Python frameworks to choose from. However, when it comes to distributing your trained model to something other than a Python environment, the number of options quickly drops. Luckily there is Tensorflow.js , a JavaScript (JS) subset of the popular Python framework with the same name. By converting a model such that it can be loaded by the JS framework, the inference can be done effectively in a web browser or a mobile app. The goal of this article is to show how to train a model in Python and then deploy it as a JS app which can be distributed online.

Since not only the complexity of Machine Learning (ML) models but also the size of data sets continue to grow, so does the need for computer power. While most laptops today can handle a significant workload, the performance is often simply not enough for our purposes at dida. In the following article, we walk you through some of the most common bottlenecks and show how cloud services can help to speed things up.