AmoKI: Autonomous mining monitoring for sustainable land management

Extensive coal mining operations have been widely recognized as harmful to the environment, causing deforestation, soil depletion, and air pollution. Governmental measures aimed at monitoring these pits require timely procedures, which include on-site visits, among other things. With its AmoKI project, in cooperation with the Institute of Mineral Engineering at RWTH Aachen University and Geologischer Dienst NRW and funded by BMBF, dida aims to find ways to accelerate and automate these processes while making them more precise.

A key resource for our work is publicly available geospatial data from Geobasis NRW, which provides valuable topographical and geological datasets under open data licenses. These datasets include high-resolution aerial imagery and digital terrain models, which can be supplemented by open satellite data, such as the multispectral imagery provided by the Sentinel-2 satellites. dida will utilize its experience from previous computer vision projects (Artisanal and Small Mine Detection, Mining Tailings Detection, and Crop Type Classification), where machine learning models have already proven useful in classification tasks on satellite images. The goal is to extract real-time insights that would otherwise have to be collected by human professionals.

One significant challenge for all satellite image processing is the lack of depth information - in some cases, the satellite images simply may not contain enough data to make any reasonable predictions about the volume of the mine. Furthermore, even if the model is able to accurately predict the depth and extent of the mine, calculating its volume is not always straightforward - it often requires information from the mining permit, as well as other local domain knowledge. Moreover, naturally occurring artifacts in aerial and satellite images (such as clouds, poor resolution, and extraneous objects) need to be addressed to prevent them from polluting the output.

Sometimes even the least cloudy images in a given month can obscure the target object.

Our approach focuses on automating the detection and assessment of mining activities. We employ machine learning algorithms that analyze open geospatial datasets from Geobasis NRW, along with high-resolution aerial images and satellite data. These models make accurate predictions of the topographical structure of mines and can be used to calculate volume changes over time. By automating parts of the workflow, we reduce reliance on manual site visits, offering a safer, more efficient, and faster way to identify critical changes. Our methods could also be extended to detect environmental indicators such as vegetation loss, landform alterations, and potential pollution markers, facilitating timely interventions.

To ensure widespread accessibility, we will create a frontend demo to showcase how the models make predictions, similar to our ASMSpotter demo.

The core model underlying this project is a Monocular Depth Estimation Model, which looks at objects in a single image and tries to determine the distance of these objects from the camera. In our case, the images we are using are aerial and satellite images, and the objects are features of the landscape (trees, hills, houses, mines, etc) and the ground itself. The prediction of distance to the camera therefore corresponds to the height of the terrain, otherwise known as a Digital Elevation Model (DEM). Although sophisticated depth estimation models already exist, they are heavily focused on making predictions of standard photographs involving people, traffic, and other everyday experiences - in other words, they aren’t trained to make predictions on satellite data.  We will finetune a selection of these preexisting models using our custom aerial and satellite image datasets, thus adapting these depth estimation models to be better suited to our specific problem.

Pretrained models provide a good starting point for monocular depth estimation but require fine-tuning to provide accurate results. 

In order to alleviate some of the pain points identified in the Challenges section, we have chosen to incorporate multiple data sources and multiscale predictions. For example, high-resolution aerial images may provide a detailed view of the landscape and allow for predictions of the local changes in elevation but may lack a reference elevation. These images can be supplemented with reference elevations from the Sentinel-2 satellites to help provide a baseline for the predictions. Alternatively, multiscale training can allow for a reference elevation to be first estimated based on high-resolution data that has been downscaled to a low resolution. These initial predictions can then be refined by passing higher and higher-resolution images.

If you would like to speak with us about this project, please reach out and we will schedule an introductory meeting right away.