The supply of resources extracted adhering ecological, economic and social norms are a central element for maintaining the competitiveness and jobs of the German/European industry. Particularly in view of the growing competition from Asian countries and trade conflicts with far-reaching consequences for the procurement of raw materials.

The exploration of new economic deposits that fulfil these criteria is the riskiest phase of mining projects due to the high probability of failure and high cost. In 2018 alone, expenditure on exploration for non-ferrous metals totalled US$ 10.1 billion (S&P Global, 2019). The global demand for mineral raw materials, combined with the declining material content of primary deposits (natural mineral enrichment), leads to a supply risk with regard to the availability of extractable resources.

We have been granted research funds from the BMBF to investigate the automated assessment of tailings (mine waste storage sites) utilizing satellite imagery as a viable new way of finding extractable resources.

Why would we remotely analyze tailings:

• The surface often is predominantly uncovered, which makes remote sensing possible in the first place.

• Large volumes of high-quality economically strategic metals might lie in tailings piles and ponds, as processing technologies were more inefficient in the past compared to today.

• They may contain metals such as rare earth elements that were not considered worth mining when the ores were originally processed, but whose value and usability have since increased.

• Mines are usually located in geographically remote locations, which precludes both time-consuming and costly conventional exploration methods.

• Even fee-based databases lack precise GPS locations with volume estimates and data on the overall mineral composition of tailings piles and ponds, meaning that a great deal of manual effort and on-site expeditions are required to collect precise data and identify promising tailings piles.

Within this project, the aim was to create a database that provides the information necessary for targeting promising tailing candidates for further exploration endeavors. This entails:

1. Finding the tailing object and delineating its shape using a computer vision model, which classifies pixels into a tailing category or background (Semantic Segmentation)

2. From the found shape the volume is estimated using elevation data

3. The surficial elemental concentrations are estimated from the satellite pixels located within the delineated shape

4. The location, estimated volume and amount of resources per element is returned

Task

For the training of the semantic segmentation model, the exact tailing shapes need to be delineated. Furthermore, as coordinates in the databases are often imprecise, the tailing object needs to be found and distinguished from similar non-tailing objects.

Solution

Our partners from MRE used their expert knowledge to discriminate tailing objects in the vicinity of database coordinates, and delineated a corresponding shape in Google Earth.

Task

To analyze the tailing and its content, first it needs to be found and delineated. Based on multispectral satellite data, tailing objects are discriminated from a general background and classified.

Data

The labelled dataset showed a severe class imbalance among tailing types. As well as regional biases. These insufficiencies are handled through the training setup.

As model input several satellite sources were combined. The multispectral Sentinel 2 data provides 12 spectral channels as well as additional masks such as cloud coverage. In addition digital elevation model data from the JAXA mission is used to detect the distinct elevation profiles of tailings and binary segmentation masks from the 2019 world settlement footprint to weed out some false positives.

A crucial addition to the dataset, which increases performance are hard negatives. Here we include objects like lakes, dams etc. labelled as background, which can easily be confused with tailings due to their topography and shape.

Model

As core deep learning architecture, an ensemble of UNet++ models is used. Each UNet++ uses an SK ResNet encoder backbone pretrained on Imagenet as feature extractor using spatial and channel 'squeeze & excitation' attention, batch norm, and the ELU activation. The loss was given as a sum of differently weighted combinations of Dice and focal loss with adapted weights for different channels.

Post-Processing

The soft segmentation predictions from the deep learning architecture are subjected to an elaborate post-processing algorithm customized for the best possible detection of tailings. The post-processing consisted several steps.

1. A combination of classical image processing algorithms such as the SLIC (Simple Linear Iterative Clustering) algorithm for finding superpixels and edge detection using Sobel filters

2. Extraction of polygons from the pixel wise segmentation probabilities

3. Filtering polygons, which area is smaller than a predetermined threshold

4. Polygon smoothing

5. Removing "holes" from polygons

6. Removal of obvious False Positives with respect to settlement data.

Result

A tailing is considered to be found, if a considerable fraction of the a predicted polygon covers the object area.

Due to the imbalanced training dataset, ultimately a performance categorization based on ecoregion yields strong results for desert region, acceptable for temperate and mediterranean, whereas the tropics are a very challenging region.

Task

To understand what amount of resources can potentially be extracted from the tailing, a volume analysis is required.

Problem

From satellite data alone, there is no way to exactly determine the volume of a tailing. Also in practice databases contain estimated values.

Particularly inconvenient for a model evaluation is the considerable spread of volume estimates between different databases.

Solution

Based on satellite elevation data on the found tailing as well as the surrounding area, the volume is automatically estimated. The general solution is to estimate the ground underneath the tailing structure.

Dependent on the tailing type, which is classified in the segmentation step, different methods are applied. If the tailing is classified as mass piled on top of flat ground, a plain is fitted to the surrounding flat surface to recover the ground underneath.

If the tailing material is poured into a valley, the surface underneath is triangulated using cubic splines that are fitted using the surrounding surface. 

After the ground estimation, its height is subtracted from the tailing surface to calculate the volume.

Result

An average relative error of around 30% is achieved with this method, while the relative ordering of tailings according to size, reveals very strong performance of the method.

Task

Determine the elemental concentrations within the tailing, such that an estimate of the amounts of relevant resources can be estimated taking the volume into account.

Data

As model input, one needs satellite imagery with as much information as possible. The hyper-spectral satellite Prisma was used.

Unfortunately there is no sufficiently large, publicly available database, which relates point coordinates on a tailing to elemental concentrations, found at that coordinate. This would allow for model training in a straight forward fashion. Regarding tailings, we only had access to reduced concentrations for an entire tailing. These values however are not consistent, there is no methodology attached and it is unclear how the reduced concentrations relate to the surface material.

The found solution consists of taking databases from USGS as well as GeoROC and training our model on non-tailing point samples.

Model

As tailings are often covered by tailing ponds or vegetation, as a first step, the input pixels are filtered using the NDVI and NDWI indices.

Then a one-dimensional convolutional neural network is applied for each valid input pixel.

The returned elemental concentrations are averaged over all pixels to yield a final estimate for the entire tailing.