Training coarse, predicting fine

Most administrative data is available at only some highly aggregated level (provinces, districts, counties, etc) and those units can be both irregularly shaped and wildly different in size. In this paper, we show that be using vectorized representations of a satellite image (an earth embedding) we can represent an adminstrative polygon by averaging it’s feature values over space. This allows for large and irregularly shaped polygons to be represented by a single vector that contains much of the information in the satellite imagery for that polygon.

Next we show that it’s possible to use that vector to estimate HDI and its componenents at the native (provincial/ADM1) resolution. Finally, we show that the model trained on provincial administrative imagery can be used to make estimates for finer (municiplaity/ADM2) administrative polygons.

In this way, we demonstrate that satellite imagery can be used to train on arbitrarirly shaped and coarse resolution, but then have predictive efficacy at much finer resolution. you actually need.

Think about a crime film, where an investigator take a fuzzy image and then “enhances it” to magically produce an image at better resolution. We show that using satellite image embeddings as an auxillary dataset, it’s possible to do this exact thing!

Validating our efficacy

To test whether the province-trained model actually generalizes downward, we check its predictions against ground-truth municipal HDI in Mexico, Brazil, and Indonesia — three countries where municipal-level HDI data actually exists.

For each county, we find that our model substantially improves the granularity of the provincial (ADM1) HDI data, even though it was only trained on data at this level. Specifically, we are able to explain 45-61% of country-relative municipal variation, and 20-53% of province-relative variation, the latter number purely describing the amount of increased granularity we achieve from the satellite image-based method.

Fig. 2C-D from the paper: how well the province-trained model predicts municipal HDI for three countries.

An HDI dataset at municipal (ADM2) resolution

The map below shows the coarse level of data that was used to train the model and the much finer resolution dataset that we are able to produce using the auxillary satellite imagery. Specifically, the top row shows HDI at provincial (ADM1) resolution, as computed directly from raw survey data by Smits and Permanyer. The bottom row shows the finer-resolution global gridded dataset — about 819,000 tiles — that we produce in this research.

Province-level HDI (top) versus grid-level HDI at roughly 10km resolution (bottom).