Lesson 1 · 09:30–10:30

Where is it hot?

A satellite can't touch the ground with a thermometer. It measures the invisible heat glow (thermal infrared) every surface gives off, and that glow is turned into surface temperature: how hot roofs, roads, trees and lakes are, not the air we breathe.
Landsat 705 km up: a camera for heat glow sun roofs and roads ≈ 40 °C, glows most trees ≈ 37 °C lake ≈ 32 °C, glows least Bengaluru's surface temperature (Landsat), 10:40 am, middle value of March–May 2022–26

Finding the pictures

Landsat 8 and 9 (NASA and USGS) each pass over Bengaluru every 16 days, together every 8, always at 10:40 am. Their thermal camera sees 100 m squares, and USGS turns each picture into surface temperature on 30 m squares, correcting for the air in between. Free online catalogues (STAC) list every picture:
import planetary_computer
import pystac_client

catalog = pystac_client.Client.open(
    "https://planetarycomputer.microsoft.com/api/stac/v1",
    modifier=planetary_computer.sign_inplace,  # free, no account needed
)
search = catalog.search(
    collections=["landsat-c2-l2"],
    bbox=[77.45, 12.82, 77.80, 13.155],  # Bengaluru: west, south, east, north
    datetime="2024-03-01/2024-05-31",
    query={
        "platform": {"in": ["landsat-8", "landsat-9"]},
        "eo:cloud_cover": {"lt": 40},
    },
)
items = search.item_collection()

pd.DataFrame({
    "date": [i.datetime.date() for i in items],
    "satellite": [i.properties["platform"] for i in items],
    "cloud %": [i.properties["eo:cloud_cover"] for i in items],
})
date satellite cloud %
2024-05-29 landsat-8 39.52
2024-05-21 landsat-9 37.68
2024-05-05 landsat-9 0.02
2024-04-27 landsat-8 0
2024-04-19 landsat-9 20.6
2024-04-11 landsat-8 0.15
2024-04-03 landsat-9 2.06
2024-03-26 landsat-8 0.04
2024-03-18 landsat-9 0.3
2024-03-10 landsat-8 1.76
2024-03-02 landsat-9 10.69

Reading one picture

We download only the pixels over Bengaluru and turn the stored numbers into surface temperature in °C:
import rasterio
from rasterio.windows import from_bounds

# the clearest picture, and the city's edges in metres (the same map grid as the picture)
scene = min(items, key=lambda item: item.properties["eo:cloud_cover"])
city = datapack.grid30().bounds

with rasterio.open(scene.assets["lwir11"].href) as image:  # lwir = the thermal infrared camera
    window = from_bounds(*city, image.transform)
    dn = image.read(1, window=window)  # only the pixels over the city

# whole numbers -> surface temperature in °C (0 means no data)
lst = np.where(dn > 0, dn * 0.00341802 + 149.0 - 273.15, np.nan)
Surface temperature at 10:40 am on 27 April 2024 (Landsat-8, the clearest picture of March–May 2024): pale is cooler (lakes, parks and the leafy centre), dark is hotter (bare farmland at the edges). See it, get it. Every Landsat picture of the world is on Planetary Computer's Explorer and USGS EarthExplorer (free login to download). This picture's thermal file is 88.5 MB and covers 231 × 227 km; our code reads only the 38 × 38 km over Bengaluru, about 1/36 of it, straight from the cloud. To keep the whole file, paste this into a new cell:
import requests
with open("landsat_thermal.tif", "wb") as out:  # 88.5 MB; open it in QGIS or with rasterio
    out.write(requests.get(scene.assets["lwir11"].href).content)

At night: another satellite

Landsat only passes in the morning. MODIS, a camera on NASA's Terra and Aqua satellites, also passes at night, in 1 km squares; each MODIS file is the average of the clear nights in 8 days. We load the Terra file for the same 8 days as the Landsat picture:
import odc.stac

nights = catalog.search(
    collections=["modis-11A2-061"],  # MODIS surface temperature, one file per 8 days
    bbox=[77.45, 12.82, 77.80, 13.155],
    datetime=f"{scene.datetime:%Y-%m-%d}",  # the 8-day files that include the Landsat picture's day
).item_collection()
terra = [i for i in nights if i.id.startswith("MOD")][0]  # MOD = Terra, MYD = Aqua

night = odc.stac.load(
    [terra], bands=["LST_Night_1km", "Night_view_time"], bbox=[77.45, 12.82, 77.80, 13.155],
    crs="EPSG:32643", resolution=1000,  # Landsat's map grid, in 1 km squares
)
dn_night = night["LST_Night_1km"].values[0].astype("float32")
lst_night = np.where(dn_night > 0, dn_night * 0.02 - 273.15, np.nan)  # -> surface temperature in °C

# the file also records when the satellite looked, in sun time; India's clock runs 20 minutes ahead of it here
seen = night["Night_view_time"].values[0]
seen_at = np.median(seen[seen < 255]) * 0.1 + 20 / 60  # hours after midnight, India time
Same week, two satellites: the surface temperature is 20 °C lower at night, and the centre stays warmest. See it, get it. NASA Worldview maps MODIS surface temperature for the whole world, day by day (search its layers for "land surface temperature"); the files download from NASA Earthdata. Each file covers a tile of about 1,100 × 1,100 km, and again we read only Bengaluru's part.

Five summers in one map

We repeated this for 44 Landsat mornings and 105 MODIS 8-day files (Terra at about 10 pm, Aqua at about 2:30 am), March–May 2022–26, and kept each square's middle value. So your browser can draw the whole city, the download averages each 8 × 8 block of 30 m squares into one 240 m square:
squares = datapack.table("pixels")  # 12,449 squares of 240 m, from the workshop GitHub page
squares[["ward", "lst_day", "lst_night"]].head()  # surface temperature (°C) by day and at night
ward lst_day lst_night
North 3 · Chowdeshwari ward 36.8 21.8
North 3 · Chowdeshwari ward 35.3 21.9
North 3 · Chowdeshwari ward 39 21.9
North 3 · Chowdeshwari ward 35.3 21.9
North 3 · Chowdeshwari ward 32.3 21.9
Toggle day and night; hover to see the ward.
How this file was made · making it reads 44 Landsat passes and 105 MODIS 8-day files from Planetary Computer, about 12 minutes

The chart reads a small prepared file (94 kB): your copy in data/ if you have one, otherwise it downloads once. To make it yourself from the original data, run uv run wardheat datapack build --step landsat --step modis --step landcover --step canopy --step features --force. This is the code that runs:

def step_landsat(force=False):
    from . import landsat, stac

    if datapack.exists("lst_day") and not force:
        return
    g = city_grid(30)
    items = stac.landsat_items()
    _log(f"landsat: {len(items)} scenes (Mar-May {config.YEARS[0]}-{config.YEARS[1]})")
    ds = landsat.load(items, g)
    phys = landsat.to_physical(ds)
    comp = landsat.median_composite(phys)
    save_geotiff(_out("lst_day"), comp["lst"].values.astype("float32"), g, source="Landsat C2 L2 ST_B10")
    _log(f"landsat: done, median city LST {np.nanmedian(comp['lst'].values):.1f} °C")


def median_composite(phys: xr.Dataset, min_obs: int = 3) -> xr.Dataset:
    """Per-pixel median across all clear scenes; pixels with fewer than ``min_obs`` clear looks -> NaN."""
    n = phys["lst"].notnull().sum("time")
    comp = phys[["lst"]].median("time", skipna=True).where(n >= min_obs)
    comp["n_obs"] = n.astype("int16")
    return comp


def step_modis(force=False):
    from . import modis

    if datapack.exists("lst_night") and not force:
        return
    g = city_grid(30)
    items = modis.items()
    _log(f"modis: {len(items)} 8-day composites {modis.platform_counts(items)}")
    r = modis.night_lst(items, g)
    save_geotiff(_out("lst_night"), r["fine"], g, n_composites=r["n"])
    src = config.data_dir() / "source"
    src.mkdir(exist_ok=True)
    save_geotiff(src / "lst_night_1km.tif", r["coarse"], r["coarse_grid"])
    _log(f"modis: done, median night LST {np.nanmedian(r['coarse']):.1f} °C")


def night_lst(items_: list, grid: Grid, coarse_res: float = 1000.0, which: str = "LST_Night_1km") -> dict:
    """Median night LST on a ~1 km UTM grid, plus a smooth bilinear version on ``grid``.

    Returns ``{"coarse": array, "coarse_grid": Grid, "fine": array on grid, "n": count}``.
    """
    import odc.stac

    coarse = grid.coarsen(int(round(coarse_res / grid.res)))
    # one layer per file: Terra's and Aqua's files for the same 8 days share a date, and grouping by date would merge them
    ds = odc.stac.load(items_, bands=[which], geobox=coarse.geobox(), resampling="nearest", fail_on_error=False, groupby="id")
    dn = ds[which].astype("float32")
    lst = (dn * 0.02 - 273.15).where(dn > 0)
    med = lst.median("time", skipna=True).values.astype("float32")
    n = int(lst.sizes["time"])
    fine = reproject_array(med, coarse.transform, coarse.crs, grid)
    return {"coarse": med, "coarse_grid": coarse, "fine": fine, "n": n}


def pixel_table(factor: int = 8, wards=None) -> pd.DataFrame:
    """The 30 m layers averaged to ``30 × factor`` m cells (240 m by default), cells inside the city only.

    Small enough (~12 000 rows) to draw every cell as an interactive chart in the browser.
    Columns: ``col``, ``row`` (cell position, row 0 = north), ``ward_uid``, ``ward``, day/night surface
    temperature, tree canopy, built-up, water and farmland shares, and the cover type.
    """
    wards = wards if wards is not None else load_wards()
    g30 = city_grid(30)
    lab = ward_labels(wards, g30.coarsen(factor))
    L = datapack.layer
    built = L("built_frac")
    layers = {
        "lst_day": L("lst_day"),
        "lst_night": L("lst_night"),
        "canopy": L("canopy_frac"),
        "built": built,
        "water": L("water_frac"),
        "farmland": L("crop_frac"),
    }
    agg = {k: block_mean(v, factor) for k, v in layers.items()}
    rows, cols = np.nonzero(lab > 0)
    df = pd.DataFrame({"col": cols, "row": rows, "ward_uid": lab[rows, cols]})
    for k, v in agg.items():
        df[k] = v[rows, cols].astype("float64")
    df.insert(3, "ward", df["ward_uid"].map(wards.set_index("ward_uid")["label"]))
    df = df.dropna(subset=["lst_day"])
    df["cover"] = np.select(
        [df.water > 0.3, df.canopy > 0.35, df.farmland > 0.4, df.built > 0.6],
        ["water", "trees", "farmland", "built-up"],
        "mixed",
    )
    for c in ("lst_day", "lst_night"):
        df[c] = df[c].round(1)
    for c in ("canopy", "built", "water", "farmland"):
        df[c] = df[c].round(2)
    return df.reset_index(drop=True)
By day the highest surface temperature is on farmland at the edges; at night it flips and the packed centre stays warmest.

From squares to wards

Each ward's surface temperature is the average of all the 30 m squares inside it:
wards = load_wards()  # the 369 GBA wards' boundaries (OpenCity)
lst_day = datapack.layer("lst_day")  # surface temperature at 10:40 am, 5-summer middle value, 30 m squares, °C

labels = ward_labels(wards, datapack.grid30())  # each square -> the number of its ward
ward_temp = zonal_stats({"lst": lst_day}, labels)  # average of each ward's squares
ward_temp.head()
lst_mean
38.482
37.656
37.15
36.699
39.471
zonal_stats drops every square into its ward's bucket and averages each bucket; here are 16 wards around Padarayanapura (a ward cut by the box edge shows the average of all its squares, also those outside the box):
West 72 · Kaveripura has the highest surface temperature (41.9 °C), West 50 · Sadashiva Nagara the lowest (35.8 °C): 6.1 °C apart.

Try it

  1. Find your ward on the map. 2. Search 2019 instead of 2024:
passes_2019 = catalog.search(
    collections=["landsat-c2-l2"],
    bbox=[77.45, 12.82, 77.80, 13.155],
    datetime="2019-03-01/2019-05-31",  # change the year here
    query={"platform": {"in": ["landsat-8", "landsat-9"]}, "eo:cloud_cover": {"lt": 40}},
).item_collection()
f"{len(passes_2019)} clear pictures in March–May 2019"

4 clear pictures in March–May 2019

  1. MODIS surface temperature by day: the same Terra file as above, with the LST_Day_1km band:
import odc.stac as _stac

_day = _stac.load([terra], bands=["LST_Day_1km"], bbox=[77.45, 12.82, 77.80, 13.155], crs="EPSG:32643", resolution=1000)
_dn = _day["LST_Day_1km"].values[0].astype("float32")
charts.images({"MODIS Terra by day, same 8 days · 1 km": np.where(_dn > 0, _dn * 0.02 - 273.15, np.nan)},
              "surface temperature, °C")
Made with open data and open-source Python · marimo · Altair