Lesson 6 · 15:25–15:45

How do we get it all back?

During the day we saved seven things into my_catalog/: a picture, a night of rain, water bodies, watersheds, buildings, a change map and embeddings. Now we open them again, starting from nothing but catalog.json.
It is the same three steps every time: open the catalog, find the item, follow its link to the file.

Open the catalog

pystac reads catalog.json and follows its links to every item.
import pystac

catalog_file = mycatalog.FOLDER / "catalog.json"     # lessons 1, 2, 3 and 5 fill it
mo.stop(not catalog_file.exists(), "Run lessons 1, 2, 3 and 5 first.")

catalog = pystac.Catalog.from_file(str(catalog_file))
pd.DataFrame([{"id": item.id, "title": item.properties["title"],
               "date": item.datetime.date(),
               "format": item.assets["data"].media_type.split(";")[0]}
              for item in catalog.get_items()])
id title date format
lake_2025 Bellandur lake, February 2025 (Sentinel-2) 2025-02-06 image/tiff
rain_flood_night Rain over Bengaluru, night of 4-5 Sep 2022 (IMERG) 2022-09-04 application/vnd.zarr
waterbodies Water bodies and their water area per year (CoRE Stack) 2025-01-01 application/vnd.apache.parquet
watersheds Micro-watersheds around the lake (CoRE Stack) 2025-01-01 application/vnd.apache.parquet
buildings Building footprints east of the lake (Overture Maps) 2025-01-01 application/vnd.apache.parquet
change_2017_2025 Change of each 10 m square, 2017 to 2025 (AlphaEarth) 2025-01-01 image/tiff
alphaearth_2025 AlphaEarth Foundations embeddings, 2025 (Google DeepMind) 2025-01-01 image/tiff
The format column (the media type) tells you which tool opens the file: geopandas for Parquet, rasterio for GeoTIFF, xarray for Zarr.

The buildings, from GeoParquet

Ask for an item by its id, then follow its data asset to the file.
import geopandas as gpd

buildings_item = catalog.get_item("buildings")
buildings_file = buildings_item.assets["data"].get_absolute_href()
buildings = gpd.read_parquet(buildings_file)
title = f"{len(buildings):,} buildings, from the catalog"
charts.buildings_map(buildings, title=title,
                     subtitle=buildings_item.properties["title"])

The picture, from a COG

import rasterio

lake_item = catalog.get_item("lake_2025")
with rasterio.open(lake_item.assets["data"].get_absolute_href()) as cog:
    lake = cog.read()                                # 3 colours x 500 x 700 pixels
when = f"{lake_item.datetime:%d %B %Y}"
charts.picture({when: lake.transpose(1, 2, 0)}, width=640)

The rain, from Zarr

import xarray as xr

rain_item = catalog.get_item("rain_flood_night")
rain_file = rain_item.assets["data"].get_absolute_href()
rain = xr.open_zarr(rain_file, consolidated=False)
night = rain.rain_mm.sum("time").to_dataframe(name="mm").reset_index()
charts.rain_map(night, LAKE, title="The flood night, back from the catalog",
                subtitle=rain_item.properties["title"])

Searching a static catalog

In lesson 1, Earth Search answered our questions because it runs a search service. A static catalog is only files, so we search it ourselves: read every item and keep the ones we want. Here, everything that covers the lake and is from 2025:
from shapely.geometry import Point, shape

lake_point = Point(LAKE)
[item.id for item in catalog.get_items()
 if shape(item.geometry).contains(lake_point) and item.datetime.year == 2025]
['lake_2025', 'waterbodies', 'watersheds', 'change_2017_2025', 'alphaearth_2025']
That is quick for seven items or a few thousand. For millions, a catalog is kept as one table instead (stac-geoparquet) and queried with DuckDB, the way we found Overture's buildings in lesson 3.

Share it

Everything above opened files on this laptop. Copy my_catalog/ to any web server, an S3 bucket, GitHub Pages or Source Cooperative and the same code works with an address in place of the path: pystac.Catalog.from_file("https://…/catalog.json"). geopandas, rasterio and xarray then read the files over the internet, only the parts they need. Paste the address into STAC Browser and anyone can look through it in a web page.

Try it

  1. Only part of the buildings. Add bbox=(77.68, 12.92, 77.69, 12.93) to gpd.read_parquet in the buildings cell. How many come back?
  2. Older things. Change the search to keep items from before 2023. Which lesson saved it?
  3. The embeddings. Find the item that has an emb:dimensions property and open its file with rasterio. Does the number of bands match the property?

Where to go from here

Thank you for spending the day with the lake. Everything here is open: try for a different location
Made with open data and open-source Python · marimo · Altair