Creating an Explorer

Explorers are OWID's multi-dimensional dashboards (e.g. ourworldindata.org/explorers/food-prices). They're authored as YAML in this repo and published by ETL at viz://explorer/<ns>/latest/<short>.

This skill is the explorer-flavored sibling of /create-multidim. They use the same engine (paths.create_chart for multidims, paths.create_explorer for explorers) and the same YAML schema for dimensions / views / definitions.common_views. The differences are:

	Multidim	Explorer
Channel	viz://chart/...	viz://explorer/...
Step file location	etl/steps/viz/chart/<ns>/latest/	etl/steps/viz/explorer/<ns>/latest/
PathFinder method	paths.create_chart(...)	paths.create_explorer(...)
Top-level config block	title:, default_selection:, default_dimensions:	config: block carrying legacy explorer settings (explorerTitle, explorerSubtitle, selection, subNavId, entityType, …)
Slug convention	underscores in file paths and short_name	underscores in file path, hyphens in URL slug and short_name argument
Verification	preview URL on staging	preview URL on staging + diff against owid-grapher/explorers/<slug>.explorer.tsv
Save call	c.save()	c.save(tolerate_extra_indicators=True) (upstream grapher datasets usually have more indicators than the explorer references)

If you're modifying an existing explorer (adjusting chart text, swapping a catalogPath, adding a dimension choice, reordering views), most of the deeper sections below don't apply — find the existing <short>.config.yml, edit, run etlr, done. The full structure is documented below for new explorers and substantial reshapes.

When to use this skill

• After a /migrate-explorer-csv, /migrate-explorer-grapher, or /migrate-explorer-indicator-legacy skill has produced (or already located) the upstream snapshot/meadow/garden/grapher chain, and now needs the explorer step.
• For a brand-new explorer where the data is already in ETL (skip directly to step 1).
• When porting an existing explorer's view layout (e.g. full-YAML → table-driven, or moving FAUST text from per-view YAML up into indicator metadata).

Step 1 — Files & directories

Every explorer is exactly two files plus a DAG entry:

etl/steps/viz/explorer/<ns>/latest/
├── <short>.py              # Python uses snake_case
└── <short>.config.yml

mkdir -p etl/steps/viz/explorer/<ns>/latest

Hyphens vs underscores (recurring source of confusion):

• The Python file path uses underscores: food_footprints.py, crop_yields.py.
• The explorer slug used in the URL and the short_name= argument keeps hyphens: food-footprints, crop-yields.
• paths.create_explorer(short_name="<short-with-hyphens>") — pass the hyphenated slug.

Step 2 — Pick a construction style

The two ends of the spectrum, plus everything in between:

	Full-YAML	Programmatic / table-driven
Where views live	Hand-listed in <short>.config.yml under views:	Auto-expanded by paths.create_explorer(tb=tb, ...) from columns whose m.dimensions is set
Where chart text (FAUST) lives	Per-view view.config.{title, subtitle, note} in YAML	presentation.{title_public, grapher_config} on each indicator's garden metadata; common defaults via definitions.common.presentation.grapher_config
Where chart-level config lives (hasMapTab, tab, yAxis, chartTypes)	Per-view view.config	Indicator's presentation.grapher_config — single source of truth, same as for any standalone chart on that indicator
Map color scale	view.indicators.y[i].display.{colorScaleScheme, colorScaleNumericBins} (semicolon-string form, explorer-flavored override)	presentation.grapher_config.map.colorScale.{baseColorScheme, binningStrategy, customNumericValues} (canonical grapher form, inherited at chart render time)
Python step content	Trivial: paths.create_explorer(config=config, short_name=...).save(...)	Loops columns to set m.dimensions, optionally post-processes (sort_choices, group_views, per-view display tweaks)

It's a spectrum, not a switch. Mix freely: use table-driven for the bulk of views, hand-list a handful of bespoke ones; or stay full-YAML but still push title/subtitle for the single-indicator views into garden metadata to remove duplication.

Strong fit for table-driven:

• Single-indicator views dominate. Each view is a thin wrapper around one indicator → that indicator's metadata is the right home for chart text. Avoids duplication between explorer YAML and the equivalent standalone chart, and keeps both in sync forever.
• Many views (>20) following the cartesian product of a few dimensions. Hand-listing them is repetitive; auto-expansion plus YAML dimensions is significantly less code.
• Upstream is a dimensional table (one row per country/year × dim_a × dim_b × …) with one indicator. create_explorer(tb=tb, indicator_names=..., dimensions=...) matches this shape directly — model: migration/latest/migration_flows.py.
• Same indicators back standalone grapher charts. Pushing FAUST upstream means explorer view and standalone chart inherit the same text — no drift over time.

Stick with full-YAML when:

• Few views (<10), all bespoke (different chart types / data sources / hand-tuned text).
• Multi-indicator views dominate. FAUST cannot live on any single indicator when a view shows multiple indicators — you have to write it explicitly per view in the explorer YAML (or build views via c.group_views(...)).
• Single-shot migration with no plan to maintain. The duplication of full-YAML doesn't matter if no one will edit it again.

Step 3 — The Python step

Full-YAML variant

"""<one-line description of what this explorer surfaces>."""

from etl.helpers import PathFinder

paths = PathFinder(__file__)


def run() -> None:
    config = paths.load_config()
    c = paths.create_explorer(
        config=config,
        short_name="<short-with-hyphens>",  # explorer slug
    )
    c.save(tolerate_extra_indicators=True)

tolerate_extra_indicators=True is the common case: the upstream grapher dataset usually carries more indicators than the explorer references, and without this flag c.save() errors on the unused ones.

Table-driven variant

"""<one-line description>."""

from etl.helpers import PathFinder

paths = PathFinder(__file__)

# Map column → dimension tuple. "na" is the conventional empty slot for conditional dimensions
# (e.g. cost_metric is meaningful only when type=cost; affordability views set cost_metric="na").
COLUMN_DIMENSIONS: dict[str, dict[str, str]] = {
    "<col_a>": {"dim1": "value_a1", "dim2": "value_a2"},
    "<col_b>": {"dim1": "value_b1", "dim2": "value_b2"},
    # ...
}


def run() -> None:
    config = paths.load_config()

    ds = paths.load_dataset("<grapher_dataset>")
    tb = ds.read("<table>", load_data=False)  # metadata only — faster, we don't need values

    for column, dims in COLUMN_DIMENSIONS.items():
        tb[column].m.dimensions = dims
        tb[column].m.original_short_name = "<unifying_indicator_name>"

    c = paths.create_explorer(
        config=config,
        tb=tb,
        indicator_names=["<unifying_indicator_name>"],
        dimensions={
            "dim1": ["value_a1", "value_b1", ...],   # explicit choice order
            "dim2": ["value_a2", "value_b2", ...],
        },
        # common_view_config={...},                   # only if not in indicator metadata
        short_name="<short-with-hyphens>",
    )

    # Optional post-processing — see "Post-processing" below.
    # c.sort_choices({"dim1": lambda x: sorted(x)})
    # c.group_views([...])

    c.save(tolerate_extra_indicators=True)

Key APIs (see etl/viz/chart/core/expand.py and etl/viz/chart/core/create.py):

• tb[col].m.dimensions: dict[str, str] — required per column. Each entry says "this column represents the (dim1=value, dim2=value) cell." Columns without m.dimensions are ignored by the expander.
• tb[col].m.original_short_name: str — the unifying indicator name. With indicator_names=[that_name] and a single name, the expander treats all N columns as one logical indicator with N dimension combinations and drops the auto-added "indicator" pseudo-dimension.
• dimensions= accepts:
  • None → all dimensions found, arbitrary order.
  • list[str] → restricts and orders dimensions, all values shown.
  • dict[str, list[str] | "*"] → restricts and orders both dimensions and choices. Use "*" for "all values, arbitrary order."
• common_view_config= is applied uniformly to every auto-expanded view. Use it for fields that are truly shared and don't live at indicator level. Prefer indicator-level presentation.grapher_config for anything that should also flow to standalone charts.

Step 4 — The config YAML

Always block style. Mappings and lists in explorer config YAML must use block style — one key per line, list items on their own line under -. Never use flow style ({ key: value, ... } or [a, b, c]) even for tiny per-view dimensions: blocks. PR review on a 45-view file is unreadable when half the views collapse to a single flow line. The only exception is markdown links inside a quoted-scalar subtitle:/note: (those [text](url) brackets are content, not YAML structure).

config:
  # Explorer settings rows — keys map verbatim from the legacy TSV settings section.
  explorerTitle: ...
  explorerSubtitle: ...
  isPublished: true
  hasMapTab: false
  hideAlertBanner: true
  hideAnnotationFieldsInTitle: true
  entityType: country         # or "food", "region", etc.
  thumbnail: https://assets.ourworldindata.org/uploads/...
  wpBlockId: "12345"
  subNavId: explorers
  subNavCurrentId: <slug>
  selection:
    - <default selected entity>
    - <another>
  pickerColumnSlugs: []        # an empty list is OK; non-empty must be block-style
  yAxisMin: 0
  # ...

definitions:
  # Shared config applied to all views. Use this list (with optional `dimensions:`
  # filter per entry) — NOT YAML anchors and `<<:` merge keys. The framework merges
  # entries at expansion time; per-view `config:` blocks override anything here.
  common_views:
    - config:
        type: DiscreteBar
        hasMapTab: false
    # Dimension-filtered overrides apply only to matching views:
    # - dimensions:
    #     metric: share
    #   config:
    #     note: "Share values sum to 100%"

dimensions:
  # one entry per dropdown / radio / checkbox the user toggles
  - slug: <snake_case>          # e.g. "metric"
    name: <human label>          # e.g. "Metric"
    presentation:
      type: dropdown             # or radio / checkbox
    choices:
      - slug: <choice_snake>
        name: "<as shown in widget>"
      - slug: <another>
        name: "..."

views:
  # one entry per (dim1=x, dim2=y, …) tuple
  - dimensions:
      <dim_slug>: <choice_slug>
      # ...
    indicators:
      y:
        - catalogPath: <table>#<short>      # short form — see "catalogPath — short forms accepted" below
          display:                          # per-view, per-indicator overrides
            colorScaleNumericBins: 0;1;2
            colorScaleScheme: PuBu
    config:
      # Only per-view overrides here. Common stuff lives in definitions.common_views.
      # No `<<:` merge keys, no `&anchor`s.
      title: ...
      subtitle: ...
      type: <chart type>         # LineChart, DiscreteBar, "LineChart DiscreteBar", StackedArea, …
      hasMapTab: false
      minTime: 1990
      yAxisMin: 0

For table-driven explorers, views: should still be present but is typically views: [] — the explorer JSON schema requires the key, and create_explorer(tb=tb, ...) populates the views at runtime.

catalogPath — short forms accepted

The Indicator.is_a_valid_path check (etl/viz/chart/model/view.py:62) accepts three forms; pick the shortest one that still unambiguously resolves:

Form	Example	When to use
table#indicator	global_carbon_budget#emissions_total	Default. Resolved against the explorer's DAG dependencies via tables_by_name — fine as long as no two dependencies expose a table with the same name.
dataset/table#indicator	global_carbon_budget/global_carbon_budget#emissions_total	When two upstream datasets happen to expose tables with the same short_name.
grapher/<ns>/<v>/<dataset>/<table>#<indicator>	grapher/gcp/2025-11-13/global_carbon_budget/global_carbon_budget#emissions_total	Only when you need to pin a specific dataset version separate from the one in the DAG — almost never the right form to write by hand.

Short forms are expanded at c.save() time by Indicator.expand_path(tables_by_name). If the table name doesn't exist in any dependency it raises Table name '<x>' not found in dependency tables; if multiple dependencies expose the same table name, it raises and asks you to disambiguate with the medium form.

Default to table#indicator when authoring YAML. The full path is verbose, drifts when upstream versions bump, and is only needed for genuinely ambiguous cases.

Top-level config: settings — the most common keys

Key	Type	Notes
explorerTitle	string	Page title above the explorer.
explorerSubtitle	string	One-liner under the title.
isPublished	bool	true to publish; false keeps it draft.
hasMapTab	bool	Whether any view shows the map tab by default.
entityType	string	country (default), or food, region, species, etc. — controls picker labels.
selection	list[str]	Default selected entities.
pickerColumnSlugs	list[str]	Picker columns shown alongside the entity name.
subNavId	string	Almost always explorers.
subNavCurrentId	string	The slug — appears as the active nav item.
wpBlockId	string	WordPress block ID for embedding (legacy). Stringify even when numeric.
thumbnail	string	URL of preview image.
hideAlertBanner	bool	Suppress the OWID-wide banner.
hideAnnotationFieldsInTitle	bool	Drop time/entity from auto-titles.
yAxisMin	number/string	Default Y-axis floor.
yScaleToggle	bool	Allow user to toggle linear/log.
originUrl	string	Path back to the topic page (e.g. /environmental-impacts-of-food).

Dimension presentation types

• dropdown: shown as <select>. Use for >4 choices or when the choices have long labels.
• radio: shown as a row of pills. Use for ≤4 mutually-exclusive choices.
• checkbox: shown as a single toggle. Two choices only — usually "off" (slug like combined/absolute/no) and "on" (slug like the field name). Pair with presentation.choice_slug_true: <on_slug> so the framework knows which slug means "checked."

- slug: by_stage
  name: By stage of supply chain
  presentation:
    type: checkbox
    choice_slug_true: stages
  choices:
    - slug: combined
      name: ""
    - slug: stages
      name: By stage of supply chain

Conditional dimensions (the "na" pattern)

When a dimension is only meaningful for some rows (e.g. cost_metric matters only when type=cost, not when type=affordability), include the dimension everywhere with an "na" slot:

• Tag the columns/views that don't use it with <dim>: "na".
• Declare the na choice with name: "" so the widget renders as empty when applicable.
• Model: agriculture/latest/food_prices.{py,config.yml}.

- slug: cost_metric
  name: Cost metric
  presentation:
    type: radio
  choices:
    - slug: na
      name: ""
    - slug: dollars_per_day
      name: $ per day

Step 5 — Push FAUST upstream (recommended for single-indicator views)

For single-indicator views, the rendered chart inherits the indicator's stored grapher_config from MySQL at render time (both standalone-chart and explorer-view paths). Push:

Per-view config	→ indicator garden metadata
title	presentation.grapher_config.title
subtitle	presentation.grapher_config.subtitle
note	presentation.grapher_config.note
map.colorScale	presentation.grapher_config.map.colorScale.{baseColorScheme, binningStrategy, customNumericValues}
hasMapTab, tab, yAxis, chartTypes, hideRelativeToggle, selectedFacetStrategy, …	presentation.grapher_config.<field>

For the chart-heading flow specifically, the priority is grapher_config.title > title_public > display.name > title (see docs/architecture/metadata/faqs.md). When migrating a chart-wrapping explorer, the chart's bespoke heading text belongs in grapher_config.title — title_public is the human-readable replacement for a dimensional indicator.title, not the chart's heading.

Cross-cutting baselines (e.g. hasMapTab: true for every indicator in a dataset) go under definitions.common.presentation.grapher_config in the garden .meta.yml. The catalog merge is recursive on presentation and grapher_config (lib/catalog/owid/catalog/core/yaml_metadata.py:_merge_variable_metadata), so each indicator inherits the common defaults plus its own overrides without manual <<: *anchor repetition.

DRY for repeated text fragments via dynamic-yaml interpolation:

definitions:
  prefix: "Long shared phrase about diet X."
  suffix: "Common closing sentence about methodology."

# In the indicator block:
subtitle: "{definitions.prefix} {definitions.suffix}"

This composes N unique full strings from a handful of building blocks. Verified via dynamic_yaml_to_dict (lib/catalog/owid/catalog/core/utils.py).

Step 6 — Post-processing the chart (table-driven only)

After paths.create_explorer() returns the explorer c, you can mutate it before c.save():

• c.sort_choices({dim_slug: lambda x: sorted(x)}) — control the order of dimension dropdowns. Useful when slugs sort poorly alphabetically.

• c.group_views(groups=[...]) — bundle multiple existing views into a new multi-indicator view. Each entry in groups:

  • dimension: the dimension whose choices are being collapsed.
  • choices: the choice slugs to combine (omit for all).
  • choice_new_slug: name for the new collapsed choice (e.g. combined, total, breakdown).
  • view_config: chart-level config for the new view (chartTypes, title, subtitle, selectedFacetStrategy, …). Title can be a template like "Population aged {age}" evaluated against params.
  • view_metadata: data-page metadata (description_key, etc.) for the new view.
  • replace=True to drop the originals; default keeps both.
  • overwrite_dimension_choice=True if choice_new_slug collides with an existing choice and you want grouped views to win.
  • Use cases: an explorer with sex={female, male} views — group_views adds sex=combined showing both timeseries on one chart. Same pattern works for age brackets, region groups, conflict types, or any dimension where users may want a single multi-line chart.

• c.edit_views([...]) — apply chart-level config to many views at once, optionally scoped by dimension. Each entry is {"dimensions": <filter>, "config": {...}, "metadata": {...}}; the framework merges entries by specificity (more dimensions in the filter = wins on conflicts). Use this in preference to set_global_config whenever you have per-slice overrides:

  c.edit_views([
      # No filter → applies to every view (defaults).
      {"config": {"type": "LineChart DiscreteBar", "hasMapTab": True}},
      # Scoped override — only this exact (gas, accounting, fuel, count) cell gets a Slope tab.
      {
          "dimensions": {"gas": "co2", "accounting": "territorial", "fuel": "all_fossil", "count": "per_capita"},
          "config": {"type": "LineChart SlopeChart DiscreteBar"},
      },
  ])
  

  Callable values inside config (e.g. "title": lambda v: ...) are evaluated against each matching view, so dimension-aware text templates still work. set_global_config is just a one-entry shortcut for edit_views; reach for edit_views once you have more than one slice to address.

• Manual loop over c.views — for things edit_views can't reach: per-indicator display blocks (numDecimalPlaces, colorScaleScheme, colorScaleNumericBins, display.color, …). These live on view.indicators.y[i].display, not on view.config. Match views via the view.matches(**kwargs) helper, which accepts a single value or a list (list = OR semantics):

  for view in c.views:
      if view.matches(gas=["methane", "all_ghg"], count="per_capita"):
          decimals = 1
      elif view.matches(gas="warming_impact", fuel=["land_use", "fossil_plus_land_use"]):
          decimals = 3
      else:
          continue
      for indicator in view.indicators.y or []:
          indicator.display = {**(indicator.display or {}), "numDecimalPlaces": decimals}
  

  Pattern: migration_flows.py's add_display_settings(c). Avoid this loop when the setting can live on the indicator's garden metadata instead.

• choice_renames={dim: {slug: display_name, ...}} (passed directly to create_explorer) — map slug → display name when you need to derive the display label programmatically. Model: chart/minerals/latest/minerals.py.

• Sidecar <short>.dims.yaml — when the column → dimensions map exceeds ~50 entries, lift it out of the Python step into a sidecar YAML loaded at module-import time. Keeps <short>.py focused on logic and turns dim-tagging changes into a 1-line YAML edit. Model: etl/steps/viz/explorer/emissions/latest/co2.{py,dims.yaml}:

  from pathlib import Path
  import yaml
  COLUMN_DIMENSIONS = yaml.safe_load((Path(__file__).parent / "co2.dims.yaml").read_text())
  

Step 7 — DAG entry

In dag/<ns>.yml:

viz://explorer/<ns>/latest/<short>:
  - data://grapher/<ns1>/<v1>/<dataset1>
  - data://grapher/<ns2>/<v2>/<dataset2>
  # ... one line per unique upstream grapher dataset

Place near related explorer entries (or alongside the upstream grapher steps) for discoverability.

Step 8 — Verify

Hand off to the user:

1. .venv/bin/etlr viz://explorer/<ns>/latest/<short> --grapher — runs the step and upserts the explorer to the staging DB.
2. Open http://staging-site-<branch>/admin/explorers/preview/<slug> and spot-check:
   • default view (no dimensions toggled)
   • every dimension switch
   • map tab (if applicable)
   • country/entity picker
   • default selection
3. For migrations: diff the resulting TSV against the legacy owid-grapher/explorers/<slug>.explorer.tsv. Cosmetic differences (column ordering, whitespace) are acceptable; structural differences (missing views, swapped dimension orderings) are not. The Wizard's apps/wizard/app_pages/explorer_diff/ page does this comparison interactively for staging vs production.
4. make check.

Hand the user the exact etlr command — don't run it yourself.

Common pitfalls