UK Data Centre Energy Consumption Profiles

# UK Data Centre Energy, Load & Spare Capacity — Dataset Pull + EDA Pulls every **keyless, open** UK source identified in the research report, profiles it, and derives the things we actually need for a flexibility model: **half-hourly load shape rescaled to 10 / 20 / 50 MW**, **available network headroom at those same thresholds**, and the **carbon / price signals** a flexible data centre would be dispatched against. | # | Source | What it gives | Access | |---|---|---|---| | 1 | UK Power Networks Open Data (`ukpn-data-centre-demand-profiles`, `ukpn-large-demand-list`, `ukpn-flexibility-dispatches`) | ~100 real UK data centre half-hourly profiles; ≥5 MVA demand pipeline; paid flexibility dispatches | metadata open, **records require free login** | | 2 | NESO Transmission Entry Capacity (TEC) register | transmission-connected project capacity in MW, status, Gate 1/2 | open CKAN | | 3 | NGED Network Opportunity Map Headroom + GSP Technical Limits | **spare demand headroom (MW) per substation** | open CKAN | | 4 | NGED Substation Demand (Substation Loading Details) | real half-hourly GB substation load → load shape, load factor, duration curve | open CKAN | | 5 | NESO Carbon Intensity API + Elexon BMRS Insights | half-hourly carbon intensity and system price — the flexibility value signal | open, keyless | **Key idea:** no open source publishes absolute MW for a *named* UK data centre. So we work in **normalised shape × capacity**: take a measured half-hourly profile, scale i…

import io, json, os, time, zipfile, urllib.request, urllib.parse
from pathlib import Path

import numpy as np
import pandas as pd
import plotly.express as px
import plotly.graph_objects as go
from plotly.subplots import make_subplots

pd.set_option("display.width", 160)
pd.set_option("display.max_columns", 60)

DATA = Path("data"); DATA.mkdir(exist_ok=True)
FIGS = Path("figures"); FIGS.mkdir(exist_ok=True)
UA = {"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) clusy-notebook/1.0"}

# Site sizes we are profiling throughout the notebook
SITE_MW = [10, 20, 50]
…

ready — 2026-10-02 14:40 BST

## 1 · UK Power Networks — the DC-specific datasets (metadata open, records login-gated) UKPN is the only GB network operator publishing **data-centre-specific** half-hourly demand. The catalogue metadata is fully open, so we pull the schema, record count, licence and refresh date programmatically. The `/records` and `/exports` endpoints return **HTTP 403 ForbiddenAccess** without a portal login, so the notebook documents that gate rather than pretending to have the rows.

UKPN_DATASETS = [
    "ukpn-data-centre-demand-profiles",
    "ukpn-data-centre-utilisation",
    "ukpn-large-demand-list",
    "ukpn-flexibility-dispatches",
]

meta_rows, schema_rows, access = [], [], {}
for ds in UKPN_DATASETS:
    m = get_json(f"{UKPN}/{ds}")
    dmeta = m["metas"]["default"]
    meta_rows.append({
        "dataset_id": ds,
        "title": dmeta.get("title"),
        "records": dmeta.get("records_count"),
        "licence": dmeta.get("license"),
        "access": ", ".join(m["metas"]["dcat"].get("accessRights") or []),
        "modified": (dmeta.get("modified") or "")[:10],
…

UKPN /records probe:
  ukpn-data-centre-demand-profiles       403 Forbidden — portal login required
  ukpn-data-centre-utilisation           403 Forbidden — portal login required
  ukpn-large-demand-list                 403 Forbidden — portal login required
  ukpn-flexibility-dispatches            403 Forbidden — portal login required

## 2 · NESO Transmission Entry Capacity (TEC) register — who holds grid capacity, and at what MW Open CKAN, NESO Open Data Licence, republished twice weekly. Every project holding a transmission connection contract, with `Cumulative Total Capacity (MW)`, `Project Status`, `Connection Site` and the `Gate` flag (Gate 1 / Gate 2, added Nov 2025). We use it to see how many contracted projects sit in the **10 / 20 / 50 MW bands** that match our target site sizes.

TEC_RESOURCE = "17becbab-e3e8-473f-b303-3806f43a6a10"

def ckan_table(portal: str, resource_id: str, page: int = 10_000) -> pd.DataFrame:
    """Page a CKAN datastore resource into a DataFrame."""
    out, offset = [], 0
    while True:
        url = (f"{portal}/api/3/action/datastore_search"
               f"?resource_id={resource_id}&limit={page}&offset={offset}")
        res = get_json(url)["result"]
        recs = res["records"]
        out.extend(recs)
        total = res.get("total", len(out))
        offset += len(recs)
        print(f"  fetched {offset}/{total}", end="\r", flush=True)
        if not recs or offset >= total:
            break
    print()
    return pd.DataFrame(out).drop(columns=["_id"], errors="ignore")
…

fetched 2197/2197

plot_df = tec.dropna(subset=[mw_col]).query(f"`{mw_col}` > 0").copy()

fig = px.histogram(plot_df, x=mw_col, nbins=70, log_x=True,
                   color_discrete_sequence=["#2a6fb0"],
                   title="NESO TEC register — contracted capacity per project<br>"
                         "<sup>2,197 projects; log scale. Dashed lines mark our 10 / 20 / 50 MW site sizes.</sup>")
for mw, col in zip(SITE_MW, PAL.values()):
    fig.add_vline(x=mw, line_dash="dash", line_color=col, line_width=2,
                  annotation_text=f"{mw} MW", annotation_position="top")
fig.update_layout(template=TEMPLATE, height=430, bargap=0.05,
                  xaxis_title="Cumulative total capacity (MW, log scale)",
                  yaxis_title="Projects", margin=dict(t=90, r=30))
fig.show()

top_sites = (plot_df.groupby("Connection Site")[mw_col].agg(["sum", "count"])
             .sort_values("sum", ascending=False).head(12).reset_index())
fig2 = px.bar(top_sites.sort_values("sum"), x="sum", y="Connection Site", orientation="h",
              text=top_sites.sort_values("sum")["count"].map(lambda n: f"{n} projects"),
…

[Output omitted — too large to include in the shared snapshot]

## 3 · NGED — spare network capacity: how much headroom exists for a 10 / 20 / 50 MW demand connection? National Grid Electricity Distribution's open portal (WPD Open Data Licence, no login) publishes the **Network Opportunity Map Headroom** file — demand and generation headroom in MW per substation — plus **GSP Technical Limits**. This is the direct answer to *"where would a 10/20/50 MW site physically fit?"* for NGED's four licence areas (East & West Midlands, South Wales, South West).