A companion to "How Sky Score works, in plain English" · 2026-09-21
Preflight is a checklist that runs on my computer before any change to the project is saved into its history. One command runs all 57 in sequence and prints one line per check: PASS, or FAIL with the reason.
Names as the report prints them; explanations in plain words. Advisory checks are in italics.
| # | Name in the report | What it asks |
|---|---|---|
| Is the code tidy? (four checks, three languages) | ||
| 1 | ESLint (8 targets) | Reads the JavaScript for mistakes a browser would not report: an unused name, a misspelt variable, a call to something that does not exist. |
| 2 | html-validate (9 pages) | Checks the nine public pages are well-formed HTML: every tag closed, every image labelled. |
| 3 | ruff (backend/lambdas) | The same for the Python that runs on Amazon. |
| 4 | ruff (scripts, tests) | And for the scripts that build the data and the tests themselves. The tests were outside this for a month once; the one folder guarding the score engine was the one nothing checked. |
| Does the arithmetic still work? (unit tests) | ||
| 5 | pytest (backend) | Hundreds of small tests of the scoring engine and the assistant: "given these inputs, the score must be 5.4"; "an invented number must be caught". Includes the ten tests of the forty-line check. |
| 6 | pytest (root) | Tests of the data scripts and the deploy tooling, including one that reads the whole deploy file (the Makefile) to make sure the runner can still understand it. |
| Do the two halves and the documents agree with the engine? (contracts) | ||
| 7 | API base-URL drift (I-N5) | Every page that talks to the API uses the same address, held in one file. A page with its own copy of the address is a page that breaks silently when the address changes. |
| 8 | score sanity (live API) | Asks the live API for 28 postcodes, one per city, and checks each resolves to the right city with a well-formed score. The only check that can catch a data fault in the live system. |
| 9 | demo key reaches only /v1/score | Asks the running API whether the free and demo passes are refused on the paid batch route. The configuration says they should be; only the running system can prove it. |
| 16 | openapi == score engine | The published description of the API (which customers generate code from) must match what the engine actually returns: every field, every city, every buyer profile. |
| 17 | worked example reproduces | The methodology page shows a worked example for SW11 1AA. This recomputes it from the engine, input by input, weight by weight. It once passed for weeks while never comparing affordability at all. |
| Do the published numbers still equal their source? (the unusual ones) | ||
| 10 | prices == HM Land Registry | Every borough's price and growth figure, re-read from the Land Registry's file and compared. Written after a month where prices matched one edition and growth matched none. |
| 11 | aircraft bands == geometry | Every borough's aircraft-noise band, re-derived from the airport positions and the measured noise footprints. |
| 12 | crime == ONS Table C4 | Every borough's crime rate against the statistics office's own table, with a per-city minimum so it cannot pass by comparing nothing. |
| 13 | flood == EA service (georef) | Asks the Environment Agency's own map service what risk it reports at sample points, and compares. Samples the edges of the map first, because a positioning error shows at the edges. The first version passed a known-broken map 9 times out of 9 until it did. |
| 14 | site == Lambda (91 boroughs) | Drives the real website, reads the score it draws for every borough, and compares with the API's answer. The check from the main guide's Part 5. |
| 15 | area names == MSOA names | Every neighbourhood label (281 of them) is checked against the official small-area names for that district. It caught six labels that had been wrong for months. |
| 48 | p8 == DfE KS4 2023/24 | Every borough's school figure against the Department for Education's file. Advisory only because that file cannot be downloaded automatically. |
| 49 | borough bands == sources | Road noise, air quality, flood, transport and healthcare shares, re-derived from the maps and registers. Advisory because the maps are large files a fresh copy would not have. |
| 50 | neighbourhood medians == PPD | The 481 neighbourhood prices, re-derived from the Land Registry's full sales file (155 MB, not kept in the project). |
| 51 | aircraft footprint == DEFRA | The measured size of each airport's noise footprint, re-derived from the noise maps. |
| 52 | quiet estimate == DEFRA | Measures how far the distance-based aircraft estimate is from the government's readings, over 35,352 London postcodes. The accuracy figure the API page publishes comes from here, so it is re-measured rather than quoted. |
| Does the website behave, on every screen? (a robot drives it) | ||
| 18 | no em dashes (all pages) | A house style rule, enforced by a check rather than by memory. |
| 20 | self-hosted fonts (9 pages) | Every page loads its typefaces from our own site, not from Google, so no page depends on a third party or leaks a visit to one. |
| 21 | extension extraction | The browser extension (a demo for property listings) still reads the address, price and coordinates from a real saved listing page. |
| 22 | extension panel per env state | The extension's panel renders correctly for each shape of answer the environment endpoint can give, including "measured as quieter than 40 dB". |
| 23 | Playwright e2e (--workers=2) | A robot opens the live site and walks the main journeys: search, panels, saved places, install prompt. |
| 24 | extension e2e | The same robot loads the extension into a real browser on a real listing and checks what it shows against the live API. |
| 25 | local smoke + registry | Opens the working copy of the site and checks it boots, every city has every registry field, and the map library is fetched exactly once. |
| 26 | degraded + offline fallbacks | Stalls the network, launches offline, and breaks the transport service on purpose, checking the site says the honest thing each time rather than a confident absence. |
| 27 | locator inset | The small UK map in the corner draws the right city markers and none of them is a keyboard trap. |
| 28 | selector tiers do not overlap | The country and city chips never overlap each other or fall off the strip, at every width. |
| 29 | every city switches | Clicks every city chip, on a phone and on a desktop, and counts the borough outlines drawn against that city's own file. Written the day six of nine cities were found dead on the live site. |
| 30 | legal links reachable on a phone | Privacy and terms can be reached from a phone. For ten days once, the mobile home page had exactly one visible link. |
| 31 | PWA installable (manifest + SW) | The site can still be installed as an app: the manifest and the offline worker are valid and every file the worker pre-caches exists. |
| 32 | changes page explains why | The page listing score changes between data editions explains each movement rather than just listing it. |
| 33 | map fits its box | At nine screen sizes including landscape phones, every city's map is drawn inside its own frame. 53 of 90 combinations were failing when this was written, with every other check green. |
| 34 | web/native layout split | The website and the phone-app build render their different layouts as intended; neither has silently become the other. |
| 35 | UK cities get UK panel content | The panel for a UK city shows UK data sources and stations, never the New York wording. |
| 36 | area pages carry real data | The 100 static borough pages each have real figures, unique titles, and no two share a block of text. A hundred near-identical pages would count against the site. |
| 37 | area pages match the live API | Those pages bake their scores in, so this compares all 99 against the live API in one request. |
| 38 | layers paint only real data | The map's colour layers (road noise, flood, air) paint a borough only where there is a reading, and the legend shows only bands that are actually painted. Once, seven cities were painted a confident colour meaning nothing. |
| 39 | panel says what it measured | Renders the borough panel and checks its caveat names the inputs it actually has. Nothing had rendered that panel before; it had shipped the word "undefined" to users. |
| 40 | favourites: keyboard + not nested | Saved places can be added and removed by keyboard, and no button sits inside another button. |
| 41 | ranking claims only real order | When boroughs' scores are too close to rank meaningfully, the ranking table says so instead of implying an order. |
| 42 | panel contrast, borough (AA) | Measures the colour contrast of every visible piece of text in the borough panel against the accessibility standard, itself, rather than trusting a scanner that reports "could not determine" as a pass. Found 46 failures on its first run. |
| 43 | panel contrast, area (AA) | The same for the neighbourhood panel, including the state where the NHS helper says nothing is nearby. |
| 44 | responsive, source | At twelve screen sizes, nothing scrolls sideways, no control is pushed off-screen, covered by something else, or clipped above the top. |
| 45 | WCAG source scan (all pages) | An accessibility scan of 109 pages at up to three sizes, with the legend expanded so hidden text is measured too. Level AA of the current standard is blocking. |
| Do the promises match reality? (the honesty checks) | ||
| 19 | log retention == privacy.html | Reads the retention promise off the privacy page, asks Amazon what it is actually doing, and fails on any difference. |
| 53 | aws perms == iam-policy.json | Exercises 18 safe Amazon actions to see which the deploy pass really has. Found a total outage on its first run: a console edit had replaced the whole permission list. "A permission is a timestamp, not a property." |
| 54 | deployed == source | Downloads 134 files from the live site and compares each with the working copy, so a forgotten upload is named. |
| 55 | site == /v1/score (6 postcodes) | The score the live website draws against the live API, for six postcodes. |
| 56 | responsive, live | Check 44, run against the live site instead of the working copy. |
| 57 | live web == tabbed layout | The live phone layout is the one that was deployed on purpose. |
| Housekeeping (advisory) | ||
| 46 | Prettier (all files deviate) | Code formatting. Deliberately advisory: it would flag every file, and a permanently red check is an ignored check. |
| 47 | npm audit | Known vulnerabilities in the third-party packages the tooling uses. |
The rule exists because twelve checks in this project's history were green for months while checking nothing. Nobody switched them off; each decayed one sensible decision at a time:
The question that finds these is not "what does this check run?" but "what would this check miss?". Names are treated as claims: a check called "all pages" that scans one page is the warning sign.
Most of the code was written with Claude Code, a coding assistant that reads and writes the project's files under direction. It runs on my computer, not on Amazon. When it needs to touch the live system, here is the path, and where it is narrow.
.env in the project folder that the version history is told to ignore. The energy token is handed to the blueprint at deploy time as a hidden value, so it never appears in Amazon's own records of the deploy.The layers are deliberately different in kind: a checklist, a filter, a judgement, a permission, a person. A single guardrail, however good, has one way to fail.
16 September. A data release morning: the Land Registry's July prices, a methodology change, two defects found by the checks. When it came to the back-end deploy, the assistant's safety classifier refused to run it from its own session. I ran the one deploy command myself. That is the arrangement working as designed: the assistant prepared and verified everything up to the line, and the line held.
21 September, the day this was written. After uploading the plain-English write-up to the site, the assistant tried to fetch the live address to compare the file's fingerprint with the original. The classifier refused that too, judging the fetch to be part of the deploy. So the assistant stopped, said what it had tried to do and why, and asked me to run the one-line check. Thirty seconds of my time, and the rule stayed a rule rather than becoming a rule with exceptions.
Sky Score · stage names from the preflight report of 2026-09-21 (45 PASS, 12 advisory, 23 min 34 s); permission facts from the deploy identity's policy file and the CI workflow; the two refusals from the sessions of 16 and 21 September 2026.