← all methods · TT-008 instrument page · raw markdown: ?raw=1
Grade: REPORTED. Counted by US customs from declarations filed under penalty. Not observed
by us, not modelled.
Two reasons. The second was added on 2026-07-30.
To check TT-001. AIS infers that cargo moved because a ship's reported draught fell. Nothing
had ever tested that inference. This counts declared mass at the same gateways, so the proxy can
be confirmed or killed. It is also the only free source that says where the cargo came from.
To be a mirror, not a query. The observatory must be able to defend a published figure years
later. Reading a live API cannot do that, for four reasons in descending order of importance:
1. Census revises prior months. Read live and a restatement is invisible. Mirrored, with the
prior value preserved, a revision becomes an observation about the source itself.
2. Reproducibility. If the API answers differently in 2028 we cannot re-argue a claim.
3. Availability. The API goes dark, and the all-ports HS2 request does not complete at all.
4. Speed. Interactive lab work against those latencies is unusable.
One row = direction × month × US port × partner country × commodity code.
There is no vessel dimension — customs records a shipment, not a ship. That is precisely why
it complements AIS instead of duplicating it, and why the two can only ever be joined on **port
and time**, never on vessel.
comm_level='TOTAL' rows are the port × country total; HS2 rows are the commodity breakdown
beneath them. Do not sum across levels or you will double count.
Every measure Census publishes at this grain is collected, for both directions.
| field | what it is |
container_weight_kg | The comparison field. Declared mass of containerised cargo — what draught change is a proxy for. |
container_value_usd | Customs value of the same. |
vessel_weight_kg / vessel_value_usd | All vessel traffic including bulk and breakbulk. The gap against container_* is a containerisation rate nobody publishes — and it is how bulk contamination in the AIS fleet gets bounded. |
air_weight_kg / air_value_usd | Air freight, weight and value. Air costs roughly ten times ocean, so air share is the market pricing ocean reliability. |
all_modes_value_usd | GEN_VAL_MO on imports, ALL_VAL_MO on exports — the two datasets differ here. |
census_last_update | Census's own revision stamp. Returns 0 on every one of 126,313 rows. See below. |
retrieved_at | When we fetched this exact figure. The only usable revision clock. |
Census revises prior months, sometimes materially. Its LAST_UPDATE field is supposed to say
when, and it does not work: verified 2026-07-30, all 126,313 rows return 0.
So the source cannot tell us when it changed its mind. We detect it by comparison instead.
Every incoming figure is checked against what we hold; when a measure moves, the prior value is
written to port_trade_revisions before the update, so the original is recoverable rather than
merely flagged.
Each archived change is classified, and the distinction is not cosmetic:
| kind | meaning |
revision | value → different value. Census genuinely restated a figure. |
withdrawn | value → NULL. Census removed something it had published. |
backfill | NULL → value. We did not hold the figure before. Not a Census change. |
Why backfill exists. Migration 001 added air_weight_kg, a measure never previously
collected, so every pre-existing row held NULL for it. The first sync saw NULL → value on 4,401
rows and recorded them all as revisions. None were. Left uncorrected, the one table whose
purpose is to prove when a source changed its mind would have begun life with 4,401 fabricated
entries. Migration 002 reclassified all 4,401 as backfill from their own recorded evidence.
Only revision and withdrawn may ever be cited as the source having changed.
PORT is the port of unlading, not the customs district where cargo cleared. Tested two ways
on 2026-07-30:
all 0. If in-bond cargo were credited where it clears, Dallas would be enormous. Chicago and
Detroit show small amounts and both are genuine Great Lakes vessel ports.
POLA's own published 530,558 loaded-import TEU. That is 6.9 tonnes per TEU, inside the
plausible 6–12 t band.
So the in-bond concern does not apply. What genuinely stays invisible is narrower: **foreign
in-transit cargo** (discharged in the US, destined for Mexico or Canada, never entered) and
empty containers, which carry no customs value and are never declared.
Census reports LA and Long Beach, New York and Newark, and Seattle and Tacoma separately,
while TT-001 watches some pairs as one coverage zone. They must be summed to the AIS zone
before comparison — comparing one Census port against a two-port AIS zone understates by roughly
half.
Note: us_seattle_tacoma is a single AIS zone spanning two ports 35 km apart. That is a
measurement error to fix on the AIS side, not to compensate for here.
India filter on 2026-07-30.*
multiplies row count by roughly six.
per year.
incomplete. sync --months=N deliberately re-fetches recent months rather than assuming a
figure is final — that re-fetch is how revisions get caught.
Census returns three overlapping layers in the same response, distinguished only by
SUMMARY_LVL, a field the mirror did not collect until 2026-07-30:
| layer | what it is |
country = '-' | TOTAL FOR ALL COUNTRIES. Marked DET. The authoritative total. |
SUMMARY_LVL='CGP' | groupings: APEC, ASEAN, OECD, ASIA, PACIFIC RIM, and the continental 1XXX–7XXX codes |
SUMMARY_LVL='DET' | individual countries |
The groupings overlap each other heavily — China sits inside APEC, Pacific Rim, Asia
and ASEAN's neighbours — so summing every row does not double count, it over-counts by a
variable factor. Los Angeles, May 2026 containerised imports:
grand total row 3,666 kt <- the truth continental groups 3,666 kt <- a correct partition of it "individual" codes 11,588 kt <- 3.2x, because APEC/OECD/Pacific Rim overlap
Two rules follow, and the second is the one that catches people:
1. For a port-level figure, read the country = '-' row and nothing else.
2. Filtering on SUMMARY_LVL='DET' is NOT sufficient. Census marks the grand total
DET as well, so a DET-only sum adds the total to its own members.
What this corrupted: per-port containerisation rates were out by up to **7.9 percentage
points* (New York read 77.7% against a true 69.8%). What it did not* corrupt: the
port-of-unlading verification above, which used the - row and stands.
| level | rows per port-month, imports | exports |
| TOTAL | 182 | 197 |
| HS2 | 5,350 | 4,832 |
| HS4 | 26,541 | 21,756 |
| HS6 | 67,649 | 45,172 |
| HS10 | request fails | not offered |
Across 11 ports, both directions, 197 months: HS2 ≈ 20M rows (~2.4 GB), HS4 ≈ 105M
(~12.6 GB), **HS6 ≈ 244M (~29 GB) — more than the free disk, and roughly two weeks of
fetching.**
So the standing archive is TOTAL + HS2, and the depth decision is made per analysis:
of physical form and is consistent within a chapter — mineral fuel 1%, meat and
cereals near 100%. Depth would add rows, not information.
too heterogeneous — Chapter 85 spans transformers at a few dollars a kilo and
semiconductors at a few thousand. At HS2 a change in $/kg is overwhelmingly product
mix, not price or quality. Fetch HS6 on demand for the specific cells a claim is
about, never as a standing archive.
to HS6, so any import-versus-export comparison is capped there regardless of budget.
containerisation rate = containerised vessel weight / ALL vessel weight
Both fields come from the same row, so the ratio is internally consistent and no
cross-source join is involved. Computed at port × direction × month, and at HS2
commodity level pooled across the gateways.
Why it is worth publishing. Anyone could compute it from the free API; nobody
publishes it as a series. It answers a question operators actually have — which trades
move in boxes, where, and how fast that is changing. And it is the direct bound on bulk
contamination in TT-009: at a gateway that is 38% containerised, an AIS filter that
cannot separate a container ship from a bulk carrier is measuring a mixed fleet.
Observed May 2026, imports: Oakland 100%, Norfolk 86%, Savannah 85%, Los Angeles 79%,
Newark 57%, Houston 38%. By commodity: meat, cereals and cocoa near 100%; iron and steel
19%; mineral fuel 1%.
Exports are UNDERSTATED and are a lower bound. Census states the containerised flag
is optional for exporters and that "some filers may omit this information", so the
export numerator is incomplete by an unmeasured amount. Never compare an export rate
directly against an import rate without saying so. The apparent import/export scissor —
Norfolk importing 86% in boxes and exporting 24% — is real in direction but its
magnitude is not trustworthy.
A zero denominator yields NULL, never zero. A port-month with no waterborne cargo
has no containerisation rate; printing 0% would assert that nothing containerised.
Never mix commodity levels. TOTAL rows already contain the HS2 rows beneath
them. Summing across levels double counts.
Thin cells cannot carry a ratio. The commodity series excludes chapters under
50,000 tonnes in a month.
An apparent rise in import containerisation from 38% (2010) to 63% (2026), against a
fall in exports from 37% to 26%, is visible in the data today. **It is not publishable
and the series are suspended.** The 2010–2023 backfill is still loading: the archive
holds twelve months for 2024 and 2025 but a partial 2010, so any cross-year comparison
weighs one loaded month against twelve. The volumes give it away — 2010 shows a sixth of
2024's tonnage, which is a loading artifact, not trade growth.
Lift the suspension only when every month from 2010-01 to the last complete month is
present, verified by a per-year month count, not by row totals.
value density = containerised customs value / containerised shipping weight
This is the method Census itself abandoned. It discontinued its unit-value trade
indices in 1989 for composition bias and replaced them with survey-based price indexes;
the IMF's manual calls unit-value indices "seriously misleading" as price proxies. So a
naive $/kg series is not a neutral choice — it revives a method the source agency
dropped, and must be built to survive that objection or not built.
Proven in our own data 2026-07-30. Chapter 85 at Los Angeles spans 1.1 $/kg
(filament lamps) to 179.8 $/kg (multilayer ceramic capacitors) — a **161× range
inside one HS2 chapter**. Its chapter figure of 12.9 $/kg is a weighted average of all
of that and moves whenever the mix moves, with no price change at all.
The SCISSOR level is sound. Import value density against export value density, same
port, same month, both computed the same way — measurement bias (shipping-weight
conventions, estimated low-value shipments) is common-mode and largely differences out.
May 2026: every gateway imports goods worth more per kilo than it exports, from about
1.0× at New York to 4.1× at Tacoma. That asymmetry is why a box arriving full often
leaves empty. Textbook qualitatively; not published as a monthly measurement.
The MOVEMENT is not interpretable at this level. Same-month year on year the ratio
runs 2.03× (2024) → 2.02× (2025) → 1.83× (2026), import density falling from 5.17 to
4.68 $/kg. At TOTAL aggregation that is indistinguishable from a mix shift, so all three
series are suspended until an HS6 decomposition can separate the within-line component
from the mix component and publish both side by side.
series is a mode-mix index wearing a price label.
country='-'), so overlapping country groupings cannotinflate either side.
directions — imports publish to HS10, exports only to HS6.
cli.js deep --level=HS6 --from=YYYY-MM [--to=] [--ports=] [--direction=]
Scope is always named explicitly and printed back before the run, so a very large pull
cannot start by accident. Measured 2026-07-30: HS6 returns **~71,000 rows per
port-direction-month** and takes ~30 s per request.
| scope | rows | disk | time |
| 2 ports, 13 months, both directions | ~3.7M | ~0.4 GB | ~30 min |
| 11 ports, 24 months, both directions | ~37M | ~4.5 GB | ~4.5 h |
| 11 ports, full 197-month history | ~244M | ~29 GB | ~2 weeks |
Two years across all ports is affordable and is the right default for anything about
recent trends; the full history at HS6 is not, and is what the on-demand design exists
to avoid.
CHK-003 — tt001.us-gateways.cargo-worked against tt008.us-gateways.import-weight.
It cannot run yet and says so: TT-001's archive begins 26 July 2026 and Census reaches only May.
The first clean shared month is August 2026, arriving in early October. Until then the checker
reports insufficient overlap rather than a number, which is the correct behaviour.
node cli.js sync 2026-05 one month, both directions, TOTAL + HS2 node cli.js sync --months=6 re-fetch the last N complete months (catches revisions) node cli.js backfill --from=2024-01 every month to the last complete one node cli.js revisions Census restatements, separated from our own backfills node cli.js status coverage, and what is missing