Harden RDAP rate-limit handling: one retry on 429 + log a counter #4

New Issue

tonym · 2026-08-17T05:15:39Z

tonym commented

2026-08-17 05:15:39 +00:00

Follow-up from the 2026-08-16/17 session (no evidence we're currently being limited — a live probe of four .com domains came back clean — but we have no way to know historically).

Today: a Verisign/registry 429 maps to status: "unknown" with detail "Registry rate-limited the lookup" (src/lib/rdap.ts), shown as grey unclear. Deliberately not cached, so the next run retries. Nothing is logged, so a past rate-limit is invisible after the fact — and a redeploy wipes the container log anyway.

Two cheap hardenings, in order of value:

One retry after ~1s on 429 in checkViaRdap. Registry limits are bursty and a single backoff usually clears it; costs nothing when there's no 429. Keep it to ONE retry — we're an anonymous client on someone else's public endpoint.
Log a one-line counter (e.g. rdap.429 <tld>) so app_logs can answer "did we get limited?" instead of guessing.

Constraints that must not be relaxed while doing this: 60 domains/request, 8-wide concurrency, per-IP token bucket, 10-minute result cache (see CLAUDE.md "Availability — the trap"). Note paging now appends (v0.6.0), so a long session issues more total lookups than before — this got slightly more relevant.

Follow-up from the 2026-08-16/17 session (no evidence we're currently being limited — a live probe of four `.com` domains came back clean — but we have no way to know historically). **Today:** a Verisign/registry `429` maps to `status: "unknown"` with detail *"Registry rate-limited the lookup"* (`src/lib/rdap.ts`), shown as grey **unclear**. Deliberately not cached, so the next run retries. Nothing is logged, so a past rate-limit is invisible after the fact — and a redeploy wipes the container log anyway. Two cheap hardenings, in order of value: 1. **One retry after ~1s on 429** in `checkViaRdap`. Registry limits are bursty and a single backoff usually clears it; costs nothing when there's no 429. Keep it to ONE retry — we're an anonymous client on someone else's public endpoint. 2. **Log a one-line counter** (e.g. `rdap.429 <tld>`) so `app_logs` can answer "did we get limited?" instead of guessing. Constraints that must not be relaxed while doing this: 60 domains/request, 8-wide concurrency, per-IP token bucket, 10-minute result cache (see CLAUDE.md "Availability — the trap"). Note paging now appends (v0.6.0), so a long session issues more total lookups than before — this got slightly more relevant.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: tonym/names#4