Crawler CommonMark refactor: converting distiller + markdown-it-py parse gate
DONE 2026-07-03 per operator. Distiller rewritten (crawlers/claude_docs/md.py): piecewise inline converter — <a>→[text](url) with absolutized+percent-encoded hrefs, strong/em/code→**/*/backticks — and spec backslash-escaping on every plain-text run (adversarially tested: ](\[evil](x) injection neutralized). New cm.py gate: markdown-it-py parses every DISTILLED doc; any token outside CommonMark core (esp. html_block/html_inline) → COMMONMARK-REJECT, write refused. Vendor markdown (source:md) written as-is with beyond-core tokens logged only. Design-system rationale encoded: core-only subset renders 1:1 in Chrome 151 canary AND swift-markdown/swift-markdown-ui (iPhone 16 Pro path). Live-proven: 10 blog pages recrawled → 10 WROTE, 0 rejects; link conversion surfaced previously-invisible nav junk → measurement fix (URLs excluded from length floor) + noise entries; platform vendor-md logged html_block/image, sha-skipped unchanged. markdown-it-py recorded in toolchain (9 entries).
created 2026-07-03 18:08:22 · updated 2026-07-03 18:08:22