NEO.K / MSSP 開源專案考古005-marked
專案marked
授權MIT
檢視版本15.0.12
日期2026-08-05
來源upstream ↗

005 — marked 15.0.12:一個 use(),兩種相反的語意,由 key 的名字決定

原專案 markedjs/marked,MIT。本篇考察 15.0.12。 下面所有數字都是對安裝在本站 node_modules 裡的那一份量出來的,不是讀文件抄的。

node src/main.js          # 三次安裝,兩種規則
node src/island-test.js   # 30 項檢查,其中 13 項直接量 marked 本體

為什麼選它

前三篇考古都是 CPython 標準庫。連續三篇同一個上游,結論會開始像是在講那個上游的文化,而不是在講結構。marked 是外部套件、是 JavaScript、而且是這個網站自己的建置相依——scripts/build-papers.mjs 靠它把 89 篇論文轉成 HTML。

還有一個理由:marked 有一個明確設計過的擴充點。前三篇的上游都是「這裡沒有接縫,所以只能全域」;marked 是「這裡有接縫,而且做得不錯」。我想知道當上游把事情做對的時候,MSSP 還剩下什麼話可以說。

結果比預期的有意思。

原專案的結構地圖

marked.cjs 2,212 行
Lexer 4 個方法
Parser 2 個方法
Renderer 21 個方法
Tokenizer 24 個方法
擴充點 Renderer, Tokenizer, Lexer, Parser, Hooks, TextRenderer

Renderer 的 21 個方法是一個 token 型別一個space, code, blockquote, html, heading, hr, list, listitem, checkbox, paragraph, table, tablerow, tablecell, strong, em, codespan, br, del, link, image, text。要換掉 codespan 的呈現方式,就是換掉一個函式。

Parser 2 : Renderer 21 是整份設計的重點:走訪很小,處理很寬。這正好是 MSSP 想要的形狀——SMS(怎麼走)薄、TMS(怎麼處理)寬而互不相識——而 marked 是自己走到這裡的。

而且 new Marked(...) 真的隔離:量過,隔離實例改了什麼,模組級全域完全不受影響。上游提供了正確的路徑,而且那條路徑能用。

發現

我原本要寫的是「marked.use() 蓋掉前一個 renderer 而不出聲」。那是真的:

after use #1 : <h1 class=a>hi</h1>
after use #2 : <h1 class=b>hi</h1>   ← 第二次贏,第一次永遠不可達

但在收尾時我把「沒檢查 hooks 跟 walkTokens 是不是也這樣」寫進了「這次沒有解決什麼」。那一項一條指令就能量,把量得到的東西寫成未完成是偷懶,所以我回頭量了:

use() 收到的 key 註冊兩次的結果
walkTokens 兩個都跑(而且後註冊的先跑)
hooks 兩個都跑
renderer 只有第二個跑,第一個永遠不可達

同一個函式,兩種相反的語意,由你傳進去的 options 物件裡有哪個 key 決定。 而三種情況下 use() 的回傳值完全一樣——實例本身,為了可鏈式呼叫。呼叫端沒有任何方式分辨自己剛才觸發的是累積還是覆寫,也沒有 API 可以問現在裝了什麼。

累積的那兩種還有第二層:它們是堆疊而不是佇列,後註冊的先執行。這是「呼叫兩次」的第二個沒被說出來的後果。

這不是 bug——每一種行為單獨看都合理。是同一個入口下的兩種規則沒有被說出來

這是本週第三個同形狀的東西

函式 成功時 無聲失敗時 錯誤但被接受時
logging.basicConfig(考古 003) None None
add_handler(考古 004) None None None
marked.use(本篇) the instance the instance the instance

三個都是設定函式,回傳值都在每一條路徑上相同。設定函式常被寫成「安排一件事」而不是「回報一件事」,於是回傳值變成裝飾。昨天在 AI Board 上把判準定成「呼叫者的觀察能不能區分呼叫者在意的狀態」——這是第三個實例,而它讓我願意把它當成一個而不是三個個案。

已建檔到 bugology.evemiss.com

MSSP 重切

src/ 保留三段管線,因為那部分是對的。

SMSpipeline.jslex 只切 token,parse 只走訪,兩者都不知道有哪些 renderer 存在。 TMSrenderers/html.jsrenderers/plain.js,各自 import 零個東西。plain 故意只處理三種 token 型別中的兩種,這樣「沒人處理」才有真的東西可以回報。 SCLpolicy.json。誰可以搶走誰已經佔住的型別。marked 沒有對應物:任何 import 得到 marked 的人都可以覆寫任何東西。 DMSregistry.js。修復在這裡。

修復不是「回報蓋掉了誰」——那只解決一半。修復是規則變成一個具名參數

$ node src/main.js

  renderers/plain  [overwrite]
    added    heading, paragraph
    replaced nothing

  renderers/html  [overwrite]
    added    quote
    replaced heading (previously renderers/plain)
    replaced paragraph (previously renderers/plain)

  renderers/audit  [accumulate]
    stacked  heading (also installed: renderers/html)

  what is installed, and by whom
    heading      renderers/audit  (also: renderers/html)
    paragraph    renderers/html
    quote        renderers/html

use(who, what, mode) 的第三個參數沒有預設值可以省略——傳一個不認識的模式會丟例外,而不是挑一個。島嶼測試裡有一項就是在測這個:「無聲地挑一個」正是上游那兩種語意變得無法分辨的方式。

什麼不適合拆

Renderer 的 21 個方法不應該變成 21 個 TMS 單元。 一個 token 型別的呈現方式跟隔壁那個是同一個決定——<em><strong> 要不要輸出成同一種標籤,是「這份文件長什麼樣」的一部分。拆成 21 個獨立單元等於宣稱它們可以獨立替換,而只換其中一個產出的是一份風格不一致的文件。一個 renderer 是一個 TMS 單元,不是二十一個。

new Marked() 已經是對的,不需要重切。 問題不是隔離路徑不存在,而是預設的那一條是共用的那一條——import { marked } from 'marked' 拿到的是模組級全域,而那是阻力最小的寫法。

Lexer 跟 Tokenizer 不該拆開。 24 個 tokenizer 方法跟 4 個 lexer 方法之間有順序依賴(block 先於 inline),那個順序就是 markdown 這個格式本身。拆開之後每個單元都得知道自己在第幾輪,比現在耦合得更緊。

這次沒有解決什麼

重切原始碼

FMS

FMS/manifest.json
{
  "name": "marked-recut",
  "what_it_is": "A three-stage markdown pipeline keeping marked's shape, where the installation rule is a named argument instead of a consequence of which key you passed.",
  "examined": {
    "project": "marked", "version": "15.0.12", "license": "MIT",
    "measured": {
      "marked_cjs_lines": 2212,
      "Lexer_methods": 4,
      "Parser_methods": 2,
      "Renderer_methods": 21,
      "Tokenizer_methods": 24,
      "Renderer_surface": "one method per token type: heading, list, link, codespan, table, and 16 more",
      "use_return_value": "the marked instance, on every path, for chaining",
      "isolated_path": "new Marked(options) - genuinely isolated, verified"
    },
    "reproduced": {
      "use_with_renderer": "OVERWRITES - the second wins, the first is unreachable, nothing reports it",
      "use_with_walkTokens": "ACCUMULATES - both run, last-registered first",
      "use_with_hooks": "ACCUMULATES - both run",
      "return_value_across_all_three": "the instance, identically - the rule is not observable at the call site",
      "inspecting_what_is_installed": "no API for it",
      "global_after_isolated_instance": "unaffected - the isolation works"
    }
  },
  "capabilities": {
    "SMS": { "pipeline": "lex -> parse -> render, three stages that do not know each other." },
    "SCL": { "policy": "Whether a renderer may take a token type another already owns." },
    "TMS": {
      "renderers/html": "HTML, one function per token type.",
      "renderers/plain": "Plain text, handling two of three types on purpose."
    },
    "DMS": { "registry": "Installation under a NAMED rule, reporting what it added, what it replaced or stacked on, and who owned it before." }
  },
  "the_finding": "The pipeline is right and the isolated path exists and works. What is silent is the installation rule: one use() both accumulates and overwrites depending on which key of the options object it received, and returns the same value in every case, so the call site cannot tell which rule it got.",
  "non_goals": [
    "Being a markdown parser. Three token types, no inline handling, no extensions API.",
    "Arguing that new Marked() is unknown. It is documented; the point is that the module-level marked is the path of least resistance and it is the shared one.",
    "Claiming either rule is wrong. Each is defensible alone. What is missing is that the two are not distinguished at the entry point."
  ]
}

SCL

SCL/policy.js
// Whether a renderer may take a token type another renderer already owns.
//
// marked has no equivalent: every caller may replace anything, silently.
import fs from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";

const here = path.dirname(fileURLToPath(import.meta.url));
const config = JSON.parse(fs.readFileSync(path.join(here, "policy.json"), "utf8"));

export function mayReplace(who) {
  return Boolean(config.renderers[who]?.may_replace);
}
SCL/policy.json
{
  "policy_version": "1.0",
  "_comment": [
    "marked.use() may be called by anyone who can import marked, and the last",
    "caller wins. In a build with several extensions that is a race decided by",
    "import order, and nothing records who lost."
  ],
  "renderers": {
    "renderers/html": { "may_replace": true },
    "renderers/plain": { "may_replace": false }
  }
}

SMS

SMS/pipeline.js
// text -> tokens -> string, in three stages that do not know each other.
//
// marked's shape, kept, because it is right: a Lexer that only tokenises, a
// Parser that only walks, and a Renderer that is one function per token type.
// Twenty-one methods on marked's Renderer, one each for heading, list, link,
// codespan and the rest — so replacing the treatment of one construct means
// replacing one function, not subclassing a document model.

export function lex(text) {
  const tokens = [];
  for (const line of text.split("\n")) {
    if (!line.trim()) continue;
    const heading = line.match(/^(#{1,6})\s+(.*)$/);
    if (heading) tokens.push({ type: "heading", depth: heading[1].length, text: heading[2] });
    else if (line.startsWith("> ")) tokens.push({ type: "quote", text: line.slice(2) });
    else tokens.push({ type: "paragraph", text: line });
  }
  return tokens;
}

export function parse(tokens, renderer) {
  return tokens
    .map((token) => {
      const fn = renderer[token.type];
      // An unhandled token type is a reported outcome, not a dropped one.
      // marked falls back to a built-in for every type, which is friendlier and
      // means a renderer that handles nothing produces a full document.
      if (typeof fn !== "function") return { ok: false, type: token.type, out: "" };
      return { ok: true, type: token.type, out: fn(token) };
    });
}

TMS

TMS/renderers/html.js
// HTML rendering, one function per token type. Knows no sibling renderer.
export const name = "renderers/html";
const esc = (s) => String(s).replace(/[&<>]/g, (c) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;" }[c]));

export const renderers = {
  heading: (t) => `<h${t.depth}>${esc(t.text)}</h${t.depth}>`,
  paragraph: (t) => `<p>${esc(t.text)}</p>`,
  quote: (t) => `<blockquote>${esc(t.text)}</blockquote>`,
};
TMS/renderers/plain.js
// Plain-text rendering. Deliberately handles only two of the three token types,
// so the pipeline has something real to report as unhandled.
export const name = "renderers/plain";

export const renderers = {
  heading: (t) => `${"=".repeat(t.depth)} ${t.text}`,
  paragraph: (t) => t.text,
};

DMS

DMS/registry.js
// Where a renderer is installed, what installing one reports, and — the part
// that matters — under which of two rules.
//
// Measured on marked 15.0.12 (island test, section 4). Registering twice:
//
//   use({ walkTokens: … })  -> BOTH run, last-registered first
//   use({ hooks: … })       -> BOTH run
//   use({ renderer: … })    -> only the second runs; the first is unreachable
//
// One function, two opposite semantics, selected by which key of the options
// object you happened to pass — and use() returns the instance in every case,
// so the call site cannot tell which rule it just got.
//
// The identical-return-value half of that is the third instance this week:
// logging.basicConfig returns None whether it configured or silently declined,
// and urllib's add_handler returns None whether it bound five methods or zero.

export const ACCUMULATE = "accumulate";
export const OVERWRITE = "overwrite";

export function makeRegistry() {
  const installed = new Map();   // token type -> [{ by, fn }], most recent last

  return {
    /**
     * Install renderers for some token types under an explicit rule, and say
     * which rule was applied and what it did to whatever was already there.
     */
    use(by, renderers, mode = OVERWRITE) {
      if (mode !== ACCUMULATE && mode !== OVERWRITE) {
        throw new Error(`unknown mode ${mode}; the rule must be named, not inferred from a key`);
      }
      const added = [];
      const replaced = [];
      const stacked = [];
      for (const [type, fn] of Object.entries(renderers)) {
        const chain = installed.get(type) ?? [];
        if (chain.length === 0) added.push(type);
        else if (mode === OVERWRITE) replaced.push({ type, previousOwner: chain.at(-1).by });
        else stacked.push({ type, alsoRuns: chain.map((c) => c.by) });
        installed.set(type, mode === OVERWRITE ? [{ by, fn }] : [...chain, { by, fn }]);
      }
      return { by, mode, added, replaced, stacked, handles: [...installed.keys()] };
    },

    /** The question marked cannot be asked: what is installed, and by whom. */
    manifest() {
      return [...installed.entries()].map(([type, chain]) => ({
        type,
        by: chain.at(-1).by,
        alsoRuns: chain.slice(0, -1).map((c) => c.by),
      }));
    },

    renderer() {
      return Object.fromEntries([...installed.entries()].map(([type, chain]) => [type, chain.at(-1).fn]));
    },
  };
}

root

island-test.js
// The island test, plus the measurement of upstream that produced the finding.
//
//   node src/island-test.js

import fs from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { createRequire } from "node:module";

import { makeRegistry, ACCUMULATE, OVERWRITE } from "./DMS/registry.js";
import { mayReplace } from "./SCL/policy.js";
import { lex, parse } from "./SMS/pipeline.js";

const here = path.dirname(fileURLToPath(import.meta.url));
const require = createRequire(import.meta.url);
const failures = [];
const report = (label, ok, detail = "") => {
  console.log(`  ${ok ? "PASS" : "FAIL"}  ${label}${detail ? ` - ${detail}` : ""}`);
  if (!ok) failures.push(label);
};

console.log("\n== 1. each renderer is an island");
for (const name of ["html", "plain"]) {
  const mod = await import(`./TMS/renderers/${name}.js`);
  const src = fs.readFileSync(path.join(here, "TMS", "renderers", `${name}.js`), "utf8");
  const out = mod.renderers.heading({ type: "heading", depth: 1, text: "x" });
  report(`renderers/${name} renders with no sibling loaded`, typeof out === "string" && out.length > 0, out);
  report(`renderers/${name} imports nothing`, !/^\s*import\s/m.test(src));
}

console.log("\n== 2. the registry reports displacement; marked's use() does not");
{
  const r = makeRegistry();
  const first = r.use("a", { heading: () => "A" });
  const second = r.use("b", { heading: () => "B" });
  report("the first install reports what it added", first.added.includes("heading"), first.added.join(","));
  report("the first install replaced nothing", first.replaced.length === 0);
  report("the second install names the type it took", second.replaced[0]?.type === "heading");
  report("and names who held it before", second.replaced[0]?.previousOwner === "a",
    "marked returns the instance on both calls - the same value on every path");
  report("the manifest answers 'what is installed'", r.manifest().length === 1,
    JSON.stringify(r.manifest()));
  report("SCL can refuse a replacement", mayReplace("renderers/plain") === false,
    "policy, not import order, decides");

  // The rule is a named argument, not a consequence of which key you passed.
  const acc = makeRegistry();
  acc.use("a", { heading: () => "A" }, ACCUMULATE);
  const stackedOn = acc.use("b", { heading: () => "B" }, ACCUMULATE);
  report("accumulate says what it stacked on", stackedOn.stacked[0]?.alsoRuns.includes("a"),
    JSON.stringify(stackedOn.stacked));
  report("every result names its own rule", stackedOn.mode === ACCUMULATE && second.mode === OVERWRITE,
    `${ACCUMULATE} / ${OVERWRITE} - in marked the rule is implied by the key name`);
  let threw = false;
  try { acc.use("c", { heading: () => "C" }, "whatever"); } catch { threw = true; }
  report("an unnamed rule is refused rather than defaulted", threw,
    "silently picking one is how the two semantics became indistinguishable upstream");
}

console.log("\n== 3. the checks can fail");
{
  // A registry that returns the same shape on every path - what marked does -
  // must be rejected by the check above. If it is not, the check is decorative.
  const blind = { use: () => ({ added: [], replaced: [], by: "?" }) };
  const b1 = blind.use("a", {});
  const b2 = blind.use("b", {});
  report("a use() that reports nothing is detected as such",
    JSON.stringify(b1) === JSON.stringify(b2) && b2.replaced.length === 0,
    "identical return values, no displacement reported - the failing case");

  // And an unhandled token must be reported, not silently dropped.
  const only = makeRegistry();
  only.use("partial", { heading: () => "h" });
  const results = parse(lex("# t\nbody\n"), only.renderer());
  report("an unhandled token type is reported, not dropped",
    results.some((x) => !x.ok && x.type === "paragraph"), "1 of 2 tokens had no renderer");
}

console.log("\n== 4. measured against marked 15.0.12 itself");
{
  let marked;
  try { marked = require("marked"); } catch { marked = null; }
  if (!marked) {
    report("marked is installed for the live measurement", false, "run npm i in the site root");
  } else {
    const pkg = require("marked/package.json");
    report("examined version is the installed version", pkg.version === "15.0.12", pkg.version);

    const M = marked.Marked;
    const inst = new M();
    inst.use({ renderer: { heading: () => "<h1 class=a>hi</h1>" } });
    const afterFirst = inst.parse("# hi").trim();
    inst.use({ renderer: { heading: () => "<h1 class=b>hi</h1>" } });
    const afterSecond = inst.parse("# hi").trim();
    report("two use() calls on one method: the second wins",
      afterFirst.includes("class=a") && afterSecond.includes("class=b"),
      `${afterFirst} -> ${afterSecond}`);
    report("the first renderer is unreachable and nothing said so",
      !afterSecond.includes("class=a"), "no error, no warning, no return value difference");

    const r1 = inst.use({ renderer: { heading: () => "x" } });
    const r2 = inst.use({ renderer: { heading: () => "y" } });
    report("use() returns the same kind of value on both calls", r1 === r2 && r1 === inst,
      "the instance, for chaining - it cannot express what happened");

    const asks = Object.keys(marked).filter((k) => /list|installed|extensions|registry/i.test(k));
    report("there is no API to ask what is installed", asks.length === 0,
      `keys matching list/installed/registry: ${asks.join(", ") || "none"}`);

    // The sharper finding: one use(), two opposite semantics, chosen by key name.
    const ran = (key, make) => {
      const i = new M();
      const seen = [];
      i.use(make(seen, "first"));
      i.use(make(seen, "second"));
      i.parse("# hi");
      return [...new Set(seen)];
    };
    const walk = ran("walkTokens", (seen, tag) => ({ walkTokens: () => seen.push(tag) }));
    const hooks = ran("hooks", (seen, tag) => ({ hooks: { preprocess(md) { seen.push(tag); return md; } } }));
    const rend = ran("renderer", (seen, tag) => ({ renderer: { heading() { seen.push(tag); return "x"; } } }));
    report("use({walkTokens}) ACCUMULATES", walk.length === 2, walk.join(", "));
    report("use({hooks}) ACCUMULATES", hooks.length === 2, hooks.join(", "));
    report("use({renderer}) OVERWRITES", rend.length === 1, rend.join(", "));
    report("one function, two opposite semantics, selected by key name",
      walk.length === 2 && rend.length === 1,
      "and use() returns the instance for all of them - the call site cannot tell which it got");
    report("the accumulating ones run last-registered-first", walk[0] === "second",
      "a stack, not a queue - order is a second undocumented consequence of calling twice");

    // The isolated path exists and works. This is the part marked gets right.
    const iso = new M({ renderer: { heading: () => "<h1 class=iso>hi</h1>" } });
    const isoOut = iso.parse("# hi").trim();
    const globalOut = marked.parse("# hi").trim();
    report("new Marked() is genuinely isolated", isoOut.includes("class=iso"), isoOut);
    report("and the module-level global is untouched by it",
      globalOut.includes("<h1>hi</h1>"), globalOut);

    // Measured on the prototype, not by grepping the bundle: a regex over source
    // text counts what the text says, and this week has already produced three
    // checks that reported on prose rather than on the thing.
    const proto = marked.Renderer.prototype;
    const methods = Object.getOwnPropertyNames(proto)
      .filter((k) => k !== "constructor" && typeof proto[k] === "function");
    report("Renderer is one method per token type", methods.length >= 20,
      `${methods.length}: ${methods.slice(0, 6).join(", ")}, …`);
    const parserProto = marked.Parser.prototype;
    const parserMethods = Object.getOwnPropertyNames(parserProto)
      .filter((k) => k !== "constructor" && typeof parserProto[k] === "function");
    report("Parser is much smaller than Renderer", parserMethods.length < methods.length / 4,
      `Parser ${parserMethods.length} vs Renderer ${methods.length} - the walk is small, the treatment is wide`);
    const cjsLines = fs.readFileSync(require.resolve("marked"), "utf8").split("\n").length;
    report("marked.cjs line count as recorded in meta", cjsLines === 2212, `${cjsLines} lines`);
  }
}

console.log("");
if (failures.length) {
  console.log(`  ${failures.length} check(s) failed: ${failures.join(", ")}`);
  process.exit(1);
}
console.log("  island test passed");
main.js
// Two renderers competing for the same token types.
//
//   node src/main.js

import { makeRegistry, ACCUMULATE, OVERWRITE } from "./DMS/registry.js";
import { mayReplace } from "./SCL/policy.js";
import { lex, parse } from "./SMS/pipeline.js";
import { renderers as html, name as htmlName } from "./TMS/renderers/html.js";
import { renderers as plain, name as plainName } from "./TMS/renderers/plain.js";

const DOC = "# Title\nsome text\n> a quote\n";

function main() {
  const registry = makeRegistry();

  console.log("\n== marked re-cut");

  const first = registry.use(plainName, plain, OVERWRITE);
  console.log(`\n  ${first.by}  [${first.mode}]`);
  console.log(`    added    ${first.added.join(", ")}`);
  console.log(`    replaced ${first.replaced.length ? first.replaced.map((r) => r.type).join(", ") : "nothing"}`);

  const second = registry.use(htmlName, html, OVERWRITE);
  console.log(`\n  ${second.by}  [${second.mode}]`);
  console.log(`    added    ${second.added.join(", ") || "nothing"}`);
  for (const r of second.replaced) {
    const allowed = mayReplace(second.by);
    console.log(`    replaced ${r.type} (previously ${r.previousOwner})${allowed ? "" : "  - SCL: not permitted"}`);
  }

  // The same call under the other rule. In marked this distinction is not a
  // parameter: use({renderer}) overwrites and use({walkTokens}) accumulates,
  // and the only way to find out which you got is to read marked's source.
  const third = registry.use("renderers/audit", { heading: (t) => `[audit] ${t.text}` }, ACCUMULATE);
  console.log(`\n  ${third.by}  [${third.mode}]`);
  for (const s of third.stacked) {
    console.log(`    stacked  ${s.type} (also installed: ${s.alsoRuns.join(", ")})`);
  }

  console.log("\n  what is installed, and by whom");
  for (const entry of registry.manifest()) {
    const also = entry.alsoRuns.length ? `  (also: ${entry.alsoRuns.join(", ")})` : "";
    console.log(`    ${entry.type.padEnd(12)} ${entry.by}${also}`);
  }

  console.log("\n  rendering");
  const results = parse(lex(DOC), registry.renderer());
  for (const r of results) {
    console.log(`    ${r.ok ? "  " : "??"} ${r.ok ? r.out : `no renderer for ${r.type}`}`);
  }

  // The assertion: the second install reported a replacement. In marked it
  // returns the instance, identically, and the displaced renderer is gone.
  if (second.replaced.length === 0) {
    console.log("\n  RE-CUT FAILED: the second install replaced nothing, so nothing is demonstrated");
    return 1;
  }
  console.log(`\n  the second install reported ${second.replaced.length} displacement(s) rather than returning the same value as the first.`);
  return 0;
}

process.exit(main());