mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-13 10:22:11 +03:00
Compare commits
2 Commits
chore/work
...
feat/i18n-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
455aade660 | ||
|
|
e89a4da2b0 |
17
AGENTS.md
17
AGENTS.md
@@ -578,31 +578,14 @@ own dedicated branch, and you MUST confirm the base branch with the operator bef
|
||||
# HARD LINKS (`cp -al`), never a symlink: ~5s for the whole tree and near-zero extra
|
||||
# disk (the inodes are shared), and unlike a symlink it does not break the dev server.
|
||||
cp -al "$(git -C <main_checkout> rev-parse --show-toplevel)/node_modules" node_modules
|
||||
# `.husky/_` is gitignored, so a fresh worktree does NOT have it and
|
||||
# `core.hooksPath=.husky/_` then points at a directory that does not exist —
|
||||
# every pre-commit gate goes silently mute. Copy it too.
|
||||
cp -a "$(git -C <main_checkout> rev-parse --show-toplevel)/.husky/_" .husky/_
|
||||
```
|
||||
|
||||
`scripts/dev/new-worktree.sh <branch> [base]` does all of the above (canonical path,
|
||||
hard-linked `node_modules`, `.husky/_`) and then **verifies** the hook is actually
|
||||
executable, so prefer it over running the steps by hand.
|
||||
|
||||
**Never `ln -s` node_modules.** Turbopack rejects a symlink that resolves outside the
|
||||
project root, so `npm run dev` dies with a FATAL panic (`Symlink [project]/node_modules
|
||||
is invalid, it points out of the filesystem root`) while typecheck, lint and the test
|
||||
runners all keep passing — the error names "filesystem root", not the worktree, so it
|
||||
reads like a Next/build bug and costs real time to trace (incident 2026-07-31, #9043).
|
||||
|
||||
**A worktree without `.husky/_` runs NO pre-commit gate — and says nothing.** `git`
|
||||
resolves `core.hooksPath` relative to the worktree top; when the directory is missing it
|
||||
simply finds no hook and commits. Nothing is printed, the commit succeeds, and the
|
||||
identity/lint/docs gates never ran. This is how 59 commits carrying a stale identity
|
||||
override (name of a contributor + the maintainer's e-mail) got past
|
||||
`scripts/check/check-git-identity.sh` between 2026-08-29 and 09-02 — they were all made in
|
||||
`cp -al` worktrees. Verify with `ls .husky/_/pre-commit` inside a new worktree, or just use
|
||||
`scripts/dev/new-worktree.sh`, which fails loudly when the hook is not executable.
|
||||
|
||||
3. **Work, commit, push, open the PR — all from inside the worktree.** Never `git checkout` a
|
||||
different branch inside a worktree another session might share.
|
||||
4. **Tear down only your own** worktree + branch when done, from the main checkout:
|
||||
|
||||
@@ -1,92 +0,0 @@
|
||||
#!/usr/bin/env sh
|
||||
# Cria uma worktree isolada seguindo o protocolo obrigatório do AGENTS.md
|
||||
# (Git Workflow → "Worktree isolation" / Hard Rule #19), incluindo os dois
|
||||
# passos que são fáceis de esquecer e falham em silêncio:
|
||||
#
|
||||
# 1. node_modules por HARD LINK (`cp -al`), nunca symlink — um symlink que
|
||||
# resolve fora da raiz mata o Turbopack com um FATAL que culpa a
|
||||
# "filesystem root" e não a worktree (incidente 2026-07-31, #9043).
|
||||
# 2. `.husky/_` copiado — é gitignored, então uma worktree nova NÃO o tem, e
|
||||
# `core.hooksPath=.husky/_` aponta para um diretório inexistente: TODOS os
|
||||
# hooks de pre-commit ficam mudos, sem aviso nenhum. Foi assim que 59
|
||||
# commits com identidade trocada passaram pelo gate entre 29/08 e 02/09
|
||||
# (ver .mailmap e scripts/check/check-git-identity.sh).
|
||||
#
|
||||
# Uso: scripts/dev/new-worktree.sh <branch> [base-branch]
|
||||
# Ex.: scripts/dev/new-worktree.sh fix/12345-algo release/v3.8.51
|
||||
|
||||
set -e
|
||||
|
||||
BRANCH="$1"
|
||||
BASE="$2"
|
||||
|
||||
if [ -z "$BRANCH" ]; then
|
||||
echo "uso: scripts/dev/new-worktree.sh <branch> [base-branch]" >&2
|
||||
echo " ex: scripts/dev/new-worktree.sh fix/12345-algo release/v3.8.51" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# O checkout PRINCIPAL, mesmo quando este script roda de dentro de outra worktree:
|
||||
# `--show-toplevel` devolveria a worktree atual, e a nova nasceria aninhada nela.
|
||||
MAIN=$(dirname "$(git rev-parse --path-format=absolute --git-common-dir)")
|
||||
cd "$MAIN"
|
||||
|
||||
# Sem base explícita, usa a release ativa (maior release/* por semver) — nunca
|
||||
# `main` e nunca "a branch em que eu estou", conforme a Hard Rule #19.
|
||||
if [ -z "$BASE" ]; then
|
||||
BASE=$(git ls-remote --heads origin 'refs/heads/release/*' \
|
||||
| sed 's#.*refs/heads/##' | sort -V | tail -1)
|
||||
[ -z "$BASE" ] && { echo "não consegui resolver a release ativa; passe a base explicitamente" >&2; exit 1; }
|
||||
echo "base não informada — usando a release ativa: $BASE"
|
||||
fi
|
||||
|
||||
DIR=".claude/worktrees/${BRANCH##*/}"
|
||||
[ -e "$DIR" ] && { echo "já existe: $DIR" >&2; exit 1; }
|
||||
|
||||
git fetch origin "$BASE" --quiet
|
||||
git worktree add "$DIR" -b "$BRANCH" "origin/$BASE"
|
||||
|
||||
# `.husky/_` PRIMEIRO: é minúsculo e é o que decide se os gates locais rodam.
|
||||
# Copiar node_modules antes seria arriscar abortar (set -e) numa árvore de ~10 GB
|
||||
# e deixar a worktree sem hook nenhum — exatamente o defeito que este script existe
|
||||
# para impedir.
|
||||
if [ -d "$MAIN/.husky/_" ]; then
|
||||
cp -a "$MAIN/.husky/_" "$DIR/.husky/_"
|
||||
else
|
||||
echo "AVISO: .husky/_ não existe no checkout principal — rode 'npm install' lá primeiro" >&2
|
||||
fi
|
||||
|
||||
# node_modules: hard links, ~5s e disco quase zero (inodes compartilhados).
|
||||
# Um `cp -al SRC DEST` com DEST já existente aninharia SRC DENTRO dele
|
||||
# (node_modules/node_modules), então DEST não pode existir aqui.
|
||||
if [ -d "$MAIN/node_modules/node_modules" ]; then
|
||||
echo "AVISO: $MAIN/node_modules/node_modules existe — resíduo de um cp -al aninhado." >&2
|
||||
echo " Ele infla a cópia e esgota o limite de hard links; convém removê-lo." >&2
|
||||
fi
|
||||
if [ -d "$MAIN/node_modules" ]; then
|
||||
# Falha parcial (limite de hard links, disco) não pode derrubar a worktree inteira:
|
||||
# os hooks já estão no lugar e o npm install continua sendo uma saída válida.
|
||||
if cp -al "$MAIN/node_modules" "$DIR/node_modules" 2>"$DIR/.cp-node-modules.log"; then
|
||||
echo "node_modules: $(ls "$DIR/node_modules" | wc -l) entradas (hard links)"
|
||||
rm -f "$DIR/.cp-node-modules.log"
|
||||
else
|
||||
echo "AVISO: a cópia de node_modules falhou parcialmente (veja $DIR/.cp-node-modules.log)." >&2
|
||||
echo " Primeiras linhas:" >&2
|
||||
head -3 "$DIR/.cp-node-modules.log" >&2
|
||||
fi
|
||||
else
|
||||
echo "AVISO: node_modules não existe no checkout principal — rode 'npm install' lá primeiro" >&2
|
||||
fi
|
||||
|
||||
# Verificação: o hook precisa estar REALMENTE ativo, não apenas presente.
|
||||
HOOKS_PATH=$(git -C "$DIR" config --get core.hooksPath || echo ".git/hooks")
|
||||
if [ -x "$DIR/$HOOKS_PATH/pre-commit" ]; then
|
||||
echo "hooks: ativos ($HOOKS_PATH/pre-commit)"
|
||||
else
|
||||
echo "AVISO: pre-commit NÃO está ativo em $DIR/$HOOKS_PATH — os gates locais não vão rodar" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "pronto: $DIR (branch $BRANCH, base $BASE)"
|
||||
echo " cd $DIR"
|
||||
@@ -119,6 +119,32 @@ export const TRANSLATION_SYSTEM = (englishName, native) =>
|
||||
`Keep punctuation and trailing whitespace identical to the source.`,
|
||||
].join(" ");
|
||||
|
||||
/**
|
||||
* Restores the ICU literal escape the backends drop around angle placeholders.
|
||||
*
|
||||
* English writes `'<name>'`: those single quotes are ICU's escape, so the span
|
||||
* renders as the literal text `<name>`. Translations come back as a bare
|
||||
* `<nome>`, which ICU then parses as an (unclosed) tag and the message fails to
|
||||
* compile — every locale in the first batch shipped two of these.
|
||||
*
|
||||
* Only messages whose English side quotes EVERY angle span are touched: when the
|
||||
* source mixes real markup (`<b>`) with a literal span there is no safe way to
|
||||
* tell which is which, so the translation is left exactly as it came back. An
|
||||
* apostrophe inside the span is doubled, otherwise it closes the literal early.
|
||||
*/
|
||||
export function preserveIcuLiteralQuotes(englishValue, translated) {
|
||||
if (typeof englishValue !== "string" || typeof translated !== "string") return translated;
|
||||
if (!englishValue.includes("'<")) return translated;
|
||||
// Every "<" in the source must be the start of an escaped span.
|
||||
for (let i = 0; i < englishValue.length; i++) {
|
||||
if (englishValue[i] === "<" && englishValue[i - 1] !== "'") return translated;
|
||||
}
|
||||
return translated.replace(
|
||||
/(?<!')<([^<>]*)>(?!')/g,
|
||||
(_m, inner) => `'<${inner.replace(/'/g, "''")}>'`
|
||||
);
|
||||
}
|
||||
|
||||
export async function translateString(englishValue, localeEntry, backend) {
|
||||
const englishName = localeEntry.english ?? localeEntry.name;
|
||||
const native = localeEntry.native ?? localeEntry.name;
|
||||
@@ -127,7 +153,7 @@ export async function translateString(englishValue, localeEntry, backend) {
|
||||
{ role: "user", content: englishValue },
|
||||
];
|
||||
const out = await callChat(messages, backend);
|
||||
return out.trim();
|
||||
return preserveIcuLiteralQuotes(englishValue, out.trim());
|
||||
}
|
||||
|
||||
// ----- Batch mode ----------------------------------------------------------
|
||||
@@ -198,8 +224,14 @@ export async function translateBatch(entries, localeEntry, backend) {
|
||||
{ role: "user", content: JSON.stringify(payload) },
|
||||
];
|
||||
const out = await callChat(messages, backend);
|
||||
return parseBatchResponse(
|
||||
const parsed = parseBatchResponse(
|
||||
out,
|
||||
entries.map((e) => e.id)
|
||||
);
|
||||
for (const entry of entries) {
|
||||
if (typeof parsed[entry.id] === "string") {
|
||||
parsed[entry.id] = preserveIcuLiteralQuotes(entry.text, parsed[entry.id]);
|
||||
}
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
@@ -380,16 +380,22 @@ const SYSTEM_PROMPT = (englishName, native) =>
|
||||
`Return ONLY the translated markdown — no preamble, no explanation, no surrounding fences.`,
|
||||
].join(" ");
|
||||
|
||||
// Splits a markdown body into chunks of <= maxChars, breaking on top-level `## ` headings only.
|
||||
function chunkMarkdown(markdown, maxChars = 6000) {
|
||||
// Splits a markdown body into chunks of <= maxChars. Top-level `## ` headings
|
||||
// are the preferred cut; a section that is still longer than maxChars is then
|
||||
// split again on `### ` headings and paragraph boundaries, never inside a
|
||||
// fenced code block. Before the second pass a single long section (README.md
|
||||
// has a 16 KB one, USER_GUIDE.md a 20 KB one) became one oversized request
|
||||
// that the slow fallback model could not answer inside the backend's
|
||||
// 10-minute fetch timeout, and the biggest docs failed on every retry.
|
||||
export function chunkMarkdown(markdown, maxChars = 6000) {
|
||||
if (markdown.length <= maxChars) return [markdown];
|
||||
const lines = markdown.split("\n");
|
||||
const chunks = [];
|
||||
const sections = [];
|
||||
let buf = [];
|
||||
let size = 0;
|
||||
for (const line of lines) {
|
||||
if (line.startsWith("## ") && size > maxChars * 0.5) {
|
||||
chunks.push(buf.join("\n"));
|
||||
sections.push(buf.join("\n"));
|
||||
buf = [line];
|
||||
size = line.length;
|
||||
} else {
|
||||
@@ -397,7 +403,65 @@ function chunkMarkdown(markdown, maxChars = 6000) {
|
||||
size += line.length + 1;
|
||||
}
|
||||
}
|
||||
if (buf.length) chunks.push(buf.join("\n"));
|
||||
if (buf.length) sections.push(buf.join("\n"));
|
||||
return sections.flatMap((section) =>
|
||||
section.length <= maxChars ? [section] : splitOversizedSection(section, maxChars)
|
||||
);
|
||||
}
|
||||
|
||||
const FENCE_LINE = /^\s*(```|~~~)/;
|
||||
|
||||
// Groups a section into blocks — a whole fenced code block, a heading-led run,
|
||||
// or a paragraph ending at a blank line — and packs them greedily. A block that
|
||||
// is itself larger than maxChars stays whole: cutting mid-paragraph or inside a
|
||||
// fence would hand the model a fragment it cannot translate faithfully.
|
||||
function splitOversizedSection(section, maxChars) {
|
||||
const blocks = [];
|
||||
let block = [];
|
||||
let inFence = false;
|
||||
for (const line of section.split("\n")) {
|
||||
const isFence = FENCE_LINE.test(line);
|
||||
if (inFence) {
|
||||
block.push(line);
|
||||
if (isFence) {
|
||||
inFence = false;
|
||||
blocks.push(block);
|
||||
block = [];
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (isFence) {
|
||||
if (block.length) blocks.push(block);
|
||||
block = [line];
|
||||
inFence = true;
|
||||
continue;
|
||||
}
|
||||
if (/^##+ /.test(line) && block.length) {
|
||||
blocks.push(block);
|
||||
block = [];
|
||||
}
|
||||
block.push(line);
|
||||
if (line.trim() === "") {
|
||||
blocks.push(block);
|
||||
block = [];
|
||||
}
|
||||
}
|
||||
if (block.length) blocks.push(block);
|
||||
|
||||
const chunks = [];
|
||||
let current = [];
|
||||
let size = 0;
|
||||
for (const lines of blocks) {
|
||||
const length = lines.join("\n").length + 1;
|
||||
if (size > 0 && size + length > maxChars) {
|
||||
chunks.push(current.join("\n"));
|
||||
current = [];
|
||||
size = 0;
|
||||
}
|
||||
current.push(...lines);
|
||||
size += length;
|
||||
}
|
||||
if (current.length) chunks.push(current.join("\n"));
|
||||
return chunks;
|
||||
}
|
||||
|
||||
|
||||
65
tests/unit/i18n-run-translation-chunking.test.ts
Normal file
65
tests/unit/i18n-run-translation-chunking.test.ts
Normal file
@@ -0,0 +1,65 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { chunkMarkdown } from "../../scripts/i18n/run-translation.mjs";
|
||||
|
||||
// The docs translator splits a page into chunks so one upstream call stays
|
||||
// short. It used to break only on `## ` headings, so a single long section
|
||||
// (README.md carries a 16 KB one, USER_GUIDE.md a 20 KB one) became one
|
||||
// oversized request that the slow fallback model could not answer inside the
|
||||
// backend's 10-minute fetch timeout — the three biggest docs of every locale
|
||||
// then failed with "fetch failed" on every retry.
|
||||
|
||||
const para = (label: string, n = 12) =>
|
||||
Array.from({ length: n }, (_, i) => `${label} sentence ${i + 1} with some filler text.`).join(
|
||||
" "
|
||||
);
|
||||
|
||||
test("a section longer than maxChars is split on sub-headings and paragraphs", () => {
|
||||
const body = [
|
||||
"## Big",
|
||||
para("a"),
|
||||
"",
|
||||
"### Part one",
|
||||
para("b"),
|
||||
"",
|
||||
para("c"),
|
||||
"",
|
||||
"### Part two",
|
||||
para("d"),
|
||||
].join("\n");
|
||||
const chunks = chunkMarkdown(body, 700);
|
||||
assert.ok(chunks.length > 1, "must split an oversized section");
|
||||
for (const chunk of chunks) assert.ok(chunk.length <= 700, `chunk too big: ${chunk.length}`);
|
||||
// Nothing lost: re-joining with blank lines reproduces every line of the source.
|
||||
const lines = (s: string) => s.split("\n").filter((l) => l.trim() !== "");
|
||||
assert.deepEqual(lines(chunks.join("\n\n")), lines(body));
|
||||
});
|
||||
|
||||
test("never splits inside a fenced code block", () => {
|
||||
const code = [
|
||||
"```ts",
|
||||
...Array.from({ length: 30 }, (_, i) => `const v${i} = ${i};`),
|
||||
"```",
|
||||
].join("\n");
|
||||
const body = ["## Code", para("x"), "", code, "", para("y")].join("\n");
|
||||
const chunks = chunkMarkdown(body, 500);
|
||||
const withFence = chunks.filter((c) => c.includes("```"));
|
||||
for (const chunk of withFence) {
|
||||
assert.equal(
|
||||
(chunk.match(/```/g) ?? []).length % 2,
|
||||
0,
|
||||
"fence must open and close in the same chunk"
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("keeps the old behaviour for short pages and for normal ## sections", () => {
|
||||
assert.deepEqual(chunkMarkdown("# Title\n\nshort", 6000), ["# Title\n\nshort"]);
|
||||
const body = ["## A", para("a", 4), "", "## B", para("b", 4), "", "## C", para("c", 4)].join(
|
||||
"\n"
|
||||
);
|
||||
const chunks = chunkMarkdown(body, 400);
|
||||
assert.ok(chunks.every((c) => c.length <= 400));
|
||||
assert.ok(chunks.length > 1);
|
||||
for (const chunk of chunks.slice(1)) assert.match(chunk, /^## /, "cuts land on ## boundaries");
|
||||
});
|
||||
43
tests/unit/i18n-translate-backend-icu-quotes.test.ts
Normal file
43
tests/unit/i18n-translate-backend-icu-quotes.test.ts
Normal file
@@ -0,0 +1,43 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { preserveIcuLiteralQuotes } from "../../scripts/i18n/lib/translate-backend.mjs";
|
||||
|
||||
// The English catalog escapes angle placeholders for ICU: the single quotes in
|
||||
// '<name>' make the span literal text. Translation backends drop them, and the
|
||||
// message then parses as an unclosed ICU tag — every locale added in batch 1
|
||||
// shipped two such strings before CI caught them.
|
||||
test("re-escapes an angle span the translation left bare", () => {
|
||||
assert.equal(
|
||||
preserveIcuLiteralQuotes(
|
||||
"Regenerate ~/.claude/profiles/'<name>'/settings.json",
|
||||
"Regenerar ~/.claude/profiles/<nome>/settings.json"
|
||||
),
|
||||
"Regenerar ~/.claude/profiles/'<nome>'/settings.json"
|
||||
);
|
||||
});
|
||||
|
||||
test("doubles an apostrophe inside the span, which would close the literal early", () => {
|
||||
assert.equal(
|
||||
preserveIcuLiteralQuotes("'<your OmniRoute API key>'", "<teie OmniRoute'i API-võti>"),
|
||||
"'<teie OmniRoute''i API-võti>'"
|
||||
);
|
||||
});
|
||||
|
||||
test("leaves a translation that already carries the quotes untouched", () => {
|
||||
const already = "'<seu token>'";
|
||||
assert.equal(preserveIcuLiteralQuotes("'<your token>'", already), already);
|
||||
});
|
||||
|
||||
test("does not touch messages whose English has a real unquoted tag", () => {
|
||||
// <b> here is markup the translation must keep as markup, not literal text.
|
||||
const translated = "Clique em <b>Salvar</b> e em '<nome>'";
|
||||
assert.equal(preserveIcuLiteralQuotes("Click <b>Save</b> and '<name>'", translated), translated);
|
||||
});
|
||||
|
||||
test("leaves messages without angle spans alone", () => {
|
||||
assert.equal(preserveIcuLiteralQuotes("Settings", "Nastavitve"), "Nastavitve");
|
||||
assert.equal(
|
||||
preserveIcuLiteralQuotes("Restricted to {count} endpoints", "Omejeno na {count} točk"),
|
||||
"Omejeno na {count} točk"
|
||||
);
|
||||
});
|
||||
Reference in New Issue
Block a user