Strot编译LLM意图并本地运行决策

The open decision compiler Compile your intent once with an LLM. Then decide on every record forever — classification, scoring, routing — as type-safe JSON with probabilities and confidence, on your own hardware, at zero marginal cost. Website · Docs · Benchmarks · PyPI Agents at compile time. Never at runtime. pip install strot-ai import strot result = strot . run ( tickets , query = "Triage incoming support tickets into the correct workflow." , action_registry = { "billing_dispute" : "Refunds, invoice corrections, double charges" , "technical_defect" : "Crashes, bugs, broken behavior" , }, llm = "anthropic/claude-opus-4-8" , # used once, on first compile only ) for o in result . outcomes : print ( o [ "action" ], round ( o [ "confidence" ], 2 )) # billing_dispute 0.97 # technical_defect 0.94 The LLM runs once , at compile time. Production is a frozen, self-contained Python artifact: a small local decision model arbitrates every record, nothing is generated, every output is bounded, and low-confidence records abstain instead of guessing. STROT = Agentic Schema Intent + Compiler & State + Code + Execution + NLU ⟹ JSON Decisions (classification · scoring · routing · typed answers) How it works COMPILE (once per intent + schema) intent + sample ─▶ synthesize ─▶ golden gate ─▶ teacher labels ─▶ calibrate (labels, (your known (annotate (fit the projection, answers must a sample, confidence handlers) pass) once) threshold) │ ▼ frozen artifact (.py) RUNTIME (every record, forever) record ─▶ projection ─▶ decision model ─▶ confidence gate ─▶ typed JSON (compiled) (144M, CPU, (decide or decision ~60 ms) abstain) Two surfaces run() — route each record to an action and execute the compiled handler. decide() — answer typed questions (choice / score / boolean) per record: result = strot . decide ( records , questions = { "is_bug_report" : { "type" : "boolean" , "instructions" : "mentions crashes/errors?" }, "customer_sentiment" :{ "type" : "score" , "min" : 1 , "max" : 5 , "instructions" : "1 angry, 5 happy" }, "routing_bucket" : { "type" : "choice" , "options" : [ "billing" , "engineering" , "spam" ]}, }) # → {"booleans": {...}, "scores": {...}, "choices": {...}} with probabilities + confidence Measured Real model (SupersonicLabs/Julia-1, 144M, CPU), real datasets, clean Docker containers: Result Grounded routing accuracy (398 records) 92.5% compiled vs 74.8% naive — the compiler is worth +18pp on the same model Semantic text (AG News, unseen labels) 86.7% compiled vs 72.7% naive (+14pp) Typed decisions vs the leading hosted decision API's own example near-identical (sentiment 1.79 vs 1.8) Latency / marginal cost / LLM calls per record ~60 ms · $0 · 0 Reproduce everything: benchmarks/ — with the losses published too. What makes it different Compile once, decide forever — content-addressed artifact cache; schema drift triggers exactly one recompile. Golden-record promotion — artifacts must pass your known answers before they serve. Fitted guarantees — temperature scaling + split conformal prediction, baked in: dispatched decisions are wrong ≤ α of all records (marginal); the rest abstain. Measured, not guessed — the compiler scores labelings, and can train a distilled head, keeping only what beats the baseline on held-out data. 0% structural hallucination — output is selection + computation over your registry; nothing is generated at runtime. Model-independent — the decision model is a slot ( nlu= ): local Julia-1 by default, hosted APIs, or any drop-in specialist. Install pip install strot-ai # core: pure stdlib pip install strot-ai[local] # + the local decision model (torch/transformers) pip install strot-ai[hosted] # + openai, for a hosted decision runtime pip install strot-ai[anthropic] # + anthropic, for the synthesizer Bake the local model into your image for offline production: RUN pip install strot-ai[local] \ && python -c "import strot; strot.prefetch_nlu(local_dir='/opt/julia-1', install=True)" ENV STROT_NLU_CHECKPOINT=/opt/jul