智能体确定性决策引擎 / Sub-35ms · 100% 排序无关 / 双系统决策结晶

面向 AI 智能体的确定性决策引擎

Sub-35ms 离散路由 · 100% 排序无关 · 双系统结晶

替代耗时较长、格式不可控的生成式路由调用。通过首 Token 条件似然提取实现 <35ms 判定,从机制上避免 JSON 解析异常;支持选项排列无关性(任意排列胜者一致),并提供双系统决策机制:常规请求走快路径,边缘样本由推理模型分析并提炼为规则缓存。

在 GitHub 查看源码 →
quickstart.py
pip install openjevpro
from openjevpro import OpenJevProClient

# Point to local engine, or pass mock=True for zero-GPU offline testing
client = OpenJevProClient(base_url="http://localhost:8000/v1")  # or mock=True
decision = client.decide_choice(
    state={"ticket": "Unauthorized login attempt from unknown IP"},
    candidates=["billing", "tech_support", "security_fraud"],
    order_invariant=True  # 100% mathematical invariance across all N! orderings
)
print(decision.value, decision.confidence)  # 'security_fraud' 0.985 (<35ms)
PIPELINE FLOW: DUAL-SYSTEM DECISION & CRYSTALLIZATION TOPOLOGY
Step 1: Input & Isolation
Isolated Candidate Scoring
100% Order Invariance (1.00)
Sub-35ms / System 2 Deliberate
Step 2: Crystallization Loop
Distilled Rules → Fast Path
Permanent Sub-35ms Promotion
STRATEGY 1 · LOCAL vLLM · Laya 322M

纯本地 / 开源独立运行

支持完全离线部署。兼容开源模型(如 Qwen3、Gemma 4)的词表语法约束解码,以及专用 Laya (ModernBERT 322M) 双向编码器;延迟在 35ms 以内,无需依赖外部云端接口。

STRATEGY 2 · GATEWAY Jev · Cloud API

企业混合网关与容灾降级

与商业 Jev 或云端接口协同运行:由边缘前置处理 70% 高频请求以降低调用成本;置信度护栏拦截歧义样本减少误判,遭遇云端超时或限流时自动熔断降级。

FLAGSHIP · EVOLUTION System 2 → 1

双系统结晶与排序无关

常规请求由快路径高效判定;歧义样本交由推理模型分析并提炼为规则,使后续同类请求直接走快路径。同时具备选项排序无关性,避免排列偏差。

Latency (P50)
<35ms
System 1 Fast Path
Order Invariance
100%
1.00 Permutation Invariant
Selective Precision
100%
Guard Harness Dual-Cutoff
Flywheel Speedup
50x+
1.8s Deliberate → Sub-35ms
REPL SIMULATION

Interactive Decision Simulator

Observe how OpenJevPro extracts normalized posterior distributions over candidate enums.

repl.openjev.pro/v1/choice
GPU ENGINE ONLINE
Model: Qwen3-4B-Instruct
DECISION RECEIPT VERIFIED
48ms TTFT
Selected Choice security
Calibrated P 92.4%
security 92.4%
support 5.1%
human_review 2.0%
Abstained: False Scaling: T=1.30 Gate: ECE Calibrated
SYSTEM ARCHITECTURE

Decision Primitives & Guard Layer

Type-safe building blocks designed for low-latency agent routing and policy gates.

PRIMITIVE 01 100% Type-Safe · Order Invariant

Choice<T>

针对离散 Enum 类型的多分类路由。通过词表掩码约束输出空间,直接计算各选项的条件对数似然,避免 JSON 解析异常;支持 order_invariant=True 消除选项排列顺序偏差。

Input State →
Grammar Mask →
Choice [T]
100% Permutation Invariance (order_invariant=True)
PRIMITIVE 02 Binary Gate

Noul

二元断言评估器($P_{true}, P_{false}$),输出经标定的概率与不确定度区间,适用于工作流门控与安全检查。

Uncertainty Margin ±1.8%
PRIMITIVE 03 Expectation

Score

有序档位与等级评分。依据各档位后验概率质量计算连续期望值,避免离散打分的不稳定抖动。

Continuous Metric E[Score]
SAFETY LAYER 100% Selective Precision

TypeSafeJevGuardHarness

运行期动态标定与阈值拦截($\tau = \max(\tau_{\min}, \alpha / K)$)。将输出概率反推为伪对数似然并进行温度缩放;置信度不足时主动输出 UNKNOWN 弃权,使有效回答准确率达 100%。

Dual-Cutoff: τ = max(0.72, 1.25 / K)
Zero Silent Errors in Benchmark
RESILIENT GATEWAY 60%+ Cloud Cost Cut · 99.99% Fallback SLA

HybridJevGateway & Fallback Circuit Breaker

双层成本分流与熔断网关。70% 以上的高频意图直接由本地模型解析(<20ms);难以确定的边缘样本再转发至商业 API。遭遇云端超时或限流时自动熔断切回本地,避免上游调用发生未捕获异常。

Traffic Routing:
Tier 1 Edge (70% @ $0) →
Tier 2 Cloud (Escalate)
Resilience: CircuitBreaker [CLOSED ↔ OPEN ↔ HALF_OPEN]
COGNITIVE FLYWHEEL Explore First, Crystallize Later (System 2 → System 1)

Deliberative Decision Flywheel & Crystallization

常规请求由快路径模型在 35ms 内完成判定;歧义样本自动交由慢思考模型进行因果推理,并将证据链提炼为判定准则写入规则库,使后续同类请求直接走快路径。

Flywheel Pipeline:
System 1 Fast (<35ms) →
System 2 Thinking (Escalate) →
Crystallization Operator →
Promoted Fast Path (<35ms)
Zero Extra Latency on Repeat
REPRODUCIBLE RESEARCH

Can Open LLMs Match Jev?

Empirical validation on PolyAI Banking77 confirms that open models with grammar decoding achieve statistical parity with dedicated Jev models.

GPT-5.6 Luna Cloud API
97.1%

General frontier model ceiling. Expensive per-call pricing, 300~600ms latency, proprietary cloud lock-in.

TypeSafe Jev (1.13.0) System 1
96.3%

Dedicated compact model. RLCD calibrated, ~25-45ms latency, but proprietary commercial closed source.

OpenJevPro (Qwen3.6-35B) Open Baseline
95.5%

Zero task fine-tuning needed. SGLang grammar decoding matches Jev within overlapping 95% confidence intervals.

PolyAI Banking77 Benchmark Results N = 36 Samples · Live Replicated
Engine Naive Acc Selective Precision Tentative Top-1 OOS Recall P50 Latency ECE Error
OpenJevPro (gemma4:cloud) 94.4% (34/36)
100.0%
100.0%
100.0%
1,084ms 0.025
Direct Open LLM (gemma4 + JSON) 97.2% 96.7% 97.2% 100.0% 630ms 0.029
TypeSafe Jev (1.13.0 Live API) 97.2% 96.7% 97.2% 100.0% 740ms 0.026
1. The Abstention Penalty Out of 36 items, only 1 in-domain query triggered abstention due to multi-clue logit smoothing; naive scoring penalizes this as 0 even though tentative top-1 choice is 100% correct.
2. Zero False Positives On ambiguous items (Sample 1 card locator), Direct LLM & Jev made silent overconfident misclassifications, whereas OpenJevPro safely abstained (100% precision).
3. Deployment Latency Generating full 7-class distribution in Ollama Cloud records 1,084ms; collocated private vLLM deployments on L4/4090 GPUs achieve sub-100ms speeds.
DEPLOYMENT STRATEGY

Recommended Model Tiers (2025–2026)

Optimized for non-thinking classification with tiered latency and cost profiles.

Tier Architecture Active Parameters Latency Recommended Workload
Tier 0: Dedicated System 1 Laya (ModernBERT-large) 322M Bi-directional < 35ms Local microservice, zero-cloud-cost ultra-fast edge routing (Apache 2.0)
Tier 1: Edge Router Qwen3-1.7B / 4B (Dense) 1.7B / 4B < 50ms High-QPS intent gating with /no_think parameter
Tier 1: Multimodal Edge Gemma 4 E4B (Vision) 4.5B Effective ~60ms Invoices, screenshots, and visual document routing
Tier 2: Production Standard Qwen3-30B-A3B (MoE) 3B Active / token 60–90ms Enterprise routing: 30B representation at 3B compute cost
Tier 2: Single-GPU Private gpt-oss-20b (MXFP4 MoE) 3.6B Active ~80ms Private on-premise single GPU deployment, Apache 2.0
Tier 3: Long-tail Fallback DeepSeek-V4.1-Flash 8B~16B Active 200–400ms Complex multi-turn tool arbitration & fallback routing
DEVELOPER ONBOARDING

Quick Start in 30 Seconds

Drop-in Python client compatible with vLLM, SGLang, and OpenAI endpoints.

from enum import StrEnum
from openjevpro.schemas import ChoiceDecision
from openjevpro.client import OpenJevProClient

class TicketRoute(StrEnum):
    BILLING = "billing"
    TECH_SUPPORT = "tech_support"
    SECURITY = "security"
    ESCALATE = "human_review"

# Connect to local vLLM / SGLang, or pass mock=True for zero-GPU CI/CD
client = OpenJevProClient(
    base_url="http://localhost:8000/v1",  # or mock=True
    model="Qwen/Qwen3-4B-Instruct",
    temperature_scaling=1.30,
    abstain_threshold=0.45
)

decision: ChoiceDecision = client.decide_choice(
    state={"ticket_text": "Brute-force login attempt from untrusted subnet."},
    candidates=TicketRoute,
    criteria="Classify incoming ticket to the appropriate response team.",
    order_invariant=True  # 100% permutation-invariant isolated scoring
)

print(f"Action: {decision.value}")       # 'security'
print(f"Confidence: {decision.confidence:.2%}") # '92.40%'
print(f"Abstained: {decision.abstained}")       # False
GOVERNANCE & TERMS

Licensing & Commercial Terms

Dual-license structure supporting open community research and compliant enterprise scale.

Source-Available

PolyForm Noncommercial 1.0

Free for personal learning, academic research, non-profit institutions, and non-commercial development.

  • ✓ Full source code on GitHub
  • ✓ Python SDK & offline calibration tools
  • ✓ Community issue discussions
Explore Repository →
Enterprise Production

Commercial License & SLA

Required for production deployments in commercial enterprises, paid SaaS products, or managed API offerings.

  • ✓ Commercial deployment authorization
  • ✓ Custom model sizing & temperature fitting
  • ✓ SLA guarantee & expert engineering support
Inquire Commercial License →
TECHNICAL CLARIFICATIONS

Frequently Asked Questions

How does OpenJevPro differ from standard LLM JSON prompting?

Standard JSON prompting autoregressively generates 20-100 tokens, risking schema malformation and uncalibrated probabilities. OpenJevPro extracts raw candidate logprobs at the first token using grammar masking, yielding 100% type adherence, mathematically calibrated confidence, and sub-100ms speeds.

What latency can be expected on local private GPU clusters?

With vLLM or SGLang running Qwen3-4B or Qwen3-30B-A3B on A10G/L4 GPUs, end-to-end time-to-first-token latency typically ranges between 35ms and 90ms, perfectly suited for high-throughput gateway routing.

How does the engine handle ambiguous or out-of-scope inputs?

All primitives include first-class UNKNOWN / HUMAN_REVIEW abstention. Combined with the TypeSafeJevGuardHarness dynamic dual-threshold (tau = max(tau_min, alpha/K)), low-confidence borderline queries are safely intercepted without risking silent misclassification.

If I already subscribe to commercial TypeSafe Jev, why do I still need OpenJevPro?

OpenJevPro functions not just as an open alternative, but as an indispensable production harness and guard for commercial TypeSafe Jev: 1) It provides TypeSafeJevGuardHarness to eliminate overconfident silent errors on borderline samples, lifting selective precision to 100%; 2) Its HybridJevGateway filters 70%+ of routine high-frequency queries locally, slashing your commercial cloud billing by 60%+; 3) Its circuit-breaker fallback SLA protects downstream agents against cloud timeouts, rate limits, and outages with zero crashes.

How does OpenJevPro eliminate candidate order bias and enable cognitive crystallization?

In regulated financial and compliance environments, standard multiple-choice prompting suffers from positional bias. OpenJevPro introduces order_invariant=True, which scores each candidate option in an isolated forward pass without competitor options, then normalizes probabilities via commutative softmax. This mathematically guarantees 100% permutation invariance across all N! candidate orderings (delta P < 1e-4). Furthermore, the Deliberative Decision Flywheel couples fast decisions with System 2 thinking models: ambiguous queries trigger deep reasoning traces, which are crystallized into discriminative boundary rules and cached, permanently promoting subsequent queries to the sub-35ms fast path.

Can I develop and test OpenJevPro without a dedicated GPU or active model server?

Yes. OpenJevPro includes a lightweight deterministic mock simulator (MockClient or OpenJevProClient(mock=True)) for zero-GPU offline development and CI/CD pipelines. You can also run 'python3 -m openjevpro.demo' in any terminal to see complete decision receipts in under 15ms.