Native local edition · One complete GGUF

8.39GB

BTL-3 Compact puts the complete 27B text model in one native artifact.

Decoder, vocabulary matrices, output correction, and behavior repair are all packed. The runtime consumes the representation directly—without downloading or reconstructing the BF16 checkpoint.

Download Compact ↗View full BTL-3Runtime source
ARTIFACT MANIFEST AVQ2
27BComplete text model · 7.82 GiB
Exact bytes
8,392,369,600
Packed tensors
2,416 verified
Representation
AVQ2 / UniSVQ
Native backends
CUDA + Metal
Original BF16 required
No
8.39 GBComplete artifact7.82 GiB on disk
92.2%Conditional retention83 / 90 teacher-correct
43.16 t/sNative generationRTX PRO 6000
84.70 t/sPrompt processing512-token probe

Measured retention

What survived
the packing.

A fresh private 100-turn tool-contract gate was authored after representation choices froze. The full model scored 90/100; Compact retained 83 of those 90 teacher-correct cases.

SingleTeacher-correct retained20 / 20
ParallelTeacher-correct retained20 / 20
SequentialTeacher-correct retained20 / 20
AbstentionTeacher-correct retained20 / 20
Parallel-multipleTeacher-correct retained3 / 10
OverallConditional retention83 / 90 · 92.2%

Representation

Small on disk.
Native in memory.

The release is a mixed representation chosen against behavioral gates—not a stock scalar quant with a new label.

Decoder64 layers

Packed AVQ2 decoder tensors with affine INT4 tensors and measured higher-precision islands.

Vocabulary pathComplete

Packed embeddings and output matrices plus a rank-32 output correction.

Behavior pathRepaired

A compact behavior adapter protects tool names, structure, stopping, and abstention.

Runtime support

One model.
Its own runner.

Stock Ollama and the stock LM Studio GGUF engine do not decode AVQ2. The release includes a native OpenAI-compatible server and bridges for familiar client workflows.

macOS arm64 · MetalNative bundleVerified
Linux arm64 · NVIDIA CUDADGX Spark packagePreview
OpenAI-compatible HTTPbtl3-serverVerified
LM StudioIncluded generatorBridge
Ollama CLIIncluded launcherBridge

Quickstart

Serve locally.
Call it normally.

The verified macOS arm64 package starts an OpenAI-compatible endpoint. Device memory, KV cache, and runtime workspace determine usable context.

Start the native server
BTL3_MODEL="$PWD/model/BTL-3-Compact-AVQ2.gguf" \
BTL3_CTX_SIZE=4096 \
  runtimes/supported/BTL-3-Compact-macos-arm64/bin/btl3-server
OpenAI-compatible request
curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "BTL-3",
    "messages": [{
      "role": "user",
      "content": "Write a retrying fetch helper with tests."
    }],
    "stream": true
  }'

Measured performance

43.16 tok/s
on RTX PRO 6000.

Exact GGUF, full CUDA offload, 128 generated tokens, mean of three runs with 0.29 tok/s standard deviation. Apple M2 compatibility smoke measured 2.48 tok/s generation.

Get the 8.39 GB model ↗

Artifact integrity

One file.
One identity.

BTL-3-Compact-AVQ2.gguf · SHA-2562ddf9527620a17a2a6739d184a7096c45712092e6589128792ec6254e94dc30c8,392,369,600 bytes · 2,416 tensor payloads byte-verified

Deployment boundaries

Parallel-multiple: retained 3/10 teacher-correct cases. Overall retention is not uniform across categories.

Runtime: AVQ2 needs BTL's packed runner. Stock Ollama and LM Studio do not load it directly.

Hardware: RTX 4090, RTX 5090, DGX Spark, Windows, and newer Apple devices remain device-specific gates.

Context: 262K is architectural. Begin at 4K–32K according to device memory and increase after measuring headroom.