v3.6 pushes the problem to its hardest setting: the whole prompt is consumed serially through dynamic segmentation, per-chunk summarization, and a KDA state machine (linear attention with a channel-wise gated delta rule), keeping only one readout package — a logical readout unit of 1,536 scalars. The decoder must restore the original text from this fixed-size state. This page gives the pipeline, the two training tasks, and the current fair-comparison anchors; all numbers are single-seed demo protocol and do not constitute performance claims.
Previous · Research question: FLUED bets that language can be translated into a continuous latent space and then restored precisely to bytes — this chapter shows how v3.6 presses that bet into a concrete pipeline.
Tracing means checking each node against real code: every card is a top-to-bottom forward pipeline whose nodes map to concrete modules in the public repository, with the generation's outcome recorded at the end. Below we start with the current mainline v3.6 (S0.5 GRPO is its training-stage extension, attached as a subordinate card rather than its own generation), then go back in time through v0.4 → v2 → v3.1 → v3.2 → v3.3 → v3.4, and finally place the competitors BLT / H-Net / BPE on the same scale.
Status: current mainline. S0 dynamic boundaries + a serial KDA state compress a 512B prompt into 1×1,536 scalars; masked 0.149 beats the HNet-DiT bottleneck arm's 0.142 (single seed, same protocol).
字节流 512B单样本全上下文 · 无窗口无 stride
byte stream 512Bone sample, full context · no window, no stride
PlainByteLookupd_byte 384 · PAD-offset
byte encoder ×3DiTStyleBlock · RoPE + ALiBi
segmentor ×9 + headS0 段落标签预训后冻结 · 教师 F1 0.886
segmentor ×9 + headpretrained on S0 paragraph labels, then frozen · teacher F1 0.886
Bottleneck note: the v36.1 decoder condition is package.mean + chunk_pos broadcast; S0.7 per-chunk conditioning has shown the retrieval bottleneck is solvable (unmasked 0.351 / PPL 12.1), but it is not the default.
S0.5 · 挂在 v3.6 主线之后 · 不改架构S0.5 · attached after the v3.6 mainline · no architecture change
GRPO 边界优化 GRPO boundary optimization 训练阶段 · 非独立架构training stage · not a separate architecture
Outcome: hard stop at 24.3 segments ≈ 21B human-marked granularity — the teacher-granularity debate is settled; quality ties the control arm, so GRPO's current value = choosing granularity, not improving quality.
v3.6 已训基线自 3K 检查点续训 2K 步
trained v3.6 baselinecontinued 2K steps from the 3K checkpoint
同前缀采 G=8 个切分方案组内排名优势 · 免 value net
sample G=8 segmentations per prefixwithin-group ranking advantage · no value net
约束走可微直接损失E[count] = Σcut_prob（不走采样计数率项）
constraint as a differentiable direct lossE[count] = Σcut_prob (no sampled count-rate term)
微调边界 + β 写入门R4 定稿；R1–R3 三轮失败教训见下方 #frontier
fine-tune boundaries + the β write gateR4 finalized; the three failed rounds R1–R3 are documented under #frontier below
历代架构（按时间序 v0.4 → v3.4）Past generations (in time order, v0.4 → v3.4)
v0.4 · flued/model.py · FLUEDAutoencoder
tied 权重自编码器
tied-weight autoencoder
结局：共享逆被三重证据判死，tied inverse 路线关闭。
Outcome: the shared inverse was ruled out by triple evidence; the tied-inverse line is closed.
byte idsvocab 257 · PAD-offset
tied Transformer blocksencoder / decoder 共享同一组权重
tied Transformer blocksencoder / decoder share one set of weights
Outcome: the 2M codec loop is learnable (recon 0.506 · length_acc 0.973 · units/byte 0.117); the latent backbone edged out the segment-mask byte baseline (0.178 vs 0.150), but early clean-encode evaluation risked information leakage.
byte ids
weak boundary startsUTF-8 / 标点 / 长度弱规则标签
weak boundary startsweak-rule labels from UTF-8 / punctuation / length
Outcome: masking at the byte input layer before encode closed the clean-readout leak; under the strict protocol the latent backbone clearly beat the byte baseline (mask_acc 0.1898 vs 0.1440, +0.046) — then the cleanest positive evidence of the v3 series.
Outcome: memory / emit failed to justify themselves and were closed; plain lookup + standalone decoder + uniform boundaries were frozen as canonical_v35.
The E20 phenomenon: without explicit compression pressure, boundaries degenerate — ~1 chunk in the DiT standard arm, ~190 chunks in the bottleneck arm, no middle ground.
The finest cuts: 148 segments on the same Chinese sample (vs ~13 for the v3.6 S0 arm); boundaries are decided by offline frequency merges, unrelated to model semantics.
离线语料统计频次最高字节对迭代合并
offline corpus statisticsiterative merging of the most frequent byte pairs
固定词表vocab 8,192
fixed vocabularyvocab 8,192
查表切分在线无参数、不看上下文
table-lookup segmentationno parameters at inference, no context
The forward pass is a one-way pipeline: encode, segment, summarize, write, read out. No window, no stride — 512 bytes are consumed as one complete sample. Highlighted nodes on the left are v3.6's core innovations, dimmed ones are conventional modules; the explanation cards on the right align vertically with their node groups.
字节流 512B单样本全上下文 · 无窗口无 stride
byte stream 512Bone sample, full context · no window, no stride
Training is constrained jointly by two tasks: one tests exact memory, the other tests whether information survives rewriting by the backbone. Masks are native byte masks at 5%, span 1-8.
任务一 · 精准还原（direct）
Task one · exact restoration (direct)
readout 包直接进共享 span decoder。
The readout package goes straight into the shared span decoder.
目标：逐字节精准还原文本，指标 direct_acc。
Goal: restore the text byte for byte; metric direct_acc.
检验状态本身是否无损携带全文信息。
Tests whether the state itself carries the full text losslessly.
任务二 · 经主干改写还原（backbone）
Task two · restoration after backbone rewriting
readout 包先经 TinyBackbone 改写，再由 decoder 全部还原。
The readout package is first rewritten by TinyBackbone, then fully restored by the decoder.
The two tasks share the same encoding and segmentation; they differ only in whether a backbone is inserted after the readout package. All headline metrics reported for v3.6 use the task-two protocol.
Comparisons only hold at the same information-transmission budget: v3.6 and the HNet-DiT bottleneck arm are compared on masked acc at comparable compression tiers; the H-Net AR reproduction serves as a next-byte ceiling anchor. The HNet-DiT standard arm's boundaries degenerate (~1 chunk), so it sits outside this compression protocol and is not used as an anchor.
模型Model口径Protocol指标Metric信息传输Transmission定位Role
FLUED v3.6 (S0)任务二 masked acc（eval）task-two masked acc (eval)0.1491,536 标量1,536 scalars主模型 · 单 readout 包main model · single readout package
At essentially tied masked acc (0.149 vs 0.142), v3.6 transmits 1,536 scalars across chunks versus 97,540 for the bottleneck arm — roughly a 60-fold gap.
两臂退化是现象不是 bugBoth degenerate arms are a phenomenon, not a bug
Without explicit compression pressure, dynamic segmentation has no middle ground: the standard arm barely cuts (~1 chunk) while the bottleneck arm over-segments (~190 chunks). The demo page shows both arms' real segmentations directly.
口径声明Protocol statement
以上均为单 seed、演示性质证据，用于锚定当前研究方向；不构成多种子或尺度上的性能结论。
All of the above is single-seed, demo-grade evidence meant to anchor the current research direction; it is not a multi-seed or scale-level performance claim.
Module-by-Module
v3.6 vs H-Net：逐模块三列对齐
v3.6 vs H-Net: three columns aligned module by module
Compared here are training regimes and operator differences per module: how boundaries cut, how gradients flow back, how chunks are pooled and compressed, and how upsampling restores bytes. Column one is FLUED v3.6; column two is the original H-Net paper mechanism; column three is our fair-comparison arm that swaps H-Net's trunk for DiT, run under the same masked-infilling protocol as v3.6.
模块ModuleFLUED v3.6H-Net 原版（论文机制）H-Net original (paper mechanism)HNet-DiT（我方公平对比臂）HNet-DiT (our fair-comparison arm)
边界算子Boundary operatorsegmentor 9 层细长 DiT → tanh 有界置信度 >0.94 下刀 + UTF-8 续字节禁切segmentor: 9 slim DiT layers → cut at tanh-bounded confidence >0.94 + no cuts at UTF-8 continuation bytes相邻隐状态余弦不相似度 0.5·(1−cos) >0.5 下刀（routing module，2 层 encoder 之上）cut where adjacent hidden-state cosine dissimilarity 0.5·(1−cos) >0.5 (routing module, above a 2-layer encoder)同 H-Net 原版（余弦不相似度）same as H-Net original (cosine dissimilarity)
边界梯度（训练方式）Boundary gradient (training regime)不走主任务梯度：S0 段落标签 BCE 独立预训后冻结；S0.5 用 GRPO（组相对策略梯度）做离散边界优化No main-task gradient: independently pretrained on S0 paragraph labels with BCE, then frozen; S0.5 uses GRPO (group relative policy optimization) for discrete boundary optimizationSTE 直通估计器端到端回传（hard boundary + (prob − prob.detach())）+ ratio loss 把边界率压向 0.2STE straight-through estimator, end-to-end (hard boundary + (prob − prob.detach())) + a ratio loss pushing the boundary rate toward 0.2同原版 STE，但训练目标换成 masked infillingsame STE as the original, but the training objective is masked infilling
汇聚 / 压缩算子Pooling / compression operatorChunkSummarizer 4-slot 注意力池化逐段产 memory → WriteHead k/v/β/α 门控 → KDA 状态机串行 delta-rule 写入ChunkSummarizer 4-slot attention pooling produces one memory per chunk → WriteHead k/v/β/α gating → serial delta-rule writes into the KDA state machine按边界硬切 + chunk 均值池化 → chunk_proj 进主干hard cuts at boundaries + chunk mean pooling → chunk_proj into the trunk同原版same as the original
主干BackboneTinyBackbone 3 层双向 Transformer，只看 1 个 readout 包（1,536 标量）TinyBackbone: 3-layer bidirectional Transformer seeing only 1 readout package (1,536 scalars)层级主干，论文为 Mamba-2；我方复现因 Windows 环境用 9 层因果 Transformer（已披露差异）hierarchical trunk, Mamba-2 in the paper; our reproduction uses a 9-layer causal Transformer due to the Windows environment (difference disclosed)7 层双向 Transformer（38.2M 对齐 39M 目标）7-layer bidirectional Transformer (38.2M, aligned to the 39M target)
解码 / 上采样算子Decoding / upsampling operatorGlobalSpanDecoder：slot+cond 经 DiTStyleBlock，与 encoder 共享字节表做 cosine logitsGlobalSpanDecoder: slot+cond through DiTStyleBlock, cosine logits against the byte table shared with the encoderdechunk：取「最后完成 chunk」的主干输出回贴字节级 + 边界概率平滑 + encoder 残差（decoder_skip）dechunk: paste the trunk output of the "last completed chunk" back to byte level + boundary-probability smoothing + encoder residual (decoder_skip)同原版；标准臂保留 byte 旁路（skip），瓶颈臂砍旁路逼信息走瓶颈same as the original; the standard arm keeps the byte bypass (skip), while the bottleneck arm removes it to force information through the bottleneck
压缩激励Compression pressure率控制 100% 归边界（CBIU 锚点协议 + GRPO 线）rate control assigned 100% to boundaries (CBIU anchor protocol + the GRPO line)ratio loss = (边界率boundary rate − 0.2)² × 0.03无显式激励 → 边界退化：标准臂塌成 ~1 chunk、瓶颈臂碎到 ~190 chunks（E20 的对照证据）No explicit pressure → boundary degeneration: the standard arm collapses to ~1 chunk, the bottleneck arm fragments to ~190 chunks (the E20 control evidence)
Bottom line: under the same protocol, v3.6 masked 0.149 ≈ the HNet-DiT bottleneck arm's 0.142, but v3.6 transmits only 1,536 scalars — about 60× information-transmission efficiency. Without explicit compression pressure, both H-Net arms' boundaries degenerate; dynamic segmentation has no middle ground.
Latest Evidence · 2026-08-04
S0.5–S0.7：粒度之争了结，瓶颈锁定 readout 通道
S0.5–S0.7: the granularity debate settled, the bottleneck localized to the readout channel
Three new pieces of evidence (E21 / E22 / E23, candidate protocol): GRPO changes segmentation granularity, not quality; the summarizer-capacity ablation came back null across the board; the real bottleneck is the package-mean conditioning channel of the downstream readout. All single-seed demo protocol — not performance claims.
Group relative policy optimization on the segmentor boundaries + the β write gate (GRPO: sample G segmentations per prefix, within-group ranking advantage, no value network). Lessons from the three failed rounds R1–R3: a pure quality reward → a fragmenting shortcut rushing to 59/64 segments; a sampled count-rate term → decoupled by temperature smearing; a deterministic penalty → zero within-group variance erased by advantage normalization. Core lesson: group-relative advantage can only optimize what varies within the group. R4 finalized: the constraint becomes a differentiable direct loss E[count]=Σcut_prob, budget 18 ≈ 24 deployed segments anchored to human granularity → masked ties the control arm on both protocols, but the boundary ruling succeeded: hard stop at 24.3 segments ≈ 21B human-marked — the teacher-granularity debate is settled. GRPO's current value = choosing granularity, not improving quality.
E22 · S0.6 summarizer 容量全因子E22 · S0.6 summarizer-capacity full factorial
A 2³ full factorial over slots / hidden / d_mem — 8 arms trained from scratch for 20K steps: main effects ≤ 0.24pp, all null. The ceiling is not in the summarizer — the bottleneck was localized to the package-mean conditioning channel of the downstream readout, pushing the question directly to S0.7's per-chunk conditioning diagnostic.
per_chunk_readout: after consuming each chunk, KDA reads that chunk's conditioning vector from the current state, replacing mean + position broadcast. Unmasked all-position acc 0.190 → 0.351 (+16pp), PPL 33.9 → 12.1 — the information is indeed in the KDA state; the mean channel is the retrieval bottleneck, and the line stays alive. Masked 0.154 → 0.141 missed the pre-registered bar (protocol note: masked measures bytes that never entered the state — inference, not retrieval). Rate correction: this arm transmits ~27K scalars, still ~3.6× cheaper than the HNet-DiT bottleneck arm (~97K).
Ruling (2026-08-04): masked stays a required metric and is not downgraded; S0.7 is not made default — canonical stays v36.1 with per_chunk_readout off by default; whether GRPO reruns under the new conditioning is undecided.
Code Anchors
每个关键节点都能落到公开代码
Every key node lands on public code
链接固定到公开分支的实现行；训练脚本中的任务定义与指标口径应和本页共同阅读。
Links pin to implementation lines on the public branch; read them together with the task definitions and metric protocols in the training scripts.
