输入保持字节级
Byte-level input
所有语言、符号和编码细节都保留在 byte stream 中，避免词表边界提前丢失信息。
All languages, symbols, and encoding details stay in the byte stream, so no information is lost early at vocabulary boundaries.
返回公开项目Back to Open Projects
Chapter 01 · 研究问题Chapter 01 · Research Question
Alethic Insight Research Archive
FLUED
Byte-to-Latent Decision Interface
FLUED 研究如何把原始字节流翻译成神经网络更容易处理的连续潜空间表示， 同时保留字节级可还原能力。它不是新的 tokenizer 宣传页，而是一条语言编码器路线： 让模型先在连续语义空间中理解和补全，再通过 decoder 精确回到 byte。
FLUED studies how to translate a raw byte stream into a continuous latent representation that neural networks handle more easily, while preserving byte-level recoverability. It is not a promotional page for a new tokenizer, but a language-encoder research line: let the model understand and complete in a continuous semantic space first, then return precisely to bytes through the decoder.
research prompt
Can byte streams be converted into latent intelligence without freezing language into a vocabulary?
Research Question
问题不是取代 tokenizer， The question is not about replacing the tokenizer, 也不是单纯的语言编码接口， nor merely about a language encoding interface, 而是更平滑的语义表示空间 but about a smoother semantic representation space
BPE 把字符串先离散切成 token，再让 embedding 表承载大量语义先验。 FLUED 的出发点是把这一步改成可训练、可还原、可压缩的 byte-to-latent 翻译过程。
BPE first discretizes a string into tokens, then lets the embedding table carry much of the semantic prior. FLUED's starting point is to turn this step into a trainable, recoverable, and compressible byte-to-latent translation process.
表示进入连续空间
Representation enters a continuous space
readout latent 承载局部语义和位置信息，让外部 backbone 在更平滑的空间中学习。
The readout latent carries local semantics and position, letting an external backbone learn in a smoother space.
输出必须可还原
Output must be recoverable
latent representation 必须能通过 tied decoder 回到 byte，不能只追求不可解释的中间特征。
The latent representation must return to bytes through a tied decoder; uninterpretable intermediate features alone are not enough.
核心判断
Core claim
语言可以先被翻译为连续潜空间表示，而不是先被永久离散化。
Language can be translated into a continuous latent representation first, instead of being permanently discretized up front.
工程约束
Engineering constraint
编码成本必须接近 tokenizer / BPE 级别，否则语义优势没有部署价值。
Encoding cost must stay near tokenizer / BPE levels, or the semantic advantage has no deployment value.
评估底线
Evaluation baseline
补全任务必须先在 byte 输入层 mask，不能 clean encode 后再遮 readout。
Completion tasks must mask at the byte input layer first — never clean-encode and then mask the readout.
Research Route
研究地图：每一代的判词都留在线上
Research map: every generation's verdict stays on record
FLUED 的迭代不是从“小模型”走到“大模型”，而是沿一个问题逐代排雷：软切分能跑通吗、训练稳吗、 严格口径下还有效吗、压到最硬还剩什么。节点是版本，判词是结局——判死的也留在线上，点击跳到对应证据。
FLUED's iteration is not a march from a small model to a large one, but a generation-by-generation clearing of one question: does soft segmentation work, is training stable, does it survive a strict protocol, and what remains at the hardest setting. Nodes are versions and verdicts are outcomes — even the ruled-out stay on record, each linking to its evidence.
v0.4
tied 自编码器tied autoencoder
软切分 + O(T²) 池化跑通 byte→latent→byte；共享逆后被三重证据判死Soft segmentation + O(T²) pooling ran byte→latent→byte; the shared inverse was later ruled out by triple evidence
历史History
v1
最小假设验证Minimal-hypothesis validation
E1v5 recon_acc 0.99999；历史 E3 BPB 1.21 vs BPE 1.48——最早强信号，仅作背景E1v5 recon_acc 0.99999; historical E3 BPB 1.21 vs BPE 1.48 — the earliest strong signal, kept as background only
历史信号Historical signal
v2
训练动力学Training dynamics
三种子重建稳定（0.9993±0.0005）；公平 D1 0.8732 落后 BPE 0.8066——稳定但落后Three-seed reconstruction stable (0.9993±0.0005); fair D1 0.8732 trails BPE 0.8066 — stable but behind
公平口径Fair protocol
v3.2.1
严格 masked-sourceStrict masked-source
先在 byte 层 mask 再 encode：latent readout 0.1898 对 byte 0.1440——v3 最干净正证据Mask at the byte layer before encode: latent readout 0.1898 vs byte 0.1440 — the cleanest positive evidence of v3
关键证据Key evidence
v3.3–3.4
模块化与消融矩阵Modularity & ablation matrix
七模块拆分逐个过消融：共享逆判死、memory/emit 关闭，canonical_v35 固化Seven modules ablated one by one: shared inverse ruled out, memory/emit closed, canonical_v35 frozen
截止点Cutoff point
v3.6
KDA 状态机KDA state machine
512B → 1×1,536 readout 包；同口径 masked 0.149 对 HNet-DiT 瓶颈臂 0.142，传输少约 60×512B → one 1×1,536 readout package; masked 0.149 vs the HNet-DiT bottleneck arm's 0.142 under the same protocol, ~60× less transmission
当前主线Current mainline
S0.5–0.6
粒度与容量Granularity & capacity
GRPO 自选停在 24.3 段 ≈ 用户手标 21B；summarizer 容量全因子无效——两案了结GRPO chose to stop at 24.3 segments ≈ 21B human-marked granularity; the summarizer-capacity full factorial came back null — both cases settled
已了结Settled
S0.7
瓶颈定位Bottleneck localization
逐段条件化 unmasked acc 0.190→0.351、PPL 33.9→12.1：检索瓶颈在 readout 通道，路线活Per-chunk conditioning lifted unmasked acc 0.190→0.351 and PPL 33.9→12.1: the retrieval bottleneck sits in the readout channel; the line stays alive
前沿Frontier
逐代展开：三代的问题意识Generation by generation: three eras of framing
先证明 soft boundary codec 能跑通
First prove a soft-boundary codec can run
v1 的价值在于把 byte stream -> latent units -> tied inverse bytes 这条链路跑通， E1v5 达到 recon_acc 0.9999、m/n 0.379、bp_std 0.443，并在历史 E3 中出现强 BPB 信号。 问题是短序列还原能力差， 压缩率、阈值和训练参数仍像 magic number，语义质量也不能只靠重建证明。
v1's value was getting the byte stream -> latent units -> tied inverse bytes chain to work: E1v5 reached recon_acc 0.9999, m/n 0.379, and bp_std 0.443, with a strong BPB signal in the historical E3. The problems: poor short-sequence recovery, compression rate, thresholds, and training hyperparameters still looking like magic numbers, and semantic quality that reconstruction alone cannot prove.
去噪增强了重建，也扰动了压缩目标
Denoising strengthened reconstruction — and disturbed the compression target
v2 加入 denoising 后三种子重建稳定，但去噪任务、压缩约束和边界分化之间存在冲突。 latent consistency 的均方误差损失曾直接触发 loss 爆炸和边界塌缩； 去噪比例扫描显示 m/n 随噪声漂移，压缩权重扫描在部分 target 上出现 NaN。
With denoising added, v2's three-seed reconstruction was stable, but the denoising task, the compression constraint, and boundary divergence conflicted. The MSE loss on latent consistency once directly triggered a loss explosion and boundary collapse; the denoise-ratio scan showed m/n drifting with noise, and the compression-weight scan produced NaN at some targets.
从自编码器转向语言编码接口
From autoencoder to language encoding interface
v3 不再把 reconstruction 当作唯一目标，而是追问 latent readout 是否真的降低外部 backbone 的学习难度。 这让 FLUED 接近 BLT、H-Net、ByteFlow、FLEXITOKENS 等方向共同面对的问题： 如何扩大计算复杂度、语义自然度、训练难度的不可能三角面积。
v3 stopped treating reconstruction as the sole goal and instead asked whether the latent readout genuinely lowers an external backbone's learning difficulty. This brought FLUED to the question it shares with BLT, H-Net, ByteFlow, and FLEXITOKENS: how to enlarge the impossible triangle of compute complexity, semantic naturalness, and training difficulty.
v1 soft-boundary autoencoder 在历史 E3 20K 设置中，相比 BPE 有约 18.1% 更低的 bits-per-byte， 并在同一历史设置下优于当时的 BLT reproduction。这是 FLUED 路线最早的强阳性信号。
In the historical E3 20K setting, the v1 soft-boundary autoencoder reached about 18.1% lower bits-per-byte than BPE, and beat the BLT reproduction of the time under the same historical setting — the earliest strong positive signal for the FLUED line.
v2 在 2048 原始 byte、100K steps、统一 downstream backbone 的矩阵下稳定但落后 BPE。 这说明 v1 信号值得重视，但不能直接外推为生产 tokenizer 替代结论。
Under a matrix of 2048 raw bytes, 100K steps, and a unified downstream backbone, v2 was stable but trailed BPE. The v1 signal deserves attention, but it cannot be extrapolated into a production tokenizer-replacement conclusion.
严格在 byte 输入层 mask 后，no-memory latent backbone 明显超过 byte baseline。 这是 v3 系列目前最干净的正向证据。
With strict masking at the byte input layer, the no-memory latent backbone clearly beat the byte baseline — currently the cleanest positive evidence in the v3 series.
v1
最小假设：soft segmentation + tied inverse decoder，证明可微 byte codec 有信号，但短序列和超参魔数暴露完成度不足。
Minimal hypothesis: soft segmentation + tied inverse decoder showed a differentiable byte codec has signal, but short sequences and hyperparameter magic numbers exposed its immaturity.
v2
大模型与去噪：328M tied model 三种子重建稳定，同时暴露 denoising、compression、boundary 之间的训练动力学冲突。
Scale-up and denoising: the 328M tied model reconstructed stably across three seeds, while exposing training-dynamics conflicts among denoising, compression, and boundary.
v3.1
语言编码器原型：readout latent / summary memory / minimal backbone，但早期 clean encode 评估有侧漏风险。
Language-encoder prototype: readout latent / summary memory / minimal backbone, but early clean-encode evaluation risked information leakage.
v3.2.1
严格 masked-source：先在 byte 输入层 mask，再 encode，确认 latent readout 对小 backbone 有正向作用。
Strict masked-source: mask at the byte input layer before encode, confirming latent readout helps a small backbone.
v3.3
架构截止点：segmentor、interpreter、memory、decoder 职责拆开，转向系统消融和公开复现。
Architecture cutoff: segmentor, interpreter, memory, and decoder responsibilities separated; the focus turned to systematic ablation and public reproduction.
v3.4
消融矩阵（2026-07）：共享逆以三重证据判死、独立 decoder 成为默认；提出 CBIU（counterfactual boundary utility，反事实边界效用）协议评估单点边界贡献。
Ablation matrix (2026-07): the shared inverse was ruled out by triple evidence and a standalone decoder became the default; the CBIU (counterfactual boundary utility) protocol was proposed to score single-point boundary contributions.
v3.6
KDA 世代（2026-07/08）：S0 segmentor 预训 F1 0.886；组件预训 +5.8pp 判死端到端；归因矩阵确认增益全部来自 S0 动态边界；S0.5 GRPO 边界优化停于 24.3 段 ≈ 用户手标 21B；S0.7 逐段条件化证明检索瓶颈可解、路线活。
KDA generation (2026-07/08): S0 segmentor pretraining F1 0.886; component pretraining beat end-to-end by +5.8pp, ruling the latter out; the attribution matrix confirmed all gains come from S0 dynamic boundaries; S0.5 GRPO boundary optimization settled at 24.3 segments ≈ 21B human-marked granularity; S0.7 per-chunk conditioning showed the retrieval bottleneck is solvable — the line stays alive.
Current Position
当前位置：v3.6 主线，四个最短入口
Current position: the v3.6 mainline in four shortest entries
地图走到这里，就是现在：v3.6（KDA 世代）把 512 字节 prompt 经动态切分、逐段摘要和 KDA 状态机串行消费后， 只保留 1 个 readout 包（1,536 标量），再由 decoder 从固定大小状态还原原文。 下面四个入口是理解当前证据的最短路径；v2 D1 等旧口径保留在研究地图，不替代当前结论。
This is where the map arrives: v3.6 (the KDA generation) consumes a 512-byte prompt through dynamic segmentation, per-chunk summarization, and a serial KDA state machine, keeps exactly one readout package (1,536 scalars), and lets the decoder restore the text from a fixed-size state. The four entries below are the shortest path into the current evidence; older protocols such as v2 D1 remain on the research map and do not replace current conclusions.
v3.6 架构：512 字节进，一个包出
v3.6 architecture: 512 bytes in, one package out
S0 动态边界、ChunkSummarizer、WriteHead 门控写入、KDA 状态机串行消费，整条 prompt 恰好读出 1×1,536 标量。
S0 dynamic boundaries, ChunkSummarizer, WriteHead gated writes, and a serial KDA state machine read out exactly 1×1,536 scalars for the whole prompt.
Read architecturemasked 0.149 打赢 HNet-DiT 瓶颈臂 0.142
masked 0.149 beats the HNet-DiT bottleneck arm's 0.142
单 seed、38–44M 同规模、同 20K 步同语料、同 masked infilling 口径；信息传输 1,536 vs ~97,540 标量，同指标约 60× 信息效率。
Single seed, matched 38–44M scale, same 20K steps on the same corpus, same masked-infilling protocol; 1,536 vs ~97,540 scalars transmitted — about 60× information efficiency at the same metric.
Open frontier边界优化停在 24.3 段 ≈ 用户手标 21B
Boundary optimization settled at 24.3 segments ≈ 21B human-marked
GRPO 组内排名优化边界与写入门，质量与控制臂打平——当前价值是选粒度而非提质量，教师粒度之争了结。
GRPO optimizes boundaries and the write gate by within-group ranking, tying the control arm on quality — its current value is choosing granularity, not improving quality; the teacher-granularity debate is settled.
Read S0.5–S0.7逐段条件化：unmasked 0.351 / PPL 12.1
Per-chunk conditioning: unmasked 0.351 / PPL 12.1
证明检索瓶颈可解、路线活；masked 未过预注册线，canonical 维持 v36.1，此臂不设默认。
Shows the retrieval bottleneck is solvable and the line stays alive; masked did not pass the pre-registered bar, so canonical stays at v36.1 and this arm is not the default.
Watch live demoEvidence
当前证据边界
Current evidence boundaries
这里保留官网摘要。完整训练历程、100-step 曲线、NaN 断点和 strict masked-source 对照表放在独立证据浏览器中，方便读者直接比较。
This page keeps the site summary. The full training history, 100-step curves, NaN breakpoints, and strict masked-source tables live in a standalone evidence browser for direct comparison.
v2 reconstruction
0.9993 +/- 0.0005
fair D1 BPE-8K
0.8066 BPB
fair D1 FLUED-v2
0.8732 BPB
v3.2.1 gain
+0.0458 mask acc
v3.6 masked acc
0.149 @ 1,536 标量0.149 @ 1,536 scalars
阶段Stage
发现Finding
研究含义Implication
v2
A-class 三种子 eval_acc 0.9991 / 0.9993 / 0.9996，均值 0.9993 +/- 0.0005；说明可微 byte-boundary / reconstruction 稳定。A-class three-seed eval_acc 0.9991 / 0.9993 / 0.9996, mean 0.9993 +/- 0.0005; differentiable byte-boundary / reconstruction is stable.
稳定边界学习成立，但不支撑“优于 BPE”。Stable boundary learning holds, but does not support "better than BPE".
v2 D1
2048-byte / 100K 公平矩阵中，BPE-8K 0.8066 BPB，FLUED-v2 0.8732 BPB，BLT theta=0.3 reproduction 2.3996 BPB。In the 2048-byte / 100K fair matrix: BPE-8K 0.8066 BPB, FLUED-v2 0.8732 BPB, BLT theta=0.3 reproduction 2.3996 BPB.
FLUED 稳定但落后 BPE，这是官网主口径。FLUED is stable but trails BPE — this is the site's main protocol.
v3.2.1
严格 byte-level mask 后，byte baseline mask_acc 0.1440，no-memory latent backbone 达到 0.1898；memory 分支接近但未稳定超过 no-memory。Under strict byte-level masking: byte baseline mask_acc 0.1440, no-memory latent backbone 0.1898; the memory branch comes close but does not stably beat no-memory.
证明 latent interface 可能降低 backbone 学习难度，但 memory gain 仍需严格消融。Suggests a latent interface can lower backbone learning difficulty, but the memory gain still needs strict ablation.
v3.4
one-shot DiT-style refinement、硬执行/软反传、并行 no-self memory、RoPE + 小 AR 与 shared inverse decoder 已实现；40K 归因显示动态边界与 hard emit 的课程仍耦合。One-shot DiT-style refinement, hard-execution/soft-backprop, parallel no-self memory, RoPE + small AR, and a shared inverse decoder are implemented; 40K attribution shows the curricula of dynamic boundaries and hard emit remain coupled.
当前公开重点是架构与梯度的可审计性，不将单种子归因写成最终性能结论。The current public focus is auditability of architecture and gradients; single-seed attribution is not presented as a final performance conclusion.
v3.6
38–44M 同规模、同 20K 步、同语料、同 masked infilling 口径（单 seed）：FLUED v3.6 masked 0.149 @ 1,536 标量，HNet-DiT 瓶颈臂 0.142 @ ~97,540 标量；HNet-DiT 标准臂 0.324 为无压缩天花板锚，不在同一压缩口径。Matched 38–44M scale, same 20K steps, same corpus, same masked-infilling protocol (single seed): FLUED v3.6 masked 0.149 @ 1,536 scalars; HNet-DiT bottleneck arm 0.142 @ ~97,540 scalars; the HNet-DiT standard arm's 0.324 is an uncompressed ceiling anchor outside this compression protocol.
同指标下约 60× 信息效率；受控单种子证据，不做多种子或尺度外推。About 60× information efficiency at the same metric; controlled single-seed evidence, with no multi-seed or scale extrapolation.
v3.6 S0.7
逐段条件化诊断臂：unmasked 全位置 acc 0.351 / PPL 12.1，masked 0.141 未过预注册线；该臂为非默认口径，canonical 维持 v36.1。Per-chunk conditioning diagnostic arm: unmasked all-position acc 0.351 / PPL 12.1; masked 0.141 missed the pre-registered bar; the arm is a non-default protocol and canonical stays v36.1.
证明 KDA 状态的检索瓶颈可解、路线活；不进入主结论。Shows the retrieval bottleneck of KDA state is solvable and the line stays alive; it does not enter the main conclusions.
Boundary Showcase
v3.6 的切分边界长什么样
What v3.6's segmentation boundaries look like
同一段文本，五种切分来源并排：FLUED v3.6 的 S0 / GRPO R4 两个学习切分臂、H-Net 复现（学习切分）、BLT 字节 LM 熵切分、BPE 分词器。 全部来自预计算的真实模型输出（单 seed 演示口径，offsets 为 UTF-8 字节偏移），不是示意图； 实时演示可以在本机对任意输入重跑其中的模型。
The same text, five segmentation sources side by side: FLUED v3.6's two learned arms (S0 / GRPO R4), an H-Net reproduction (learned), BLT byte-LM entropy cuts, and the BPE tokenizer. All are precomputed real model outputs (single-seed demo protocol; offsets are UTF-8 byte offsets), not illustrations; the live demo reruns these models on any input on your own machine.
checkpointarm_a_s0 + grpo_r4
S0 教师边界 F1S0 teacher-boundary F10.886
R4 hard 停点R4 hard stop24.3 段 ≈ 用户手标 21B24.3 segments ≈ 21B human-marked
口径Protocol单 seed 演示single-seed demo
切分对比加载中…Loading segmentation comparison…
S0 臂按教师标注粒度切（约 27B/段）；GRPO R4 臂经 S0.5 边界优化停在约 24 段 ≈ 用户手标 21B 粒度——任务奖励自选粒度，GRPO 当前的价值是选粒度而非提质量。 BPE 切得碎且无语义（zh 148 段 / en 319 段）；BLT 熵切分是纯局部统计信号（entropy > 3.5 下刀）； H-Net 是字节级模型，zh 文本存在字符内部下刀（段内出现 替换符为正常行为，数据已标注 lossless_join=false）。
The S0 arm cuts at teacher-annotated granularity (~27B/segment); the GRPO R4 arm, after S0.5 boundary optimization, settles at ~24 segments ≈ 21B human-marked granularity — the task reward chooses its own granularity, and GRPO's current value is choosing granularity, not improving quality. BPE cuts finely and is semantics-blind (148 segments zh / 319 en); BLT entropy cutting is a purely local statistical signal (cut where entropy > 3.5); H-Net is a byte-level model and cuts inside characters on zh text ( replacement chars within segments are expected; the data is marked lossless_join=false).
Architecture Trace
架构追踪：从 tied 自编码器到 KDA 状态机
Architecture trace: from a tied autoencoder to a KDA state machine
每一代架构都是一条从上而下的流水线，节点对应公开仓库里的真实模块，结局如实记录。 这里是摘要版；逐节点完整流水线、两任务口径与代码行级对照在架构追踪页。
Every generation is a top-to-bottom pipeline whose nodes map to real modules in the public repository, with outcomes recorded as they are. This is the summary; the full node-by-node pipelines, the two-task protocol, and line-level code references are on the architecture trace page.
512 字节进，一个 readout 包出
512 bytes in, one readout package out
状态：canonical v36.1。整条 prompt 压成 1×1,536 标量；masked 0.149 打赢 HNet-DiT 瓶颈臂 0.142（单 seed 同口径）。
Status: canonical v36.1. The whole prompt is compressed to 1×1,536 scalars; masked 0.149 beats the HNet-DiT bottleneck arm's 0.142 (single seed, same protocol).
- 字节流 512B单样本全上下文
- byte stream 512Bone sample, full context
- PlainByteLookup → byte encoder ×3DiTStyleBlock · RoPE + ALiBi
- segmentor ×9 + tanh 置信度S0 预训后冻结 · F1 0.886
- segmentor ×9 + tanh confidencefrozen after S0 pretraining · F1 0.886
- 动态切分conf > 0.94 · UTF-8 续字节禁切 · 容量截断
- dynamic segmentationconf > 0.94 · no cuts at UTF-8 continuation bytes · capacity truncation
- ChunkBuilder → ChunkSummarizer每段摘要成一个 memory（d_mem 512）
- ChunkBuilder → ChunkSummarizereach chunk summarized into one memory (d_mem 512)
- WriteHead → KDAStateMachinek/v/β/α 门控 · 串行 delta 规则 · 4 头 k128/v256
- WriteHead → KDAStateMachinek/v/β/α gating · serial delta rule · 4 heads k128/v256
- readout query ×1 + realign包 1×1,536 标量
- readout query ×1 + realignpackage of 1×1,536 scalars
- TinyBackbone3L pre-norm · 仅任务二经过
- TinyBackbone3L pre-norm · task two only
- GlobalSpanDecoder → bytes共享 byte 表 · cosine logits
- GlobalSpanDecoder → bytesshared byte table · cosine logits
边界在哪里下刀
Where the boundary cuts fall
- BLTbyte LM 逐字节熵超阈值下刀 · 纯局部统计信号byte-LM per-byte entropy over threshold · purely local statistics
- H-Net余弦不相似度硬下采样 + 因果主干 · 无压缩激励即退化（E20：~1 或 ~190 chunks）cosine-dissimilarity hard downsampling + causal trunk · degenerates without explicit compression pressure (E20: ~1 or ~190 chunks)
- BPE离线频次合并固定词表 · 与语义无关 · 最碎（zh 148 段）offline frequency merges into a fixed vocabulary · semantics-blind · finest cuts (148 segments on zh)
- FLUED v3.6S0 学习边界 · 为下游任务训练 · 粒度可被 GRPO 选择（24.3 段 ≈ 用户 21B）S0 learned boundaries · trained for the downstream task · granularity selectable by GRPO (24.3 segments ≈ 21B human)
逐节点流水线与 E20 退化证据见架构追踪页。
Node-by-node pipelines and the E20 degeneration evidence are on the architecture trace page.
tied 权重自编码器
tied-weight autoencoder
sigmoid(Linear(ΔH)) 软切分 + O(T²) 池化 + tied inverse。结局：共享逆三重证据判死。
sigmoid(Linear(ΔH)) soft segmentation + O(T²) pooling + tied inverse. Outcome: the shared inverse was ruled out by triple evidence.
模块化拆分
Modular decomposition
segmentor / policy / memory / decoder 七模块职责分离。结局：架构截止点，转向系统消融。
Seven modules with separated responsibilities: segmentor / policy / memory / decoder. Outcome: architecture cutoff; focus turned to systematic ablation.
单体 probe + 消融矩阵
Monolithic probe + ablation matrix
DiT block、软桥、no-self memory、rate_emit 逐个过消融。结局：memory / emit 关闭，canonical_v35 固化。
DiT block, soft bridge, no-self memory, and rate_emit ablated one by one. Outcome: memory / emit closed; canonical_v35 frozen.
Related Thinking
借鉴工作和 FLUED 的取舍
Related work and FLUED's trade-offs
FLUED 不是孤立提出的结构。它吸收了 tokenizer-free、动态分块、信息论压缩、 latent diffusion、草稿修正和控制空间映射等方向的启发，但最终目标仍是语言编码器， 不是直接复刻某一种 byte-level LM。
FLUED was not proposed in isolation. It borrows from tokenizer-free modeling, dynamic chunking, information-theoretic compression, latent diffusion, draft correction, and control-space mapping — but its end goal remains a language encoder, not a replica of any particular byte-level LM.
熵驱动动态 patching
Entropy-driven dynamic patching
启发 FLUED 关注 byte-level 动态分块和局部不确定性；FLUED 不依赖大型预训练 ByteLM 作为唯一边界来源。
Inspired FLUED's focus on byte-level dynamic chunking and local uncertainty; FLUED does not rely on a large pretrained ByteLM as its only boundary source.
用编码率约束压缩
Compression constrained by coding rate
启发 readout 数量不应只靠 reconstruction loss 控制；v3.3 保留 rate pressure 和消融入口。
Motivated the idea that readout count should not be governed by reconstruction loss alone; v3.3 keeps rate pressure and an ablation entry.
边界学习与边界质量
Boundary learning and boundary quality
启发 signed boundary、边界置信度诊断和 ROI 可视化；FLUED 避免把硬规则当作最终 tokenizer。
Inspired signed boundaries, boundary-confidence diagnostics, and ROI visualization; FLUED avoids treating hard rules as the final tokenizer.
可微分词与梯度分块
Differentiable tokenization and gradient chunking
证明“切分可以学习”这一路线有先例；FLUED 进一步要求 latent 可被 decoder 反向还原为 byte span。
Established precedent that segmentation can be learned; FLUED further requires the latent to be decodable back into byte spans.
连续 latent 空间建模
Modeling in a continuous latent space
启发把 denoise 放在 latent refinement 上，而不是把 FLUED 变成纯 next-byte 自回归模型。
Motivated placing denoising on latent refinement rather than turning FLUED into a pure next-byte autoregressive model.
小自回归头做轻量修正
A small autoregressive head for lightweight correction
启发少量串行 correction head，但 v3.3 当前先保留一阶段接口，避免训练和推理成本过早膨胀。
Inspired a few serial correction heads, though v3.3 keeps a one-stage interface for now to avoid inflating training and inference cost too early.
低维控制到高维行为
Low-dimensional control to high-dimensional behavior
启发 memory/readout 的控制与 payload 分离：memory 不直接暴露给 backbone，readout 才是外部接口。
Inspired the control/payload split between memory and readout: memory is never exposed directly to the backbone; the readout is the external interface.
层次压缩与长上下文
Hierarchical compression and long context
启发“本段高清、历史压缩”的长上下文路线；FLUED 把该思想放在 encoder memory，而不是固定 patch。
Inspired the "current segment in high resolution, history compressed" long-context line; FLUED places this idea in the encoder memory rather than fixed patches.
Claims
我们主张什么，也不主张什么
What we claim — and what we don't
可以主张
We claim
- FLUED 是 Alethic Insight 的 tokenizer-free language interface 研究项目。
- FLUED is Alethic Insight's research project on tokenizer-free language interfaces.
- v2 证明了稳定重建和可微边界的可行性。
- v2 demonstrated stable reconstruction and feasible differentiable boundaries.
- v2 公平 D1 证明 FLUED-v2 稳定但仍落后 BPE。
- The v2 fair D1 run showed FLUED-v2 is stable but still trails BPE.
- v3.2.1 在严格协议下证明 latent readout 对小 backbone 有帮助：0.1898 mask_acc vs byte baseline 0.1440。
- v3.2.1 showed under a strict protocol that latent readout helps a small backbone: 0.1898 mask_acc vs byte baseline 0.1440.
- v3.3 把问题明确推进到 byte-level language encoder 的计算复杂度、语义自然度、训练难度三角。
- v3.3 sharpened the problem into the compute-complexity / semantic-naturalness / training-difficulty triangle of a byte-level language encoder.
- v3.3 提供了清晰的开源架构、参数接口和消融入口。
- v3.3 provides a clear open-source architecture, parameter interface, and ablation entries.
- v3.6 在同规模、同步数、同语料、同 masked infilling 口径下 masked 0.149 超过 HNet-DiT 瓶颈臂 0.142，信息传输少约 60×（单 seed 受控证据）。
- v3.6, at matched scale, steps, corpus, and masked-infilling protocol, reached masked 0.149 vs the HNet-DiT bottleneck arm's 0.142 with ~60× less information transmission (controlled single-seed evidence).
暂不主张
We do not claim
- FLUED 已经替代 BPE。
- That FLUED has replaced BPE.
- FLUED 已经击败 BLT、H-Net、ByteFlow 等近期系统。
- That FLUED has beaten recent systems such as BLT, H-Net, or ByteFlow.
- memory branch 已被证明是默认主线。
- That the memory branch is proven as the default mainline.
- v1 历史 BPB 优势已经被 v2/v3 在所有新设置中复现。
- That v1's historical BPB advantage has been reproduced by v2/v3 across all new settings.
- reconstruction accuracy 等于语义质量。
- That reconstruction accuracy equals semantic quality.
- v3.3 已经彻底解决训练参数 magic number 和可塑性问题。
- That v3.3 has fully solved the magic-number hyperparameter and plasticity problems.
- v3.3 已经完成系统实验并证明优于近期 tokenizer-free 系统。
- That v3.3 has completed systematic experiments proving superiority over recent tokenizer-free systems.