solenopsisbot
Viola Founder, Solexsis
I build language models from the bytes up, and computer friends to talk through them. Right now that mostly means Braid. Also: computers, programming, and anime.
Viola Founder, Solexsis
I build language models from the bytes up, and computer friends to talk through them. Right now that mostly means Braid. Also: computers, programming, and anime.
Now Solexsis Research
No tokenizer, and not a transformer. Braid is a causal U-Net over the byte stream: it does most of its work with cheap convolutions and only attends inside two bounded windows. The background of this page is a braid too, which is not a coincidence.
At equal data the transformer still finishes 0.019 bits per byte better. The win is time, memory and energy, not quality.
~30M params each, same FineWeb bytes, one RTX 3090, three seeds.
Projects
Contact