Ternlight: How to Install a 7 MB Embeddings Engine That Runs Entirely in Your Browser
Semantic search almost always assumes a server in the middle: you embed your documents through an API, you embed the user’s query with that same API, you pay per token, and you hope the round trip is fast enough to feel instantaneous. Ternlight, a project that hit Hacker News this week, bypasses that assumption entirely. It packages a functional sentence encoder in a 7 MB WebAssembly bundle that runs the entire embeddings pipeline on the user’s own CPU — no network calls after the initial page load, no per-token cost, no backend to maintain.
What it actually is
Ternlight is a sentence encoder distilled from all-MiniLM-L6-v2 using quantization-aware training in the style of Microsoft’s BitNet b1.58 research — that is, each weight is constrained to just three values (-1, 0, +1) from the start of training, rather than compressed afterward. That’s the difference between a model that was built to be small and one that was shrunk by force, and it’s why Ternlight maintains quality despite extreme compression.
The inference engine is hand-written Rust compiled to WASM with SIMD support, and the model, tokenizer, and engine all go into a single .wasm file — no model download at runtime, no postinstall step. You feed it text and it returns a 384-dimensional vector; you compare two vectors with cosine similarity and get a semantic relationship score even when the words don’t match at all (the README example: “reset my password” and “I forgot my password” give a score of ~0.88).
Two tiers are published, same API:
npm install @ternlight/base # 7 MB on the wire, ~5 ms/embedding (reported by author)
npm install @ternlight/mini # 5.5 MB on the wire, ~2 ms/embedding, slightly lower quality
import { embed, cosineSim } from '@ternlight/base';
const v1 = await embed("reset my password");
const v2 = await embed("I forgot my password");
cosineSim(v1, v2); // ~0.88
// Building a simple local search index
const index = docs.map((d) => ({ d, v: embed(d.text) }));
const q = embed(query);
index.sort((a, b) => cosineSim(q, b.v) - cosineSim(q, a.v));
It runs anywhere there’s V8 or a browser engine — Node ≥18, Cloudflare Workers, Vercel Edge, Deno, Bun, and any frontend with a bundler (Vite needs vite-plugin-wasm; Webpack 5 needs asyncWebAssembly: true). The live demo indexes 2,000 pages of React documentation and runs search-as-you-type entirely on the client side.
Where this is really useful
Because nothing leaves the browser, Ternlight fits naturally into: FAQ and support search widgets, note-taking or knowledge base apps that want offline search, browser extensions, and any case where you’d prefer not to send user text — HR notes, legal drafts, support tickets — to a third-party embeddings API. It’s a retrieval layer, not a generation layer: the natural pattern is “embed and rank locally, and send only the top results to an LLM API if you need a generated response.”
What to keep in mind before using it
Before treating this as production-ready infrastructure, here are a few things to consider:
- It’s brand new. The repository was published alongside the Hacker News launch this week. The code is real and the demo works, but it’s day-one traction, not a battle-tested library — it’s worth treating it as an early-stage, fast-moving project, not settled infrastructure.
- The benchmarks are self-reported. Numbers like the Spearman correlation score of ~0.844 and embedding speed figures come from the author’s own evals folder. The author mentioned that direct comparisons against similarly-sized models like
gte-smallare on the roadmap but haven’t been published yet — so “how it compares against alternatives” is an open question, not a settled one. - Performance may vary more than the headline numbers suggest. At least one comment in the Hacker News thread reported embedding speeds well below what the author claims on their hardware, possibly related to a non-SIMD fallback. If throughput matters to you, benchmark it on your actual target browsers — especially mobile Safari on older devices — before committing.
None of this disqualifies the project — a hobby-scale ternary WASM encoder with a working demo and an author actively responding in the comments is a genuinely interesting technical achievement. It just means the right frame for now is “interesting to keep an eye on and prototype”, not “replace your embeddings API today.”