NL-2610.003Result[nl.SM]
A 132 MB embedding model picks the right tool from a plain request
The Nook team
Submitted 5 October 2026 · 3 min read
In Nook you ask for a Nooklet, a small ready-made job such as transcribing a recording or editing a PDF, in plain words, and a model picks it. We wanted that to work on any computer with nothing to set up, so we used multilingual-e5-small, an embedding model of 132 MB in 8 bits, running on the processor through llama.cpp. It compares the request with example requests written for each Nooklet. It passed all 30 cases of our live test, including a request in Turkish, and requests that matched no Nooklet scored 0.83 to 0.85 against the 0.88 we treat as a sure match. The margin is narrow, which is why near-misses ask before they act.
- Comments
- Processor only, llama.cpp CPU build. 30 test cases written by us; thresholds set by hand.
- Subjects
- Small models (nl.SM)
- Cite as
NL-2610.003· usenook.ai/labs/small-model-picks-the-tool.html
Full text
A 132 MB model on the processor is enough to send a plain-language request to the right Nooklet: 30 of 30 on our test set. Picking from a short menu does not need a chat model.
The question
Nook's Nooklets are small jobs you reach by saying what you want: “make a PDF out of these photos”, “transcribe this meeting”. Something has to turn the request into the right Nooklet. A chat model could do it, but it would need a graphics card and gigabytes of download before the first Nooklet even opens. We wanted the smallest model that does the job well on any computer.
How it works
- Model
- multilingual-e5-small, 8-bit GGUF (q8_0), 132 MB, pinned to one revision
- Engine
- llama.cpp's
llama-server, processor build, embedding mode with mean pooling - Graphics card
- None needed
- Test
- Nook's live test
finds_with_the_real_model: 30 requests, each with the Nooklet it should find
- Every Nooklet has a few example requests, written in several languages.
- The request gets the prefix
query:, which is how e5 models expect a search query, and the model turns it into a list of numbers, an embedding. - That embedding is compared with each example's by cosine similarity. A Nooklet's score is its best match among its examples, nudged up by 0.03 for each word in the request that points to it, such as “PDF” for Edit a PDF.
- At 0.88 or more, Nook is sure and opens the Nooklet. Between 0.85 and 0.88 it offers it as a “may be”. Below 0.85 it says it found nothing.
Until the model has downloaded, Nook matches words alone, and after ten idle minutes the model stops to give its memory back.
Results
The first test set of 19 requests passed 19 of 19. When three more Nooklets arrived, we added 15 cases, and all 30 passed. Some of the requests, checked against the real model:
| Request | What the finder chose |
|---|---|
| Make a PDF out of these photos | Convert documents, to PDF |
| Bu videoyu Türkçeye çevir (translate this video into Turkish) | Translate speech, into Turkish |
| Change the date on this scanned form | Edit a PDF |
| Write me a poem | An unsure “may be” for Read aloud |
Requests that matched no Nooklet scored between 0.83 and 0.85: just under the line where Nook offers a “may be”, and close to it.
One thing that tripped us
Not every GGUF conversion of the same model loads. The first conversion we tried stopped llama.cpp with bert model needs to define token type count. The conversion published by TwinSunsLLC loaded and works, and Nook pins it.
Limits
Run it yourself
$env:NOOK_TEST_LLAMA_CPU = "C:\path\to\llama-server.exe" # the processor build
$env:NOOK_TEST_FINDER = "C:\path\to\multilingual-e5-small-q8_0.gguf"
cargo test -p nook-core finds_with_the_real_model -- --ignored --nocapture