Portable Model Evidence
Embedding model bake-off
Public screening evidence selected potion-code-16m-v2 for the full retrieval matrix.
This screening used one repetition. The released-binary resource report adds repeated latency and resource measurements.
Screening peak RSS uses cumulative child-process resource counters. It does not isolate enhancement RSS. The repeated report measures each enhancement child separately.
Candidate results
| Profile | Status | Queries | nDCG@10 | Warm p95 | Neural build | Peak RSS | Decision |
|---|---|---|---|---|---|---|---|
potion-code-16m-v2 | selected-default | 100 in 4 tasks | 0.7125 | 593.80 ms | 1.74 s | 247.57 MiB | Highest nDCG@10, MRR@10, and recall@20 of the candidates with a complete screening matrix, with the smallest model download. It builds vectors more slowly and uses more memory than static-retrieval-v1. Promoted to the full public matrix. |
code-minilm-l12-v1 | resource-stop | 25 in 1 task | 0.5815 | 1511.43 ms | 614.64 s | 4.06 GiB | Stopped after one task because it exceeded both screening budgets, with the longest vector build of the candidates. Its single-task nDCG@10 is the highest of the transformer candidates and below potion-code-16m-v2 on the same task. |
code-minilm-l6-v1 | rejected | 25 in 1 task | 0.5617 | 884.49 ms | 309.75 s | 4.04 GiB | Crossed both laptop screening limits on its first completed task; the partial quality result is a single-task result. |
general | rejected | 25 in 1 task | 0.5758 | 879.73 ms | 261.88 s | 3.13 GiB | Crossed both laptop screening limits on its first completed task. |
static-retrieval-v1 | previous-default | 100 in 4 tasks | 0.7058 | 578.11 ms | 1.18 s | 199.08 MiB | Default through v1.2.16. Scores slightly lower than potion-code-16m-v2 on the same screening matrix and needs a model download almost four times larger; it builds vectors faster and uses less memory. Still available as an opt-in profile (IVYGREP_MODEL_PROFILE=static). |
Salesforce/SFR-Embedding-Code-400M_R | excluded | - | - | - | - | - | Non-commercial license and custom architecture are incompatible with the shipped default. |
jinaai/jina-code-embeddings-1.5b | excluded | - | - | - | - | - | The 1.5B-parameter custom model exceeds portable laptop download, memory, and inference budgets. |
nomic-ai/CodeRankEmbed | excluded | - | - | - | - | - | Custom NomicBERT architecture is not supported by the portable local inference backend. |
Scope
potion-code-16m-v2 is the selected default. Complete screening matrix: potion-code-16m-v2, static-retrieval-v1. Candidates that crossed a screening limit were stopped after one completed task; those rows are single-task observations and are not comparable with complete rows.