Here is a question worth taking seriously, because the answer tells you what a vector database actually is: can a text file be one?
Yes. And I measured exactly where it stops working, which turns out to be more interesting than the yes.
A vector database has two jobs. Store a pile of vectors — lists of numbers that represent the meaning of something — and, given a new vector, find the ones closest to it. That is the whole contract. Everything else a vector database sells you is an optimisation on job two.
A text file does both. Put one vector per line. To search, read every line, compute the similarity to your query, keep the best ten. That is exact nearest-neighbour search. You have built a vector database, and unlike most of the ones you can pay for, yours is guaranteed to return the actual right answer.
So where does the text file break?
I ran it. Same vectors, same exact search, three containers, cold from disk each time. 1,536 dimensions, which is a common embedding size.
vectors format on disk cold query
1,000 JSON text 34 MB 404 ms
1,000 binary (.npy) 6 MB 2 ms
10,000 JSON text 342 MB 4,104 ms
10,000 binary (.npy) 61 MB 16 ms
50,000 JSON text 1,709 MB 25,447 ms
50,000 binary (.npy) 307 MB 1,059 msA text file is a perfectly good vector database for about a thousand vectors. It is uncomfortable at ten thousand and hopeless at fifty.
But look at why, because this is the part that matters. It is not the searching. The arithmetic is identical in both rows. It is the parsing — the cost of turning the characters 0.0234567 back into a number, two hundred times over per vector, before any comparison can happen. The text file also spends 5.6× the disk space storing the same values.
Change one thing — write the numbers as bytes instead of as text — and the identical brute-force search runs comfortably past a hundred thousand vectors. Same algorithm. Same exact results. No index, no service, no tuning.
What you are actually buying
If brute force gets you to six figures, what does a vector database sell?
An index — and specifically an approximate one. The dominant algorithm is HNSW, published by Malkov and Yashunin in 2016. [1] It builds a navigable graph so a query can hop toward the right neighbourhood instead of touching every vector, and it scales logarithmically rather than linearly.
The first word in “approximate nearest neighbour” is doing real work. These indexes trade correctness for speed. They will sometimes miss the genuine closest match, and they will not tell you when they have.
Algorithms were ranked on the throughput they achieve, as long as the recall@10 is at least 90%.
That is the pass mark from the Big ANN benchmark leaderboard — the competition the field runs against itself. [2] Read it plainly: at the operating point the discipline uses as its bar, finding nine of the true top ten is a pass. That is often fine. It is worth knowing it is the deal.
What approximate actually costs, in the vendors’ own words
You do not have to take a sceptic’s word for this. The clearest examples are published by the vector database companies themselves.
ivfflat.probes defaults to 1. The same document recommends starting at the square root of your list count — which on a million rows is about 32. [4] The shipped default searches a thirty-second of what its own authors advise. FAISS ships a default search effort of 16. [5] And hnswlib warns that the accuracy setting “is currently not saved along with the index” — tune your recall, save, reload in production, and the tuning is silently gone. [6]None of this is scandalous. Defaults have to pick something, and speed demos better than accuracy. But it does mean the recall you are getting is a number you chose by not choosing.
Where the crossover actually is
Here the sceptical case gets unexpected support — from the people selling indexes.
Qdrant ships a default that refuses to build an index on small collections. Their operations documentation gives the reason: “if the number of points is less than 10000, using any index would be less efficient than a brute force scan.” [7] That is not marketing copy; it is a shipped configuration value with the rationale attached.
FAISS — Meta’s vector search library — opens its index-selection guide by telling you not to index. The first branch of the decision tree reads: “If you plan to perform only a few searches… the index building time will not be amortized by the search time. Then direct computation is the most efficient option.” And on exactness: “The only index that can guarantee exact results is IndexFlat… The flat index does not require training and does not have parameters.” [8] There is a companion page titled, without irony, Brute force search without an index, which states that “the best indexing method is to not index at all” and links to notebooks showing you how to do it in plain numpy. [9]
pgvector performs exact search by default and describes it as providing “perfect recall.” Its own troubleshooting advice is to “monitor recall by comparing results from approximate search with exact search” [4] — which is to say, run brute force anyway, as your ground truth.
The tiers, with the number that ends each one
- A text file — up to about 1,000 vectors. Ends in parsing cost, not search cost. Genuinely fine for a personal index of notes or bookmarks.
- A binary file and fifteen lines of numpy — to roughly 100,000–250,000. Ends in memory: a hundred thousand 1,536-dimension vectors is about 614 MB. Exact, no dependencies, nothing to keep in sync.
- SQLite with sqlite-vec — hundreds of thousands. Still brute force in the stable release, and its author is refreshingly clear about why: “most applications of local AI or embeddings aren’t working with billions of vectors… Most of my little data analysis projects deal with thousands of vectors, maybe hundreds of thousands.” [10] Worth knowing it is still pre-1.0, came back in March 2026 from a year-long pause, and its approximate indexes remain in alpha. [11]
- Postgres with pgvector — the same range, exactly, plus everything a database gives you. The reason to move here is usually not vector count. It is that your data already lives in Postgres and you want one system, transactions, and real filtering.
- An approximate index — past a million. FAISS’s own guidance switches to composite indexes above roughly a million vectors. [8] This is where an index stops being premature.
- A dedicated vector database — when the problem is operational, not arithmetic. Multi-tenancy, replication, managed uptime, billion-scale, or a team that should not be maintaining this.
When you genuinely do need one
This article would be dishonest if it only argued one side, so here is the strongest version of the other case.
- Metadata filtering at scale is genuinely hard. Combining “semantically similar” with “and written after March, by this author, in this workspace” is a real research problem. Stanford and Berkeley researchers built ACORN specifically for it and report throughput improvements of two to a thousand times over prior methods at fixed recall [12] — a spread that tells you how badly naive approaches handle it.
- Multivector retrieval breaks the arithmetic entirely. Late-interaction methods produce one embedding per token rather than per chunk. A hundred thousand documents becomes tens of millions of vectors overnight, and every number in this article stops applying. [13]
- Concurrency, replication and someone else’s pager. These are database problems rather than vector problems, and they are real.
The honest landing point is that the sceptic and the vendor agree on roughly where the line sits: somewhere between a hundred thousand and a million vectors, depending on your dimensions and how fast you need answers.
A useful sanity check on scale. BEIR is the standard benchmark suite the retrieval research community assembled to represent diverse, realistic search tasks. Of its nineteen corpora, five are under 100,000 documents and eleven are under a million. [14] That is a statement about what researchers consider representative, not about what is deployed — there is no credible public survey of production corpus sizes, and anyone quoting one is probably quoting a vendor. But it does suggest that “small enough to brute force” describes a lot of real retrieval problems.
The receipts on my own shelf
I have been running four knowledge bases on Pinecone’s free tier since February 2026. The review is here, and the numbers are: 748 files, 2,367 vectors, and $0.
Two thousand three hundred and sixty-seven vectors. At that size the exact search in the table above takes well under a millisecond, and even the text file — the deliberately silly option — would load in about a second and then be instant.
What it cost instead was not money. It was two rebuilds, one of which nothing warned me about, a console that will not tell me how many documents I have, deletion that does not reliably work, and documentation thin enough that I guessed at things. I do not regret learning it. But I was not paying for scale, because I did not have any. I was paying in evenings for infrastructure my corpus never needed.
Run it on your own corpus
The only number that matters is yours. This is the whole benchmark:
import numpy as np, time
M = np.load("vectors.npy") # shape (n_vectors, n_dims), float32
M /= np.linalg.norm(M, axis=1, keepdims=True)
q = M[0] # any query vector, normalised
t = time.perf_counter()
scores = M @ q # every comparison, exactly
top10 = np.argpartition(-scores, 10)[:10]
print(f"{len(M):,} vectors in {(time.perf_counter()-t)*1000:.2f} ms")If that prints a number you are happy to wait for, you have your answer, and it is exact. If it does not, you now know what you are shopping for and roughly what recall will cost you.
A vector database stores vectors and finds the nearest ones. A text file does that, for about a thousand vectors, until parsing costs overwhelm it. A binary file and one line of numpy does it exactly, to a few hundred thousand. SQLite does it. Postgres does it. All of them return the right answer every time, because none of them are guessing.
What you buy with a dedicated vector database is an approximate index — genuinely necessary past a million vectors, genuinely valuable for filtered and multivector search, and a trade you are making whether or not anyone told you. Qdrant will not build one below ten thousand points. FAISS opens its guide by suggesting you do not index at all. pgvector tells you to check its answers against brute force.
Measure your corpus first. Most people never reach the point where the index earns its complexity, and the ones who do would rather find out from a number than from a rebuild.
Sources & References
ivfflat.probes default of 1 against its own sqrt(lists) recommendation; the advice to monitor recall against exact search. github.com/pgvector/pgvectorfaiss/impl/HNSW.h — SearchParametersHNSW default efSearch = 16. github.com/facebookresearch/faissHow the measurements were made. Every timing in this article is first-party and reproducible with the fifteen lines above. Vectors were randomly generated and L2-normalised at 1,536 dimensions, float32, exact top-10 cosine similarity, one query at a time with no batching, run on four ARM cores with 3 GB of RAM — modest hardware, deliberately. Your laptop will be faster. The text-versus-binary timings are cold from disk and include parsing. The ratio between them, which is the finding that matters, holds regardless of machine.
A note on this article. An earlier version was a generic explainer with no sources and no numbers. It was rewritten in August 2026 after a fact-check found nothing in it that could be checked.