Vector Databases at scale: Real time updates

In February 2026, Notion wrote that its vector search had grown to 10 times its scale in two years, at a tenth of the cost.^[1] At launch, in November 2023, it could onboard only a few hundred workspaces a day.

Engineers who keep a vector database up to date face three problems:

Every document has to be chunked and embedded before it can be found.
An edit to one word sends the whole document to be embedded again.
Chunking and embedding run on different machines, so the work is paid for twice.

In this blog we will see how Notion keeps its vector database up to date, and the changes that cut its cost to a tenth.

1. Vector search

How does Notion AI find the pages that answer your question? It doesn't match your words. It matches what they mean.

Notion uses vector search to answer user queries. The query embedding and the document embeddings are semantically close to each other, so the relevant documents are retrieved during search.

Each point is a page, and the star is the query, Q3 financial goals. The dotted circle holds the four pages nearest to it, all in the finance cluster. The design pages lie outside the circle and are not returned.

A query for "team meeting notes" can then find a page titled "group standup summary". A keyword search would miss it.

2. Two paths into the vector database

Where do those embeddings come from? A page can't be found until it's embedded. Every page in Notion needs one.

The documents are chunked using a batch job running on Apache Spark. Large documents are broken into chunks, each of which has its own embedding. These embeddings are then bulk loaded into the vector database.

When a document is edited, the updated chunk is passed to a Kafka message queue. The queue is later processed to update the vector database.

One page of four lines, played step by step. It is cut into four chunks and stored as four vectors, then two added words go through the Kafka message queue and only the two chunks they landed in get new vectors. Last, the tag changes to permissions: public on all four rows, and the vectors stay as they are.

Take a page of four lines, tagged "permissions: Herman". The batch job cuts it into four chunks, and each chunk gets its own vector. Add two words to it, "exactly how", and the edit goes through Kafka to the vector database.

With Airflow scheduling and Spark job tuning, Notion's daily onboarding capacity grew 600 times. It cleared its Q&A waitlist by April 2024.

3. Serverless

What does it cost to onboard 600 times more workspaces? The embeddings workload ran on dedicated hardware, billed for as long as it was up.

The engineers at Notion tuned Apache Spark to process more than tens of thousands of workspaces daily, and they moved to a serverless architecture to reduce costs by 50%.

Serverless hardware is billed differently: it separates storage from compute, and charges for usage. Notion moved there in May 2024, and cut cost by 50% from peak usage.

In late 2024 it moved its search engine again, to turbopuffer. That cut its search engine spend by 60%.

4. A hash for each chunk

Was there less work, though? No. Moving to turbopuffer cut the price of the work, not the amount of it. On every edit, the whole page was chunked, embedded and uploaded again, even when one character changed.

Your two words cost four embeddings!

To speed up real-time processing, whenever a document has a text edit, only the relevant chunks are picked up for updates.

This is done by hashing the chunks, where a change in the text changes the hash value of the chunk. For a single document, if the hash value of the chunk has not changed, then that chunk does not need an update.

Those hash values are 64-bit xxHash. Notion caches the hashes in DynamoDB, one record per page.

Your page's record holds four hashes, and your two words changed the last two. Two embeddings are made instead of four.

5. Metadata apart from text

Is the text all that a chunk holds? No. A chunk also carries the page's metadata, such as who may read it.

If the document permissions are changed, which is basically changing the metadata of the document, every chunk of the document requires an update, since the metadata of the document is propagated across chunks.

Notion solved this problem by splitting every chunk into two pieces, the metadata and the textual content. Whenever there was a metadata change, only the metadata would be updated, speeding up permission changes in any document.

This resulted in a 70% reduction in data volume, which saved money on both embedding API costs and vector database write costs.

Notion keeps a second hash per chunk, over its metadata fields. When the text hashes match and the metadata hashes don't, it skips the embedding and sends the vector database a PATCH.

That PATCH is all your page needs when its tag changes to "permissions: public". Four rows get the new tag, and no embedding is made.

One edit on its way to the vector database, played twice. A Notion page sends its edit to a Kafka queue, then to a check of the chunks' hashes in DynamoDB. First two words are added: chunks 3 and 4 are embedded again and chunks 1 and 2 are skipped, two embeddings instead of four. Then the permissions change: all four chunks get a metadata update, and nothing is embedded again.

6. Ray

What did the embeddings themselves cost? Notion chunked on Spark, then paid an embeddings API provider per token. Its engineers call this the "Double Compute problem".

Finally, the engineers at Notion decided to move to Ray, which is a product used by many companies, including OpenAI initially, for large-scale data processing.

The benefit of Ray is that CPU and GPU operations can be performed in a single node. So chunking, which runs on the CPU, and generating embeddings, which runs on the GPU, can both run on the same node. That reduces communication cost.

One Ray node holds three CPUs, a buffer and two GPUs. Partition 4 is on a CPU, partition 3 waits in the buffer, and partitions 1 and 2 are on the GPUs. A finished partition lands in the tray at the lower left.

Ray also lets Notion run open source embedding models itself, instead of paying per token. Notion began the move in July 2025, and anticipates a 90% reduction in its embeddings infrastructure costs. That number is a forecast.

The final result is that Notion is able to generate 10x the number of embeddings at one-tenth the cost.

Summary

Notion's vector search returns the pages whose embeddings are closest to the query's.
A Spark batch job chunks and embeds pages in bulk, and edits reach the vector database through Kafka.
A move to serverless cut cost by 50%, and turbopuffer cut search engine spend by 60%.
A hash of each chunk's text means only changed chunks are embedded again.
A second hash of the metadata turns a permission change into a PATCH. The two hashes cut data volume by 70%.
Ray runs chunking on CPUs and embedding on GPUs in one node, for an anticipated 90% cut in embeddings cost.

Sources

Two years of vector search at Notion: 10x scale, 1/10th cost, Preeti Gondi and others, Notion blog, 19 February 2026.
Ray Data: Scalable Data Processing for AI Workloads, Ray documentation.