pgvectorscale vs pgvector: StreamingDiskANN vs HNSW in Postgres (2026)

Most teams that put embeddings in Postgres start with pgvector and an HNSW index, and for the first few hundred thousand rows that is exactly the right call. Then the table grows, the index no longer fits in RAM, a multi-tenant WHERE clause starts returning three rows instead of ten, and someone asks whether a separate vector database is now unavoidable. Often it is not, but the answer depends on understanding what actually differs between pgvectorscale vs pgvector, because they are not rivals in the way the headline suggests.

pgvector provides the vector type, the distance operators and two index types. pgvectorscale is a second extension that depends on pgvector and adds a third index, diskann, which implements what its authors call StreamingDiskANN, plus a compression scheme called Statistical Binary Quantization (SBQ). This article walks through the mechanisms, the memory and build-time arithmetic, the filtering behavior that most often decides the choice, runnable SQL for both, and the places where each one hurts. Vendor benchmark claims are labelled as such, and anything I could not verify is marked.

What this covers: how each index works, how filtering differs, a sizing worked example, tuning knobs, hosting and maturity caveats, a decision matrix, and a checklist for choosing.

Context and Background

Postgres became a credible vector store through pgvector, an open-source extension that stores embeddings in a vector column and answers nearest-neighbor queries with operators such as <-> (L2), <=> (cosine) and <#> (negative inner product). Without an index, a query is an exact scan: the planner reads every row and sorts by distance. That is perfectly accurate and scales linearly, which is fine at 50,000 rows and painful at 50 million.

To go faster, pgvector offers two approximate nearest neighbor (ANN) indexes. IVFFlat partitions vectors into lists and probes only the lists nearest the query. HNSW, the Hierarchical Navigable Small World graph from Malkov and Yashunin (arXiv 1603.09320), builds a multi-layer proximity graph and walks it greedily. According to the pgvector README, IVFFlat builds faster and uses less memory but has lower query performance than HNSW, which is why HNSW became the default choice for most production workloads. If you want the broader landscape of how these index families compare, our deep dive on HNSW vs DiskANN vs IVF-PQ index internals and quantization covers the algorithms in isolation; this post stays inside Postgres.

The two limits that drive people toward alternatives are memory and filtering. HNSW performs best when the graph is resident in memory; the README notes that index builds are much faster when the graph fits in maintenance_work_mem, and Postgres prints a notice when it no longer does. And because approximate indexes in Postgres apply WHERE filters after the index scan, a selective filter can starve the result set. The README is blunt about this: with the default hnsw.ef_search of 40, a condition matching 10 percent of rows yields about four rows on average.

pgvectorscale, developed by Timescale (now branded TigerData), is the extension aimed at those two limits. Per its repository it is written in Rust using the pgrx framework and released under the PostgreSQL License. The same README describes the project as “still at an early stage”, a phrase worth remembering when we reach the recommendations. For a Postgres-versus-dedicated-engine framing rather than an index-versus-index one, see our pgvector vs Qdrant hybrid search and reranking ADR.

One framing point before the mechanisms. “pgvector vs pgvectorscale” is a slightly malformed question, because installing pgvectorscale with CREATE EXTENSION vectorscale CASCADE also installs pgvector. You keep the same table, the same column type and the same operators. What you choose is which index sits on top of that column, and sometimes you keep both.

How pgvector HNSW and pgvectorscale StreamingDiskANN Actually Differ

pgvector HNSW and pgvectorscale’s StreamingDiskANN are both graph indexes that answer nearest-neighbor queries by walking edges toward the query. They differ in graph shape (layered versus single layer), in what is stored in the index pages (full vectors versus compressed codes), and in how a scan terminates (a fixed candidate budget versus a stream that continues until Postgres says stop).

Figure 1: Two query paths through the Postgres planner. HNSW fetches a fixed candidate set and then filters; diskann walks a compressed single-layer graph, streams results, and rescores from the heap.

Figure 1 shows the two paths side by side. In both cases the planner sees an ORDER BY embedding <=> $1 LIMIT k query and chooses an index scan if the operator class matches. The divergence is inside the index access method.

pgvector HNSW: a layered graph that wants to live in memory

An HNSW index stores each vector as a node. Upper layers are sparse “express lanes” with long edges; layer zero contains every node. A search enters at the top, greedily descends, and at layer zero maintains a priority queue of size ef_search (default 40 in pgvector) from which the final top-k is taken. Two build parameters shape the graph: m, the maximum connections per node per layer (default 16), and ef_construction, the candidate list size while inserting (default 64). Raising either improves recall at a given ef_search and costs build time and index size.

Everything the index needs to compute distances is stored in the index itself: the full-precision vector for each node plus its neighbor lists. That is why HNSW queries are fast when the index is cached: no heap fetch is needed to rank candidates. It is also why the index is large. For a vector(768) column, each indexed element carries 3,072 bytes of float data before neighbor lists and page overhead, so the HNSW index is at least as big as the vector data itself.

pgvector can reduce this footprint through type choice. The README documents halfvec (16-bit floats) indexable up to 4,000 dimensions, bit up to 64,000 and sparsevec up to 1,000 non-zero elements, against a 2,000-dimension indexing limit for plain vector. Casting to halfvec inside an expression index halves the stored precision cost with typically small recall impact; I recommend measuring on your own embeddings rather than assuming.

StreamingDiskANN: a single-layer graph with compressed codes

DiskANN, from Microsoft Research (Subramanya et al., NeurIPS 2019), replaces the hierarchy with a single-layer graph built so that long-range edges exist alongside short ones, which reduces the number of hops and therefore the number of random page reads. It was designed so the graph and full vectors live on SSD while only compressed codes sit in RAM. Timescale’s description says StreamingDiskANN implements DiskANN with their own improvements and stores part of the index on disk rather than requiring it all in memory.

The build parameters mirror the HNSW ones in spirit. From the pgvectorscale README: num_neighbors (default 50) is the maximum neighbors per node, search_list_size (default 100) is the greedy search width during construction, and max_alpha (default 1.2) controls how aggressively the pruning rule keeps long edges. Notice the neighbor count: 50 versus HNSW’s 16 per layer, which is possible because each stored edge points at a node whose representation is small.

The second difference is the storage_layout. The default, memory_optimized, stores SBQ-compressed vectors in the index; plain stores uncompressed vectors. With the default layout the graph walk computes approximate distances on compact binary codes, then rescores a configurable number of top candidates with full-precision vectors fetched from the table.

The “streaming” part

The word streaming is not marketing fluff; it describes the scan contract. Timescale’s write-up describes a get_next() style interface: the index returns the next-closest candidate each time Postgres asks, and Postgres keeps asking until the executor has collected LIMIT rows that survive the remaining filter quals. There is no ef_search-style preset cutoff. Compare this with HNSW, where the scan collects a fixed candidate list and hands it back; if the filter rejects most of it, you are short.

This distinction matters less for unfiltered top-k queries and a great deal for filtered ones, which is why we treat filtering in its own section below. For a broader treatment of the pre-filter versus post-filter design space, including ACORN-style approaches, see our post on filtered vector search with pre-filter, post-filter, HNSW and ACORN.

Where the planner fits

Both indexes are ordinary Postgres index access methods, so they participate in cost-based planning, WAL logging and crash recovery, and the same EXPLAIN workflow applies. One practical consequence: pgvectorscale operates on pgvector’s vector type, and its 0.9.1 release notes say operator classes now bind explicitly to pgvector’s schema and that DiskANN indexes can no longer be created on columns without a valid vector(N) typmod. Declare your dimension on the column. Doing so is good hygiene anyway.

Statistical Binary Quantization, Memory Arithmetic and Build Cost

SBQ turns each float dimension into one or two bits using the statistics of your own data rather than a fixed threshold of zero. A 768-dimension float32 vector occupies 3,072 bytes; at two bits per dimension its SBQ code occupies 192 bytes, a 16x reduction, and at one bit 96 bytes, a 32x reduction. The index then ranks most candidates using cheap bit operations and rescoring fixes the ordering at the top.

Figure 2: The SBQ pipeline. Per-dimension statistics are learned, vectors are encoded to two-bit codes, the graph walk uses Hamming distance, and the top candidates are rescored with the full vector from the heap.

How SBQ works

Plain binary quantization (BQ) sets a bit to 1 if the value is above 0.0 and compares codes with XOR plus popcount. That throws away a lot when the dimension’s distribution is not centered on zero, which is common for real embedding models. Timescale’s explanation of SBQ proceeds in steps. First, a pass over the data computes a mean per dimension and uses it as the threshold instead of zero. Second, for the two-bit variant, each dimension’s z-score (distance from the mean in standard deviations) is split into three regions encoded as 00, 01 and 11, so that adjacent regions differ by exactly one bit and Hamming distance respects ordering.

The pgvectorscale README states the default: num_bits_per_dimension is 2 when the vector has fewer than 900 dimensions and 1 otherwise. Timescale’s post gives one data point for the benefit of the second bit on a 768-dimension dataset: recall rose from 96.5 percent to 98.6 percent when moving from one-bit to two-bit encoding. That is a single vendor-reported figure with no dataset name disclosed in the article, so treat it as directional. The exact z-score cut points are not specified in the sources I could access, and I will not guess them.

Why does statistical thresholding matter? Most modern text-embedding models produce vectors where individual dimensions have small, non-zero means and different variances. A zero threshold wastes bit entropy on dimensions that are nearly always positive. Centering per dimension spreads the codes across the bit space, which preserves more neighbor ordering. That reasoning is mechanistic and plausible, and it is consistent with Timescale’s description, but the measured gains will vary by embedding model.

Rescoring is not optional in practice

Compressed distance is an approximation. The query-time GUC diskann.query_rescore (default 50 in the README) sets how many candidates get re-ranked using full-precision vectors; 0 disables rescoring. Rescoring means heap fetches, so it trades latency for recall in a way HNSW does not have to, because HNSW already holds full vectors in the index. The 0.8.0 release notes mention that rescoring can be disabled, which is useful if you only need coarse candidates for a downstream reranker.

The other query-time knob is diskann.query_search_list_size (default 100), the number of candidates considered during graph search. In practice you tune these two together: widen the search list to find the right neighborhood, raise rescore to order it correctly.

A worked sizing example

The following numbers are my own arithmetic, not benchmark results. Assume 10 million vectors from a 768-dimension embedding model.

Item	Calculation	Approx. size
Raw `vector(768)` data	10M x 768 x 4 B	30.7 GB
Raw `halfvec(768)` data	10M x 768 x 2 B	15.4 GB
SBQ 2-bit codes	10M x 192 B	1.9 GB
SBQ 1-bit codes	10M x 96 B	0.96 GB
diskann neighbor lists, rough	10M x 50 x ~6 B	~3 GB (estimate)

The neighbor-list line assumes roughly 6 bytes per stored edge, matching the size of a Postgres tuple identifier. I have not verified pgvectorscale’s on-disk edge encoding, so read it as an order-of-magnitude estimate only. Even so, the shape of the result is clear: an HNSW index over vector(768) at this scale needs on the order of 30 GB or more of cache to run at its best, while a memory-optimized diskann index plausibly keeps its hot working set in single-digit gigabytes, with full vectors read from the heap only during rescoring.

That asymmetry is the whole economic argument. If your instance has 64 GB of RAM and also hosts the application tables, a 31 GB HNSW graph competes with everything else for shared buffers and the OS page cache. Cache misses turn into random 8 KB reads, and HNSW’s pointer-chasing pattern makes each hop a potential disk read. A compressed index leaves headroom.

Build time and the maintenance_work_mem trap

The pgvector README says HNSW builds are much faster when the graph fits in maintenance_work_mem. When it does not, the build spills to a slow path and prints a notice. Parallel HNSW builds exist (added in pgvector 0.6.0 per the changelog), controlled by max_parallel_maintenance_workers, which defaults to 2 plus the leader; the README example raises it to 7 and notes max_parallel_workers (default 8) may need to rise too.

For pgvectorscale, the README says parallel builds happen only when four conditions hold: the feature is compiled in, the table has no labels, the storage layout is SBQ, and the row count reaches diskann.min_vectors_for_parallel_build (default 65,536). Label-filtered indexes therefore build single-threaded. Index builds can also be memory-intensive, and 0.8.0 made builds respect maintenance_work_mem. The 0.9.0 notes added concurrent index builds, the equivalent of CREATE INDEX CONCURRENTLY, which matters for production tables you cannot lock.

I cannot give you a verified build-time ratio between the two. Timescale’s published material did not report build times in the sources I read, and I have seen no independent head-to-head I would cite. Plan to measure: build both indexes on a 1 million row sample, record wall time and index size with pg_relation_size, then extrapolate cautiously.

Filtered Search: Iterative Scans, Streaming and Labels

Filtering is where pgvectorscale vs pgvector decisions are usually won or lost, because real RAG and recommendation systems almost never run unfiltered top-k. Tenant ids, document types, language codes and access-control lists all become WHERE clauses, and an approximate index that ignores them until the end can silently return too few rows.

Figure 3: Filtered query behavior. HNSW returns its candidate batch and relies on iterative scans to ask for more; StreamingDiskANN pulls candidates one at a time until the limit is met.

The post-filter problem in pgvector HNSW

Walk through the README example. You ask for 10 rows where category_id = 123, and that predicate matches 10 percent of the table. HNSW returns its ef_search candidates (40 by default). On average 10 percent survive the filter, so you receive about four rows, not ten. At 1 percent selectivity the same arithmetic gives 0.4 rows per query, which in practice means many queries return nothing.

Raising hnsw.ef_search helps linearly but not forever, and it makes every query slower, including unfiltered ones. This is the failure mode behind a lot of “my RAG system sometimes finds nothing” tickets, and it is invisible in unit tests that use a small, uniformly distributed dataset.

pgvector 0.8 iterative index scans

pgvector 0.8.0 (released 2024-10-30 per the changelog) added iterative index scans, the direct answer to the above. With SET hnsw.iterative_scan = relaxed_order (or strict_order), if the first batch of candidates does not yield enough rows after filtering, the scan continues into more of the graph. It stops when it has enough rows or reaches hnsw.max_scan_tuples, which defaults to 20,000. strict_order guarantees results are in exact distance order; relaxed_order may return them slightly out of order in exchange for better recall, and the README suggests that a materialized CTE can re-sort if you need strict ordering. There is a related hnsw.scan_mem_multiplier (default 1, a multiple of work_mem) bounding scan memory. IVFFlat has its own ivfflat.iterative_scan, supporting relaxed_order, and stops at ivfflat.max_probes.

Iterative scans make pgvector HNSW filtered search far more robust than it was. They still operate by repeatedly widening the same graph walk, so a very selective filter (a tenant with 200 rows in a 50-million-row table) can exhaust max_scan_tuples before finding ten matches. For that pattern, the README suggests other techniques: index the filter column for low-percentage conditions, use partial indexes when there are few distinct values, or partition when there are many.

StreamingDiskANN’s two filtering mechanisms

pgvectorscale offers two things here. The first is the streaming model already described: the diskann scan produces candidates on demand, so generic post-filters on any column keep pulling until LIMIT is satisfied. Timescale’s post asserts this avoids the accuracy degradation of fixed-candidate HNSW; the argument is by design, and the post reports no measured HNSW comparison. Streaming is not magic either. With a very selective filter the scan must still traverse many candidates, so latency grows, and the compressed distance means more rescoring work.

The second is label-filtered search, added in pgvectorscale 0.7.0 (the release notes call it Filtered DiskANN) and followed by 0.7.1, which fixed poor recall when using label filters. It builds on the Filtered-DiskANN research (Gollapudi et al., WWW 2023), where labels are incorporated into the graph construction so that traversal can respect them. In SQL you add a SMALLINT[] column to the index and query with the array-overlap operator:

-- labels is SMALLINT[], values in -32768..32767
CREATE INDEX ON documents
  USING diskann (embedding vector_cosine_ops, labels);

SELECT id, title
FROM documents
WHERE labels && ARRAY[1, 3]
ORDER BY embedding <=> $1
LIMIT 10;

The README attaches important limits. Labels must fit smallint. Other WHERE conditions beyond the label overlap are applied as post-filters after the vector search. Null labels are treated as empty arrays. Label-filtered indexes cannot use parallel builds. Because labels are a small integer vocabulary, this is a good fit for categories, tenant groups or language codes you can map to integers, and a poor fit for high-cardinality tenancy where every customer is its own label.

Choosing among the filter strategies

The strategies stack in a rough order of how much work they ask of you. Iterative scans on pgvector are one SET away and need no schema change. Partial indexes give exact, fast results for a handful of hot values but multiply index count. Partitioning handles many tenants cleanly and isolates their indexes, at the cost of planner and operational complexity. Label-filtered diskann sits in between: one index, small integer vocabularies, and a vendor-extension dependency.

A useful test is selectivity. If your typical filter keeps more than about 20 percent of rows, pgvector HNSW with iterative scans is probably enough. If it keeps 1 to 10 percent, measure result counts and recall under load, with iterative scans on, before choosing. Below 1 percent, consider partitioning, partial indexes or label filtering instead of relying on any traversal-time filtering. These thresholds are my rule of thumb from the arithmetic above, not a published result.

Runnable SQL, Tuning and What the Published Benchmarks Say

This section puts both indexes on the same table, shows the parameters that matter, and then reads the one widely cited vendor benchmark critically.

Side-by-side setup

CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE;  -- also installs pgvector

CREATE TABLE docs (
  id        bigserial PRIMARY KEY,
  tenant_id int NOT NULL,
  labels    smallint[],
  body      text,
  embedding vector(768) NOT NULL
);

-- Option A: pgvector HNSW (full precision, graph wants RAM)
SET maintenance_work_mem = '8GB';
SET max_parallel_maintenance_workers = 7;
CREATE INDEX docs_hnsw ON docs
  USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

-- Option A2: halfvec expression index, half the stored precision
CREATE INDEX docs_hnsw_half ON docs
  USING hnsw ((embedding::halfvec(768)) halfvec_cosine_ops);

-- Option B: pgvectorscale StreamingDiskANN with SBQ (default layout)
CREATE INDEX docs_diskann ON docs
  USING diskann (embedding vector_cosine_ops)
  WITH (num_neighbors = 50, search_list_size = 100, max_alpha = 1.2);

The values shown for m, ef_construction, num_neighbors, search_list_size and max_alpha are the documented defaults; they are written out for clarity, not as tuned recommendations. A halfvec expression index requires the query to use the same cast, otherwise the planner will not match it.

Query-time settings

-- HNSW: widen the search and turn on iterative scans for filtered queries
SET LOCAL hnsw.ef_search = 100;
SET LOCAL hnsw.iterative_scan = relaxed_order;
SET LOCAL hnsw.max_scan_tuples = 20000;

SELECT id FROM docs
WHERE tenant_id = 42
ORDER BY embedding <=> $1 LIMIT 10;

-- diskann: widen graph search and rescoring
SET LOCAL diskann.query_search_list_size = 200;
SET LOCAL diskann.query_rescore = 100;

SELECT id FROM docs
WHERE tenant_id = 42
ORDER BY embedding <=> $1 LIMIT 10;

SET LOCAL scopes the change to one transaction, which is the right tool when different endpoints in the same application need different recall targets. Both extensions document per-session and per-transaction settings. Remember the diskann ordering caveat from the README: results use relaxed ordering and may be slightly out of order, so wrap the query in a materialized CTE and re-sort if strict ordering matters downstream.

Measuring recall honestly

Recall@k for an ANN index is the fraction of the true k nearest neighbors, found by an exact scan, that the index also returns. Build the ground truth once with the index disabled (SET enable_indexscan = off forces a sequential scan), store the exact top-10 ids for a few thousand real queries, then sweep ef_search or query_search_list_size and plot recall against p95 latency. Report the filtered and unfiltered curves separately; a single unfiltered curve will flatter HNSW, and a single synthetic filter will flatter whichever index suits it.

What the one big benchmark does and does not show

The most-cited number comes from Timescale’s own post, which compares PostgreSQL with pgvector and pgvectorscale against Pinecone on 50 million Cohere 768-dimension embeddings. It reports, against Pinecone’s storage-optimized s1 index at 99 percent recall, 28x lower p95 latency and 16x higher query throughput, and against the performance-optimized p2 index at 90 percent recall, 1.4x lower p95 latency and 1.5x higher throughput. It also gives monthly costs of about $835 for self-hosted Postgres on AWS EC2 versus $3,241 (s1) and $3,889 (p2) for Pinecone.

Read it with these caveats, all stated or implied by the post itself. It is vendor-published by a company that sells a competing hosted service. The comparison target is Pinecone, not pgvector HNSW; the post reports no HNSW recall, latency, build-time or index-size numbers. Hardware specifications, instance type and index parameters were not reported in the post I read, and a figure caption refers to p1 where the text says p2. Pinecone’s older pod indexes offered no accuracy tuning, which the post uses to justify comparing at different recall levels. The cost figures exclude operational labor. None of that makes the result wrong; it makes it a claim about pgvectorscale versus a pod-based managed service, not evidence that diskann beats HNSW on your table.

That last point is the honest summary of the evidence. The mechanism-level argument (compressed graph, smaller working set, streaming filters) is sound and I find it convincing. A quantified head-to-head between pgvector HNSW and pgvectorscale diskann at matched recall, on your embedding model and filter mix, is something you should produce yourself. Independent third-party comparisons such as ANN-Benchmarks cover graph algorithms generally, but I did not verify that they include pgvectorscale, so do not assume they do.

A note on rescoring and the heap

Because the memory-optimized layout stores only codes in the index, rescoring reads full vectors from the table heap. If the heap is cold, each rescored candidate is a random read. For a workload with query_rescore = 50 and a 30 GB heap on a 16 GB machine, the rescoring step can dominate latency. Two mitigations: keep the heap hot with pg_prewarm, or use the plain storage layout when memory is plentiful and you want to skip compression error entirely (note that the README says inner-product indexes are incompatible with plain storage).

Hosting, Versions and Maturity: The Decision Matrix

Index internals matter, but the decision is often made by where Postgres runs and how much operational risk you can carry. This section collects what I could verify about versions and availability, then lays out a decision framework.

Figure 4: A decision flow for choosing between pgvector HNSW and pgvectorscale StreamingDiskANN. Extension availability and filter shape gate the choice before raw size does.

Versions and licensing, as verified this run

pgvector’s README shows 0.8.7 as the install tag at the time of reading, while its changelog lists 0.8.6 as the last dated release (2026-07-29) and 0.8.7 as unreleased; check which you are actually running. Notable fixes in the 0.8.x line include possible HNSW index corruption during vacuuming (fixed in 0.8.3) and an hnsw graph not repaired error during vacuuming (0.8.4), so staying current on patch releases is not optional if you update or delete rows heavily. The README I fetched did not itself name the license; the repository carries a LICENSE file, and I recall it being the PostgreSQL License, but verify before redistributing.

pgvectorscale’s releases page lists 0.9.1 as the latest, with the page showing day and month but not year. 0.9.0 added PostgreSQL 18 support, dropped PostgreSQL 13 and added concurrent index builds; 0.9.1 added security hardening against malformed vector data and type confusion, and requires ALTER EXTENSION vectorscale UPDATE TO '0.9.1', which may require dropping and recreating pre-existing invalid indexes. I could not confirm the year of 0.9.1 from the page, so I will not state one. The repository states a PostgreSQL License and Rust implementation via pgrx.

Hosting availability

The pgvectorscale README documents three routes: a pre-built TimescaleDB Docker image, a source build with Rust and cargo-pgrx, and Timescale Cloud, where you enable the extension on a service. It notes that macOS on Intel is unsupported for source builds. The README does not list other managed Postgres providers, and I did not verify whether AWS RDS, Google Cloud SQL, Azure or other vendors allow the vectorscale extension. This is the single most important thing to check, because an extension that cannot be installed on your managed service is not an option no matter how good its index is. Managed providers broadly allow pgvector, but confirm your provider and version.

The decision matrix

Situation	pgvector HNSW	pgvectorscale diskann	Notes
Under ~1M vectors, index fits in RAM	Strong default	Unnecessary	Simplicity wins
10M+ vectors, RAM-constrained	Large cache needed; halfvec helps	Compressed codes shrink working set	Measure rescoring cost
Multi-tenant, few category labels	Iterative scans or partial indexes	Label-filtered index fits	Labels limited to smallint
Many distinct tenants, strict isolation	Partitioning plus per-partition HNSW	Streaming post-filter, slower when selective	Partitioning often best
Managed Postgres without the extension	Available widely	Not an option	Verify per provider
Heavy writes, frequent deletes	Mature, patched vacuum paths	Newer, “early stage” per README	Test update behavior
Strict exact ordering needed	strict_order mode	Relaxed ordering, re-sort needed	CTE re-sort for diskann
Embeddings above 2,000 dims	halfvec to 4,000, bit to 64,000	Matryoshka via num_dimensions	Check model dims

The row on write behavior deserves emphasis. The pgvectorscale README does not describe how inserts and updates behave after index creation, and its changelog shows crash fixes for concurrent inserts and scans with NULL values in 0.8.0. That is a normal maturity trajectory, not a condemnation, but it argues for a staging soak test with your real write pattern before production.

When to run both

Nothing prevents keeping an HNSW index on a small hot partition (recent documents, say) and a diskann index on the large historical partition. Partitioned tables can carry different index types per partition, and the planner handles the append. This is more moving parts than most teams need, but it is a legitimate way to get HNSW’s full-precision latency where it fits in cache and diskann’s footprint where it does not. If you outgrow Postgres entirely, our comparison of Milvus 3.0 and 2.6 lake-native vector search shows what a dedicated engine adds.

Trade-offs, Gotchas, and What Goes Wrong

Compression error is a recall tax you pay up front. SBQ is lossy. Two-bit codes on 768 dimensions keep far more information than one-bit codes, but they are still approximations, and the recall ceiling of the graph walk is bounded by how well the codes preserve neighbor ranks. Rescoring recovers ordering among the candidates the walk surfaces; it cannot recover a true neighbor that the compressed walk never reached. If your measured recall plateaus below target as you raise query_rescore, widen query_search_list_size or switch to plain storage.

Ordering is relaxed. The pgvectorscale README warns that results may be slightly out of order. For retrieval feeding a reranker this is irrelevant. For a user-facing “nearest first” list, or for pagination with OFFSET, it produces visible inconsistencies. Re-sort in a materialized CTE.

Selective filters still hurt. Streaming removes the hard candidate cutoff but not the cost of walking many candidates to find rare matches. A filter keeping 0.1 percent of rows turns a 5 ms query into something that may need to traverse a large fraction of the graph. Both extensions need an escape hatch (partitioning, partial indexes or labels) for extremely selective predicates.

Label limits bite late. smallint[] labels are fine for dozens of categories. A team that designs around labels and later needs per-user access control with hundreds of thousands of principals will find the model does not stretch. Decide whether your filter vocabulary is small and stable before committing, since changing the index definition means a rebuild.

Build memory surprises. For HNSW, an index that exceeds maintenance_work_mem slows the build sharply. For diskann, index builds can be memory-intensive per the README, and label indexes forgo parallelism. Both are solved by scheduling builds on a larger instance or replica and by using concurrent builds in production, but nobody enjoys discovering this at 2 a.m. during a migration.

Operator and type constraints. pgvectorscale requires a typed vector(N) column as of 0.9.1, does not support UNLOGGED tables (index creation errors with ambuildempty: not yet implemented), does not index null vectors and cannot do inner-product indexes with plain storage. pgvector’s limit is 2,000 dimensions for indexing plain vector columns, so 3,072-dimension models need halfvec, dimensionality reduction or Matryoshka truncation.

Upgrade coupling. pgvectorscale depends on pgvector. A pgvector upgrade changes the schema objects the extension binds to, and 0.9.1 explicitly tightened that binding. Upgrade both in staging first, and read release notes for invalid-index warnings.

Vacuum and corruption fixes. pgvector’s own 0.8.3 and 0.8.4 notes record HNSW vacuum bugs including possible index corruption. If you run an older 0.8.x on a table with churn, the practical advice is simple: upgrade, then REINDEX if you suspect damage.

The benchmark trap. Comparing the two at default settings measures defaults, not capability. HNSW’s ef_search of 40 and diskann’s search list of 100 with rescore 50 sit at different points on different curves. Always compare at equal measured recall.

Practical Recommendations

Start from the boring default. If your table is under a few million rows and the HNSW index fits comfortably in memory, use pgvector HNSW, keep hnsw.iterative_scan = relaxed_order on for filtered queries, and move on. You gain a mature, widely available index and one fewer extension to patch.

Reach for pgvectorscale when one of three things is true: the HNSW working set no longer fits your memory budget, your filters are mid-selective and drawn from a small stable vocabulary that maps cleanly to labels, or you want the streaming scan contract for generic post-filters. In each case, you must also be able to install the extension where you run Postgres.

Whatever you pick, treat the decision as reversible. Both indexes sit on the same column, so you can build the second beside the first and switch by dropping the one you no longer need, without changing application SQL beyond the session settings.

Confirm your managed provider allows vectorscale before benchmarking anything.
Declare vector(N) dimensions explicitly on the column.
Build both indexes on a 1M-row sample; record build time, pg_relation_size, and recall@10 against exact scans.
Test with your real filters, including the most selective tenant, not a synthetic uniform filter.
Sweep ef_search or query_search_list_size and query_rescore, and compare at equal recall.
Soak-test writes, updates and deletes for at least a day on staging.
Pin and patch both extensions; read changelogs for vacuum and corruption fixes.
Revisit annually: both projects change quickly.

Frequently Asked Questions

Is pgvectorscale a replacement for pgvector?

No. pgvectorscale is an additional extension that depends on pgvector and installs with it when you use CREATE EXTENSION vectorscale CASCADE. You keep the vector type and the distance operators and add a diskann index access method. You can run both index types on the same column and compare them, and pgvector’s HNSW and IVFFlat indexes remain available. The practical choice is which index serves each query pattern, not which extension survives.

Is StreamingDiskANN faster than pgvector HNSW?

It depends on recall target, memory and filters, and I know of no independent head-to-head I can cite. Timescale’s published benchmark compares against Pinecone, not HNSW, and is vendor-run. Mechanistically, diskann should win when HNSW’s graph exceeds RAM or when filters starve HNSW’s candidate list, and HNSW should win when everything is cached and full-precision vectors avoid rescoring. Benchmark your own embeddings, filters and recall target.

What is statistical binary quantization?

SBQ is pgvectorscale’s compression scheme. It learns a mean and standard deviation per dimension, then encodes each value relative to that statistic instead of relative to zero, using one or two bits per dimension (two by default below 900 dimensions). Distances on the codes are computed with bit operations during the graph walk. A 768-dimension float32 vector shrinks from 3,072 bytes to 192 bytes at two bits. Top candidates are rescored with full vectors.

How does pgvector handle filtered vector search?

By default, approximate indexes scan first and filter afterward, so selective filters can return fewer rows than requested; with the default hnsw.ef_search of 40 a 10 percent filter yields roughly four rows. Since pgvector 0.8.0, iterative index scans (hnsw.iterative_scan) keep scanning until enough rows pass or hnsw.max_scan_tuples (default 20,000) is reached. Partial indexes and partitioning cover very selective or high-cardinality filters.

Can I use pgvectorscale on managed Postgres?

The pgvectorscale README documents Timescale Cloud, a TimescaleDB Docker image and source builds. I did not verify support on other managed providers such as AWS RDS, Google Cloud SQL or Azure, so check each provider’s extension list before designing around it. Because pgvector is more widely supported, a common pattern is to start with pgvector HNSW and adopt diskann only where the extension is installable.

Does pgvector support quantization?

Yes, through types rather than a graph-level scheme. The README documents halfvec for 16-bit floats (indexable to 4,000 dimensions), bit for binary vectors (indexable to 64,000 dimensions) and sparsevec. pgvector 0.8.1 also improved binary_quantize performance. Binary quantization in pgvector is something you apply yourself in an expression index, typically with a rerank step, whereas pgvectorscale’s SBQ is built into the index and learns statistics from your data.