The State of Serene, September 2026

Welcome back to The State of Serene, our monthly note on what shipped and where SereneDB is heading.

September went into the engine underneath search. Full-text queries got faster across the board: on search-benchmark-game SereneDB now answers 1.8x to 3.2x faster than Lucene 10.5 and about 2x faster than tantivy 0.26. Every scan uses every core, even on one big segment, and queries on a cold cache run 1.7x faster over a billion logs. Regex, wildcard and fuzzy search always answer now: a 2,433-branch regex that used to return nothing after 11 seconds answers correctly in 25 ms. Text analysis became an expression language, HNSW joined IVF, the documentation moved into the binary and SereneDB learned to take OpenTelemetry directly. The SQL layer took its biggest update yet, with MATCH_RECOGNIZE and a parser 5x to 6x faster, and schema changes now commit in one transaction with the data. Benchmark season went on with the Postgres family, the Lucene family and ClickHouse at ten billion logs.

This release changes the storage format, so read the upgrade note before you point it at an existing data directory.

v26.10.1 is the latest release.

Star UsContact Us

What happened in September

Search, rebuilt

We rebuilt the execution layer every full-text query runs on, and every kind of query got faster: counts, unions, intersections, phrases and top-k by relevance alike. A count no longer computes scores it never returns, wide OR queries work through thousands of documents at a time and top-k queries skip far more documents that can't make the cut: on the benchmark's top-100 intersections SereneDB scores a quarter of the matching documents instead of all of them. A filter on a stored column keeps that skipping on, and queries with exclusions run in about half the time, with counting +bass -fish -guitar 6x faster. You get the same documents and counts as before, except for fuzzy matches, which now count exactly what Lucene counts.

On search-benchmark-game, 962 queries over English Wikipedia on one thread, each cell is how many times faster SereneDB answers than Lucene 10.5 / tantivy 0.26:

query type	COUNT	TOP_100	TOP_100_COUNT
union (301)	2.4x / 6.3x	1.7x / 3.2x	4.6x / 2.5x
intersection (300)	1.5x / 1.8x	1.4x / 2.0x	1.5x / 1.5x
phrase (300)	2.2x / 1.3x	2.0x / 1.5x	2.2x / 1.3x
everything else (61)	1.7x / 1.3x	1.7x / 1.8x	2.0x / 1.9x
all 962	2.1x / 2.0x	1.8x / 2.0x	3.2x / 1.9x

SereneDB is ahead in every cell. When we first published this benchmark in March, the lead over Lucene 10.3 was 1.7x on counts and 1.4x on the top 100. Since then Lucene got 3% to 10% faster and tantivy about a quarter faster. The lead over Lucene still grew and the lead over tantivy held, because our own build got 1.3x faster on counts, 1.4x on the top 100 and up to 1.95x on unions.

Interactive results: search-benchmark-game.

Scores you can compare with Lucene's

BM25 scores match Lucene's now. Every score is the old one divided by 2.2, every ranking stays the same and the values agree with Lucene 10.5 to six decimals, so a hybrid bm25 + vector formula or a constant(N) branch tuned against Lucene or Elasticsearch carries over as is. The rest of the scoring follows suit:

• Prefix, range, wildcard and regex matches score a constant by default, as in Lucene. Add ::score(...) to score each expanded term, and drop sdb_scored_terms_limit, which is gone.
• Fuzzy matches score and count like Lucene's, and sdb_levenshtein_max_terms defaults to 50.
• Numeric, boolean, IS NULL and geo clauses score constant(1), and an excluded clause never adds to a score.
• A rare term is scored against the whole field, not only against the segments it appears in.
• Sloppy phrases match and score like Lucene's, so a b c matches a c b at slop 2.

Docs: scoring and fuzzy search.

Every core on every query

A full-text or analytical scan used to get one worker per index segment, so one big segment left most cores idle. Now a scan spreads over all cores whatever the segment layout. Top-k queries finish inside the scan and read full rows only for the results they return, and ORDER BY <column> LIMIT stops as soon as the rest of the data can't beat what it already has.

On SearchBench that took 39% off the cold geometric mean at 100M logs and 34% at a billion, and 13% and 5% off the hot one. The biggest wins are where one worker per segment hurt most: top-k queries 2.5x to 3.5x faster cold, ORDER BY Timestamp LIMIT 3x to 5x cold and 2x to 4x hot. Many clients at once gained the most. On 20M rows with a pool of 96 threads:

clients × query	throughput	p99 latency
64 × one-term count	10.5x higher	49x lower
32 × top-k of 4 terms	1.6x higher	5.1x lower
96 × 4-term AND count	2.4x higher	9.6x lower

You can cap parallelism per session or per transaction:

SQL

SET threads = 2;
SET LOCAL threads = 1;

SET GLOBAL threads resizes the pool for everyone, and row_group_size sets how finely an index splits up for a parallel scan.

Docs: scan settings and SET.

Star UsContact Us

Faster on a cold cache

The first queries after a restart, or over data that isn't in memory yet, used to read the disk one small page at a time. SereneDB now reads ahead whenever a query reads sequentially, and hot queries pay nothing for it. Cold runs over a billion logs, fresh process and empty page cache, summed per query family:

query family	queries	faster by
count	32	2.2x
top-k	21	2.3x
group by	14	2.1x
recent rows	16	1.9x
join	9	1.4x
all	92	1.7x

Hot runs did not move, and on a cloud SSD cold top-k got 3.3x faster. The very first query after a start is quicker too.

Regex, wildcard and fuzzy search that always answer

Regex, wildcard and fuzzy search used to give up on big patterns: past a size limit the search returned nothing. Now every pattern answers, however large, and its size only affects speed. \b, \B and anchors work inside a term, a pattern that can only match a few words costs what ts_any over those words costs and a prepared statement compiles its patterns once. With sdb_pattern_cache_size set, the server keeps compiled patterns across queries as well. The string functions and LIKE got faster too:

query	change
2,433-branch ts_regexp over 379k company names	0 rows after 11 s → 24.5 ms
27-branch .*\bX\b.* filter over job titles	killed after 10 min → 0.47 ms
count(regexp_match(txt, 'пере([а-я]+)'))	35x faster
three LIKE '%…%' patterns joined with OR	6.6x faster
regexp_matches(txt, 'ость')	4.2x faster
ts_levenshtein('дом', 2)	3.1x faster
regexp_full_match(txt, '.*tion.*'), English	2.8x faster

The first two rows come from a bug report. The string functions were measured over six kinds of text (Russian page titles, English, Chinese and Japanese Wikipedia, URLs and source code): 72 of 84 shapes got more than 5% faster and none got slower. Pattern searches on a cold cache got faster as well, a cold .*ени.* by 6.4x.

Spell correction got a shortcut: ORDER BY ts_dict_score DESC LIMIT n over a fuzzy match looks at only the best n terms of each segment and still returns the exact answer, and EXPLAIN shows the cap. A ten-term lookup got 3.9x faster:

SQL

SELECT unnest(ts_dict_agg(w)) AS t, unnest(ts_dict_score(w)) AS s,
       unnest(ts_dict_count(w)) AS c
FROM fl_words_idx WHERE w @@ ts_levenshtein('cats', 1)
ORDER BY s DESC LIMIT 1;
-- bats  0.75  1

On a column indexed with generate_wildcard_ngrams, ts_regexp returns exactly what the pattern matches and uses the n-grams to find candidates fast:

SQL

SELECT id, term FROM terms_idx WHERE term @@ ts_regexp('(re)?sea.*') ORDER BY id;

Over a million log lines most ts_like and ts_regexp patterns on such a column got at least 2x faster, GET /api/% 217x faster and a case-insensitive (?i).*outofmemoryerror.* 1,492x faster. One limit: a pattern with around 500 candidates is 23% to 48% slower.

ts_regexp also works as a part of a phrase, between plain words:

SQL

SELECT a FROM tsq_idx WHERE b @@ ('quick' ## ts_regexp('(brown|red)') ## 'fox') ORDER BY a;
-- 1, 2

Docs: ts_regexp and friends, spell correction with ts_dict_score and generate_wildcard_ngrams.

Star UsContact Us

Text analysis as expressions

A text search dictionary is now an expression, which makes analyzers far easier to build: | chains stages into a pipeline, a list merges analyzers, any SQL function or lambda can be a stage and a stored dictionary can be built on. Whatever SQL can do with a string, an analyzer can do:

SQL

CREATE TEXT SEARCH DICTIONARY english_expr AS
    split_text_csv(',') | split_text(case := 'lower') | stem_words('en_US.UTF-8') | remove_stopwords(['the'])
    WITH (frequency, position);

SELECT ts_lexize('english_expr', 'The,Quick,Foxes');
-- {quick,fox}

CREATE TEXT SEARCH DICTIONARY split_stem AS
    (lambda x: string_split(lower(x), ',')) | stem_words('en_US.UTF-8');

CREATE TEXT SEARCH DICTIONARY exact_and_grams AS [keyword(), generate_ngrams(2, 2)] WITH (frequency);

Every template is also a scalar function of the same name, so nesting functions is the pipeline and you can try an analyzer stage by stage in a plain SELECT:

SQL

SELECT stem_words(normalize_tokens(split_text('The Runners Café', case := 'lower'), 'en_US.UTF-8', accent := false), 'en_US.UTF-8') AS tokens;
-- {the,runner,cafe}

New templates and more than a dozen new options came with it. strip_html pulls the text out of HTML, decoding character references and dropping script and style, and filter_tokens keeps tokens by length or by any lambda. split_by_non_alpha splits on ASCII, on letters and digits of any script, on letters alone or on whitespace. split_text splits grapheme clusters, normalize_tokens does full case folding and the NFD, NFKD and NFKC_Casefold forms and generate_ngrams makes edge n-grams. The Snowball stemmers cover 34 languages instead of 28. Synonyms and other tokens stacked at one position count as alternatives in ts_phrase, phraseto_tsquery, plainto_tsquery, ts_all and ts_any, and a wildcard dictionary works on LIST and ARRAY columns.

Phrase search uses shingles. On a column indexed with generate_shingles, a phrase the column stores as one shingle is a single term lookup and a longer phrase is covered by shingles. That holds for ts_phrase, phraseto_tsquery, quoted phrases in to_tsquery and ##, and highlighting marks the whole phrase:

SQL

CREATE TEXT SEARCH DICTIONARY shingle_dict AS
    generate_shingles(split_text_csv(' '), 2, 2) WITH (frequency);

CREATE INDEX test_shingle_idx ON test_shingle USING inverted(id, body shingle_dict);

SELECT id FROM test_shingle_idx WHERE body @@ ts_phrase('quick brown') ORDER BY id;

With two- and three-word shingles, a three-word phrase answers about 200x faster than over word positions, and three frequent words in a row about 13,000x faster.

A query can also name its own fields. With the relation's tableoid on the left of @@, one boolean spans several columns and each term is analyzed with its own column's dictionary:

SQL

SELECT a FROM tsf_idx WHERE tableoid @@ to_tsquery('title:dog AND body:fox') ORDER BY a;

Docs: CREATE TEXT SEARCH DICTIONARY, tokenizer functions and fields in a query.

Faster text analysis and indexing

Getting text into an index got faster from end to end. SearchBench loads its logs 1.6x faster at 100M and 1.5x faster at a billion than with the August release, and every layer contributed: analyzers run as one pipeline, the tokenizers were rewritten for speed, the indexer was rebuilt and the new Unicode layer beats the ICU library it replaces. A column of repeated values is analyzed once per distinct value.

what	faster by
loading SearchBench, 100M logs	1.6x
loading SearchBench, 1B logs	1.5x
normalize_tokens on non-ASCII text	9.2x
a Unicode column through an analyzer	8.9x
splitting on a set of delimiters	up to 7.5x
indexing a column of repeated values	6.7x
Snowball stemming	3x to 5x
collation keys	2.9x
split_text on Cyrillic	2.5x to 3.9x
CREATE INDEX over a compressed text column	1.7x to 2.1x

Star UsContact Us

HNSW next to IVF

July rebuilt vector search on IVF because it scales with disk. When the vectors fit in memory, though, HNSW is the usual answer, and the inverted index takes either now, declared the same way and queried with the same operators:

SQL

CREATE INDEX hnsw_index ON hnsw_vectors USING inverted (id, emb hnsw (metric = 'l2'));

SELECT id FROM hnsw_index ORDER BY emb <-> [0, 0, 0]::FLOAT[3] LIMIT 2;

quant is sq8 by default, or sq4, tq or none. m and ef_construction shape the graph and sdb_hnsw_ef_search trades speed for recall at query time.

An HNSW query takes a LIMIT or a radius. With either index the query vector can be written as a string such as '[0, 0, 0]', as in pgvector, and array_distance, array_cosine_distance and the other array_* and list_* distance functions use the index just like the operators. The distance functions and operators take plain lists as well as fixed-size arrays.

Docs: HNSW and vector functions.

The documentation ships inside the binary

The documentation now ships inside serened with its own search, so it always matches the version you run and works offline. There are three ways in.

.docs in serened shell and serened psql needs no connection. Give it a section, a function name, a call copied out of a query, a question or an error message pasted from the terminal:

BASH

serened shell -c ".docs how do I highlight matches"

From SQL, sdb_docs.search() ranks sections with BM25 and sdb_docs.objects catalogs more than 1,300 documented objects, functions and settings among them:

SQL

SELECT path, title, breadcrumb, score FROM sdb_docs.search('inverted index options', 5);
SELECT path, title, breadcrumb FROM sdb_docs.object('date_trunc');

Over MCP, a listener with ?api=mcp gives an agent six tools at /_mcp. Five read the documentation. The sixth, check_sql, plans a statement without running it, so an agent can see whether an inverted index serves its query before it sends it:

BASH

serened ./data --listen 'postgres://127.0.0.1:7890,http://127.0.0.1:8080?api=mcp'

How often the right page comes first and how often it is in the top five, against grep over the same pages:

queries	built in	grep
62 hand-written questions	0.69 / 0.97	0.43 / 0.81
300 object names	0.89 / 1.00	0.71 / 0.95
300 object summaries	0.98 / 1.00	0.90 / 1.00
319 pasted sentences	0.97 / 1.00	0.82 / 0.99

The search on serenedb.com/docs finds pages the same way. The docs there have a refreshed design, a configuration reference grouped by subsystem and a page for the Elasticsearch-compatible API.

Docs: documentation functions and .docs in the shell.

OpenTelemetry, straight in

A listener with the otel API makes serened the OTLP endpoint, with no collector component to install:

BASH

serened ./data --listen 'postgres://0.0.0.0:7890,http://0.0.0.0:4318?api=otel'

POST /v1/logs, /v1/traces and /v1/metrics take both OTLP encodings, protobuf and JSON, so any OTel SDK, or a collector's stock otlphttp exporter, points straight at it. Telemetry lands in seven search tables, so a log body is searchable and rankable with BM25 as soon as it is refreshed. The tables are created at startup, in another database or schema if you pass db= or schema=. Errors follow the OTLP retry rules: rejected data answers 400, which clients don't retry, and a transient condition answers 503, which they do.

Docs: OpenTelemetry.

Star UsContact Us

HTTP compression

Every HTTP API compresses both ways now, the Elasticsearch-compatible one and OTLP alike, with nothing to configure. A client asks for zstd, brotli, gzip, zxc, lz4 or snappy and gets it, and can send its payloads in any of them, so a collector or an Elasticsearch client that compresses its batches works as is. Snappy covers Prometheus remote write, while zxc and lz4 are there for the fastest decode and the fastest encode. A decompressed body is held to the same 64 MiB limit as an uncompressed one, and a malformed one gets a clear error while the connection stays open for the next request.

Each HTTP request also runs as the role it authenticated as, even on a reused keep-alive connection, and an index created through the Elasticsearch API sees SQL DELETE and UPDATE on its table.

Docs: HTTP compression and the Elasticsearch API.

Search over Iceberg, sturdier and faster

Indexes over Iceberg tables, BigLake's included, got a round of work from running them in production. An index-only search never contacts the catalog, so a search with a stale table cache got 62x faster and eight concurrent searches after an idle spell 45x faster. Catalog and storage requests reuse their connections, which made the median BigLake request 1.4x faster and two refreshes at once 1.9x faster.

REINDEX picks up another writer's commit on its first pass and applies it as a delta. serenedb_reindex waits for a running pass instead of failing, and a reindex that can't reach its source keeps the index as it is. A view-backed index keeps its rows across restarts and sees every delete, in large files and after compaction alike. Reads keep working past the hour a vended credential lasts, with credentials scoped to the transaction. Views that spell file columns in a different case work, and a small reindex over 3,072-dimension vectors no longer spends seconds on clustering.

Docs: refreshing an index over Iceberg and BigLake.

Time histograms

Grouping by time is the most common analytical query over logs, and it got much faster. GROUP BY date_trunc('hour', ts) takes a fast path now, and so do time_bucket, date_bin, strftime, monthname, dayname, last_day, casts to DATE, numeric bins such as floor(x / 100) * 100, epoch arithmetic, CASE keys, AT TIME ZONE, coordinate sets such as year(ts), month(ts) and SELECT DISTINCT over them. Time zones stopped costing a calendar lookup per row, and a filter such as time_bucket(...) = c becomes a range the inverted index can use. 100M rows, 32 threads:

query	faster by
hourly histogram in Europe/Berlin	33x
daily strftime in Europe/Berlin	25x
daily strftime	10.8x
time_bucket = c over an inverted index	6.2x
5-minute time_bucket	4.0x
monthly histogram	3.4x
hourly histogram	1.6x

Grouping search results by a string column got faster too: GROUP BY SeverityText over 10.8M hits by 2.4x on one thread, and SearchBench's group-by queries by 1.5x.

Docs: timestamp functions.

More SQL and a new parser

The SQL layer underneath SereneDB took its biggest update yet, about 1,200 changes in one go. New SQL you can use today:

• MATCH_RECOGNIZE, row pattern matching from SQL:2016. Sessions, funnels and runs of events over ordered rows become one clause instead of a chain of self-joins, and match_recognize_max_states bounds how far one match may search.
• A nearest-neighbour join. JOIN t NEAREST k BY DISTANCE ..., or BY SIMILARITY, pairs every row with its k nearest rows on the other side.
• Functions and syntax. json_set, json_insert, json_replace and json_remove edit JSON values, lttb downsamples a series to a given number of points for a chart and binom returns binomial coefficients. @ is absolute value, FETCH FIRST n ROWS ONLY works, UNNEST can go in GROUP BY, ORDER BY takes expressions over the result of a UNION and max_memory takes a percentage.

Queries got more parallel and read less. UNION ALL branches run in parallel when preserve_insertion_order is off, a materialised CTE feeds all of its readers at once and inserts and deletes keep running while a checkpoint writes their table. Aggregates are reused across joins, and string, substring, GLOB, date and IN-list filters skip more data.

Parsing is 5x to 6x faster on our parse-heavy benchmarks, and huge generated queries plan fast: a 20,000-branch UNION ALL takes 0.3 s where v26.09 took 17 s, and EXPLAIN over 2,000 branches takes 0.09 s instead of 2.1 s.

SereneDB no longer needs the ICU library. Word and sentence breaking, normalization, case folding, collations and Unicode classes in regexes are built in, from the ICU 78.3 data and Unicode 17.0, and run faster than the library did. One behaviour change to know: string type modifiers such as varchar('10') are an error.

Docs: collations.

Star UsContact Us

Schema and data, one transaction

Schema changes and data commit together now. One transaction can create a table, a text search dictionary and an inverted index, load rows, grant access and change owners, and it commits as a whole or not at all. That holds across databases and roles too: a transaction that writes to several databases, or creates a role next to its tables, commits atomically, and a crash at any point brings back either all of it or none of it. A crash in the middle of creating or dropping a database, an index or a search table leaves no stray files behind, because the next start cleans them up. A randomized stress suite runs parallel DDL and DML against the server, crashes it on purpose and checks the result against a model, so it stays that way.

Docs: transactions.

Search tables, rounded out

CREATE INDEX works on a search table that already holds data. It builds in parallel while inserts and reads keep running:

SQL

CREATE TABLE bf (id BIGINT PRIMARY KEY, tag TEXT, body TEXT)
  WITH (storage = 'search', refresh_interval = 0, compaction_interval = 0);

INSERT INTO bf VALUES
  (1, 'red',   'quick brown fox'),
  (2, 'blue',  'lazy dog jumps'),
  (3, 'red',   'fox runs fast'),
  (4, 'green', 'nothing here');

VACUUM (REFRESH_TABLE) bf;

CREATE INDEX bf_tag_idx ON bf USING inverted (tag);

SELECT id FROM bf_tag_idx WHERE tag @@ 'red' ORDER BY id;
-- 1, 3

search_backfill_group_bytes bounds how much of the table is rewritten at once. Indexing a populated search table rewrites it, so over 10M rows it takes about 2.5 times as long as indexing a transactional table.

Big writes that run on one thread no longer hold the whole statement in memory. 5M rows in one statement:

statement	time	memory
INSERT	2.1x faster	15x lower peak
UPDATE	2.0x faster	28x less growth

Search tables compact in the background by default, and ALTER TABLE ... SET changes the compaction options of a running table. optimize_top_k works as a table option, so top-k queries on a search table skip what can't make the cut. USING COMPRESSION and force_compression apply to stored columns, which made the text of our own docs index 12x smaller with zstd. @@ works on the table itself and not only on its index relation, and so do ts_dict_*, facets and ts_offsets. ALTER TABLE handles comments, renamed columns, defaults, NOT NULL and CHECK.

Docs: maintenance and ALTER TABLE.

Spatial functions

The geospatial index has been around since June. The function library around it is new: over 110 ST_* functions with PostGIS names and PostGIS argument order, covering constructors, accessors, predicates, measures, overlays, affine transforms, linear referencing and spheroid distances:

SQL

SELECT ST_Distance_Sphere('POINT(0 0)'::GEOMETRY, 'POINT(1 0)'::GEOMETRY)::bigint AS meters;
-- 111195

Clients get a two-dimensional GEOMETRY in exactly the hex WKB PostGIS sends, and \d shows geometry, so clients built for PostGIS can read it. Results are checked against PostGIS 3.6 across all seven geometry types. Affine transforms keep Z and M coordinates, and a spatial predicate the index can't serve still runs, just without the index.

As in PostGIS, <-> between two geometries is their planar distance, the same value as ST_Distance, and works in any expression, ORDER BY included. A radius in metres that the index answers is ST_Distance_Centroid(field, c) < r or ST_Distance_Between.

The coordinate reference system belongs to the type, so ST_CRS and ST_SetCRS take strings where PostGIS has SRIDs, and EPSG:4326 is written as its full WKT2 definition.

Docs: geometry functions and geospatial search.

Star UsContact Us

AI functions in SQL

SereneDB had ai_embed and nothing else. Now it has the whole family: ai_generate, ai_classify, ai_classify_labels, ai_extract, ai_filter, ai_translate, ai_redact, ai_score, ai_rerank, ai_similarity and the aggregates ai_agg and ai_summarize_agg.

SQL

CREATE SECRET local_chat (
    TYPE openai,
    base_url 'http://localhost:11434',
    model 'qwen2.5:0.5b'
);

SELECT id, ai_classify(body, ['positive', 'negative', 'neutral'],
                       secret_name := 'local_chat') AS sentiment
FROM reviews
ORDER BY id;

SELECT id, body
FROM reviews
WHERE ai_filter(body, 'the customer is unhappy', secret_name := 'local_chat');

They speak the OpenAI wire protocol, so OpenAI, Ollama, vLLM, OpenRouter and Gemini's compatible endpoint all work. Requests run concurrently without tying up query threads, with a per-query concurrency cap, retries that honour Retry-After, request and token quotas and batching. Row text goes in its own message and the structured functions demand strict JSON, which makes prompt injection harder but not impossible. A plain http:// endpoint is refused for anything but the local machine unless you allow it, because the prompts and the API key would travel unencrypted. ai_system_one asks a TypeSafe Jev decision model, or a self-hosted Kev server, for calibrated probabilities instead of prose.

Docs: AI functions.

Closer to Postgres

A pg_dump-style restore of a SERIAL table works, from the sequence through OWNED BY to the setval:

SQL

CREATE TABLE pd_t (id integer NOT NULL, v text);
CREATE SEQUENCE public.pd_t_id_seq AS integer START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1;
ALTER SEQUENCE public.pd_t_id_seq OWNED BY public.pd_t.id;
ALTER TABLE ONLY public.pd_t ALTER COLUMN id SET DEFAULT nextval('public.pd_t_id_seq'::regclass);
SELECT pg_catalog.setval('public.pd_t_id_seq', 3, true);
ALTER TABLE ONLY public.pd_t ADD CONSTRAINT pd_t_pkey PRIMARY KEY (id);

Sequences behave as in PostgreSQL: CACHE n and currval belong to the session, ALTER SEQUENCE ... RESTART works, setval is durable when it returns and a committed value is never handed out twice, crash or not. TRUNCATE takes several tables and RESTRICT, CASCADE or RESTART IDENTITY, and a concurrent writer's rows can't survive it. Also:

• EXPORT DATABASE and IMPORT DATABASE round-trip every kind of object except text search dictionaries, which are next.
• SQLAlchemy's create_all and reflection work, and json_build_object and json_build_array exist under their PostgreSQL names.
• pg_get_indexdef, pg_get_viewdef, pg_get_constraintdef and the rest of the pg_get_*def family return full definitions, an inverted index's dictionaries and options included, so database GUIs show them.
• Reading a table takes USAGE on its schema as well as the table's own privilege, as in PostgreSQL. public grants USAGE to everyone.
• DROP COLUMN drops the indexes that use the column, as PostgreSQL does. Renaming or retyping a column works while views use the table, as long as none of them reads it, and a refusal lists the views in the way.
• ALTER TABLE ... ADD COLUMN ... PRIMARY KEY works, a column can carry several named constraints and a unique index with INCLUDE columns is unique on its key columns.
• USE switches the database for everything in the session, from CREATE SERVER and REINDEX to privilege checks.
• Metadata functions have sdb_* names, such as sdb_tables() and sdb_columns().
• CREATE DATABASE name WITH (BLOCK_SIZE = n, ROW_GROUP_SIZE = n) sets a database's storage options.

Docs: sequences, TRUNCATE and transactions.

Storage that can evolve

From this release on, SereneDB's files follow one set of compatibility rules. A newer release reads what earlier ones wrote, and an older release reads what a newer one wrote unless it uses something the older one lacks. A file a release can't read is refused with an error that names it, never misread, and checksums make a damaged index file fail to open instead of returning wrong results.

Upgrading to v26.10.1. Getting there took one break, and there is no migration path: v26.10.1 can't read data directories, inverted indexes or search tables written by earlier releases. Start it on a fresh data directory and load your data again. Indexes over Parquet, Iceberg or an attached database are the easy case: recreate the server, the view and the index, which then rebuilds from data that never moved. Dictionaries need the expression syntax above, with no store_tokens or filler_token on shingles, and BM25 scores come out at 1/2.2 of what they were, ranked the same. A geo radius query written as field <-> c < r becomes ST_Distance_Centroid(field, c) < r, because <-> is the planar distance now.

Docs: storage compatibility and versioning.

Star UsContact Us

Benchmark season, continued

August promised more benchmarks and September brought three posts, each with the same 92 SearchBench queries over OpenTelemetry logs.

The Postgres family: vanilla Postgres 18, ParadeDB with pg_search and TigerData with pg_textsearch. At a billion logs SereneDB answers all 92 queries at a 35.5 ms median. ParadeDB's median is 413 ms with 77 of 92 finished, and TigerData and Postgres 18 finish 33 and 12 within the 60-second cap.

The Lucene family: Elasticsearch 9.5, its new columnar mode, OpenSearch 3.8 and CrateDB 6.4. At a billion logs SereneDB's median is 32.5 ms on 112.1 GiB of disk against Elasticsearch's 98.4 ms on 153.2 GiB, and nobody else finishes all 92.

ClickHouse with its new text index, on one instance, up to ten billion logs. At ten billion SereneDB answers all 92 at a 173.5 ms median on 1,122 GiB of disk. ClickHouse with a sorting key finishes 60 at a 1.9 s median on 1,678 GiB, and 48 without one. SereneDB is the only engine in the series that answered every query at every scale.

Every raw result is at playground.serenedb.com/searchbench. More are coming in October.

SereneDB in LangChain

langchain-serenedb is on PyPI. It makes SereneDB a LangChain vector store with the rest of the database behind it: BM25 and vector search fused in one query, metadata filters served by the index and the surrounding data in the same instance. The announcement walks through a RAG pipeline end to end.

Docs: LangChain.

Also shipped

• Deleted rows cost less. Queries skip deleted rows faster, deleting from a large index no longer rewrites its whole list of deletions and a parallel bulk insert commits with one sync to disk instead of dozens.
• Sessions end cleanly. A connection whose setup fails gets its error instead of waiting forever, a failed COMMIT in a multi-statement query answers instead of hanging and logins keep working while a role is renamed.
• Re-attaching a just-detached file waits for the transactions still holding it instead of refusing.
• Parquet filters return every matching row. A WHERE expression such as hash(id) % 3 = 1 over a dictionary-encoded Parquet column used to drop some of them.
• Partitioned COPY ... TO with ORDER_BY is reliable, where it used to fail now and then with an internal error.
• Lucene-syntax queries are safe to run concurrently.
• Top-k keeps zero-score hits, so LIMIT over a keyword column without a text dictionary returns its rows.
• Hotfix releases have their own numbers. A fix shipped on top of v26.09.1 is v26.09.1.1, which leaves v26.09.2 free for the next ordinary release.

What to expect in October

• More benchmarks. SearchBench reruns, for this release and for the other engines: the ClickHouse team sent a pull request with a newer ClickHouse and their own setup, which they report does better. Then vector benchmarks and maybe a few more. Stay tuned.
• Vector search that feels great. Better quantization, filtered HNSW, settings that tune themselves and better IVF recall with SOAR and AVQ.
• Faster, smaller search indexes. A new postings format makes queries 1.1x to 1.25x faster hot and 1.25x to 1.7x faster cold, and shrinks positions by 10% to 15% and postings with frequencies by about 20%.
• Logical replication subscriptions, promised in August, will land.
• Row-level security, also from August.
• Faster, more compact string compression for stored columns.
• Scheduled jobs.
• EXPORT DATABASE for text search dictionaries and inverted indexes.
• Backups of a whole server.
• Primary keys and unique constraints on search tables.
• A new term dictionary format.
• Better REINDEX for indexes over Iceberg tables and files.
• Parallel ingest over HTTP and COPY FROM STDIN.
• Faster compaction.
• Search tables and inverted indexes ordered by an expression, such as a column or a Hilbert or Z-order curve.
• Search indexes on S3. Inverted indexes and search tables that keep their files in object storage.

Star UsContact Us

Kudos

aksel2904 made ts_regexp exact on wildcard columns (#1301), up to 290x faster (#1308) and fast for case-insensitive patterns too (#1309). After the regexp filter in June and sloppy phrases in August, that is the third search feature from the same contributor.

romanpovol gets credit for work that landed months after they wrote it. Their RE2 speedups gave us ideas behind this month's faster string functions, and their edge n-grams, a stemmer fix and faster ASCII normalisation landed in the new analysis pipeline.

Our students took most of the tasks we opened for them, and the first ones are in. zhemalb made sloppy phrases match like Lucene (#1152). AlmazAlmaz is indexing numeric tuples and geometry on Hilbert curves (#1317, #1320) and NotFelixDzerzhinsky is speeding up the Snowball stemmers (#1201).

Thanks as well to OrlovEvgeny, whose fix makes CREATE DATABASE refuse a taken name before it opens any storage (#1252) and whose regression test for Parquet filters that dropped rows is part of this release (#1313).

In review and in progress: a fix from deadtrickster for CREATE INDEX IF NOT EXISTS scanning a whole table only to discard it (#1159), the database TEMPORARY privilege for temporary tables plus macOS ARM64 build fixes from dndungu (#1322, #1323) and faster interval search from deymon-d (#704).

Want to be in the next one? We tag beginner-friendly work with good first issue, so grab one, ask questions in the issue and we'll get you going.

---

v26.10.1 is the current release. Grab it and point it at something. If you'd rather look before you install, the code search demo is live, ⌘K on the docs gets you SereneDB searching SereneDB's documentation and .docs in serened shell does the same offline. Every raw benchmark result is at playground.serenedb.com/searchbench.

If you like what you see, ⭐ star us on GitHub. Hit a rough edge? Open an issue, or send the fix like everyone in Kudos did.

See you in the next State of Serene.