Real-Time Analytics Database with SereneDB
A database for real-time analytics makes new data queryable as it arrives, not after the next batch load. In SereneDB a row committed to a table is visible to the next SELECT, and one SQL statement can search it and GROUP BY it.
Full-text search is near real-time: it sees the row after the next index refresh, every second by default. Open source, one binary, PostgreSQL wire protocol.
What Is a Real-Time Analytics Database?
Real-time analytics is the analysis of data as it is generated: dashboards, alerts and ad-hoc queries about what is happening now rather than yesterday. A real-time analytics database is the store under a real-time analytics platform. It takes a continuous stream of writes (logs, metrics, orders), makes each one queryable within seconds, and aggregates fresh and historical data in the same SQL.
Two numbers define it: data freshness, from write to queryable, and query latency, from question to answer. Batch analytics gives up the first: an ETL job copies data into a warehouse on a schedule, so every chart is as old as the last run.
Why Batch Pipelines Fall Short for Real-Time Analytics
The nightly-job problem
A scheduled load decides how fresh every dashboard is: an error at 12:01 stays invisible until the job runs, and the job is one more system to operate. A shorter interval moves the problem without removing it.
The search-plus-warehouse problem
Logs and events need text search and a GROUP BY. Split across a search cluster and a warehouse, that is two copies, two query languages and a sync job between them.
How SereneDB closes both
The application writes each row once, to one table, and one statement on one engine runs the search predicate and the GROUP BY over it.
How SereneDB Runs Real-Time Analytics
The real-time data analytics architecture is one process. Rows arrive over the PostgreSQL wire protocol or OTLP/HTTP, land in a table with an inverted index beside the columnar data, and every question is a plain SELECT.
Parquet · CSV · JSON
snapshot isolation
search tables · OTel
GROUP BY · OVER
Two clocks: commit and refresh
A committed transaction is visible to every transaction that starts after it and, in persistent mode, written to persistent storage. The inverted index is eventually consistent: a background refresh publishes new rows every refresh_interval (lower it for fresher search), and VACUUM (REFRESH_TABLE) publishes them now. Background compaction merges the small segments that frequent small batches leave.
CREATE INDEX events_idx ON events
USING inverted (id, ts, service, level, message logs_dict)
INCLUDE (latency_ms)
WITH (refresh_interval = 1000); -- the default, in ms
-- publish pending rows now, e.g. after a bulk load
VACUUM (REFRESH_TABLE) events;Data ingestion over COPY or OTLP
Stream rows with the PostgreSQL COPY … FROM STDIN protocol, which every driver exposes and which is far faster than row-by-row INSERT. An otel listener serves OTLP/HTTP itself, with no collector; its search tables show rows at the next refresh.
serened ./data --listen \
'postgres://0.0.0.0:5432,http://0.0.0.0:4318?api=otel'Watch the refresh backlog
sdb_metrics reports num_buffered_docs per index, the rows not yet searchable; if it stays high, or avg_commit_time_ms climbs, maintenance is falling behind the write rate.
SELECT c.relname AS index, m.value AS buffered
FROM sdb_metrics m
JOIN pg_class c ON c.oid = m.relation_id
WHERE m.metric = 'num_buffered_docs';Step-by-Step: Real-Time Analytics on Fresh Events in 5 Minutes
Create the dictionary, table and index
INCLUDE stores latency_ms in the index, so a search query returns it without a lookup against the table.
CREATE TEXT SEARCH DICTIONARY logs_dict AS
split_text(case := 'lower')
WITH (frequency, position, norm);
CREATE TABLE events (
id BIGINT PRIMARY KEY,
ts TIMESTAMP,
service VARCHAR,
level VARCHAR,
latency_ms INTEGER,
message VARCHAR
);
CREATE INDEX events_idx ON events
USING inverted (id, ts, service, level, message logs_dict)
INCLUDE (latency_ms);Write the events
Six rows stand in for a feed; a live producer would stream them over COPY … FROM STDIN.
INSERT INTO events VALUES
(1, '2026-09-27 12:00:05', 'checkout', 'INFO', 42, 'order placed'),
(2, '2026-09-27 12:00:31', 'checkout', 'ERROR', 950, 'payment gateway timeout'),
(3, '2026-09-27 12:00:48', 'search', 'INFO', 18, 'query served'),
(4, '2026-09-27 12:01:02', 'checkout', 'ERROR', 1210, 'payment gateway timeout after retry'),
(5, '2026-09-27 12:01:15', 'search', 'WARN', 480, 'upstream timeout, served stale'),
(6, '2026-09-27 12:01:40', 'auth', 'INFO', 25, 'token refreshed');Aggregate on commit
A query on the table sees every committed row: in the first minute checkout logged two events, one error and a quantile_cont p95 of 904.6 ms. Rollups and grouping sets are on the OLAP database page.
SELECT time_bucket(INTERVAL '1 minute', ts) AS minute,
service,
count(*) AS events,
count(*) FILTER (WHERE level = 'ERROR') AS errors,
quantile_cont(latency_ms, 0.95) AS p95_ms
FROM events
GROUP BY ALL
ORDER BY minute, service;Search and GROUP BY in one statement
A full-text @@ predicate reads the index, which sees the rows after a refresh: two timeouts in checkout (worst 1210 ms), one in search.
VACUUM (REFRESH_TABLE) events; -- or wait for refresh_interval
SELECT service, count(*) AS timeouts, max(latency_ms) AS worst_ms
FROM events_idx
WHERE message @@ 'timeout'
GROUP BY service
ORDER BY timeouts DESC, service;A rolling window over the latest rows
A three-row moving average per service: checkout climbs 42, 496, 734 ms. Window functions buffer their whole input, so filter to the range you chart.
SELECT ts, service, latency_ms,
avg(latency_ms) OVER (PARTITION BY service ORDER BY ts
ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) AS rolling_ms
FROM events
ORDER BY service, ts;Use Cases for Real-Time Analytics
Observability on OpenTelemetry
Point an OTel SDK or the collector's otlphttp exporter at SereneDB. Logs, traces and metrics land in otel_* tables, log bodies searchable with BM25; p95 per service is a plain query.
Live dashboards in Grafana
Grafana connects through its built-in PostgreSQL data source. Panels are raw SQL (the query builder is not supported), so one panel mixes a full-text filter, time_bucket and a percentile.
Fresh files in a data lake
Rows another engine commits to Iceberg become searchable after REINDEX, which indexes only the delta; reindex_interval runs it on a loop. See search over a data lake.
When a Streaming Database or Distributed OLAP Is the Better Choice
You need results kept current as data arrives
Materialize and RisingWave update results incrementally as data arrives; ClickHouse runs incremental materialized views on insert. SereneDB has no MATERIALIZED VIEW: each aggregate runs at query time.
You ingest straight from Apache Kafka
ClickHouse's Kafka table engine subscribes to topics, and Druid's Kafka indexing service ingests topics continuously, with arriving rows queryable in real time. SereneDB documents no Kafka connector: a consumer writes over COPY … FROM STDIN.
You need a cluster
ClickHouse shards and replicates tables across a cluster. SereneDB's docs describe a single-node deployment, with no sharding, no replication and no managed cloud.
Frequently Asked Questions
A database that makes new data queryable within seconds of arrival and answers low-latency analytical queries over it, so dashboards and alerts track the present. SereneDB is a search-OLAP engine: one SQL statement searches the fresh rows and aggregates them.
The best database for real-time analytics is the one that fits the workload. Incrementally maintained results point to a streaming database, Kafka-fed dashboards on a cluster to ClickHouse or Druid. Search and aggregation over the same fresh rows, in SQL on one node, is the case SereneDB is built for.
Data is usually called real-time when it is queryable within seconds of the event. In SereneDB a row committed to an ordinary table is visible to every transaction that starts after the commit. Search predicates, and any query on a search table such as the OpenTelemetry ones, see it after the next refresh (refresh_interval, 1000 ms by default) or after VACUUM (REFRESH_TABLE).
Batch processing runs on a schedule, so results are as fresh as the last run; real-time analytics queries streaming data as it arrives. Batch still suits large historical reports, so one rarely replaces the other.
No. A streaming database such as Materialize or RisingWave keeps query results incrementally up to date as events arrive. A real-time analytics database stores the events and answers ad-hoc queries over them; some, like ClickHouse, add incremental materialized views. SereneDB has none: every aggregate runs at query time.
Often. In SereneDB the same SQL aggregates rows committed a second ago and Parquet, CSV or JSON read from object storage, with no warehouse to copy into. It does not replace a scale-out cluster: the docs describe one node.
Not natively: the docs describe no Kafka, Kinesis or CDC connector. A consumer can write batches over COPY … FROM STDIN, and OpenTelemetry data can go to the built-in OTLP/HTTP receiver.
Get Started with SereneDB
One binary, one table, one index: four steps from an empty database to a search-filtered GROUP BY over rows committed a moment ago.