Skip to main content
SereneDBSereneDB
897Contact Us

Synonyms

Shoppers and your catalog rarely use the same word. Someone types "telly", the product says "television" and nothing matches. A synonym dictionary closes that gap: you declare the words that mean the same thing and SereneDB expands them at analysis time, on both the indexed text and the query, so either side finds the other.

Synonyms are a step in a text search dictionary. You wrap a tokenizer and a synonym filter in a pipeline: the tokenizer lowercases and splits the text, then the synonym step rewrites the tokens. There are two filters. expand_solr_synonyms takes a map you write by hand and keeps the surface words, and expand_wordnet_synonyms normalizes words to a shared sense from the WordNet database. Most of this recipe uses solr; the last section shows WordNet.

Four products sit in a catalog indexed on name, behind a dictionary that treats tv, television and telly as one word and files every laptop under notebook.

Schema and sample data
SQL
CREATE TEXT SEARCH DICTIONARY syn AS    split_text(case := 'lower')    | expand_solr_synonyms('tv, television, tellylaptop => notebook')    WITH (frequency, position);
CREATE TABLE catalog (id INTEGER PRIMARY KEY, name VARCHAR);
CREATE INDEX catalog_idx ON catalog USING inverted (id, name syn);
INSERT INTO catalog VALUES(1, 'wall mounted tv'),(2, 'smart television screen'),(3, 'gaming laptop'),(4, 'wireless headphones');
VACUUM (REFRESH_TABLE) catalog;

Find the word the shopper did not type​

The catalog never says "telly", but the query does. Expansion runs on the indexed text too, so the tv and television rows both carry "telly" and the search lands.

SQL
SELECT id, name FROM catalog_idx WHERE name @@ 'telly' ORDER BY id;
 id | name
----+-------------------------
  1 | wall mounted tv
  2 | smart television screen

It works both ways​

A comma group is symmetric, so it does not matter which member is in the document and which is in the query. Searching "tv" finds the row that says "television" just as well.

SQL
SELECT id, name FROM catalog_idx WHERE name @@ 'tv' ORDER BY id;
 id | name
----+-------------------------
  1 | wall mounted tv
  2 | smart television screen

Rewrite one word onto another​

Use laptop => notebook when you want a one-way rewrite rather than a group. Every "laptop" is filed as "notebook" so a search for "notebook" finds it, without dragging unrelated notebook senses back onto laptops.

SQL
SELECT id, name FROM catalog_idx WHERE name @@ 'notebook' ORDER BY id;
 id | name
----+---------------
  3 | gaming laptop

See what a word expands to​

ts_lexize runs a single value through the dictionary so you can see the expansion the index and the query both get. A two-way group returns every member, a one-way rule returns the target.

SQL
SELECT ts_lexize('syn', 'telly') AS telly, ts_lexize('syn', 'laptop') AS laptop;
 telly                 | laptop
-----------------------+------------
 {television,telly,tv} | {notebook}

Normalize to a shared sense with WordNet​

The expand_wordnet_synonyms filter takes a different tack. Instead of adding synonyms to the surface word it replaces each word with the numeric id of its WordNet sense, so words that mean the same thing collapse to one term and match each other. Here "couch", "sofa" and "divan" share a sense, so any of them finds all three.

SQL
SELECT id, name FROM furniture_idx WHERE name @@ 'sofa' ORDER BY id;
 id | name
----+---------------
  1 | leather couch
  2 | fabric sofa
  3 | velvet divan

WordNet only knows the words in its map. A word with no sense passes through unchanged, so ts_lexize returns the word itself and it stays searchable as written; only the words the map knows meet their synonyms. Load the full WordNet database when you want broad coverage, or reach for solr when you would rather hand-write a few groups.

SQL
SELECT ts_lexize('wn', 'sofa')  AS sofa,       ts_lexize('wn', 'couch') AS couch,       ts_lexize('wn', 'table') AS unknown;
 sofa        | couch       | unknown
-------------+-------------+---------
 {100000001} | {100000001} | {table}

See also​

Edit this page ↗