Photo by Steven Lelham How fast is fast enough? In the early days of the web, 1 second was thought to be the limit for a responsive website. But more recent research shows that any latency above 13 ms has an increasingly negative impact on human performance and that even 100 milliseconds of latency can…

Photo by Sergio Jimenez Posting list compression is an important building block of every search engine. It reduces index size and query latency, as it reduces the amount of data to be loaded from disk or ram (memory bandwidth). Compression algorithms like roaring bitmaps compress the document ID and are…

Photo by Mateusz Wacławek This is part 3 of a 4-part blog post series on Vector vs. keyword search: War of the worlds vs. we come in peace Data structures and algorithms LSMT-IVF for Billion-Scale Approximate Nearest Neighbor Search Benchmarking Vector vs. keyword search 3: LSMT-IVF for Billion-Scale Approximate Nearest Neighbor Search While in the first part we concentrated on the more…

Photo by Mateusz Wacławek This is part 2 of a 4-part blog post series on Vector vs. keyword search: War of the worlds vs. we come in peace Data structures and algorithms LSMT-IVF for Billion-Scale Approximate Nearest Neighbor Search Benchmarking Vector vs. keyword search 2: Data structures and algorithms Now lets’s take a closer look at the inner workings…

Photo by Mateusz Wacławek This is part 1 of a 4-part blog post series on Vector vs. keyword search: War of the worlds vs. we come in peace Data structures and algorithms LSMT-IVF for Billion-Scale Approximate Nearest Neighbor Search Benchmarking Vector vs. keyword search 1: War of the worlds vs. we come in peace Vector search has lived in the shadow for quite some…

Technology becomes more complex and faster changing, data become more massive, and competition denser by globalization. There are different approaches to cope with these challenges. One is to build your application based on available libraries and best practices. This high-level approach on top of existing building blocks allows you to…

TL;DR: A single search server with SeekStorm technology equals the energy saving of switching off 1,000,000 lightbulbs Sustainability and the reduction of the carbon footprint is a giant, global task. We will succeed only if we all do our best within our domain, where we are experts and at home…

The Pruning Radix Trie is a novel data structure, derived from a radix trie — but 3 orders of magnitude faster. After I published SymSpell, a very fast spelling correction algorithm, I have been frequently asked whether it can be used for auto-completion as well. Unfortunately, despite its speed, SymSpell…

Exploring the implications of Artificial Intelligence, Consciousness and Free will TL;DR AI will replace most jobs. Human labor becomes worthless, the biggest devaluation in history. Democracy will crumble as people lost their negotiating power. Consciousness will spontaneously emerge in AI by Darwinian evolution, not by human engineering. Superintelligence will surpass the…