INDEXING ALL OF WIKIPEDIA, ON A LAPTOP

Summary from the TLDR Web Dev newsletter

Cohere released a dataset containing all of Wikipedia, making it practical to create a semantic, vector-based index of Wikipedia on a laptop. This developer was able to index the entire database on a laptop by making sure they had enough free disk space and RAM and disabling swap. They used an indexing library that supported indexing larger-than-memory datasets by compressing vectors.