TLDR Web Dev · OPINIONS & ADVICE
YOU DON'T ALWAYS NEED INDEXES
Summary from the TLDR Web Dev newsletter
Sometimes you have a lot of data. One approach to support quick searches is pre-processing it to build an index so a search can involve only looking at a small fraction of the total data. The threshold at which it's worth switching to indexing, though, might be higher than you'd guess. Unless you know from the start that you'll be searching hundreds of millions of records, consider starting with simple scans and only add indexing if you can't get acceptable performance. And even then, if queries are rare and highly varied it may still be better to do the work at query time instead of ingestion time.