Faster to re-calculate than retrieve from database
As you know, SearchTempest allows you to search craigslist by distance from your zip code. To do this, we keep database tables (MySQL) of zip codes and craigslist cities, each with latitude and longitude.
In the past, we also had one large table containing the distance between every single zip code and every craigslist city. Although this table takes a long time to put together, the logic is it only needs to be done once. When you actually do your search, all the distances for your zip code have already been calculated, so the server just has to grab them from the database.
I’ve had a suspicion for a while and just decided to test it. Long story short, it turns out it’s actually faster to recalculate these 500-odd distances on the fly, than it is to simply pull them out of a database table on the hard drive. Basically, processors are so fast compared to hard drives that minimizing hard drive access is practically the only thing that matters. Pretty cool!
(Yes, if the database were on an SSD the result might be different, but at the moment it takes approximately 0.01s to pull up all the distances, so that’s probably an unnecessary investment at this point! 🙂 )