SearchTempest Update

As mentioned in recent posts, craigslist has been making some changes lately that we have had to adapt to. (We have no affiliation with craigslist at all, so we don’t get any warning or help from their end when they change something; we just have to figure it out.)

In the past, we allowed you to open results from multiple craigslist cities at once, all on the same page using ‘iframes’. A couple months back, CL added some code to their site to prevent their pages being opened in iframes, so we had to find a new way. As a result we developed two new results modes:

  • Separate Cities – Similar in appearance to our old setup, but inside each city ‘frame’, instead of a results page straight from craigslist, we showed results from a third-party provider called 3taps. 3taps is an ‘exchange data API’, which got craigslist listings from the google cache, so they did not have to actually scrape craigslist.

  • Direct Results – The goal of this mode was to preserve the straight-from-craigslist feature of the frames. Since we could no longer link to craigslist results pages in frames, instead we created an interface whereby they could be conveniently opened in new windows. Essentially we open up a small navigation window on the left, with links to craigslist results that open in a larger window on the right. Since both remain visible at the same time, it lets you conveniently flip through results from multiple cities.

Recently there has been another development. craigslist has apparently instructed google to stop caching their results. (Note, that doesn’t mean their results have been removed from google, just that google doesn’t store copies of entire craigslist pages.) This effectively cuts off 3taps’ source for craigslist data. For us, it meant we needed to find yet another new way to do the Separate Cities mode.

So, what we’re doing now is powering that mode with a Google Custom Search. Basically this allows us to customize a Google search so it only returns results from certain sites. So we take your search at SearchTempest, then send a request to google with your keywords, along with a set of sites to search based on the zip code, distance, and category you entered. Google returns a bunch of results fitting those parameters, which we display for you. When you click on one of the results links, it your browser opens that posting right at craigslist in a new window.

There are a couple downsides to this google-powered approach. The main ones are that it does not support filtering by price or displaying prices in the results, and that you can’t filter by ads with pictures. Also, preview images only work if the ad was created with embedded html images, not the standard craigslist pics. If you need those features, we suggest using the Direct Results mode mentioned above, which should do everything you need!

So in summary, the Separate Cities mode is now powered by Google. It’s probably the most convenient option for most people. If you find yourself missing those price or image filters though, or if you just want to get your results straight from the source, rather than via google, you should give the Direct Results mode a try.

Next thing on our agenda, thanks to user feedback, is simplifying the results page. We’re going to be removing a bunch of tabs, consolidating the Separate Cities and All Cities results modes into one (since they’re both now powered by Google), and just generally making everything a lot more straightforward. Hope to have that ready for you soon!