Cities stuck “Retrieving Results from Google”

Update: Fix released

We have released an update that should make this significantly less of a hassle.  We now have built-in timers on in SearchTempest to avoid hitting this Google limit, so instead of being timed out and having everything break for five minutes, the worst you’ll experience is a few seconds delay (with a message explaining why).  This still might be an annoyance for larger searches, so we do have a couple other options for you as well.  Please see here for more info:

Original post

Some of you have probably run across this recently: view a few pages of results fine, and then all the cities just get stuck “Retrieving Results from Google”.  This issue just came up a few days ago, but what we’ve been able to determine so far is that Google have added a new limit to how many queries can be sent to their Custom Search API (which powers our results) in a short period of time.  We’re in the process of determining what those limits are, so that we can put our own limits in place to avoid hitting them.

For the moment, as long as you’re taking the time to read each page of results before moving on to the next, it’s unlikely you’ll have this problem.  Where it is more likely is if you’re searching for something rare, so a bunch of cities are skipped through quickly.  For now we suggest just taking your time with those searches.  Loading a chunk of 50 or so cities (including ones with no results), waiting 30 seconds, then loading the next chunk.  That should avoid hitting any limits.

Another option is to switch your sort order (using the “Sort” drop down on the results page) to one of the single list options.  That way a separate query doesn’t need to be sent to Google for each city, which will guarantee no limits are hit.

Again, we’re still actively working on this and hope to  have better work-arounds in place soon.  You can find more info here:

Default Results Sort Change

Update: There has been… significant… reaction in favor of sticking with Stacked Cities as the default, so we’re going to switch back, and look at ways to improve that view.  As always though, the sort order is entirely your choice, and can be changed using the “Sort” dropdown on the results page, just below the search bar.

SearchTempest is powered by a Google Custom Search.  Basically Custom Search is an API that Google offers, allowing other websites to run customized Google searches.  Like any search engine though, Google sometimes misses results, especially new ones.  We don’t like missing results, so recently we added Bing search as a backup option, to try to catch anything Google misses.

Unfortunately, it’s not possible to search each city separately with Bing like we do with Google.  We can search all the cities in your search area at once though, and show the results in a single list.  And we have found that for most searches, this Google + Bing combination returns better results than the “Separate Cities” searches we do with Google.  Therefore, we’re switching the default sort to a single list, including Google and Bing.

The “Separate Cities” sort options aren’t going away though!  If you preferred that way of viewing results, changing back is as easy as clicking the “Sort” dropdown box on the results page, just below the search bar.  We’re also looking at creating a sort of hybrid view that could offer the best of both worlds.


Finally, we also still do offer our Direct Results view, which doesn’t miss anything, so that might be of interest to some of you, especially if you like having the results separated by city.  You can learn more about Direct Results here.


I recently read a great essay by Tim Urban at WaitButWhy on beating procrastination. (I found it, as you may have recently, via this great reading list.) Among the illustrative images, one thing stood out to me, here:

The Dark Woods - Credit:
The Dark Woods – Credit:

This image is used to build an analogy for beating procrastination, in which one must forgo guilty leasure in the “Dark Playground” by entering the “Dark Woods”, a path representing work and accomplishment. If one can make it all the way through the dark woods and complete the task without being enticed back into the leisure of the dark playground, one reaches the “Happy Playground”, a place of well-earned, satisfying leisure.

However, there is an alternative: Flow. As Urban puts it,

You occasionally even end up super-engaged with what you’re working on and enter a state of Flow, where you’re so blissfully immersed in the task that you lose track of time.

Flow is a fantastic thing: hugely productive and fulfilling. Just as the Dark Woods are surrounded by the Dark Playground, meaning that one is constantly enticed by the ability to quit the task at hand and return to procrastination, the Flow path is surrounded by the Happy Playground; it is possible to break out of flow at any point and enjoy your accomplishments.

However, I found it telling that the end of the Flow path wasn’t pictured. I’ve found that just as the Dark Woods can escape from the Dark Playground, the Flow path can “escape” from the Happy Playground. At that point it no longer feels like an option to just quit and relax. You’re *accomplishing* things! Leisure time would just be a *waste*. The Flow path also continues to spread out, becoming less focused, less productive. Eventually, you end up at a point where you’re no longer in Flow, you’re just *Floating* (or “Flowting”, if you like).

I see Floating as the flip-side of the procrastination coin. When you procrastinate, you know you should be working, but your instincts betray you and cause you to seek out unfulfilling entertainment instead. When you’re Floating, you know you’re no longer being productive and you should take a break, but now your instincts are telling you that you can’t waste time; you have to keep pushing, keep working.

I wouldn’t be surprised if there’s a correlation between susceptibility to procrastination and Floating. One of the aggravating factors of the Floating state is the knowledge that a period of beneficial rest could ultimately lead to procrastination. Given their similarities, perhaps the remedies are similar as well. Perhaps the solution to Floating is structured recreation. I’m not sure. I do know it’s a real issue though. Perhaps now that I have something to call it, it will become easier to recognize and combat.

How to pin a Steam game to the start menu in Windows 10 (without special software)

This has nothing to do with Tempest or web development, but I just figured this out, and it might be useful to someone. For some reason, Steam lets you create desktop shortcuts to individual games, but there is no straightforward way to add a tile pointing to the game in your start menu. There are apps that will do this, but they require access to your steam account, and there is an intermediate step in opening the game.

There is a way to manually create shortcuts to steam games that you can put anywhere though, including the start menu. Here are the steps:

  1. Open Steam, right click on the game in your library, and choose “Create Desktop Shortcut”
  2. Right click on the new shortcut, go to Properties, and then copy the ‘URL’ section. It should be something like “steam://rungameid/212680”
  3. Right click on the desktop and go to New->Shortcut
  4. When prompted for the location, type “%windir%\explorer.exe” (no quotes), followed by a space, and then paste the URL you copied from the Steam shortcut. So, what you end up with should look something like, “%windir%\explorer.exe steam://rungameid/212680”
  5. Set the name of the shortcut to the name of the game.

You now have a working shortcut that you can pin to your start menu or taskbar. As a final step though, you can give it an icon matching the game. To do that, right click on the shortcut and choose “Change Icon…”. Browse to C:\Program Files (x86)\Steam\steam\games (assuming the default location for your steam install). In there you should see a number of Icon files with random strings of characters as names. One of them should match your game. Select it, and you will be able to set your new shortcut’s icon to match.

And that’s it! A bit of work, but it should only take a minute in total once you know the steps, and no special software is required.

One last note though: once you pin it to the start menu, don’t delete the original shortcut, or strange things will happen, like losing the icon. If you don’t want it hanging around on your desktop, move it to a different folder, like %AppData%\Microsoft\Windows\Start Menu\Programs (your start menu folder), before right-clicking on the shortcut and choosing ‘Pin to Start’.

Marimedia Ad Network Does Not Pay

Over the course of running SearchTempest, I have had the opportunity to try out quite a number of ad networks. Unfortunately I’ve found that most do not have sufficient quality of ads nor earnings to warrant using them alongside Google Adsense. (One notable exception is Sovrn, formerly Lijit, which has been quite good for us.)

However, at least all the networks we’ve used so far do pay their publishers the amounts owed… except for one. We tried out Marimedia for a short time in early 2014. They now owe us several hundred dollars, which has been due for a year, and remains unpaid. We have sent multiple follow-up requests, and have been promised payment several times, but it never comes.

A few hundred dollars isn’t the end of the world, and certainly isn’t worth pursuing legal action, but I felt other publishers deserved to be warned. At least in our case, Marimedia does not pay its bills. I would avoid them.

If the overdue invoices are ever paid, I will update this post.

Update: About a week after writing this post, I was contacted by someone at Marimedia, and about a month after that I did finally receive payment. Since then I have been contacted twice by other Marimedia representatives with no knowledge of our past relationship, asking me to become a publisher for them. Also once after the invoice was paid, to arrange paying it. So, it seems like there’s nothing underhanded going on; they’re just really disorganized.


Hiring Manager

Of all the titles I hope to never have in my life, “Hiring Manager” is right up there with “Sanitation Engineer”. So if you’re applying to our job posting, please, just call me Nathan. Or use our company name if you prefer. But please, no “Dear Hiring Manager.” 🙂

Craigslist blocking Feedly again

It looks like craigslist is once again blocking Feedly from accessing its RSS feeds. This happened a few months ago when Feedly traffic apparently got high enough to hit an automated block on craigslist’s end. They made some changes to reduce their traffic, and things started working again… until now. As far as I know there’s no official word from craigslist, but it seems likely that Feedly has simply grown to the point where they’re hitting the block again.

Unfortunately this means that at the moment RSS feeds from craigslist (like the ones from our RSS Feeds Tool) are not updating on Feedly. Most likely they will get this sorted out in the next few weeks, but if you don’t want to wait, there are a couple options. First off, other popular readers, such as NewsBlur and TheOldReader appear to be working for now, although as people move over from Feedly it likely won’t be long before they’re having the same problems.

A long-term solution, although with a bit more effort, would be to host your own RSS server. If you like NewsBlur, you can actually self-host it on your own server (or home computer), which will not only avoid getting caught up in these kinds of blocks, it will also save you the annual subscription fee. Code an installation instructions can be found on Github here.

Another popular option with a bit more detailed installation instructions is TinyTinyRSS. There’s a primer on MakeUseOf here. Whichever reader you choose, self-hosted or not, you should be able to import the OPML files generated by SearchTempest’s RSS Feeds Tool. Let us know how it goes in the comments!

Edit: It sounds like a number of people have been having trouble lately with self-hosted RSS as well. This thread at TheOldReader might shed some light. Apparently craigslist recently made a change to how they redirect RSS urls, which TheOldReader says isn’t supported by many other readers. So if your RSS reader isn’t picking up craigslist feeds, you might want to ask them to look into that.


Quick tip for other businesses out there. Regardless of what business you’re in, or what kind of website you have, if your site goes down for “maintenance” every night, you’re doing it wrong.

Oh, and if your website is “closed” during non-business hours, you’re really doing it wrong, although it appears the standard culprits there are government agencies.

(Sheesh, don’t people know 2am is a prime working hour for us entrepreneurs? ;))

Standardized password-change path

Heartbleed is making me realize how much the internet needs a standardized way to change your password on websites. Right now if you want to change your password on a given site, the process looks something like

Go to homepage > Search for login option > search for account/settings/profile/options section > search for password change prompt > oh, looks like “account” was wrong… try “profile” > hmm.. nope… maybe it’s in a submenu somewhere > gah.. maybe I can google where their password change page is…….

How great would it be if there was a standard like, the place to change your password on any website?


How Namecheap is preventing thousands from reaching our site

Until yesterday, had used for both domain registry and DNS. They are one of the least expensive registrars out there that isn’t named GoDaddy, and generally have a good reputation, so this seemed like a reasonable choice. And DNS came free with domain registration, so we didn’t see any need to look elsewhere.

That was until several recent complete outages of their DNS servers. Now we don’t blame namecheap for that. Their business isn’t distributed DNS, and they certainly didn’t DDoS themselves. However, it did demonstrate our need for a more robust solution.

We settled on DNS Made Easy. They appear to provide a very robust, globally distributed, fast, user-friendly, and inexpensive solution. But this post isn’t about them. It’s about what happened when we tried to switch from namecheap’s internal DNS servers to the ones from DNS Made Easy.

The right way to transfer DNS is pretty straightforward, but it’s important that it be followed to avoid apparent downtime. Generally nameserver records (the locations of the nameservers themselves) are cached for 24 hours. So, when you want to change your nameservers without downtime, you just follow these steps:

  1. Configure the new nameservers with all necessary records.
  2. Point the domain at the new nameservers.
  3. Wait 48 hours* for the cache period to expire.
  4. Remove the records from the old nameservers.

*or whatever the TTL of the NS records is

This process is explained pretty succinctly in the first section here, for example:

But pay attention to the fact, that the NS records of your parent DNS servers are usually cached for 48 hours. Thus you should keep your old nameservers online for at least 48 hours after making the changes to your NS records.

The problem is, at namecheap, when we performed step #2, they immediately did #4: removing our records from their DNS. That means anyone who has accessed the site within 48 hours suddenly has a stale cache and is unable to get there again, unless they know to flush their dns, or wait 48 hours. (And if it’s their ISP that cached the DNS info, they have no choice but to wait.)

I immediately contacted Namecheap support, hoping that they could reinstate our records for the remainder of the 24 hour period, but they repeatedly gave me the canned (and incorrect) response that downtime is inevitable with DNS transfers, and I should simply wait 24 hours (apparently oblivious to the fact that a 24 hour outage of a busy website is kind of a big deal, and the fact that their NS records actually had a 48 hour TTL).

Eventually, after two fruitless rounds with namecheap “tech” support, I was able to establish that they should have preserved our records, and that it is in fact their policy to do so for a period of 5 days. However I now couldn’t convince them that this had not, in fact, happened.

Finally, with a bit of help from DNS Made Easy (which appears to have very competent tech support), we figured out the problem. Namecheap has two sets of nameservers, which they call “DNS v1” and “DNS v2”. The problems we had a couple weeks ago were with v2, so we switched to v1 at that point, while we sought out a more permanent solution. However, when we transferred yesterday, they preserved our records on their v2 servers (which we haven’t even been using for weeks!), but not on v1 where they need to be. I was finally able to explain this to the third namecheap tech I spoke to, who told me that the v1 servers are controlled by a separate provider, and there must be a problem on their end. She apparently sent them a ticket.

That was now 13 hours ago, with no resolution. I apologize profusely for the inconvenience users of are suffering. Hopefully it’s some consolation that I’m at least as frustrated myself. If you’re unable to access, you could try flushing your DNS cache. The easiest way to do that is to restart your computer. If that doesn’t help, unfortunately the only options are to call your ISP and ask them to flush the nameserver records for from their cache, or to wait until the cache expires – potentially until tomorrow afternoon.

Otherwise, all I can do at this point is warn others to avoid the same pitfall. Go ahead and use namecheap for domain registration, but switch to an external DNS immediately, before your website has traffic. It is easily worth a few bucks a month to avoid these kinds of problems. If you’re already using their DNS services, it should be possible to transfer out without downtime, but make sure you’re on v2 before transferring out. And good luck.


I just followed up with Namecheap tech support for the fourth time, to ask why our records still haven’t been restored on their partner’s web servers. Unfortunately it sounds like the response they got from their partner was almost identical to the canned response they repeatedly gave me:

When you change nameservers for a domain name, these changes are not accepted instantly all over the world. It may take up to 24 hours (in rare cases more) for local ISPs to update their DNS cache, so that everyone can see your website. Since the caching time varies between ISPs, it takes time for DNS changes to be totally in effect. Unfortunately this process cannot be influenced or sped up because of its automated nature.

Once again ignoring the real problem. We know DNS propagation is not instantaneous. But if they leave the records on the old nameservers until the TTL (time to live) of the old NS (nameserver) records has passed, everyone will still be able to access the site while the propagation takes place. What’s more, according to at least one of the tech support reps I spoke with, that is in fact their policy. It’s becoming clear though that the cache period will have expired long before I will be able to find someone willing and able to make the 10 second change that would fix this problem.