What a day

I had plans for today. I was going to do some coding.. add the new Google +1 thinger to SearchTempest, update the FAQ, port some stuff from that FAQ over to the AutoTempest FAQ, tweak the ads on SearchTempest, even work on a new feature or two! … my server had other ideas. I had an email waiting for me this morning from someone who couldn’t find an ad they had posted using SearchTempest. These are fairly frequent, and 98% of the time it’s for one of the reasons covered here. When the emailer claims to have already read the FAQ, that number drops to about 97%. ๐Ÿ™‚ This case was no different. What was different was that when I pulled up SearchTempest to verify its omniscience, it took about 47 seconds for the homepage to come up. I pay way too much for screaming fast internet, so this was definitely not normal. Took a look at the apache logs, and noticed a bunch of entries like this:
mod_fcgid: can’t apply process slot for [filename]
mod_fcgid is a fastcgi module for apache, which allows apache (the web server) to talk to things like php without having to load them inside the server itself. This is great for a bunch of reasons that I won’t get into here. What wasn’t great was that it was out of processes. The server is configured such that it should handle many times the current peak load of the sites (just in case I get slashdotted or what have you.. ๐Ÿ™‚ ), so that shouldn’t happen. At this point I’m kind of freaking out because a) all you fine people can’t access SearchTempest, b) the other fine people who probably aren’t reading this because they’re new users are now thinking that SearchTempest is crap and doesn’t work!, and c) I’m nearing the end of my very limited LAMP (Linux, Apache, MySQL, PHP) debugging abilities. I sent off a somewhat panicked message to the support folks at the company I lease the server from, then, as is my way, proceeded to get in their way trying to debug the problem myself at the same time. Read More

More on Inspiration

While I’m on the topic, here’s a great TED talk I saw today. Theoretically it’s about entrepreneurship, but it touches on all kinds of interesting concepts. If nothing else, skip to the last couple minutes and listen to his question at the end. But don’t do that, because the whole talk is great. ๐Ÿ˜‰ If you’re not familiar with TED talks (or if you are and love them as much as I do), here are some more I’ve enjoyed: Dan Gilbert – Why are we happy? and Mistaken expectations Paul Collier on the “bottom billion” Benjamin Zander on music and passion And about a million more. Feel free to share your favorites in the comments!

My Secret Inspiration

Well, not secret anymore I guess. The about page talks about how I got into this Tempest business, but it doesn’t really cover why I decided to try my hand building a website in the first place. I don’t know if I should admit it, but the real catalyst was the Million Dollar Homepage. You may remember it from a few years ago. A college student in the UK had the brilliant idea of selling a million pixels for a dollar each (in 10×10 blocks). After some shrewd marketing it went viral, and a few months later he’s a million dollars richer. Clearly, I wasn’t the only one inspired by this. Soon after, there was a rash of ‘Ten Thousand Dollar Homepages’ and such. (After all, why pay $1 per pixel when you can pay 1ยข? :)) Unsurprisingly, these didn’t do so well. But it got me thinking – you’re not going to get rich copying what’s already been done, but there are plenty of other cool ideas out there. The only difference with the million dollar homepage was that the guy who had the crazy idea actually followed through and did something about it. And hey, what’s the risk? Worst case I learn a bit about web design – there are worse ways to spend your time. So I did some brainstorming, and came up with a fantastic idea. It was something.. about.. showing ads at various times of day, and people would.. guess what ad was coming up.. or something. Fantastic idea. ๐Ÿ™‚ Only thing was, it seemed a bit ambitious for a starter project. I figured I needed a little test project to get started. Not anything with real potential, just something nice and simple to do while I learned what I was doing. So I figured I’d just make a simple little site to help search multiple cities on craigslist. (You know, just until I learned the basics and could switch to working on something with real potential…) Mostly I just wanted something that would help with my air conditioner search. I may have had my priorities a bit backwards, but with a little luck, things turned out pretty well. The key is that I did something. So should you. You don’t need to start with a perfect idea; my first one was terrible. You don’t need to know how to start a business; you’ll learn as you go. You don’t need a lot of money; you can keep your job and get started part time. You just have to start something. In the words of Michael Masterson, “Ready, Fire, Aim!” Fortunately I also have a few other sources of inspiration that you might actually find inspirational. ๐Ÿ™‚ Premier among those is Paul Graham. I’ll write more about him and others later, but for now, go check out one of his essays. Just pick a title that sounds good, and I promise you an enlightening read.

If it ain’t broke… it can probably still be optimized

Any time I change anything on SearchTempest (or any of the Tempest sites for that matter), I get two broad categories of responses. Some people write to thank me for the new feature, let me know how useful it is, etc. (We like those people.) Others write to inform me that I must be most mind-numbingly stupid person on the face of the planet to think that anyone, ever would use this ridiculous new feature. (OK, they’re not all that extreme, but that’s the gist.) I admit I enjoy the comedic effect when both varieties of emails arrive at the same time. ๐Ÿ™‚ To an extent, I can understand where they’re coming from. It’s the same reason people hang on to ancient web browsers or other software when new, better, free alternatives are available. When you’ve found a solution that works for you, there’s little motivation to take the time out of your life to learn something new. With a website though, you don’t have a choice, which can be frustrating. On the other hand, we don’t have to look far to see what happens when you take “If it ain’t broke, don’t fix it” to extremes. You get craigslist. Craigslist is fantastic, but I’m guessing you wouldn’t be here if you thought it was perfect. So what’s the solution? Well, first I’ve had to accept that I can’t please absolutely everyone. Instead, I have a rough algorithm for any changes I’m considering:
Likelihood of change = difficulty level * (users helped – users annoyed) / total users
Difficulty factors in because it’s better to do a bunch of things that each help 10 people and take a day than one thing that helps 20 but takes a week. Second, when I’m contemplating any change, I think on how I can minimize the annoyance factor. Mainly, this involves making things optional. For example, the SearchTempest results originally showed just one craigslist city per page, with Previous and Next links to switch between them. Nowadays the default mode shows 20 cities per page, which lets you flip through them more quickly. The original mode is still available though – just click the Display Options tab on the results page. About 10% of searchers still use it, so clearly it’s worth keeping around. That’s not always the case. In theory, making everything optional sounds perfect. Everyone gets exactly what they want! There are two significant drawbacks though. First, each additional option means additional development and maintenance time. And every minute I spend keeping a legacy feature working is a minute I’m not spending on a cool new feature, or even a cool new site, like MovieTempest.com. Second, every additional option makes the site incrementally more complex, which makes it appear harder to use. Even if all these extra options can be safely ignored (which they can due to intelligent defaults), a new user doesn’t know that. And losing new users to other sites because they have less features is an irony I prefer to avoid! ๐Ÿ™‚ So in the end it’s all about balance. I will always keep innovating, because if I didn’t, we’d still be stuck with craigslist. I’ll make changes optional whenever possible, because I want the sites to be useful for as many people as possible. But occasionally I will likely do things you don’t like. I hope you can find it in your heart to forgive me. ๐Ÿ˜‰ And if not, I at least hope your email’s funny.

How to buy stuff on sale – even when it’s not

Of course, the best way to find great deals is to use SearchTempest to search craigslist and eBay. But occasionally I admit it makes sense to buy from an old-fashioned store. (Like Best Buy for example. ๐Ÿ™‚ ) A few months back I was helping my sister find a new laptop. She was on a tight budget, so did a bunch of shopping around and eventually found one that fit the bill at Best Buy. And it was on sale – perfect. Didn’t get a chance to tell her until the next day though, and the sale price had vanished from the website. No worries. I just pulled up Google and did a search for that product. For example, HP E-350 laptop site:bestbuy.com Then, I opened the cached version of that page. Generally Google’s cache has a couple of days’ delay, so there was the sale page. All my sister had to do was print off that page and take it in, and they gave her the sale price, no problem. Got me thinking, you could take this a step further. Before buying any new electronics, why not head over to the wayback machine and see if it’s been on sale recently? Even if they won’t give you the sale price, you might get some bargaining room, and at the very least, you’ll know whether it might be worthwhile to wait for the price to drop again. Related tip: before buying a common product on eBay, be sure to search for it in ‘completed listings’ (under their advanced search). You’ll see what prices they’ve sold for recently, which can help you make a smart bid!

Setting your own hours is great, but not for the reasons you’d think

Now that I’ve moved from my old life to working on Tempest full time, I finally get to set my own hours. I always figured that would be great. No more dragging myself out of bed too early in the morning. No staring bleary-eyed at the clock, watching the second hand drag along until it’s finally 5:30. Work when I want. Sleep when I want. And do whatever I want the rest of the time! Great, right? And it IS. But not like I expected. Getting up when you’re rested is awesome, except that I’m someone who could sleep 10 hours a night if I let myself. And without a boss expecting you to show up at a reasonable hour, it’s surprisingly (or unsurprisingly) easy to do exactly that. Before, I had to get up for work. Now, I have to get up, because… I have some vague understanding that if I don’t, over time, I’ll feel bleh. It’s a bit harder to wrap your head around that when the alarm goes off in the morning. (Fortunately my wife rather likes me to be around during the daylight hours too, so that helps. ๐Ÿ™‚ ) So the ‘worst’ part about setting my own hours is that I get to sleep in. The best part? I can work really long hours! That’s right. The best part is if it’s 1am and I’m getting some really great work done, I don’t have to quit because I’m expected to be ‘at work’ the next morning. Or when 5:30 rolls around, I don’t feel compelled to pack it in, since I’ll have to be back at the same time regardless. If things are going well, I can grab a bite then get back to it. When I’ve finally gotten the entirety of a complex problem shoehorned into my brain, or when I’m on a creative roll and the ideas are coming one after another, I don’t have to stop. And that is great. I honestly think there are days now where I do the equivalent of a week’s work in my old 9-5 jobs.
When I’ve finally gotten the entirety of a complex problem shoehorned into my brain, or when I’m on a creative roll and the ideas are coming one after another, I don’t have to stop. And that is great. I honestly think there are days now where I do the equivalent of a week’s work in my old 9-5 jobs.
Of course, you can’t do that forever. And the other benefit is that you don’t have to. I spend periods of days or even weeks working what feels like every waking minute, but follow them with periods of similar length where the extent of my work is replying to emails. I recharge, have fun, and then, start having ideas. Eventually, I can’t wait to get back to work and let that creative energy flow into making something cool. If you’re lucky enough to set your own hours and would like to try something that might boost your productivity and enjoyment of your work, Steve Pavlina suggests one way to get started. That schedule ironically ended up being too rigid for me, but it’s at least a good primer. Paul Graham’s essay on Good vs Bad Procrastination also has some great insights. In fact, while you’re there, check out some of his other stuff. It’s all fantastic. And if you’re an employer, think about the benefits of giving real flex hours to your employees. Not “show up any time between 8 and 9”. More like “as long as you’re here for scheduled meetings, come in most days, and get the work done, do whatever works for you.” Because really, is it more important to make certain everyone’s working exactly 8 hours a day, 40 hours a week, or would you rather have happy employees producing higher-quality results, faster? Obviously there are challenges to work around, but it’s worth it. I’ll write more about that side of things in a later post.

My all-time Favorite AutoTempest Blog Post

… is not this. ๐Ÿ˜‰ Actually it’s not even on this blog. It’s this one over at wisebread.com. And not just because they call AutoTempest possibly the “best search engine ever for cars”… although that helps. It’s because of how they found the site.
The site is one I found quite by accident. I’m not in the market for a used car, I was actually looking for a quote from Bill Shakespeare’s “The Tempest” and stumbled upon this gem. Ahh, I love the Internet.
How awesome is that? The best part is, Kevin Purdy at Lifehacker apparently read that post, so the next day, AutoTempest was on Lifehacker and got it’s most visits in a single day, by about 300%. Better to be lucky than good, huh? Although I guess ‘you’ve got to be good to be lucky’ might apply too. Let’s go with that one. PS: My second-favorite AutoTempest coverage is this interview with Keith Griffin of about.com. Mostly because, as an engineer turned web programmer, you don’t tend to do a lot of interviews. ๐Ÿ™‚

SearchTempest vs The Rest

MakeUseOf.com just wrote an article comparing several craigslist search sites. Naturally SearchTempest was first on the list. ๐Ÿ™‚ What struck me though were the cases made for some of the other sites. Crazedlist – has some of the features of SearchTempest, but ‘provides instructions that will help you’ get around blocks imposed by craigslist. That’s great, except that their instructions only work for Firefox, and besides, with SearchTempest things just work – no fiddling with your browser required. Doesn’t sound like much of a selling point! Craigzoom and Search All Craigs – these sites are both essentially front ends on top of Google custom searches. And that’s a good idea. In fact, SearchTempest also lets you tap into the power of Google. Just click the ‘All Cities Together’ tab on the results page. Only with SearchTempest, you still have access to all your advanced search options – particularly the ability to search by distance from your zip code, or pick and choose which cities to search. (Or, of course, search everywhere, if that’s what you want!) You can’t be everything to everyone, but we do try! ๐Ÿ™‚

Working around craigslist’s all-negative-keywords deficiency

Craigslist is great, but if you’ve used it at all, you’ve likely noticed it has a number of … imperfections. One of these is that you can not do a search with only negative keywords. For example, a SearchTempest user wrote on our forums recently that he was having trouble with his motorcycle search. He basically just wanted to browse the motorcycles/scooters category, but didn’t want to get all the atv and scooter posts. He knew he wanted a bike, and the price range, but otherwise wanted to browse. Logically this would be a simple matter of using -atv -scooter as your keywords. Unfortunately, craigslist barfs when you do this. (And since SearchTempest passes your keywords through to craigslist, we got some barf on us too. Never fun.) Fortunately, there’s a work-around. Every craigslist post ends with “You may [not] contact this user for commercial purposes”, or something to that effect. And for whatever reason, their search system actually indexes that part of the post. (A bit of a waste, but I won’t complain since it turns out to be useful. ๐Ÿ™‚ ) So for the search above, we can use contact -atv -scooter as our keywords and we’ll match every post, except those with the words atv or scooter in them – just as we wanted! And now, SearchTempest takes it one small step further. It now automatically checks for search strings like -atv -scooter and automatically adds that ‘contact’ in there for you.