At Least The Idea Was Good

Subscribe by email:

No real tips here, just something I was thinking about tonight that sort of made me chuckle.  Back at the turn of the millennium (isn’t it cool that we can say that?) I was working with a couple of guys on a system to measure and track a web site’s popularity.  What we were trying to do was provide clients a service that would give them a score for their site based on two major components–links from other sites and search engine rankings. We were going to get mega-rich doing this of course.

The first side of the equation, links from other sites, worked basically the same way PageRank works now. Every page on the web would be assigned a score, and that score would be determined by the scores of the pages that linked to it. Remember, the web was a lot smaller back then. It goes without saying that we underestimated the enormous size of the web just a few years later, and we hadn’t even started to think about the small complexities that make this problem so difficult or the amount of computing power it would take to tackle it. We also had no idea that guys a lot smarter than us from Stanford were doing this and planned on giving it away as part of a huge advertising company.

That’s right–Google’s an advertising company.  They just happen to have a lot of stuff you want to look at and use. And they make sure their ads are involved somehow. Google’s search engine just happens to be one of the tools they offer that you like to use.

Search was the other side of our scoring system. What’s crazy to think about now is the number of  search engines we were factoring in. I want to say our first prototype was analyzing results from 35 different search engines. Back then, Yahoo! was the big dog, but several other search engines had decent market share.  To top it off, the guys who were heading this project up and funding it were in Europe, so we had to factor in the engines that were being used there. We were going after the long tail before we  knew there was a long tail.

We were looking at the first 10 pages of results for each search term on each search engine. And we were planning on each customer having 10 to 20 different search terms each. That’s at least 35,000 results analyzed for each customer if you’re keeping score. And we were going to do this once a day. And if a search engine changed the layout of their results?  Well, we’d have to re-write our method of parsing those search results. Like I said, we hadn’t really though out all the details.

Having pulled several all-nighters attacking this problem, I’m utterly amazed that now Google gives away their PageRank score not only to site owners but to anyone who cares to look at it. And after all of that time spent writing software capable breaking down pages and pages of search results from different engines, isn’t it funny that all that really matters now is being on the first page (really, in the first three results) of Google?

If you’ve checked out your anaylitcs, you’ve probably noticed that Google is the only search engine that matters, at least for now.

I’m glad somebody got mega-rich on this stuff. I at least feel better knowing it was a good idea.  😀

Similar Posts:

Be Sociable, Share!
Tags: , ,


  1. Bill Woodall said February 7, 2009 | Permalink

    If you’re going to rely totally on google-analytics, then of course Google is the only search engine that matters.

    If you want to be sure of your reach, go ahead and put some bait in for the older engines, especially Yahoo-Inktomi and Lycos… there are a lot of corporate portals built on those older technologies.

    IF you want to know what is really going on, learn to use logfile analysis tools as well. Proxies (intermediate buffering and caching systems) can and do strip off the embedded scripts which google-analytics uses; thus you may have a substantial audience at AOL or in the APO/FPO realm which will not really register on your radar.

  2. Scott Adcox said February 7, 2009 | Permalink

    Very good point Bill. What I probably should have said is that in general, Google is what matters. But you’re right…I have one site that gets a lot of traffic from MSN and AOL. I suspect a big reason for that is that the demographic interested in the content is more likely to use their search engines than they are Google. However, I also rank higher for my keywords on MSN and AOL than I do on Google (something I’m working on), and I think that plays a major factor as well.

    Even with that situation, Google accounts for ~60% of the SE traffic on that site, which is relatively in line with their market share they own.

  3. Blue Collar Muse said February 9, 2009 | Permalink

    This sort of reminded me of the tech guy from one of my favorite movies, “The Italian Job”.

    When he sets in motion the over-the-interwebz-attack on LA’s traffic system from a laptop at the airport, he sends the message “You’ll Never Shut Down the Real Napster” springing from his contention that he was the owner of the Napster idea and it was stolen from him by his roommate while he was “napping”.

    Nice to know the tech side of this project is thinking in the same terms the most successful minds in online tech are thinking.

    Nice job, Scott!

  4. Scott Adcox said February 9, 2009 | Permalink

    But then again, I’m thinking more along the lines of, ‘It would be cool if someone…”

    The real all stars are the people thinking, “It would be cool if I…”


Your email is never published nor shared. Required fields are marked *




  • July 2011
  • June 2011
  • August 2009
  • July 2009
  • June 2009
  • May 2009
  • April 2009
  • March 2009
  • February 2009
  • January 2009
  • December 2008
  • Categories

  • Categories