Posts filed under 'Metrics Series'
This is an installment of the ongoing “Metrics Series.”
A bunch of weeks ago, we made some infrastructure changes that really sped up our site. Our pages were pretty slow before that. The immediate change was that the percentage of “quality visits” (a metric we use to look at the number of people who find value on Boxxet) went up 10%.
But a more qualitative metric of our slow page speed can be seen as a loss of a “star” in a review. The week we made the page speed improvement, a review of Boxxet on LAPTOP Magazine came out. It was a very positive review, but one of the criticisms was “the service is in its infancy, so it is not up to speed yet-literally. Performance was sluggish last time we visited…” Yeah, they were right.
Filed under: Boxxet, Metrics Series
May 30th, 2007
You Mon Tsang
This is an installment of the ongoing “Metrics Series.”
When is being FIFTH 5x better than being TENTH? When you get in the Top Five of search results, of course! We’ve watched the power of being a Top Five search result for a very popular search and, in real time, watched what happens when we fall out of the Top Five.
Let me explain. Boxxet is still a very young site (been out for two months) and so we are still working our way up as an highly ranked site in Google and other search engines.
Several times, we notice that a certain news event would drive a lot of traffic to our site through a high ranking on a Google search. We would be ranked in the Top 3 or so. Partially, this is due to our crawlers doing a great job getting stories quickly. And partly this is luck (Google crawled that part of our site at the right time). We can easily measure the hourly rate of inbound traffic for such a search.
But as other more highly-ranked sites get the same story and get indexed, the Boxxet result falls down the rank. First to 6-10. Then off the first page to the mid-teens. And we can measure the traffic change.
Here is our estimate of what it means to be in the Top 5 / 10 / 20 of search results.
- Top Five is 5-6x better than Top Ten (6-10)
- Top Ten is 10x better than Top Twenty
I have seen references to more formal studies out there, but there is nothing like direct experience.
Filed under: Boxxet, Metrics Series
March 21st, 2007
You Mon Tsang
This is an installment of the ongoing “Metrics Series.”
Like many other folks, I think Alexa data is just plain wrong.
When I compare Boxxet’s numbers (using two totally different methods) to Alexa’s numbers, there is absolutely NO correlation. As many others have reported, Alexa is heavily skewed with tech users and it shows. One quick example, Boxxet’s most recent biggest day, according to Alexa, was our launch date (Jan 16, 2007). Yes, we were pleased with that day, and we got lots of attention from the tech bloggers. However, Jan 16 generated only 1/3 the visits of our actual best day (Feb 12). And Alexa reports Feb 12 to be 1/7 of Jan 16. Alexa is off by 21x.
Despite my frustration of our incorrect Alexa numbers, and the near-daily exasperation I have with the many people who rely on it to make snap judgements (including me in the past), I chose not to blog about it. Also, there may nothing better out there and I do like Amazon.
But this weekend, I saw that Alexa had an upgrade to their service and in their blog announcement, they said:
I expect that this new traffic data will be particularly useful to potential advertisers. It will help to size up an audience before making an ad buy.
Until now, I wasn’t sure why Alexa was doing what it was doing. I figure it was some sort of giving back to the community. But with this statement, Alexa sees itself has a useful business tool. PLEASE STOP, YOU ARE NOT!
Bad data is more than an inconvenience, it can be downright hurtful to a business. Great web sites may be shut out of advertising, business deals may never happen because a company did not pass a fictional Alexa test, potential employees may choose not to join a company b/c of its Alexa numbers, adversaries may aim their resources at the wrong competitors.
Amazon/Alexa, you are encouraging people to make business decisions using your very suspect data. I believe you have the resources to make Alexa site traffic numbers great. Please please do so.
Filed under: Metrics Series
February 19th, 2007
You Mon Tsang
This is an installment of the ongoing “Metrics Series.”
Boxxet spent a long time in a closed beta, longer than we expected. Some reasons were normal: stamping out bugs, making sure the application scales, and determining weak areas that need improvement or protection. More importantly, after our first wave of users of the closed beta, we felt that we needed to add more automation to the system.
There are, of course, pluses to an open beta (get quick feedback from the real world on what works or does not) and a closed beta (you make sure your first impression to the masses is a good one). We felt we did the right thing by staying in a closed beta. It allowed us to make large changes without worry about massive disruption, shore up our infrastructure and experiment with new algorithms at will.
However, the downsides are significant and I’d like to share some metrics for others who need to make such a decision. Perhaps the numbers will help folks make a more educated determination about how to proceed with a beta.
When we announced Boxxet was going into beta, we got slashdotted. We estimate that approximately 40% of the people who came to our front page signed up for the beta. Frankly, this is more than I expected. However, upon letting people know that they have been given access to the beta (within a few weeks), 60% returned to take a look. In some cases, we waited a few months before we gave people access and the drop off was even greater than 60%.
Of the people who came to our front door, 40% signed up and 60% returned when we emailed them. That means 76% of the people who stop by to a closed beta did not immediately return to see Boxxet when we let them in.
Of course, if we did not effectively serve these 76% (slow pages, crashed servers), we may have done something worse: lose them forever.
But this is a choice and I wanted to make sure that others understood the numbers on both sides of the equation. As we all know, the cost of user acquisition is quite high and not giving 76% a first impression hurts.
Filed under: Boxxet, Metrics Series
February 12th, 2007
You Mon Tsang
This is an installment of the ongoing “Metrics Series.”
A few weeks after we opened up Boxxet to search engines (about 7 weeks ago), I was worried. Traffic from Google started to come in at a good clip, but traffic from Yahoo, MSN and Ask was miniscule to non-existent. Given the latest search engine stats, I wanted non-Google traffic to give us roughly half of what Google was giving us. Here’s what Hitwise estimates as search engine share in Dec 2006.
We believe we went through the appropriate steps to let search engines know about us and told them through available APIs and submissions that we were ready to be crawled. But Google completely dominated the organic search traffic to Boxxet. We talked to friends at the search engine companies, scoured for tips and decided that maybe time will solve the problem.
But time has not brought change and at Boxxet, Google continues to bring in over 90% of all organic search engine traffic.
But recently, I have also noticed that other high-profile sites have similar organic search share. For instance, the top 11 referrals at Techcrunch in Dec 2006 look like:
1. google[organic] 391,034
2. (direct)[(none)] 326,796
3. digg.com[referral] 192,774
4. google.com[referral] 78,960
5. news.bbc.co.uk[referral] 46,621
6. netvibes.com[referral] 32,444
7. techmeme.com[referral] 25,561
8. stumbleupon.com[referral] 22,294
9. reddit.com[referral] 22,035
10. my.yahoo.com[referral] 19,643
11. techcrunch.com[referral] 18,869
On this list, there is no other organic search site besides Google, so I will make the following assumption:
12. yahoo [organic] 18,000
13. Others search engines: 18,000
So Techcrunch’s organic search share looks very similar to Boxxet’s.
Over at Search Engine Land, the folks there shared their top 25 referrals from Dec 2006 and Google is the only organic search site! So here again, I estimate Yahoo and the others as follows:
Google 6,172
Yahoo* 280
Others* 280
Search Engine Land also looks the same as Boxxet (as well as Techcrunch).
The organic search share at Boxxet, Techcrunch and Search Engine Land are virtually identical and heavily skewed towards Google. So the next step in solving this supposed anomaly is looking for commonalities.
- Boxxet and Search Engine Land are pretty new sites so perhaps the non-Google crawlers haven’t figured us out yet. On the other hand, Techcrunch has been around for almost two years.
- Techcrunch and Search Engine Land are tech sites, but Boxxet is focused mainly on entertainment and sports.
- Techcrunch and Search Engine Land use blogging software (looks like Wordpress and Movable Type). Boxxet has its own presentation engine.
- Only thing that we share is Google Analytics, but let’s pray that is impartial.
Bottom line, I am starting to draw a heart-stopping conclusion: Google’s search share is > 90%. I am open to any other thoughts or theories, but I am preparing for this new world. Rich Skrenta has a very thoughtful take on the era of Google dominance and that was assuming 70% dominance!
When a well-known and well-branded site like Techcrunch gets more people coming in through a Google search than by typing in “www.techcrunch.com,” I get a chill up my spine. Will future successes on the Web be determined by a bot originating from an office park in Mountain View?
Filed under: Boxxet, Metrics Series
February 6th, 2007
You Mon Tsang
This is an installment of the ongoing “Metrics Series.”
We launched Boxxet last week and we got covered in a fair bit of the Web 2.0 media, including from Rafe Needleman at Webware (part of c|net), Dan Farber at ZDNet, Liz Gannes at GigaOm, Pete Cashmore at Mashable, Matt Marshall of VentureBeat, and Chris Sherman at Search Engine Land.
Certainly, many outlets did not write about us, but, all in all, we got fairly broad coverage in the Web 2.0 space. We were expecting a fairly large spike in visits, hopefully the biggest in our recent history and we were on full alert to make sure everything worked properly.
Well, launch date wasn’t our biggest day in January (despite what Alexa says, but that’s a post for another day). Not even close.
A few days before the launch, we were a relatively high search result for a breaking Jennifer Aniston story. A few days after the launch, we were a prominent search result for a breaking Brad Pitt story. Both those days blew away our launch day. In the chart below, I break out visits to Boxxet from people interested in Jennifer, Brad and our launch. This charts says nothing about the quality nor the long-term benefits of the different visits (I’ll take the Boxxet coverage traffic if you must know), but it does say, “oh yeah, Jennifer and Brad are really really big.”
We started Boxxet because we felt that the most popular searches and topics on the Web could be made better and better with good technology and fan participation. Why is becoming clear is just how much big some of these topics are. And, as I have always said about metrics, it can certainly put things in perspective. Wait! As I am writing this, we are seeing a sudden wave of visitors interested in Tyra Banks…
Filed under: Boxxet, Metrics Series
January 30th, 2007
You Mon Tsang
A few years back, while I was working in enterprise software, I thought, “wouldn’t it be great to grow a software company past $1B in revenue?” That would make us one of the biggest software companies in the world. Top 20 for sure, maybe even close to Top 10.
Shortly thereafter, I read that the Martha Stewart line of products at Kmart accounted for $1.5 billion of revenue per year. The revenue from towels, bedding and household goods by Martha would be a Top 10 software company!
Sure, margins are higher in software, blah blah blah. But my point is that numbers can really put things in perspective – really shake up one’s own world. My career has been focused on business intelligence, metrics and content analysis and management. Getting good information, whether for fun or business, has always been part of my goal.
So I am starting a “Metrics Series” where I plan to share some fun and hopefully useful metrics around Boxxet. Watch out for the first post shortly.
Filed under: Metrics Series
January 29th, 2007
You Mon Tsang
