How to measure the performance of the Top 100 UK e-tailers in 11min 28secs.

Every 6 months Experian Hitwise publishes it’s “Hot Shops” list – the Top 100 e-retailers in the UK, neatly ranked in order by site traffic.

I thought it would be interesting to measure the performance of the UK Top 100 to get a sense of the “state of the art” amongst the retailers who really should be looking to optimise their site performance.

Luckily, Experian Hitwise gives you the opportunity to register and download a PDF containing the list, which you can then extract into Excel to get the list in a usable format.

So, I’ve got a list… now what’s the best way to measure the performance of 100 pages?

Well, I could use webpagetest.org and either manually test each one or use some of the scripting functions to automate it, but luckily I have access to an easier way to do it – a commercial tool by Site Confidence called Performance Analyser.

I say “luckily” but it’s not luck – I spent 6 months last year working with the team within Site Confidence that built the tool, hence it has all the cool features I wanted to enable me to move beyond “single page” analysis.

Performance Analyser can test a single page, a group of pages, or crawl your website (to some arbitrary depth or number of pages), and it can do it cross-browser (IE6, IE7, IE8, FF 2, FF3, FF3.6, FF4).

So… paste the list into the “multi-test” option, pick my browser, hit “run test” and go get a coffee and wait for the “completed” email to appear in my Inbox, which is does just under 12mins later!

Name	Experian Hitwise Top 100 e-Retailers May 2011 #4 FF3.6
Created at	2011-06-07 13:47:57 UTC
Completed at	2011-06-07 13:59:25 UTC
Job Owner	Stephen Thair
Type of Test	Multi Page
Type of Run	Self Run

So, what information do I get?

Well, I get some nice summary graphs by size, speed and a nice combo of size/speed.

I can also get the information as text data, which I can then export out as CSV.

You can also get an “all objects” view, which is great for finding those ridiculously large objects. In the screen shot below I have filtered it to only show “image” MIME types and you can see the largest item across the 100 pages tested was a 382K image on the Monsoon.co.uk website.

So lets drill down into a test so let’s look at that large spike on the size vs speed graph. I can click through on the graph and find out exactly what what site was affected, and I can see it was www.matalan.co.uk.

The waterfall graphs shows an obvious problem  - two request which failed to connect.

In fact, we can click through and find out more details about the suspect object and get the URL(s) - https://isisaccreditation.imrg.org/8025745000669E37/accredited_idis.gif and https://isisaccreditation.imrg.org/8025745000669E37/accredited_isis.gif.

It turns out that they are 3rd party “accreditation” seals that are in the Matalan footer that are slowing down the firing of the onLoad event. An easy fix there then!

Although Matalan could do with some domain sharding…

As well as some “combining files” too, since they have 14 Javascript files, about 30 images files and 5 CSS files!

If we drill down into the HTTP headers for the base HTML page we can see that it’s compressed and that they’ve set some cache control headers, albeit with no validator (last-modified or eTag), and the presence of “Pragma:no-cache” might indicate some degree of confusion…

So what is the average performance of the UK Top 100 retailers (at 2Mbps on Firefox 3.6)?

	Min	Average	Max
Total Time (sec)	0.89	4.89	19.23
Page Size (Bytes)	83,000	675,007	1,829,590
Object Count	14	83	283

And what’s the performance difference in Firefox 4? Well, the average time is 4.58 vs 4.89. So in the “real world” the performance difference is only (4.89-4.58)/4.89=6.3%.

Technorati Tags: web performance,WPO,Site Confidence,performance analyser,Experian Hitwise,UK Top 100 Websites

The future of the WPO/WCO marketplace?

In an earlier post I discussed the current shape of the Web Performance Optimisation (WPO) / Web Content Optimisation (WCO) marketplace based on material presented by Josh Bixby at the London Web Performance Meetup (see The Taxonomy of the WPO/WCO Market for more details).

So if we gaze into the crystal ball what does the future shape of the WPO/WCO market look like?

Firstly, as demonstrated by Limelight’s acquisition of Acceloweb, the established players in the “Delivery” Kingdom are going to make sure that they develop or acquire someone in the “Transformation” space to create hybrid service offerings.

So all the established CDN’s like Limelight, Akamai, Level 3 will offer “Transformation” services to optimise the (static and dynamic) content and then use the usual CDN edge-caching to optimise the delivery.

Ditto the major network hardware vendors. We’ve already seen a partnership between Citrix (NetScaler) and Aptimize, and there is no doubt that other network vendors will follow suit. It’s worth pointing out that, based upon Cisco acquiring Fineground back in 2005, that might not be the best for the market or your product as Cisco doesn’t seem to have done much with their 6 year head start…

To be honest, a wave of consolidation as the market matures is Economics 101, and it’s really what many of the startups bank on as their “exit strategy” anyway. Sell your technology off to a big company, pay off your VC’s and walk away with your £££ (or $$$) after your earn-out period. For most entrepreneurial types that a lot more fun than the hard slog of building a enterprise class, mature technology company.

So what about the actual structure within the “Transformation” Kingdom? You might recall that that was divided up based on the “location” of the transformation service (“server, network or cloud”).

My gut feeling is that these divisions will persist for the foreseeable future as they represent different market niches and different architectural patterns.

For example, some people would say that using a server-based solution is more distributed, easier to scale and it “makes sense” to transform the content close to the origin (assuming that most of the “transformed” content is cached elsewhere, so the amount of “origin” requests is minimised).

I agree… but conversely once you start to have large numbers of servers managing the installation and configuration of the “transformation engines” across the servers starts to become problematic (unless the management of the transformation engines is centralised a la the way anti-virus or patching solutions are today). Of course, the performance overhead of the transformation engine on the origin server needs to be carefully considered… like SSL encryption or GZIP some people might prefer to offload that to another device.

Enter the “network-based” solutions (e.g. Strangeloop).

Dedicated appliances should, in theory, be faster than purely server-based software, especially if the major network players get into this space and can move some of the processing into dedicated silicon. This is the market opportunity for the F5, Cisco, Netscaler etc if they can truly integrate their solutions into their existing (huge) market presence.

From a network architecture point-of-view if I can “drop in” an upgraded load-balancer or router that handles the existing “delivery” role as well as “transformation” with a similar level of performance, reliability and vendor support then that makes the “optimisation decision” much easier.

Of course, my web servers are still churning out the “un-optimised” content, which has its own performance impact on both the server and the network between the web servers and the hybrid “delivery + transformation” device.

Which is better? Totally up to you and your application architecture, traffic patterns and personal preference.

One suggestion though… ask yourself “who do you trust more” – your “network admin” or your “server sysadmin” since how well it all works will probably depend on their skills and abilities. Likewise, think about your change control policies… if you have to jump through more hoops to change your server config or your network device config that might inform your decision where to put the solution.

So what about “the cloud”? What does the future for all the little start-ups like Torbit, Yottaa, and Blaze hold?

Well, most of them will probably cash-out by being acquired by the big players as discussed earlier but there is probably still a lucrative market by sweeping up the “long tail” of small volumes sites where the economics of investing in a larger WPO/WCO solution doesn’t add up.

A SaaS model matches the SME cash flow, lowers the barrier to entry and is easy to implement (by just re-pointing your DNS).

Of course, “long tail economics” requires economies of scale – you need to sum up lots of little sites and be able to deliver that service at a low cost to make money. So the question is can one of these cloud players achieve enough market share to start to generate the economies of scale with distributed points of presence around the world, or will they eek out their lives at the margins (a la the many small niche web analytics players who make a living at the margins of the major analytics players like Webtrends and Omniture)?

Of course, it’s worth considering what the existing “cloud service providers” will do. We’ve already seen GoDaddy trying to integrate mod_pagespeed into their hosting offerings and it would make sense for Amazon, Rackspace, Azure, GoGrid or any of the other major cloud providers to integrate a WPO/WCO solution as a “value-add” offering into their hosting solution.

The “hosting provider” scenario is another one where “network-based” solutions might be preferable, since you don’t have to touch the customer’s servers and can just do it within your core network in the same way you might already offer firewall, IDS and IPS services.

Of course, if they did offer “WPO/WCO as a Service” within their existing cloud that would have enormous repercussions on the market economics, although it would certainly “legitimise” web performance as a mainstream, “must have” offering.

Don’t forget, it’s easier to make a living at the margins of a large market than try to duke it out for serious market share in a smaller one.

I look forward to discussing some of these issues at the Velocity Conference next week and likewise it will be interesting to look back at the predictions next year at Velocity 2012 to see what’s happen in a year.

Technorati Tags: web performance,WPO,WCO,webperf,Aptimize,Strangeloopnetworks,yottaa,Torbit,Blaze.io,Velocity Conference,Limelight,Acceloweb,Fineground.

The Taxonomy of the WPO/WCO Market

The slides for Joshua Bixby’s presentation at the London Web Performance Meetup are up now on Slideshare and are well worth a read.

I want to focus on one area of Josh’s presentation, particularly since it was an area that I asked him to include in the presentation, and that is “The structure (taxonomy) of the WPO/WCO market”.

Across the pond here in the UK we don’t always get to hear about the new players in the market nor always understand where they might fit in to the growing ecosystem around web performance. 

Josh lives in that market, knows most of the players, attends the trade shows etc so I was keen to get his opinion on how he thinks it all fits together. I urge you to read Josh’s presentation in its entirety but I wanted to pull out some bits to hopefully stimulate some debate and get some other opinions from the WPO community.

Firstly, Josh’s taxonomy has two Kingdoms that represent a fundamental divide between “Delivery” and “Transformation” – between those solutions that do (or don’t) “touch your stuff on the way past”. Josh puts it this way…

Historically the “Delivery” community were quite strident in declaring that “we don’t mess with your content/pages/stuff” since at the time that was a way of reducing the fear many site owners had about letting 3rd parties (e.g. CDN’s) near their content.

But as the WPO movement has developed we have seen the rise of the “Transformation” providers such as Strangeloop or Aptimize who say that “the only way to make it truly faster is to optimise it by transformation”.

Josh divides the Delivery market into 2 major phylum's (“Load-balancers” and “CDN’s”) and one miscellaneous phylum of “other stuff”.

You’ll recognise the players in the next slide that further breaks it down into different Classes within the major phyla.

So, what about the “Transformation” market?

Well, here we have 3 major phylum’s, based mainly one the location of the transformation service – at the server (e.g. Aptimize), network (e.g. Strangeloop) or “in the cloud” (e.g. Acceloweb, now part of Limelight).

As you can see from below the “cloud” space has a lot of players at the moment, most of whom Joshua places in the “beta” category. You could just has easily placed them in the “hype” category but hopefully some of them will mature into decent products, not least for the commodity, low-end of the market.

So? What do you think of this taxonomy?

Has Joshua used the right characteristics to segment the market?

Should there be more Kingdom’s that just “Delivery” versus “Transformation”?

Are there some major players missing off the list? Where would you put them in the taxonomy?

And the final question – where is the market going? (That’s one topic I will tackle in a later post on the economics of WPO as a follow up to my earlier post on “Web Performance is not just for Christmas”).

Technorati Tags: web performance,WPO,WCO,Strangeloopnetworks,Aptimize,Accelloweb,market structure

Web Performance Analytics using Pion & WebTuna

I’ve been playing around with Jing to create some “how to” and “demo” videos so I thought I would whip up a quick demo on how you can use Atomic Lab’s Pion and WebTuna to build a real-time performance analytics solution for your company.

Atomic Lab’s Pion is a software solution that passively sniffs the HTTP traffic off your network via a SPAN port or network tap and uses that data to generate real-time web and performance analytics which you can then send to Webtrends, GA or your analytics provider of choice (or store in in your own SQL data warehouse if you want).

In this video I am showing a custom integration with a 3rd party performance analytics provider called WebTuna. The integration with WebTuna was whipped up in about half a day using the Pion Python script “reactor” to send the data to WebTuna.

This only covers the very basics of what you can do with Pion & WebTuna but it’s a good place to start for an overview of the kind of performance data it can provide your online business.

Anyway, have a watch and please feel free to post feedback in the comments section.

Real-time web performance analytics with Pion & WebTuna

Technorati Tags: Web Performance,WebTuna,Atomic Lab's Pion,web performance analytics,WPO

Is WPO doomed not to learn from history?

I read an interesting blog today in response to my post about Who should be on your WPO team?

Alexander Podelko’s post is entitled “WPO: A New Wave of Performance Engineering?” in which he tries to contextualise WPO as another in a long series of “waves” within the performance engineering discipline as a whole. Alexander is a Director of the non-profit Computer Measurement Group (CMG) and is a performance expert at Oracle, so he is well positioned to put things in perspective.

Alexander laments:

“So, while it is very promising and exciting that we get a new wave of people dedicated to performance, it is a little sad that it looks like it often gets started from a scratch inventing new terminology and ignoring what existed before. For me it would be better if we get all these waves together to enrich each other with the area of performance engineering they specialize in.”

I concur… which is why I presented on WPO at the UK CMG conference earlier this year as a way to try and bridge the gap between “old school back-end performance optimisation” and “new school front-end performance optimisation”.

I agree with Alexander that you can’t ignore either back-end or front-end optimisation when you are looking to improve the performance of your website!

I hope that WPO isn’t seen as “elitist” and at risk of not learning from the experiences of the earlier “waves” of performance engineering.

So… if there is a CMG group in your area why not reach out to them and offer to give them a presentation on the “new wave” and/or invite them to attend your WPO events?

Hug a CMG member today, people!