I put together this quick overview on web “user experience” KPI’s for a client but I thought it was worth re-posting it here for future reference… what they wanted to know what “how long did a given page request take on the website”, with the idea of defining certain KPI’s e.g. login must take less that 5 seconds, or saving your user profile less than 10 secs etc.
“I had a look at some of the end-user KPI's that were requested for the web side of things so I thought I would drop you a quick email as I have a fair bit of experience in this area (when I used to run www.totaljobs.com website).
There are basically 3 ways of doing this - external site monitoring, page tagging (javascript) and appliance-based.
External site monitoring is basically where a 3rd party simulated a user page request every 5 or 10 minutes and records how long it took. Basically they "GET" a web page URL. Companies such as Site Confidence (UK), Gomez or Axzona are leaders in this area.
"Page tagging" is where you insert some client-side javascript into the page that is sent to the client's browser. This then sends that information back to a collector server in the data centre that stores it in a database and you can generate reports etc. This is a much closer measure of REAL end-user experience for all of your clients and gives you more information to troubleshoot e.g. performance for users in Italy is slow, or users with IE7 on Vista is slow but Firefox is fine. The downside is that you generally have to make code level changes to insert the tags into the code. Solutions in this space include CA Wily Customer Experience Manager and Gomez's Actual Experience Manager XF. For .Net sites Avicode UX Monitor doesn't need application changes as it just hooks directly into the .Net Management framework.
Appliance-based solutions are basically devices (they look a bit like a router box) that plug-in to the main routers in the data centre and basically monitor the traffic on the way in/out to measure performance and the user experience. They can also actually insert "page tagging" type code "on the fly" and thus avoid application-level changes. Downside is getting the networking set up so you can tap into a promiscuous port can be a pain. Leaders in this area are TeaLeaf and Coradiant."
It is worth noting that you also need your classic “web analytics” (Webtrends / Omniture / Hitbox or even Google Analytics if your on a zero budget) to give you the classic unique users, page impressions etc stuff.
There are obviously many more KPI’s involved in running a website(customer service, system availability, email deliverability, SEO, marketing conversion etc etc) but at the end of the day I think that the “user experience” KPI’s sit at the top of the KPI pyramid.
Well, except for for the net profit KPI, of course :-)
Friday, April 15, 2011 at 11:22AM 