Creating an Performance SLA with your customers - Betfair’s Customer Charter

We are all used to creating SLA (Service Level Agreements) with our partners, suppliers and internal customers but how many of us have an explicit SLA with our internet customers?

My fellow web performance meetup organiser Jonathan Klein gives some great advice over on the Yottaa Blog on creating SLA’s but Betfair have gone one better – they’ve created an entire customer charter with explicit web performance goals.

From a web performance perspective the two key commitments are 5.1 (Availability) and 5.2 (Performance) -

5.1 We'll focus on the quality and reliability of our services.-

Quality and reliability is more important than technical enhancements to both us and you. Our target uptime is 99.9% for the Exchange and 99.0% uptime for our Portfolio products and we will publish our performance against these targets here at the end of each month.

5.2 Our products will meet the needs of our Customers by delivering a quick and responsive experience.-

After reliability, we believe that speed is a key feature of our products. We also acknowledge that we have a long way to go but we are working on it. In simple terms we commit to ensure our site becomes faster. To be more specific, we aim for 99.9% of bets placed in less than a second and our aspirational website SLA is as follows. Under peak loads, with performance measured at the 95th percentile, for typical user bandwidths and a 0% error rate, our users shall experience Visual Progress (header loaded) in less than 1 second, Time to Interact with useful content within 1.5 seconds and full page loads within 3 seconds. (There is room for improvement on this front as our current sports home page loads in approx. 18 seconds at the 95th percentile). We will publish our aggregate stats here monthly.

A 99.9% uptime is a fairly standard goal these days, although one that doesn’t give you much wiggle room with only 43 minutes per month to play with, even when you exclude “scheduled downtime”.

The performance goal is much more interesting – we can break it down into 3 separate performance goals:

1. our users shall experience Visual Progress (header loaded) in less than 1 second
2. Time to Interact with useful content within 1.5 seconds
3. and full page loads within 3 seconds.

The first goal – “shall experience Visual Progress” – would appear to translate to a “render start” time, and aligns well with the goals promote in my Web Performance 101 slides of a “render start time of <750ms”.

The second goal – “Time to interact” – is harder to define and probably somewhat subjective but I am assuming it relates to “Above the Fold” time (as discussed by Josh Bixby over here in his WebPerformanceToday blog).

The third goal – “full page load” – might be the Browser “OnLoad” Event but I imagine it’s closer to the “HTTP Load Time” metric used by HTTPWatch = “HTTP Load marks the completion of all HTTP or HTTPS requests made by the page.”

Again, slightly more complicated to measure with a very dynamic page with lots of AJAX requests a la the Betfair pages (as they dynamically load betting information) but still a very useful metric. Again, the 3 second goal aligns with that suggested in my Web Performance 101 slides! 

The key part of Betfair’s commitment to my mind is their commitment to transparency – the promise to publish monthly metrics to show their commitment to the charter - https://promotions.betfair.com/product-and-technology-dashboard-uk 

Transparency was one of the topic I discussed with the @Watchmouse guys at Velocity this year.

You might be familiar with @Lennysan’s “Transparent Uptime” blog but the “torch has now been passed” over to the “Transparent Performance” site organised by Watchmouse.

As with Betfair’s customer charter and their performance dashboard the goal of the “Transparent Performance” initiative is to make it easier for people to find out about the status of popular sites and services. As they put it -

Public transparency induces and instills a higher trust in a company and its brand – not less. It also gets the message across quickly and efficiently, so it can then be relayed across social networks, instead of leaving it up to the guesses of the public or the media. Finally, it will save serious money in the company’s contact center, as it sets the right expectations.

The Transparent Uptime blog is dedicated to educating companies on how to be more transparent. We will highlight stellar examples of transparency, opine on transparency trends, report on how well company’s respond to downtime in real time, have expert contributors, share articles, and more.

Hopefully more site will follow Betfair’s lead and ideally support the “Transparent Performance” initiative too by publishing their own “status dashboards” (aka “stashboards”).