How does your site stack up?

I pulled together the results of the tests that have been run on WebPagetest over the past year and did a bunch of aggregate analysis. There were over 24,000 unique urls tested in that time so the data is a pretty wide sampling across different types of sites. You can see the full details here: http://www.webpagetest.org/forums/thread-22.html

One of the particularly useful things you can do is look at your own test results and compare them in the distribution graphs to see where you land. For example, are you slower than 95% of the sites that were tested? How do the number of requests and bytes stack up? How about the optimizations?

Looking at straight averages across all of the tests:

Load Time: 10.1 seconds
Time to First Byte: 1.1 seconds
Time to Start Render: 3.8 seconds

Page Size: 510 KB
Number of Requests: 50
Number of Redirects: 1

Perhaps more interesting are observations on the distributions:

Page Measurements

Load Time: 35% of the sites tool longer than 10 seconds to load (and there's a pretty long tail that goes out to 60 seconds with 5% of the sites taking longer than 30 seconds). On the positive side, 33% of the sites loaded in under 5 seconds.

Time to First Byte: Looked surprisingly good with 76% of sites coming in under 500ms. More confirmation that the back-end on most sites works well and the work needs to be done on the front-end (content). That said, 9% took over 2 seconds so there are some sites that still have some back-end work to do.

Time to Start Render: There is a lot of room for improvement and this is probably one of the most useful measurements (and unique to Pagetest). The user doesn't see anything display before this point so it doesn't matter how fast the back-end is if there is a lot of js and css code loading in the head that prevents the page from rendering (even worse is not much code but lots of files). 60% of the sites take over 2 seconds to start rendering with 20% of the sites taking over 5. If you're going to focus on optimizing anything, this is the first number you should be looking at.

Page Size: I feel sorry for anyone still using dial-up. 30% of the sites were over 500KB.

Number of Requests: This is usually the most impactful measurement because most of the time in making a request is wasted and not actually downloading content so the more requests on the page the more time that is being wasted not downloading content. 33% of the sites have 50 or more requests with 12% having 100 or more (with a really scary tail out to 400).

Number of Redirects: 66% of the sites had no redirects and in general things looked really good. The 2% of sites with over 8 redirects should probably look at reducing them though.

Optimizations

I won't go through all of them but I will hit the high points.

Most sites are doing a good job with persistent connections. Only 5% of the sites are not leveraging keep-alives at all.

On compression, 50% of the sites could save 50% or more of their text bytes by enabling gzip compression. This helps both the end user and saves bytes on the wire which goes directly to bandwidth costs the site owners have to pay.

The biggest impact for most sites comes in combining several js and css files into a singlee file (of each type). This goes directly to the start render time and 20-30% of the sites have a large number of files to combine.

The last one I'll touch on is caching of static assets. A full 25% of the sites don't use any expires or cache-control headers at all. That makes the repeat view of the site almost as slow as the first view (and makes a lot of unneeded requests to the site). If you don't want people to keep coming back, this is a sure way to encourage that :-)

There are a ton of charts and a lot more data in the full analysis so if you want more information head over there. I also have an excel spreadsheet of the raw data available at the end of the analysis if you want to run any different kinds of analysis on it.

Optimization impact

As I look through the tests that come through WebPagetest I've been wondering if all of the optimization checks really make sense or if we should really be focusing on just the top 3 or 4 things that have the largest impact and not worrying about the smaller things. A LOT of the pages I see come through don't even have the basics in place and they may be overwhelmed by the checklists, etc (and by the time you start worrying about a few bytes from cookies, that may not be the real bottleneck for your site and you may be wasting optimization time).

This was particularly tweaked as YSlow 2 was released and added more automated checks from the 34 best practices. Are people going to be trying for an A and over-optimizing or optimizing the wrong part of their site? Are people at the beginning of the curve going to be overwhelmed about what they have to do and miss the opportunity for a large payoff for minimal effort?

Ryan Doherty wrote a really good article where he did a step-by-step optimization of a fake social network portal in "Optimizing openSpaceBook". He did miss persistent connections and I'd argue that even minifying probably wasn't necessary but he hit the big hitters and documented the improvement from each. I decided to take that framework and walk through optimizing a real-world site going through the steps from easiest to implement to the most difficult and just focusing on the changes that would result in large gains.

My proposed optimization path that everyone should take at a minimum (and that should be universally beneficial) is:

1. Enable Persistent Connections: This is a simple configuration setting on most web servers, requires no changes to the site and has almost no risk (the only risk is if you are running a site at close to capacity you may not be able to keep the connections open for long).
2. Properly Compress your Content: This includes both gzipping your html/javascript/css and properly compressing your images (jpegs can often be saved at a lower quality with no sacrifice in quality - we use Photoshop quality level 50 as the baseline at AOL). For the gzip compression it is again usually just a matter of configuration on the web server to enable it. If you pay for bandwidth this can also save you real money on hosting costs.
3. Allow the Browser to Cache your Static Content: Now we're starting to stray into possibly requiring code changes and this won't have any impact on the initial load time bot for repeat visits the savings can be significant.
4. Reduce the Number of HTTP Requests: This one is a bit more ambiguous but the most important cases for this are to collapse your CSS and JS down to a single file of each and to use image sprites for your page element graphics. This definitely requires more work than the other 3 optimizations but the payoff is usually well worth it.
   

I'd argue that these 4 optimizations will get you 90+% of the improvement for most of the sites and anything left is going to be very specific to each individual site (javascript optimizations, etc).

I decided to take a corporate portal and walk through an optimization exercise much like Ryan did to see what the benefit was for each step. It didn't take long to find one that was in pretty bad shape - I decided to look at a portal for a national web design company and it turns out that their home page pretty much failed everything except for the persistent connections so I cloned their site, broke the persistent connections and started optimizing.

First, the baseline - With everything broken I measured the site and this is what it looked like:

First View

Repeat View

If you ever see one of your repeat view waterfalls with a lot of yellow on it it means you REALLY need to do a better job of letting the browser cache your page. All of those requests are wasted round trips.

And the numbers from the baseline:

	Load Time	Start Render	Requests	Bytes In
First View	18.446s	8.174s	87	500 KB
Repeat View	13.176s	7.281s	87	20 KB

You'll notice that the repeat view is not much faster than the first view even though it only downloads 20KB of data - that's because of the 87 requests which are really slowing things down.


Step 1: Enable Persistent Connections (keepalives)

A quick tweak to the Apache configuration to turn on the keepalives and we eliminate one round trip from each of the requests. The waterfall essentially looks the same, just without the little orange bits at the beginning of each request but look at what happened to the times:

	Load Time	Start Render	Requests	Bytes In
First View	10.591s	4.922s	87	503 KB
Repeat View	7.431s	4.336s	87	23 KB

That's close to a 50% improvement in load times with 5 minutes worth of work and NO changes to the page itself.


Step 2: Compression

Again, just a quick tweak to the Apache configuration to enable gzip compression and re-compress a few of the jpeg images and we get:

	Load Time	Start Render	Requests	Bytes In
First View	9.698s	4.610s	87	354 KB
Repeat View	7.558s	4.351s	87	26 KB

We got another second or so in first view times and saved 150KB of bandwidth for the site. This particular page did not have a lot of text or javascript and the images were already in pretty good shape so the improvement wasn't as big as it would be on several sites I have seen but the effort required is minimal and there is no downside to doing it.


Step 3: Cache Static Content

This will not have any impact on the first view performance but if users ever come back to your site it can have a huge impact on the performance of your site. We are starting to cross the line into "may require some application work" though to make sure it is safe for your static assets to be cached for long periods. If so, then actually enabling it is again just a configuration setting on the server.

Here is what the waterfall looks like for the repeat view after we let the browser cache everything:

There are only 2 requests (and one of those is generated by javascript and canceled right away). More importantly, here are what the times look like:

	Load Time	Start Render	Requests	Bytes In
First View	9.751s	4.533s	87	361 KB
Repeat View	0.788s	0.753s	2	0 KB

As expected, no impact to the first view times, but the repeat view times got 90% faster.


Step 4: Reduce the number of HTTP requests

Now we're finally into the realm of having to do actual development work on the page. This page only had a few javascript files (5) but there were a TON of individual images for the various page elements. Combining the javascript and css files is a pretty trivial effort (and there are even modules that can do it for you). Changing the page to use image sprites instead of discrete images is a lot more work but WELL worth it (best if you can just plan to do this before you build a site but also worth it when retrofitting).

Here is what the waaterfall looked like after combining the files together:


And the numbers:

	Load Time	Start Render	Requests	Bytes In
First View	3.910s	1.079s	15	344 KB
Repeat View	0.773s	0.819s	2	0 KB

That's another 60% improvement in the first view times (and I only did the easy combining - it could have been refined even more).


Wrap-Up

As you can see, with just 4 core rules you can take a page from 18 seconds to load all the way down to 4 seconds (an 80% improvement). If that doesn't demonstrate the 80/20 rule, I don't know what does. Are the other best practices worth implementing/checking? I'd strongly content that it's good to know them but once you actually implement even these 4 basic rules you're either going to be fast enough or you're going to be doing some more advanced testing and analysis to see what is making the site slow (probably by manually looking at the waterfalls and looking for the bottlenecks).

I'm not making any changes yet but I'm strongly considering changing the checklist on WebPagetest to focus on these 4 rules as critical to implement and then provide the other details more as informational checks but not present them as prominently as they currently are.

Thoughts? Leave a comment here or discuss them in the WebPagetest Forums

Large sites without persistent connections

Sort of as a follow-up to the last article, over the last couple of days there were some particularly interesting sites that are just failing horribly:

Verizon's DNS error hijack - Verizon recently decided to start monetizing fat-fingered urls by taking over DNS failures and redirecting them to a search page. I'm using their default DNS servers (for now) for the online pagetest so some tests of invalid urls have returned the Verizon search page which ends up being a great example of a page that should be WAY faster than it is. Here are the full results of a test run I kicked off on purpose: http://performance.webpagetest.org:8080/result/PX8/

Here is the waterfall:


They fail pretty horribly for not using persistent connections but it's also a perfect example for image sprites and the html and css aren't gzipped either. The whole thing really should have been done in 2 or 3 requests and could be completely loading in under a second instead of a little over 3. None of those images have an expires header either so even repeat views take just as long.

Yahoo Japan - This one is mostly interesting because Yahoo is notoriously good at site optimization but somehow they seem to have missed the Japanese portal. They do pretty good on everything except for the persistent connections and gzip. Here are the full results: http://performance.webpagetest.org:8080/result/PXA/

and the waterfall:


That js is particularly bad as it's 85KB but can be reduced to 22 with gzip but the biggest crime is the persistent connections. They could cut the load time almost in half just by enabling persistent connections.

Easy ways to speed up your site

Pagetest has been online for 8 months now with close to 26,000 pages tested. I generally look through the test log daily and see what the results look like and it's been pretty frustrating because a significant number of the sites being tested could easily be twice as fast without changing the content or code but I have no way to reach out to the owner and tell them. The test results are probably a bit overwhelming for most people and they don't know where to start. so, with that.....


If you do nothing else for performance make sure you at least do these:


Persistent Connections - There is absolutely no reason that a site should not be using persistent connections yet I see them come through testing every day. Assuming you are not using a CDN and most of the content for your page is served from your server you can get close to a 50% improvement in performance just by enabling them. Here is a sample from a test that was run recently (site chopped out to protect the innocent):


I cropped it down but the whole site continued, opening new connections for 102 requests. The orange section of each request is the time used to open the connection and all but 2 would be eliminated just by enabling persistent connections. In this case the "start render" time (time when the user first sees something on the page) would go from 3.8 seconds down to roughly 2 seconds and the time to fully load the page would go down from 16 seconds to closer to 9 seconds. This sample was even from an Apache server so there's really no reason for not having them enabled.

GZIP (aka Free Money) - Just like with the persistent connections, enabling GZIP for text responses (html, css, js) is literally just a server configuration and there is very little reason for not doing it. Early versions of IE didn't react well to JS being compressed but it's easy enough to exclude them and not a good enough reason to penalize everyone else.

GZIP not only helps the end user get the pages faster but it also saves you significant bytes and if you pay for your network bandwidth, not having it enabled is throwing away free money.

It's not just the small sites that have this problem either. My favorite example is www.cnn.com which serves 523KB of uncompressed text. They could save 358KB of that just by enabling gzip compression and since most of the text for a page is all in the base page or js and css referenced in the head it all has to get downloaded before the user sees anything:

The blue bars in the waterfall are where the browser isi downloading uncompressed text for CNN.

Combining CSS and JS files - In case it's not obvious from the two waterfalls shown so far, it's not uncommon for a site to reference several different js and css files. There are several solutions available that will let you keep the files separate but download them all in a single request. David Artz has a pretty good write-up on mod_concat and some of the other options here. This does require modifying the html to reference the combined URL instead of each file individually but that is a very worthwhile effort given the massive improvement in the loading of your site.

The JS and CSS files usually get downloaded before anything is shown to the user and JS in particular has a nasty habit of only being downloaded one at a time so anything you can do to reduce the number of each that you are loading will have a huge impact to the user.

Spammers and Script Kiddies

Sigh, webpagetest.org apparently got some visibility somewhere in the spam and script kiddie/vulnerability scanner communities.

A week or so ago I started getting some "link spam" where a group of people had automated bots to kick off "tests" of their link farms. All of the hyperlinks out from webpagetest are "nofollow" links though so I'm still not sure what they hoped to gain from it. ModSecurity to the rescue and I have all of the current spam attempts locked down and cleared out the previous runs from the history.

Then today it looks like I started getting some activity from some automated compromise scans. The access logs were starting to get all sorts of bizarre requests that weren't legit, some coming from the same source IP, some from bot nets. They weren't successful since webpagegtest is completely custom code but it is on a shared host with a joomla board that I run for my neighborhood which is probably what made them try to compromise it (again, good old ModSecurity and latest patches to keep things safe).

It does make me question the benefit of using public libraries though. All it takes is a vulnerability in a version of the library and everyone who used it can get compromised pretty quickly (and it gets added to the script kiddie scanners prpetty quick). At least with custom code, someone would have to be explicitly targeting your site to compromise it which unless you're running a high-profile site it's a lot less likely (in which case you also have a team responsible for keeping things secure).

I honestly don't see how sites run by amateurs survive (well, they probably don't which is why there are so many compromised hosts out there being used for staging attacks). I was debating adding forum support directly to the webpagetest host but at this point it's probably not worth the effort and risk since those are usually swiss cheese on security.

Anyway, if you get a 403 "Access Denied" message when you're trying to do something, just shoot me a note. It's probably because I tightened down the screws a little too tight and caught you by accident.

Pagetest Optimization Tutorial

Dave Artz put together a great tutorial on using pagetest (the web version) for optimizing sites. It's fairly lengthy but well worth the time to make sure you get the most out of the tool:

http://www.artzstudio.com/2008/07/optimizing-web-performance-with-aol-pagetest/

Help make the web a faster place

Back on June 24th Eric Goldsmith presented pagetest at the Velocity conference. It was well received and he learned some critical presentation lessons (the most important being not to put up an url for something you're going to demo on the first slide with an audience full of geeks with laptops). See if you can identify the traffic spike from the conference presentation:


Normally something like 3000 page views wouldn't be a problem but there were a few hundred tests initiated all within the 30 minutes of the presentation window and each test takes close to a minute to complete (though several can be run in parallel). Hopefully people were patient because the queue did clear within an hour or so but there was quite a backlog for a while. I'll be adding a little ajax to the page to at least give people an indication of how many tests are ahead of them in the queue and set some expectations for how long the results will take.

As you can see, the traffic has maintained a boost after the conference and seems to be spreading more by word of mouth and blogs. It also looks like there are a fair number of die-hard users actively tweaking their pages. One interesting thing to browse is the test history link from the main page where you can see what tests were run for various pages. Some look to be pretty well optimized and are testing various options and some are hideous (even some really high-profile sites).

I'm also fairly impressed by the variety of locations that testing is coming from (particularly given that it's all in English):



Finally, on the short-list for upcoming features is better png optimization checking. We're going to be embedding optiPng into pagetest and providing byte-size improvements like we do with jpeg and gzip compression.