Thursday, May 31, 2012

EC2 Performance

WebPagetest makes EC2 AMI's available for people to use for running private instances and makes fairly extensive use of them for running the testing for the Page Speed Service comparisons.  We have tested that the m1.small instances produce consistent results but we aren't necessarily sure if they are representative of real end-user machines so I decided to do some testing and see.

This is a very specific test that is just looking to compare the raw CPU performance for web browsing (single threaded) of various EC2 instance sizes against physical machines.  It is not meant to be a browser comparison or a statement about EC2 performance beyond this very-specific use case.

Testing Methodology

I ran the SunSpider 0.9.1 benchmark 5 times on each of the different machines using Chrome 19 (it is important to keep the browser and version consistent since changes to the JavaScript engine will affect the results).

Results


As you can see, the m1.small instances are significantly slower than desktop systems from the last 5 years or so  but it is somewhat faster than more recent low-end laptops.  Netbooks and tablets are significantly slower still with times typically in the 1000+ range.

Conclusion

Unfortunately there isn't a clear-cut answer of what you should use to test if you are trying to test on a "representative" system because both the smaller and larger instances are representative of different ends of the computing spectrum.

My general feeling is that websites should not be CPU constrained and if they are then things will look exponentially worse on the tablets, chrome books and other cheap devices that are starting to flood the market.  If you test using the larger instances then you will be testing on systems more representative of desktops which might be a good thing to do if that is specifically what you are targeting.  The small instances are more likely to expose CPU constraints that will crop up in your user base (much like Twitter noticed).

Call for Help

The systems I tested were machines that I had easy access to but are probably not representative of a lot of systems people have at home.  If you could run SunSpider using Chrome 19 on any systems you have lying around and share the results as well as the system specs in the comments below I'll update the chart and see if we can build a more representative picture.

*update - chart has been updated with the user-submitted results, thank you
Posted by at

21 comments:

  1. Mehdi DaoudiMay 31, 2012 at 1:14 PM

    here are my results - Mac Pro (8 CPU, 64 GB ram)
    http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B11,11,11,11,10,11,11,11,11,11%5D,%223d-morph%22:%5B11,10,10,10,10,10,10,12,10,10%5D,%223d-raytrace%22:%5B14,14,15,15,15,14,14,15,16,14%5D,%22access-binary-trees%22:%5B2,2,2,2,2,2,2,2,2,2%5D,%22access-fannkuch%22:%5B9,10,10,9,9,9,10,10,9,9%5D,%22access-nbody%22:%5B6,6,6,5,6,6,5,8,7,7%5D,%22access-nsieve%22:%5B3,3,3,3,3,3,3,4,3,3%5D,%22bitops-3bit-bits-in-byte%22:%5B1,1,1,1,1,1,1,1,2,1%5D,%22bitops-bits-in-byte%22:%5B5,5,5,5,5,5,5,5,5,5%5D,%22bitops-bitwise-and%22:%5B3,3,4,4,4,3,4,4,4,4%5D,%22bitops-nsieve-bits%22:%5B6,6,5,6,6,7,6,8,6,6%5D,%22controlflow-recursive%22:%5B3,3,3,2,2,2,2,3,3,3%5D,%22crypto-aes%22:%5B7,8,7,8,8,7,8,8,9,9%5D,%22crypto-md5%22:%5B6,6,7,6,6,6,6,7,8,6%5D,%22crypto-sha1%22:%5B6,7,6,7,6,6,6,7,6,6%5D,%22date-format-tofte%22:%5B12,13,12,12,11,11,11,15,14,13%5D,%22date-format-xparb%22:%5B9,10,9,9,10,9,9,10,10,9%5D,%22math-cordic%22:%5B3,3,3,3,3,3,3,4,3,3%5D,%22math-partial-sums%22:%5B12,12,12,12,12,12,11,14,14,12%5D,%22math-spectral-norm%22:%5B3,4,3,4,4,4,4,4,4,3%5D,%22regexp-dna%22:%5B11,11,11,11,11,12,11,11,11,11%5D,%22string-base64%22:%5B7,7,7,6,6,6,6,8,7,7%5D,%22string-fasta%22:%5B10,11,10,10,11,10,11,10,11,10%5D,%22string-tagcloud%22:%5B27,27,26,27,27,25,24,27,26,27%5D,%22string-unpack-code%22:%5B34,35,34,36,36,35,36,37,36,35%5D,%22string-validate-input%22:%5B15,15,16,16,16,15,15,15,16,15%5D%7D

    ReplyDelete
  2. Patrick MeenanMay 31, 2012 at 1:23 PM

    @Mehdi - what CPU?

    ReplyDelete
    Replies
    1. Mehdi DaoudiJune 1, 2012 at 4:34 PM

      Model Name: Mac Pro
      Model Identifier: MacPro5,1
      Processor Name: Quad-Core Intel Xeon
      Processor Speed: 2.4 GHz
      Number of Processors: 2
      Total Number of Cores: 8
      L2 Cache (per Core): 256 KB
      L3 Cache (per Processor): 12 MB
      Memory: 64 GB
      Processor Interconnect Speed: 5.86 GT/s

      Delete
  3. robzillaMay 31, 2012 at 1:43 PM

    Pentium 4 640 (3.4GHz), 4GB RAM, 80GB SSD, Windows 7 (64-bit): http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B30,30,29,29,30,29,29,29,34,30%5D,%223d-morph%22:%5B26,26,26,23,26,26,25,26,27,26%5D,%223d-raytrace%22:%5B39,38,36,38,36,36,36,37,38,36%5D,%22access-binary-trees%22:%5B4,4,4,4,5,5,4,4,4,4%5D,%22access-fannkuch%22:%5B23,23,24,24,25,24,24,24,23,24%5D,%22access-nbody%22:%5B19,15,16,15,15,15,16,15,16,15%5D,%22access-nsieve%22:%5B7,7,7,7,9,7,7,7,7,7%5D,%22bitops-3bit-bits-in-byte%22:%5B3,3,0,3,3,3,5,3,4,3%5D,%22bitops-bits-in-byte%22:%5B11,11,11,10,14,10,11,11,11,10%5D,%22bitops-bitwise-and%22:%5B9,9,8,8,8,9,8,8,9,9%5D,%22bitops-nsieve-bits%22:%5B20,20,22,20,20,21,20,21,22,22%5D,%22controlflow-recursive%22:%5B7,6,7,6,7,6,6,6,7,7%5D,%22crypto-aes%22:%5B23,22,23,22,23,23,21,22,22,22%5D,%22crypto-md5%22:%5B18,18,17,18,19,17,17,17,18,17%5D,%22crypto-sha1%22:%5B17,18,17,17,17,17,17,16,17,16%5D,%22date-format-tofte%22:%5B40,42,40,41,38,37,38,40,38,38%5D,%22date-format-xparb%22:%5B32,35,30,32,33,31,33,35,31,34%5D,%22math-cordic%22:%5B8,9,8,9,9,9,9,8,8,9%5D,%22math-partial-sums%22:%5B37,37,36,36,36,34,34,35,35,35%5D,%22math-spectral-norm%22:%5B10,10,10,10,10,11,11,10,10,11%5D,%22regexp-dna%22:%5B22,22,23,22,21,22,21,24,22,22%5D,%22string-base64%22:%5B15,16,15,15,14,14,15,14,15,14%5D,%22string-fasta%22:%5B28,26,28,27,29,28,27,28,29,28%5D,%22string-tagcloud%22:%5B54,48,52,56,48,44,48,50,49,51%5D,%22string-unpack-code%22:%5B64,58,59,58,58,60,60,58,61,62%5D,%22string-validate-input%22:%5B43,46,45,43,48,45,45,45,46,44%5D%7D

    ReplyDelete
  4. sajalMay 31, 2012 at 2:02 PM

    Surprising that i got such a low benchmark vs more inferior CPUs... perhaps chrome/webkit isint tuned much for Linux.
    User-Agent : Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5
    Result : 434.4ms - http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B17,21,20,25,37,101,23,27,55,24%5D,%223d-morph%22:%5B24,12,12,30,34,16,16,13,35,15%5D,%223d-raytrace%22:%5B40,31,26,59,50,22,21,19,56,20%5D,%22access-binary-trees%22:%5B5,3,3,10,6,3,3,3,9,2%5D,%22access-fannkuch%22:%5B12,17,10,14,20,21,26,22,21,19%5D,%22access-nbody%22:%5B10,14,12,24,22,17,25,23,42,17%5D,%22access-nsieve%22:%5B6,7,5,10,7,7,7,7,11,7%5D,%22bitops-3bit-bits-in-byte%22:%5B4,3,3,3,5,4,4,5,4,7%5D,%22bitops-bits-in-byte%22:%5B9,6,6,7,10,9,9,9,10,10%5D,%22bitops-bitwise-and%22:%5B7,3,6,3,16,11,13,15,16,12%5D,%22bitops-nsieve-bits%22:%5B10,33,7,7,51,33,34,31,34,42%5D,%22controlflow-recursive%22:%5B5,3,4,4,6,4,3,4,6,4%5D,%22crypto-aes%22:%5B19,10,9,19,11,11,9,9,12,13%5D,%22crypto-md5%22:%5B8,4,5,4,5,5,4,5,6,4%5D,%22crypto-sha1%22:%5B7,4,5,5,5,3,4,4,5,4%5D,%22date-format-tofte%22:%5B17,21,19,19,14,11,12,10,11,13%5D,%22date-format-xparb%22:%5B20,17,24,17,12,8,12,13,10,9%5D,%22math-cordic%22:%5B15,19,5,40,18,17,36,60,31,22%5D,%22math-partial-sums%22:%5B37,36,33,17,16,16,19,36,19,23%5D,%22math-spectral-norm%22:%5B11,9,9,7,6,4,6,15,7,7%5D,%22regexp-dna%22:%5B12,11,14,10,9,11,11,11,12,11%5D,%22string-base64%22:%5B14,14,11,9,5,10,6,14,12,11%5D,%22string-fasta%22:%5B26,24,19,14,15,14,14,21,20,17%5D,%22string-tagcloud%22:%5B26,26,29,23,22,23,15,25,25,16%5D,%22string-unpack-code%22:%5B43,27,79,35,27,29,34,70,34,37%5D,%22string-validate-input%22:%5B36,17,16,16,24,16,17,32,22,34%5D%7D
    CPU : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz - http://pastie.org/4003098

    ReplyDelete
  5. sajalMay 31, 2012 at 2:02 PM

    Surprising that i got such a low benchmark vs more inferior CPUs... perhaps chrome/webkit isint tuned much for Linux.
    User-Agent : Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5
    Result : 434.4ms - http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B17,21,20,25,37,101,23,27,55,24%5D,%223d-morph%22:%5B24,12,12,30,34,16,16,13,35,15%5D,%223d-raytrace%22:%5B40,31,26,59,50,22,21,19,56,20%5D,%22access-binary-trees%22:%5B5,3,3,10,6,3,3,3,9,2%5D,%22access-fannkuch%22:%5B12,17,10,14,20,21,26,22,21,19%5D,%22access-nbody%22:%5B10,14,12,24,22,17,25,23,42,17%5D,%22access-nsieve%22:%5B6,7,5,10,7,7,7,7,11,7%5D,%22bitops-3bit-bits-in-byte%22:%5B4,3,3,3,5,4,4,5,4,7%5D,%22bitops-bits-in-byte%22:%5B9,6,6,7,10,9,9,9,10,10%5D,%22bitops-bitwise-and%22:%5B7,3,6,3,16,11,13,15,16,12%5D,%22bitops-nsieve-bits%22:%5B10,33,7,7,51,33,34,31,34,42%5D,%22controlflow-recursive%22:%5B5,3,4,4,6,4,3,4,6,4%5D,%22crypto-aes%22:%5B19,10,9,19,11,11,9,9,12,13%5D,%22crypto-md5%22:%5B8,4,5,4,5,5,4,5,6,4%5D,%22crypto-sha1%22:%5B7,4,5,5,5,3,4,4,5,4%5D,%22date-format-tofte%22:%5B17,21,19,19,14,11,12,10,11,13%5D,%22date-format-xparb%22:%5B20,17,24,17,12,8,12,13,10,9%5D,%22math-cordic%22:%5B15,19,5,40,18,17,36,60,31,22%5D,%22math-partial-sums%22:%5B37,36,33,17,16,16,19,36,19,23%5D,%22math-spectral-norm%22:%5B11,9,9,7,6,4,6,15,7,7%5D,%22regexp-dna%22:%5B12,11,14,10,9,11,11,11,12,11%5D,%22string-base64%22:%5B14,14,11,9,5,10,6,14,12,11%5D,%22string-fasta%22:%5B26,24,19,14,15,14,14,21,20,17%5D,%22string-tagcloud%22:%5B26,26,29,23,22,23,15,25,25,16%5D,%22string-unpack-code%22:%5B43,27,79,35,27,29,34,70,34,37%5D,%22string-validate-input%22:%5B36,17,16,16,24,16,17,32,22,34%5D%7D
    CPU : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz - http://pastie.org/4003098

    ReplyDelete
  6. sajalMay 31, 2012 at 2:03 PM

    http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B17,21,20,25,37,101,23,27,55,24%5D,%223d-morph%22:%5B24,12,12,30,34,16,16,13,35,15%5D,%223d-raytrace%22:%5B40,31,26,59,50,22,21,19,56,20%5D,%22access-binary-trees%22:%5B5,3,3,10,6,3,3,3,9,2%5D,%22access-fannkuch%22:%5B12,17,10,14,20,21,26,22,21,19%5D,%22access-nbody%22:%5B10,14,12,24,22,17,25,23,42,17%5D,%22access-nsieve%22:%5B6,7,5,10,7,7,7,7,11,7%5D,%22bitops-3bit-bits-in-byte%22:%5B4,3,3,3,5,4,4,5,4,7%5D,%22bitops-bits-in-byte%22:%5B9,6,6,7,10,9,9,9,10,10%5D,%22bitops-bitwise-and%22:%5B7,3,6,3,16,11,13,15,16,12%5D,%22bitops-nsieve-bits%22:%5B10,33,7,7,51,33,34,31,34,42%5D,%22controlflow-recursive%22:%5B5,3,4,4,6,4,3,4,6,4%5D,%22crypto-aes%22:%5B19,10,9,19,11,11,9,9,12,13%5D,%22crypto-md5%22:%5B8,4,5,4,5,5,4,5,6,4%5D,%22crypto-sha1%22:%5B7,4,5,5,5,3,4,4,5,4%5D,%22date-format-tofte%22:%5B17,21,19,19,14,11,12,10,11,13%5D,%22date-format-xparb%22:%5B20,17,24,17,12,8,12,13,10,9%5D,%22math-cordic%22:%5B15,19,5,40,18,17,36,60,31,22%5D,%22math-partial-sums%22:%5B37,36,33,17,16,16,19,36,19,23%5D,%22math-spectral-norm%22:%5B11,9,9,7,6,4,6,15,7,7%5D,%22regexp-dna%22:%5B12,11,14,10,9,11,11,11,12,11%5D,%22string-base64%22:%5B14,14,11,9,5,10,6,14,12,11%5D,%22string-fasta%22:%5B26,24,19,14,15,14,14,21,20,17%5D,%22string-tagcloud%22:%5B26,26,29,23,22,23,15,25,25,16%5D,%22string-unpack-code%22:%5B43,27,79,35,27,29,34,70,34,37%5D,%22string-validate-input%22:%5B36,17,16,16,24,16,17,32,22,34%5D%7D

    Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5 on http://pastie.org/4003098

    ReplyDelete
  7. p83822May 31, 2012 at 2:12 PM

    Total: 221.9ms +/- 2.0%
    Total: 220.4ms +/- 2.6%
    Total: 219.0ms +/- 2.6%
    Total: 222.7ms +/- 3.2%
    Total: 221.8ms +/- 3.3%

    EliteBook 8650w (Core i7 @ 2.70 Ghz)

    ReplyDelete
  8. p83822May 31, 2012 at 2:29 PM

    Total: 407.5ms +/- 1.3%
    Total: 392.7ms +/- 0.8%
    Total: 396.3ms +/- 1.1%
    Total: 404.7ms +/- 1.9%
    Total: 402.1ms +/- 2.6%

    19.0.1084.52

    ThinkPad X301 (Core 2 Duo @ 1.6 GHz)

    ReplyDelete
  9. robzillaMay 31, 2012 at 2:29 PM

    Another one, Pentium Dual-Core 1.6GHz, 3GB RAM, 80GB HDD, Windows 7 (32-bit): http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B20,20,20,21,20,19,19,20,20,20%5D,%223d-morph%22:%5B31,19,19,23,19,19,16,20,19,23%5D,%223d-raytrace%22:%5B30,27,29,27,26,27,27,27,27,27%5D,%22access-binary-trees%22:%5B3,3,3,3,4,4,4,3,4,3%5D,%22access-fannkuch%22:%5B19,18,18,18,17,19,18,18,18,18%5D,%22access-nbody%22:%5B13,11,13,12,12,12,13,12,11,13%5D,%22access-nsieve%22:%5B5,6,5,6,6,7,6,6,6,6%5D,%22bitops-3bit-bits-in-byte%22:%5B3,2,3,2,2,2,2,2,2,2%5D,%22bitops-bits-in-byte%22:%5B11,0,9,8,0,8,9,0,8,9%5D,%22bitops-bitwise-and%22:%5B6,5,6,6,5,6,5,6,6,6%5D,%22bitops-nsieve-bits%22:%5B10,10,13,10,12,10,10,10,10,11%5D,%22controlflow-recursive%22:%5B6,5,5,5,5,5,5,5,6,6%5D,%22crypto-aes%22:%5B14,13,15,14,13,14,15,14,15,13%5D,%22crypto-md5%22:%5B11,11,12,11,12,10,13,11,11,11%5D,%22crypto-sha1%22:%5B12,11,11,11,11,12,12,11,35,11%5D,%22date-format-tofte%22:%5B23,22,21,22,19,20,21,21,20,20%5D,%22date-format-xparb%22:%5B15,16,20,15,16,18,23,15,16,15%5D,%22math-cordic%22:%5B5,5,5,6,6,5,6,6,6,5%5D,%22math-partial-sums%22:%5B23,24,24,23,24,22,23,22,24,35%5D,%22math-spectral-norm%22:%5B7,7,6,7,7,8,7,8,6,7%5D,%22regexp-dna%22:%5B17,17,16,18,17,17,16,18,18,17%5D,%22string-base64%22:%5B12,10,12,11,11,10,11,11,11,11%5D,%22string-fasta%22:%5B6,19,18,17,18,19,18,18,18,18%5D,%22string-tagcloud%22:%5B58,47,47,47,45,43,48,44,45,45%5D,%22string-unpack-code%22:%5B58,60,59,60,60,62,72,62,63,63%5D,%22string-validate-input%22:%5B25,24,25,25,25,24,32,24,24,24%5D%7D

    ReplyDelete
  10. robzillaMay 31, 2012 at 3:49 PM

    That last one's a Pentium E2140, by the way. Both are desktop PCs.

    Here's a Toshiba laptop with a Pentium T2080 Dual-Core processor, 2 GB RAM and Windows Vista: http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B25,25,23,22,23,24,21,25,21,24%5D,%223d-morph%22:%5B24,23,22,19,20,22,20,20,19,19%5D,%223d-raytrace%22:%5B32,31,30,31,36,29,37,34,34,30%5D,%22access-binary-trees%22:%5B4,8,3,4,4,3,3,3,7,4%5D,%22access-fannkuch%22:%5B27,23,23,27,29,27,29,21,19,26%5D,%22access-nbody%22:%5B16,13,17,17,17,16,13,13,12,12%5D,%22access-nsieve%22:%5B7,7,10,7,7,7,11,7,7,7%5D,%22bitops-3bit-bits-in-byte%22:%5B3,8,2,3,2,5,2,2,3,3%5D,%22bitops-bits-in-byte%22:%5B9,8,9,9,11,9,9,13,8,8%5D,%22bitops-bitwise-and%22:%5B12,11,11,12,9,12,11,9,12,10%5D,%22bitops-nsieve-bits%22:%5B16,15,16,17,13,15,19,17,12,15%5D,%22controlflow-recursive%22:%5B5,7,5,8,5,5,6,8,5,5%5D,%22crypto-aes%22:%5B18,19,17,16,18,18,15,15,21,17%5D,%22crypto-md5%22:%5B14,16,11,12,14,11,14,16,14,13%5D,%22crypto-sha1%22:%5B15,14,15,14,12,14,19,12,16,12%5D,%22date-format-tofte%22:%5B28,25,29,27,22,22,23,24,23,20%5D,%22date-format-xparb%22:%5B17,17,24,25,22,26,16,20,22,17%5D,%22math-cordic%22:%5B6,7,10,9,7,7,14,6,7,7%5D,%22math-partial-sums%22:%5B26,26,31,29,24,28,28,24,29,24%5D,%22math-spectral-norm%22:%5B8,8,9,8,8,8,9,11,8,10%5D,%22regexp-dna%22:%5B18,20,21,25,18,26,19,23,26,23%5D,%22string-base64%22:%5B15,13,13,15,17,17,11,12,14,13%5D,%22string-fasta%22:%5B22,22,27,22,22,27,28,20,24,22%5D,%22string-tagcloud%22:%5B50,52,56,62,58,55,52,61,60,54%5D,%22string-unpack-code%22:%5B67,67,69,67,68,76,82,75,79,81%5D,%22string-validate-input%22:%5B30,33,29,29,29,28,30,28,28,30%5D%7D

    ReplyDelete
  11. sajalMay 31, 2012 at 4:24 PM

    i5-2410M CPU @ 2.30GHz laptop with 4GB memory + SSD
    Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5
    407.3ms
    http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B24,62,16,60,25,24,28,33,25,27%5D,%223d-morph%22:%5B24,17,18,14,13,16,15,39,27,14%5D,%223d-raytrace%22:%5B48,24,23,42,19,19,38,46,19,22%5D,%22access-binary-trees%22:%5B6,4,2,3,3,3,4,9,2,5%5D,%22access-fannkuch%22:%5B12,10,10,17,20,20,22,22,24,22%5D,%22access-nbody%22:%5B32,14,8,10,18,19,26,51,16,22%5D,%22access-nsieve%22:%5B7,6,8,8,8,9,13,8,11,11%5D,%22bitops-3bit-bits-in-byte%22:%5B4,4,6,5,6,8,7,6,6,6%5D,%22bitops-bits-in-byte%22:%5B8,9,8,10,11,16,14,14,10,10%5D,%22bitops-bitwise-and%22:%5B7,6,6,7,16,15,21,16,16,17%5D,%22bitops-nsieve-bits%22:%5B8,8,13,27,30,38,40,33,35,32%5D,%22controlflow-recursive%22:%5B3,5,7,5,4,4,4,5,4,3%5D,%22crypto-aes%22:%5B35,11,11,12,10,14,26,15,18,14%5D,%22crypto-md5%22:%5B10,6,9,6,4,11,8,9,7,5%5D,%22crypto-sha1%22:%5B6,6,5,5,4,9,6,6,6,4%5D,%22date-format-tofte%22:%5B11,14,14,12,10,10,23,11,17,11%5D,%22date-format-xparb%22:%5B10,9,14,17,10,12,21,11,8,18%5D,%22math-cordic%22:%5B7,5,15,9,18,23,50,22,31,43%5D,%22math-partial-sums%22:%5B18,17,12,13,18,15,32,17,15,31%5D,%22math-spectral-norm%22:%5B9,8,7,7,6,9,7,5,5,17%5D,%22regexp-dna%22:%5B9,8,9,10,8,8,10,9,11,13%5D,%22string-base64%22:%5B11,6,13,9,12,11,6,9,10,12%5D,%22string-fasta%22:%5B17,13,17,16,16,17,18,19,19,25%5D,%22string-tagcloud%22:%5B26,16,15,15,21,15,20,21,21,22%5D,%22string-unpack-code%22:%5B29,26,47,32,30,32,25,26,25,26%5D,%22string-validate-input%22:%5B23,28,19,20,18,22,19,25,20,17%5D%7D

    ReplyDelete
  12. AnonymousMay 31, 2012 at 5:34 PM

    2011 MacBook Air: 2.13 Core 2 Duo, 4Gb 1067 DDR3. Consistent numbers across many runs: ~327

    Surprising difference compared to the 2012! (at least, I didn't expect such a gap)

    http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B20,14,16,14,14,13,14,14,14,14%5D,%223d-morph%22:%5B12,14,15,12,21,12,13,14,12,12%5D,%223d-raytrace%22:%5B20,22,20,19,19,19,19,18,18,18%5D,%22access-binary-trees%22:%5B2,2,3,3,3,2,2,2,2,3%5D,%22access-fannkuch%22:%5B14,14,13,23,13,15,13,13,14,13%5D,%22access-nbody%22:%5B11,8,10,9,7,8,8,8,7,7%5D,%22access-nsieve%22:%5B6,4,5,4,4,4,4,4,4,4%5D,%22bitops-3bit-bits-in-byte%22:%5B2,1,2,2,1,2,2,2,1,2%5D,%22bitops-bits-in-byte%22:%5B8,7,6,6,6,7,6,6,7,7%5D,%22bitops-bitwise-and%22:%5B5,5,4,5,5,5,4,5,4,4%5D,%22bitops-nsieve-bits%22:%5B11,7,15,7,13,8,8,8,7,8%5D,%22controlflow-recursive%22:%5B7,4,4,4,4,4,5,4,4,5%5D,%22crypto-aes%22:%5B12,11,10,10,10,11,10,12,10,14%5D,%22crypto-md5%22:%5B10,8,8,10,8,9,8,8,8,8%5D,%22crypto-sha1%22:%5B9,10,9,10,10,10,14,12,12,10%5D,%22date-format-tofte%22:%5B19,16,16,17,15,17,16,15,15,14%5D,%22date-format-xparb%22:%5B12,17,17,11,12,12,11,12,12,12%5D,%22math-cordic%22:%5B6,3,4,3,3,4,4,4,5,5%5D,%22math-partial-sums%22:%5B19,15,18,15,15,15,15,14,15,14%5D,%22math-spectral-norm%22:%5B5,5,5,5,5,5,4,4,5,5%5D,%22regexp-dna%22:%5B13,13,13,15,13,13,14,13,14,12%5D,%22string-base64%22:%5B8,10,7,8,8,9,8,8,8,9%5D,%22string-fasta%22:%5B14,15,14,12,13,13,15,12,14,13%5D,%22string-tagcloud%22:%5B37,35,37,35,36,36,45,35,39,33%5D,%22string-unpack-code%22:%5B48,47,45,46,44,45,46,53,44,46%5D,%22string-validate-input%22:%5B25,24,20,21,21,21,22,19,20,25%5D%7D

    ReplyDelete
  13. AnonymousMay 31, 2012 at 5:34 PM

    2011 MacBook Air: 2.13 Core 2 Duo, 4Gb 1067 DDR3. Consistent numbers across many runs: ~327

    Surprising difference compared to the 2012! (at least, I didn't expect such a gap)

    http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B20,14,16,14,14,13,14,14,14,14%5D,%223d-morph%22:%5B12,14,15,12,21,12,13,14,12,12%5D,%223d-raytrace%22:%5B20,22,20,19,19,19,19,18,18,18%5D,%22access-binary-trees%22:%5B2,2,3,3,3,2,2,2,2,3%5D,%22access-fannkuch%22:%5B14,14,13,23,13,15,13,13,14,13%5D,%22access-nbody%22:%5B11,8,10,9,7,8,8,8,7,7%5D,%22access-nsieve%22:%5B6,4,5,4,4,4,4,4,4,4%5D,%22bitops-3bit-bits-in-byte%22:%5B2,1,2,2,1,2,2,2,1,2%5D,%22bitops-bits-in-byte%22:%5B8,7,6,6,6,7,6,6,7,7%5D,%22bitops-bitwise-and%22:%5B5,5,4,5,5,5,4,5,4,4%5D,%22bitops-nsieve-bits%22:%5B11,7,15,7,13,8,8,8,7,8%5D,%22controlflow-recursive%22:%5B7,4,4,4,4,4,5,4,4,5%5D,%22crypto-aes%22:%5B12,11,10,10,10,11,10,12,10,14%5D,%22crypto-md5%22:%5B10,8,8,10,8,9,8,8,8,8%5D,%22crypto-sha1%22:%5B9,10,9,10,10,10,14,12,12,10%5D,%22date-format-tofte%22:%5B19,16,16,17,15,17,16,15,15,14%5D,%22date-format-xparb%22:%5B12,17,17,11,12,12,11,12,12,12%5D,%22math-cordic%22:%5B6,3,4,3,3,4,4,4,5,5%5D,%22math-partial-sums%22:%5B19,15,18,15,15,15,15,14,15,14%5D,%22math-spectral-norm%22:%5B5,5,5,5,5,5,4,4,5,5%5D,%22regexp-dna%22:%5B13,13,13,15,13,13,14,13,14,12%5D,%22string-base64%22:%5B8,10,7,8,8,9,8,8,8,9%5D,%22string-fasta%22:%5B14,15,14,12,13,13,15,12,14,13%5D,%22string-tagcloud%22:%5B37,35,37,35,36,36,45,35,39,33%5D,%22string-unpack-code%22:%5B48,47,45,46,44,45,46,53,44,46%5D,%22string-validate-input%22:%5B25,24,20,21,21,21,22,19,20,25%5D%7D

    ReplyDelete
  14. AnonymousMay 31, 2012 at 5:35 PM

    MacBook Air (2011): 2.13 Ghz Core 2 Duo, 4Gb 1067Mhz DDR3

    Consistent score across many runs: ~320. I'm surprised by the 50% slower results compared to the 2012 model!

    http://www.webkit.org/perf/sunspider-0.9.1/sunspider-0.9.1/results.html?%7B%22v%22:%20%22sunspider-0.9.1%22,%20%223d-cube%22:%5B20,14,16,14,14,13,14,14,14,14%5D,%223d-morph%22:%5B12,14,15,12,21,12,13,14,12,12%5D,%223d-raytrace%22:%5B20,22,20,19,19,19,19,18,18,18%5D,%22access-binary-trees%22:%5B2,2,3,3,3,2,2,2,2,3%5D,%22access-fannkuch%22:%5B14,14,13,23,13,15,13,13,14,13%5D,%22access-nbody%22:%5B11,8,10,9,7,8,8,8,7,7%5D,%22access-nsieve%22:%5B6,4,5,4,4,4,4,4,4,4%5D,%22bitops-3bit-bits-in-byte%22:%5B2,1,2,2,1,2,2,2,1,2%5D,%22bitops-bits-in-byte%22:%5B8,7,6,6,6,7,6,6,7,7%5D,%22bitops-bitwise-and%22:%5B5,5,4,5,5,5,4,5,4,4%5D,%22bitops-nsieve-bits%22:%5B11,7,15,7,13,8,8,8,7,8%5D,%22controlflow-recursive%22:%5B7,4,4,4,4,4,5,4,4,5%5D,%22crypto-aes%22:%5B12,11,10,10,10,11,10,12,10,14%5D,%22crypto-md5%22:%5B10,8,8,10,8,9,8,8,8,8%5D,%22crypto-sha1%22:%5B9,10,9,10,10,10,14,12,12,10%5D,%22date-format-tofte%22:%5B19,16,16,17,15,17,16,15,15,14%5D,%22date-format-xparb%22:%5B12,17,17,11,12,12,11,12,12,12%5D,%22math-cordic%22:%5B6,3,4,3,3,4,4,4,5,5%5D,%22math-partial-sums%22:%5B19,15,18,15,15,15,15,14,15,14%5D,%22math-spectral-norm%22:%5B5,5,5,5,5,5,4,4,5,5%5D,%22regexp-dna%22:%5B13,13,13,15,13,13,14,13,14,12%5D,%22string-base64%22:%5B8,10,7,8,8,9,8,8,8,9%5D,%22string-fasta%22:%5B14,15,14,12,13,13,15,12,14,13%5D,%22string-tagcloud%22:%5B37,35,37,35,36,36,45,35,39,33%5D,%22string-unpack-code%22:%5B48,47,45,46,44,45,46,53,44,46%5D,%22string-validate-input%22:%5B25,24,20,21,21,21,22,19,20,25%5D%7D

    ReplyDelete
  15. Andy DaviesJune 1, 2012 at 7:57 AM

    Total: 210.2ms +/- 1.5%
    Total: 210.1ms +/- 1.5%
    Total: 211.7ms +/- 1.0%
    Total: 215.0ms +/- 3.8%
    Total: 210.4ms +/- 1.0%

    Chrome 19.0.1084.52, Feb 2011 MacBook Pro i5 2.3Ghz, 8GB RAM

    ReplyDelete
  16. Patrick MeenanJune 1, 2012 at 2:10 PM

    Thanks everyone. I updated the chard with the data you all provided. Given we tend to skew high-end I'm comfortable recommending small instances for lower-end typical user testing and m1.medium for higher-end.

    ReplyDelete
  17. p83822June 1, 2012 at 7:01 PM

    Dell Latitude D630 (Core 2 Duo @ 2.0GHz)

    Total: 330.9ms +/- 3.9%
    Total: 325.6ms +/- 1.1%
    Total: 329.7ms +/- 5.6%
    Total: 321.6ms +/- 1.2%
    Total: 326.8ms +/- 5.1%

    ReplyDelete
  18. GuypoJune 4, 2012 at 1:39 PM

    Good test, and I think the SunSpider results do have some impact on page load, though I agree they shouldn't be the core.
    How about running a benchmark of websites through medium and small instances over a certain course of time and seeing if the load times are different?

    ReplyDelete
  19. GuypoJune 4, 2012 at 1:41 PM

    Good test. I think SunSpider results do have some impact on load time, but I agree it's not the bottlenecks for almost all websites (according to my experience).

    To get a better answer to your exact question, how about running a suite of websites through medium and small instances to see the effect? Enough runs+websites spread over time should give pretty accurate results.

    ReplyDelete
  20. Patrick MeenanJune 4, 2012 at 8:32 PM

    I wasn't necessarily concerned with how the different instances performed for fully loading web pages, I just wanted to tease out the raw CPU power for javascript to see how the different instances performed when compared to physical hardware (particularly when looking at CPU spikes and waterfall gaps to make sure it isn't too underpowered).

    ReplyDelete

All comments are moderated and may take a while to appear.

Note: Only a member of this blog may post a comment.

Subscribe to: Post Comments (Atom)