EC2 AMD Epyc vs Xeon Introduction
The reservation for the t2.nano that powers this and a number of other websites finished this week. Of course I wanted to try the new t3 series EC2 instances which are meant to be both cheaper and faster than t2, and I was curious about the new t3a instances. I did a little benchmarking to work out how the t3 and t3a performed against the t2 instances, and checked the m5 vs m5a while I was doing it.
The websites run on Nginx, PHP 5.6 (I must upgrade at some point), and MySQL on a nano instance with 512MB of RAM, and 512MB of swap. The CPU is mostly idle other than during the regular backups. RAM is a bit tight, and you need to optimize MySQL carefully and remove the performance schema to make it work inside the memory available.
Testing Methodology
I set up an m5.large instance to run tests, located in the same availability zone as my primary production web server. I used the Siege benchmarking tool to test performance. The production web server is lightly enough loaded that I was comfortable testing in production.
Siege was given URLs for six pages on three of my websites – each home page, a blog category page, an “about us” page, and a page generated by custom PHP code. I modified the testing server hosts file so Siege hit the server directly, rather than going via CloudFlare which does caching and DDOS protection for the sites.
Two runs were done for each instance. The first has Nginx with one worker thread, making it effectively single threaded, though PHP and MySQL can run on any other CPU cores available. On the second run Nginx was allowed to use all CPU threads.
The Nginx page cache was in use, so this benchmark is really an Nginx benchmark, with a bit of PHP for page generation.
Testing Results
Here’s the key results from the testing.
| Instance Type | Single Thread Response Time (s) | Single Thread Transaction Rate (t/sec) | Multithreaded Response Time (s) | Multithreaded Thread Transaction Rate (t/sec) |
| t2.nano | 0.12 | 83.6 | N/A | N/A |
| t3.nano | 0.08 | 117.75 | 0.08 | 125.68 |
| t3a.nano | 0.09 | 101.22 | 0.08 | 114.36 |
| m5.large | 0.08 | 114.18 | 0.08 | 118.83 |
| m5a.large | 0.10 | 100.49 | 0.09 | 110.11 |
| m5.4xl | 0.06 | 173.08 | 0.03 | 288.70 |
| m5a.4xl | 0.07 | 134.77 | 0.04 | 245.84 |
| c5.4xl | 0.05 | 192.20 | Not run | Not run |
Conclusion
Based on both the response time and the transaction rate the AMD instances appear to have 10 – 20% lower performance than equivalent Intel Xeon CPUs.
The AMD based servers appear to scale a little better: when more threads are used the gap between Intel and AMD seems to narrow.
Both of the t3.nano instances are around 30% faster than the t2.nano single threaded. Performance improves around 10% when the second thread is used.
Given the “A” instances cost 10% less than the Xeon instances the AMD instances seem like a good option for lightly loaded servers.
Sysbench
I also ran Sysbench for a general idea of performance. The CPU and I/O tests all came out within 1% of each other, so the tests didn’t tell us all that much.
Testing Raw Data
Note that I changed the number of tests run during my testing period as I was running out of CPU credits, but I kept the number of testing threads the same.
t2.nano benchmarking
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 24.76 secs
Data transferred: 39.28 MB
Response time: 0.12 secs
Transaction rate: 83.60 trans/sec
Throughput: 1.59 MB/sec
Concurrency: 9.80
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 3.39
Shortest transaction: 0.00
t3a.nano
Nginx single core
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 20.45 secs
Data transferred: 39.28 MB
Response time: 0.09 secs
Transaction rate: 101.22 trans/sec
Throughput: 1.92 MB/sec
Concurrency: 9.59
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 2.69
Shortest transaction: 0.00
Nginx multicore
iege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 18.10 secs
Data transferred: 39.28 MB
Response time: 0.08 secs
Transaction rate: 114.36 trans/sec
Throughput: 2.17 MB/sec
Concurrency: 9.37
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 3.04
Shortest transaction: 0.00
Nginx multicore
t3.nano benchmarking
Nginx single core
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 17.58 secs
Data transferred: 39.28 MB
Response time: 0.08 secs
Transaction rate: 117.75 trans/sec
Throughput: 2.23 MB/sec
Concurrency: 9.77
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 2.30
Shortest transaction: 0.00
Nginx multicore
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 16.47 secs
Data transferred: 39.28 MB
Response time: 0.08 secs
Transaction rate: 125.68 trans/sec
Throughput: 2.39 MB/sec
Concurrency: 9.51
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 2.52
Shortest transaction: 0.00
m5.large benchmarking
Nginx single core
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 18.13 secs
Data transferred: 39.28 MB
Response time: 0.08 secs
Transaction rate: 114.18 trans/sec
Throughput: 2.17 MB/sec
Concurrency: 9.69
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 2.46
Shortest transaction: 0.00
Nginx multicore
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 17.42 secs
Data transferred: 39.28 MB
Response time: 0.08 secs
Transaction rate: 118.83 trans/sec
Throughput: 2.26 MB/sec
Concurrency: 9.53
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 2.70
Shortest transaction: 0.00
m5a.large benchmarking
Nginx single core
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 20.60 secs
Data transferred: 39.28 MB
Response time: 0.10 secs
Transaction rate: 100.49 trans/sec
Throughput: 1.91 MB/sec
Concurrency: 9.73
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 2.65
Shortest transaction: 0.00
Nginx multicore
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 18.80 secs
Data transferred: 39.28 MB
Response time: 0.09 secs
Transaction rate: 110.11 trans/sec
Throughput: 2.09 MB/sec
Concurrency: 9.59
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 3.35
Shortest transaction: 0.00
m5a.4xl benchmarking
Nginx single core
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 15.36 secs
Data transferred: 39.28 MB
Response time: 0.07 secs
Transaction rate: 134.77 trans/sec
Throughput: 2.56 MB/sec
Concurrency: 9.57
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 1.33
Shortest transaction: 0.00
Nginx multicore
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 8.42 secs
Data transferred: 39.28 MB
Response time: 0.04 secs
Transaction rate: 245.84 trans/sec
Throughput: 4.67 MB/sec
Concurrency: 9.12
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 1.35
Shortest transaction: 0.00
m5.4xl benchmarking
Nginx single core
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 11.96 secs
Data transferred: 39.28 MB
Response time: 0.06 secs
Transaction rate: 173.08 trans/sec
Throughput: 3.28 MB/sec
Concurrency: 9.56
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 1.07
Shortest transaction: 0.00
Nginx multicore
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 7.17 secs
Data transferred: 39.28 MB
Response time: 0.03 secs
Transaction rate: 288.70 trans/sec
Throughput: 5.48 MB/sec
Concurrency: 9.27
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 1.15
Shortest transaction: 0.00
c5.4xl
Nginx single core
siege -f ./siege -d0 -r10 -c10
** SIEGE 4.0.4
** Preparing 10 concurrent users for battle.
The server is now under siege…
Transactions: 2070 hits
Availability: 100.00 %
Elapsed time: 10.77 secs
Data transferred: 39.28 MB
Response time: 0.05 secs
Transaction rate: 192.20 trans/sec
Throughput: 3.65 MB/sec
Concurrency: 9.63
Successful transactions: 2070
Failed transactions: 0
Longest transaction: 1.08
Shortest transaction: 0.00