Introduction
This multi-part tutorial is a complete guide to setting up WordPress on AWS. This part of our tutorial we’ll set up HTTPS and HTTP/2 using free Let’s Encrypt certificates, and update WordPress to work on the new URL.
Links to other parts of our tutorial
- Part One, Introduction
- Part Two, Setting up AWS for WordPress with RDS, Nginx, HHVM, PHP, SSMTP
- Part Three, Migrating WordPress Sites Into AWS, including RDS Setup
- Part Four, Wordpress Website Optimization, Nginx Caching and DNS Changes to Go Live
- Part Five, Setting up HTTPS and HTTP/2 using Free Let’s Encrypt Certificates
- Part Six, Setting up CloudFlare free CDN for global caching
Page Table of Contents / Index
- HTTPS with Let’s Encrypt Official Client
- ACME Let’s Encrypt Client
- ACME Let’s Encrypt Problem Solving
- Nginx HTTP/2 Configuration
- Updating WordPress
HTTPS and HTTP2 for WordPress using Lets Encrypt
Public key cryptography is complicated, and Let’s Encrypt is fairly immature. This part of the tutorial took a long time, partly because of immature software. The Let’s Encrypt official client had huge problems under Amazon Linux, so I ended up with an alternate client – I tried a few.
Note that the general process is something like this
- Get Nginx working on port 80. Configure it to listen for requests to the /.well-known/acme-challenge/ folder
- Configure Let’s Encrypt and request your certificate
- Add the appropriate https server block to your configuration, referencing the https certificate you obtained. Of course you can add this earlier, but it won’t work.
NB: if you have CloudFlare turned on for your domain you will want to disable any page rules that do https forwarding, and you might as well put CloudFlare into development mode. Sometimes caching can turn things screwy.
Install Let’s Encrypt Official Client and Request a Certificate
NB: I suggest you skip the official Let’s Encrypt client, jump down to the alternate client Acme below.
Let’s request a let’s encrypt certificate which we’ll use later on. A couple of notes:
- Let’s Encrypt is still a newish system, so expect problems. You can request a traditional paid certificate, and there are other free certificate providers around.
- The “–debug” may be required flag is because Amazon Linux support is currently experimental.
- At one point I had to pause Cloudflare, before I did a “standalone” certificate request.
- The client will keep getting better. It may be able to install the certificate into nginx/Apache for you soon, but I wanted to do everything manually so I understand how it’s set up, since I have to support this myself and I want to avoid downtime.
- If you have any problems try stopping your web server temporarily : “sudo service nginx stop” then “sudo service nginx start”
sudo bash yum update cd /opt yum install git cd letsencrypt git checkout amazonlinux (NB: not sure if this is required or not, it switches you to the Amazon Linux beta client) git pull (only if the command above is done) ./letsencrypt-auto certonly -a standalone -d aws.photographerstechsupport.com (NB: you can add other domains for the certificate in there : just add another "-d www.aws.photographerstechsupport.com" or similar to the end
My certificate was installed in /etc/letsencrypt/live/aws.photographerstechsupport.com/fullchain.pem
Alternative Let’s Encrypt Client : acme
The Let’s Encrypt library is frankly pretty awful on Amazon Linux, at least at the time of writing. Here’s how to use an alternate client acme. The documentation is pretty good and the author replied immediately when I asked a question, and it worked moderately easily. There’s more information here as well. Note that even with good tools this can be a complex area
First we need to set up a part of the web server that works for the challenge/response system Let’s Encrypt uses. Basically a file is put on the web server, Let’s Encrypt downloads it to verify you own the domain, the file is deleted, then the certificate is issued. If it can’t get the file (eg before DNS propogates) you don’t get the certificate.
This location has to be put into each http server. Note that the https location is not used at all.
# This server directly serves ACME / certificate redirects. All other requests are forwarded the https version of the page
server {
listen 80;
server_name example.com;
access_log /var/log/nginx/access.log main;
# Let's Encrypt certificates with Acmetool
location /.well-known/acme-challenge/ {
alias /var/www/.well-known/acme-challenge/;
}
location / {
return 301 https://www.photographerstechsupport.com$request_uri;
}
}
Now create that location and set up permissions, and create a file to check the setup has worked
mkdir -p /var/www/.well-known/acme-challenge
chmod -R user:www-data /var/www/acme-challenge/*
find /var/www/acme-challenge/ -type d -exec chmod 755 {} \;
vi /var/www/acme-challenge/.well-known/acme-challenge/text.html (add "hello world" or similar)
Restart nginx and test you can download the file, which will be at http://www.example.com/.well-known/acme-challenge/text.html
Now download the acme client and run it
sudo -i (this is run as root) cd /opt wget https://github.com/hlandau/acme/releases/download/v0.0.62/acmetool-v0.0.62-linux_386.tar.gz (NB check for newer versions here) tar -xzf acmetool-v0.0.62-linux_386.tar.gz cd acmetool-v0.0.62-linux_386/bin cp ./acmetool /usr/local/bin /usr/local/bin/acmetool quickstart
Now you have to follow the quick start process through. To start with choose the staging server, and your root directory is the one you created above. This is what you need to enter into the “quick start” process
Webroot method
/var/www/.well-known/acme-challenge/
Once you’ve done that you request the certificate. Certificates are stored in /var/lib/acme/certs and sometimes it seems to tell you the name of the certificate it creates, sometimes it doesn’t and you have to poke around. You can add any subdomains of the main domain you would like onto the end of this command.
/usr/local/bin/acmetool want example.com www.example.com
If something goes wrong you can run acme in debug mode, and I suggest using “tail -f” on your access log in another putty window so you can see any website access that Let’s Encrypt tries to access your site. This outputs to the file in /tmp/dump
/usr/local/bin/acmetool --xlog.severity=debug > /tmp/dump 2>&1 want example.com www.example.com
That file is fairly complex, this command will help bring up the important information
fgrep -v fdb: /tmp/dump | fgrep -v storageops: > dumpout
A cron job is automatically installed that runs every day to check whether certificates need to be reissued, as Let’s Encrypt certificates are only valid for three months. If the certificate is going to expire in two weeks or less (a month for earlier versions) the certificate is renewed, and you don’t need to do anything. You can check this with
crontab -e
After updating Acmetool you may have to run the following in the console and accept the latest terms of service
/usr/local/bin/acmetool
If you ever want to manually renew a certificate, then test it was renewed, you can use the following. Make sure your /etc/hosts file has the correct internal IP for the server set, if you’re using CloudFlare and it goes to the public IP you’ll get the date of the CloudFlare certificate expiry which isn’t helpful.
/usr/local/bin/acmetool want example.com www.example.com
echo | openssl s_client -connect example.com:443 2>/dev/null | openssl x509 -noout -dates
If you want to check the certificate manually use a command like this (note that certificates, at least on my system, are stored in /var/lib/acme/live/(sitename)
openssl x509 -enddate -noout -in certfilename
Acme is updated regularly. Here’s how I do my updates
Get the update URL from: https://github.com/hlandau/acme/releases wget https://github.com/hlandau/acme/releases/download/v0.0.58/acmetool-v0.0.58-linux_386.tar.gz tar xvfz acmetool-v0.0.58-linux_386.tar.gz cp ./acmetool-v0.0.58-linux_386/bin/acmetool /usr/local/bin/acmetool
That’s it – pretty simple 🙂
Acmetool and Let’s Encrypt problem solving
Certificates and encryption can be tricky. I had an incorrect Nginx configuration around certificate challenges, it took me a couple of hours to track down. This issue on the Acmetool github page outlines things well, but i’ll copy the core parts below in case it disappears.
The first step is to run Acmetool interactively – sometimes it just wants to prompt you to enter infomation.
/usr/local/bin/acmetool want example.com www.example.com
The next step is to run it in debug mode with verbose output sent to a file for you to look through.
acmetool --xlog.severity=debug > /tmp/dump 2>&1 want example.com www.example.com fgrep -v fdb: /tmp/dump | fgrep -v storageops: > /tmp/dumpout
This line in the acmetool debug output was a clue to solving one problem
http-01 self test failed: non-200 status code when doing self-test
I tracked the cause of this down looking at the web server logs. I could see that Nginx was looking in the directory “/var/www/acme-challenge/.well-known/acme-challenge”, but the acmetool debug output showed acme was writing to “/var/www/acme-challenge/”. It was clear that either acme needed to write to a different directory (couldn’t do that, no idea where the config file is) or Nginx needed to look in the right place. The nginx directive “root” tells Nginx to append the directories too the file system root when looking for a file, “alias” tells it to just look for the filename requested ignoring paths.
Here’s the line that was my major clue (tidied up a bit from the logs)
open() "/var/www/acme-challenge/.well-known/acme-challenge/p64-Tlqu327rd2E0ALgwx5Fc4WuMTCuybTYMbPWUBWA" failed (2: No such file or directory), request: "GET /.well-known/acme-challenge/p64-Tlqu327rd2E0ALgwx5Fc4WuMTCuybTYMbPWUBWA HTTP/1.1",
Looking in the “/var/lib/acme/live/” directory was helpful, to show me the state of the certificates.
You can look at the expiry date of a cert file with this command, and the following command shows general certificate information for a given domain name
openssl x509 -enddate -startdate -noout -in cert openssl s_client -showcerts -connect example.com:443
You can connect directly to the server to look at the certificate it’s presenting using this command. The “servername” argument is for server name indication, as most servers run many websites. If you’re running a CDN or proxy you’ll need to use the IP address where it says HOST below, otherwise you can use your domain name.
openssl s_client -showcerts -connect HOST:443 -servername photographerstechsupport.com
Acmetool Self Test Failing #2
Another problem I had was acmetool failing the self test. The solution was to disable the self test
vi /var/lib/acme/desired/www.example.com
(add this to the end of the file)
request:
challenge:
http-self-test: false
I’m not 100% sure that this was the root problem, because I changed a few things, but it’s worth recording.
Acmetool Problem Solving #17
I have fairly regular problems with certificate renewals, for a variety of reasons. Today the problem appears to have been that I’d set CloudFlare pagerules to 301 redirect all requests to https, but I hadn’t set up an https listener. Once I turned off this 301 redirect things started working.
Another problem I had today was that I’d removed a subdomain from DNS, but it was still on a certificate. I put in a request for just the www and non-www domains, but Acmetool seemed to have sent in a request for all the domains. I solved this by deleting the request file from /var/lib/acme/desired/(domainname)
A useful diagnostic tool is inotifywait. It allows you to see all the file actions happening in a folder – file creation, modification, deletion, etc. This lets us see that Acmetool is creating challenge files, which we can match with web server requests for those files.
yum install inotify-tools libnotify-bin
inotifywait -m /var/www/.well-known/acme-challenge
Acmetool Staging Certificate Problem
Another problem I had was after issuing a test certificate using the Let’s Encrypt staging server I tried to issue a production certificate. It seemed to work, connections failed. When I used openssl to diagnose the problem I could see the certificate was still using the fake staging Let’s Encrypt intermediate certificate
issuer=/CN=Fake LE Intermediate X1
I ended up going to the folder below and using rm -rf to delete the listed folders and all related resources. I then issued the certificate again with the “acmetool want” command and it worked fine.
Before you start
- MAKE SURE YOU TAKE A BACKUP BEFORE YOU DO THIS
- Go into the folder /var/lib/acme/live/ and using “ls -l” make a note of the cert folder that the correct domains link to, both www and non-www. For example the folder example.com could be a symbolic link to ../certs/svf5gqsyfgbkpefe2gehsmg35tbsneac7qut
/var/lib/acme/desired/example.com* /var/lib/acme/live/example.com and www.example.com /var/lib/acme/live/certs/ID-GOES-HERE (www and non-www variant)
Upgrading Acmetool
This is how I upgrade acmetool to the newest version. First I check for a new version of Acme, and I change the numbers below as needed. You don’t need to run quickstart again.
cd /tmp wget https://github.com/hlandau/acme/releases/download/v0.0.62/acmetool-v0.0.62-linux_386.tar.gz tar -xzf acmetool-v0.0.62-linux_386.tar.gz cd acmetool-v0.0.62-linux_386/bin cp ./acmetool /usr/local/bin
Nginx HTTP/2 Configuration – Adding a Let’s Encrypt HTTPS Certificate to Nginx
The next step is to add the certificate to nginx so it can do https and http2. HTTP2 only runs over TLS1.2 / HTTPS. You
server {
server_name www.example.com;
# listen 80; # NB you can listen to two ports at a time if you want to
listen 443 ssl http2;
# This is where you put in the key that was just generated
ssl_certificate /var/lib/acme/live/EXAMPLE.COM/fullchain;
ssl_certificate_key /var/lib/acme/live/EXAMPLE.COM/privkey;
# Set up preferred secure protocols and ciphers. TLS1.2 is required for HTTP/2
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES:!ADH:!AECDH:!MD5;
# etc
}
# Redirect all variations to https://www domain
server {
listen 80;
server_name example.com www.example.com;
return 301 https://www.example.com$request_uri;
}
# redirect https://example.com to https://www.example.com
server {
listen 443 ssl http2;
server_name example.com;
ssl_certificate /var/lib/acme/certs/INSERT_DIRECTORY/fullchain;
ssl_certificate_key /var/lib/acme/certs/INSERT_DIRECTORY/privkey;
# Set up preferred protocols and ciphers. TLS1.2 is required for HTTP/2
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES:!ADH:!AECDH:!MD5;
return 301 https://www.example.com$request_uri;
}
You also need to add the following to your nginx.conf in the http block, as it only needs to be defined once. 1MB is enough of a cache for 4000 sessions, so 5M is enough for 20,000 sessions, you may want to alter it.
# This is a cache for SSL connections to make things faster for returning visitors ssl_session_cache shared:SSL:5m; ssl_session_timeout 60m;
Of course you then restart nginx to activate it.
Updating WordPress URLs
Once you’ve moved your site onto https you need to update your database so all the links point to the https URL, rather than the old http URL. The old links still work, but you’ll get browser warnings about insecure elements on your page. Go into your database and run these commands, tailored for your URLs of course:
UPDATE wp_posts SET post_content = replace(post_content, 'http://www.example.com', 'https://www.example.com'); UPDATE wp_posts SET guid = replace(guid, 'http://www.example.com', 'https://www.example.com'); UPDATE wp_postmeta SET meta_value = replace(meta_value,'http://www.example.com', 'https://www.example.com');
You also need to go into settings -> general and change your URL to https rather than http.
Next Steps
At this point you should a fast, well optimized WordPress server running on your AWS instance, with a Let’s Encrypt certificate enabling HTTP/2, and you could easily go live at this point. Read the next part of our tutorial (available Mid Feb 2016) to learn how to add the CloudFlare content distribution network, which will further accelerate our website as well as protect against various security threats.