Tune The Web

HTTP Public Key Pinning (HPKP)

Introduction

The SSL/TLS digital certificate used to provide HTTPS security (the green padlock) to a website has several functions which basically boil down to:

  1. It provides confidence that the website you are browsing, is the real website for that company.
  2. It encrypts network traffic so eavesdroppers cannot listen in to network traffic, and also that the message has not been tampered with between client and server.

It is this first item that Public Key Pinning attempts to improve.

It's not that difficult to redirect traffic from a real website to a dummy copy. Say a hacker sets up a free wi-fi hotspot called "Free Wi-fi" from their computer and goes along to their local Starbucks. People will likely connect to your hotspot to browse the web on their coffee break. The hacker can set up the DNS of that wi-fi network so that requests to http://mail.google.com would instead be sent to a separate website that they are running that either looks exactly like the real Gmail, or just forwards requests back and forth to the real website after listening in to the traffic (known as a Man In The Middle (MITM) attack). The browser would still show http://mail.google.com so the innocent coffee drinker would be none-the-wiser, and would happily enter their username and password which the hacker would then have access to. A few hours in Starbucks and they could have all sorts of passwords, banking details... etc.

The above scenario is made a good bit more difficult for the hacker when a website uses HTTPS (like https://mail.google.com does). In this case the traffic is encrypted with Google's certificate and the hacker will not be able to understand the traffic even if they listen in to it. To get around this the hacker will either need to keep it on HTTP rather than HTTPS (which the user might notice, as there would be no green padlock and which HSTS attempts to address) or use their own certificate which they have the key for so can decrypt. This will need to be a real certificate issued by a trusted certificate authority (CA) so the web browser trusts it and shows the green padlock rather than an error. The main issue that HPKP looks to address is that any trusted CA can issue a certificate for any website and your web browser will trust it. Google actually issue their own certificates for sites like Gmail, but there is nothing to stop another CA also issuing a certificate for that website. You would hope a CA would confirm the person requesting the certificate is from Google and not a hacker, so this may sound far fetched, but it has happened already.

So to ensure the correct certificate is used, Google can "pin" the correct certificate to the https://mail.google.com domain. This is done by returning a HTTP Header with each request telling the web browser the certificates that this website should allow. If the hacker tries to use a certificate that is not on this list, then the browser will show an error message and so the attack will be stopped.

How to set it up

Public Key Pinning works by getting a hash of the certificate using the following openssl command (assuming your server certificate is stored in a file called server.crt):

openssl x509 -pubkey -in server.crt | openssl pkey -pubin -outform der | openssl dgst -sha256 -binary | base64

This will give you a Base64 encrypted value like this:

TLWu29Y83wlFocu4Vo9Lcj34h3eLB3+BpBPb654LS8o=

Repeat the process for your backup cert. Note there is a minimum of two separate hashes, based on two completely independent certs. This is a protection to stop you accidentally shooting yourself in the foot - see below. If you do not have a backup cert, you can also do it for a backup CSR using the following command (which will save you paying for a backup cert until you need it):

openssl req -pubkey -in backup_server.csr | openssl pkey -pubin -outform der | openssl dgst -sha256 -binary | base64

Alternatively to save getting a cert or even a csr, you can also get the hash directly from the key itself using the following:

openssl rsa -pubout -outform der -in server.key | openssl dgst -sha256 -binary | base64

No matter how you get your two hashes, you then need to add the HTTP header. In Apache the HTTP header can be added with config like the following:

Header always set Public-Key-Pins: "pin-sha256=\"TLWu29Y83wlFocu4Vo9Lcj34h3eLB3+BpBPb654LS8o=\"; \ pin-sha256=\"kYYpAGMOnb4tDu2wbNuYcB2ch3vR7Djmnffb/rNRnNE=\"; \ max-age=60;"

Note: this should only be set on your HTTPS site and not the HTTP version. The RFC tells browsers to ignore this for HTTP sites, but still best to set it up properly. I've also set this up with a short max-age (60 seconds) so I don't block my site for all users for very long if there turns out to be a problem with this

Where to Pin

You do not need to pin your site certificate, as that does cause problems when you renew (see below). You could instead pin, the intermediate certificate, or perhaps the root certificate instead. As long as it's a certificate used to validate your certificate. This may be safer, so you can purchase a new certificate from the same CA without having to change your HPKP policy. This however is not without it's own issues (see below), and also the backup pin needs to be independent of this one so that is easier when pinning the site cert itself.

Reporting

You can also have the web browsers report back errors using the report-uri argument (a good way of spotting if you're blocking lots of users for a reason):

Header always set Public-Key-Pins: "pin-sha256=\"TLWu29Y83wlFocu4Vo9Lcj34h3eLB3+BpBPb654LS8o=\"; \ pin-sha256=\"kYYpAGMOnb4tDu2wbNuYcB2ch3vR7Djmnffb/rNRnNE=\"; \ max-age=60; report-uri=\"http://report.tunetheweb.com/api/report/hpkp-report.json\""

Note that, unlike the CSP equivalent syntax of "report-uri url;" this has a "report-uri=url" syntax with an equals sign. However it should be noted that, at the time of writing, only Chrome supports this and even then only just in the latest version (version 46) (see support below for details). In fact if more browsers did support this, it would be good to use the Public-Key-Pins-Report-Only header initially for a while until you confirm you're not getting any false reports:

Header always set Public-Key-Pins-Report-Only: "pin-sha256=\"TLWu29Y83wlFocu4Vo9Lcj34h3eLB3+BpBPb654LS8o=\"; \ pin-sha256=\"kYYpAGMOnb4tDu2wbNuYcB2ch3vR7Djmnffb/rNRnNE=\"; \ max-age=60; report-uri=\"http://report.tunetheweb.com/api/report/hpkp-report.json\""

Either reporting method, when it becomes more supported, requires a service to be set up to listen to the reports. Checkout Scott Helme's excellent report-uri tool for this, to save you setting up your own one. In fact, while you are there, Scott also has a great page on HPKP which is well worth a read too.

If you do fancy setting up your own service then you simply need something listening for JSON reports and then decide what to do with them. For example a NodeJs service which listens for CSP and HPKP violation reports and logs them to log file could be as simple as this:

'use strict' var express = require('express'), bodyParser = require('body-parser'), debug = require('debug')('ReportService'), errorhandler = require('errorhandler'), morgan = require('morgan'), compression = require('compression'); var app = express(); app.disable('x-powered-by'); app.use(errorhandler({ dumpExceptions: true, showStack: true })); //app.use(express.json()); app.use(morgan( 'dev', { immediate: true })); app.use(compression()); app.use(function(req, res, next) { //Some browsers (Chrome at least) doesn't set the content type for HPKP //Reports which causing the body parser to ignore the content, so add the content type. if(typeof(req.headers['content-type']) === 'undefined'){ req.headers['content-type'] = "application/json; charset=UTF-8"; } next(); }) // parse various different custom JSON types as JSON app.use(bodyParser.json({ type: 'application/json' })); app.use(bodyParser.json({ type: 'application/csp-report' })); app.get('/test/', function (req, res) { res.send('I am running!\n'); }); app.get('/report/csp-report.json', function (req, res) { res.status(403); res.send('Should only be a POST request!'); }); app.get('/report/hpkp-report.json', function (req, res) { res.status(403); res.send('Should only be a POST request!'); }); app.post('/report/csp-report.json', function (req, res) { debug('debugReport for CSP:' + JSON.stringify(req.body)); res.send('Thanks for reporting'); }); app.post('/report/hpkp-report.json', function (req, res) { debug('debugReport for HPKP:' + JSON.stringify(req.body)); res.send('Thanks for reporting'); }); app.listen(3002); debug(JSON.stringify(3002) + ":port starting");

There are some additional considerations when setting up your reporting location - in particular if using the same website it's trying to report on. If the report-uri uses HTTPS then it will need to be able to contact that site to report the error and HPKP validation is also completed for reports. As the only time it should report, is if there is a failure with the site certificate, you can see how using a HTTPS report-uri on the same site which has the HPKP error would lead to problems as it will not be able to connect to send the report! The RFC does state that the web browser "MAY attempt to re-send the report later.", but to me the better option is to either use a different domain for reports (or perhaps a separate tool like report-uri), or to use HTTP version (like this website does). However if you decide to use a HTTP site, then that may mean some extra set up to allow that if you are using HSTS on your site.

IncludeSubDomains

There is also the includeSubDomains option, which can force the pinning on all subdomains. Like for the comment on this in the HSTS post, this should be used with caution.

Testing

Testing HPKP is actually quite tricky. You can test that it works, by visiting the websites, but just because you get no error doesn't mean that it works - it can also mean the browser hasn't implemented it. To really test that it works you need to use another certificate that is not pinned - but is still valid (as the certificate checking will run first). However that means buying another certificate. If you think you could try using a self signed cert, then that doesn't work as locally installed certs, ignores HPKP (see downsides below). Chrome does allow you to view and set HPKP policies in the chrome://net-internals/#hsts settings page (HPKP being closely linked to HSTS) so you can set up a fake policy there to test - which also has the added advantage of not breaking the site for your other users. This is probably the best way of testing your policy, but that doesn't help for testing other browsers and at the time of writing Chrome did not allow you to add report-uri settings to manually set policies so testing of any report-uri service is even trickier.

Support

Web server support is not an issue since this is just another header and every major web server has the ability to add custom headers as response codes. Browser support is only OK, though, at the time of writing the report-uri part is not supported at all. Chrome have implemented reporting of HPKP errors in the next version, but not other browsers such as Firefox. And that obviously means they do not support "Public-Key-Pins-Report-Only" header either. As setting up a Report policy first is recommended, this makes HPKP difficult to recommend at present. Who knows what weird device doesn't do this properly, or if some proxy server causes your site to break.

The downsides

One of the main issues is that it is a trust on first use policy. There's nothing to stop the hacker removing that HPKP Header. To avoid this you need to have visited the real website first, so the browser has loaded the Public Key Pinning policy into it's settings, and uses that going forward. You also need to set a long expiry on the policy so it stays loaded in the website. However common websites people visit often from their laptops and phones, the risk is much reduced.

A bigger issue, to me at least, is the risk of accidentally blocking visitors to your website. Once you put up a HPKP header you are saying only those certificates can be used for this website. Now the policy will only be accepted when it passes, so just setting up a bad policy shouldn't cause an issue - though without first running in report only mode, which is not widely supported, that's difficult to be sure.

When the policy is accepted then there are issues around when you want to replace your key or certificate:

I am also concerned that HPKP encourages key reuse. As the key is effectively pinned, rather than the certificate, you can get a new certificate for the same Key or same CSR without having to think about HPKP. Best practice states that you should reissue your key when renewing the certificate, but that requires doing it in advance as discussed above. You could use your backup pin on next reissue, but then you need to create a new backup and, until the max-age time passes, live without a backup or use your old key as the backup. Or you could have two backups. So ways around it but no denying we've complicated the processes!

Finally some systems do "legitimate" MITM processing. For example if you have Avast Anti-virus installed then, by default, it scans HTTPS traffic for you. To do that it needs to subtly replace the certificate on the website with it's own copy (to which it has the key) and then sit in the middle, scanning traffic as it's sent back and forth. Similarly some corporate environments do the same for the same reasons. Now you can argue about whether they should do this (and issues like Superfish brought this argument front and centre), but the reality is they do and you don't want to break your website just because you disagree with how popular Antivirus software works. Testing seems to show that when a local cert is used, the HPKP header is ignored but, since the RFC does not mention this, it's difficult to understand how this works. It also compromises HPKP to me - the very point is you cannot use other certificates but here we have a use case which allows this! How secure that use case is, is unknown until we get full understanding of how it works. Then again this does require system access to the computer to install a root certificate that the user will trust, and after someone has that access you can argue the user is already in deep trouble, so there's still protection in this for the more likely scenario of more general intercepting of traffic.

I've a longer blog on some of the dangers of HPKP and other security headers.

Summary

HTTP Public Key Pinning is a powerful tool to ensure your website is not compromised, but with that power, comes great ability to shoot yourself in the foot! Make sure you fully understand what you are doing, and the effort you are opening yourself up to in future, before you turn this on. An alternative approach to the issues HPKP tries to address is certificate transparency, which may in the future give a lot of the benefits of HPKP with almost no risks.

Personally, until reporting is more widely supported by browsers, I think it's difficult to recommend using HPKP at all. However, given it's general support in most browsers apart from the reporting side, I do not expect it to be too long before the other major browsers support HPKP fully. In the meantime, if you are interested in this, then I would say use the Public-Key-Pins-Report-Only option now that Chrome is about to implement support for it, and keep an eye on your reports to see what comes in and whether these concerns are unfounded or not. It was only released the night before this post was put up, and hasn't yet been rolled out to all users, but already some were stating reports coming in for sites that presumably were supposed to be configured correctly. So it will be interesting to see if any technologies or proxies do not work with HPKP now we have the reporting ability to see this.

Finally you also need to ask how likely the issue HPKP is designed to protect (a MITM attack with a certificate which is recognised by the browser but not issued by a locally stored root) is? And how likely messing up a HPKP policy is and that causing you to DoS yourself? For me the second scenario is much more likely so am not sure HPKP is a good idea for most websites, though it is a welcome addition to the arsenal of tools for web site owners should they want this extra protection and be willing to manage it properly.

This page was originally created on and last edited on .

How useful was this page?

Load more comments!