Security with HTTPS and SSL (2024)

The Secure Sockets Layer (SSL)—now technically known as Transport Layer Security(TLS)—is acommon building block for encrypted communications between clients and servers. It's possible thatan application might use SSL incorrectly such that malicious entities maybe able to intercept an app's data over the network. To help you ensure that this does not happento your app, this article highlights the common pitfalls when using secure network protocols and addresses some larger concerns about using Public-Key Infrastructure (PKI).

Concepts

In a typical SSL usage scenario, a server is configured with a certificate containing apublic key as well as a matching private key. As part of the handshake between an SSL clientand server, the server proves it has the private key by signing its certificate with public-key cryptography.

However, anyone can generate their own certificate and private key, so a simple handshakedoesn't prove anything about the server other than that the server knows the private key thatmatches the public key of the certificate. One way to solve this problem is to have the clienthave a set of one or more certificates it trusts. If the certificate is not in the set, theserver is not to be trusted.

There are several downsides to this simple approach. Servers should be able toupgrade to stronger keys over time ("key rotation"), which replaces the public key in thecertificate with a new one. Unfortunately, now the client app has to be updated due to whatis essentially a server configuration change. This is especially problematic if the serveris not under the app developer's control, for example if it is a third party web service. Thisapproach also has issues if the app has to talk to arbitrary servers such as a web browser oremail app.

In order to address these downsides, servers are typically configured with certificatesfrom well known issuers called Certificate Authorities (CAs).The host platform generally contains a list of well known CAs that it trusts.As of Android 4.2 (Jelly Bean), Android currently contains over 100 CAs that are updatedin each release. Similar to a server, a CA has a certificate and a private key. When issuinga certificate for a server, the CA signsthe server certificate using its private key. Theclient can then verify that the server has a certificate issued by a CA known to the platform.

However, while solving some problems, using CAs introduces another. Because the CA issuescertificates for many servers, you still need some way to make sure you are talking to theserver you want. To address this, the certificate issued by the CA identifies the servereither with a specific name such as gmail.com or a wildcarded set ofhosts such as *.google.com.

The following example will make these concepts a little more concrete. In the snippet belowfrom a command line, the openssltool's s_client command looks at Wikipedia's server certificate information. Itspecifies port 443 because that is the default for HTTPS. The command sendsthe output of openssl s_client to openssl x509, which formats informationabout certificates according to the X.509 standard. Specifically,the command asks for the subject, which contains the server name information,and the issuer, which identifies the CA.

$ openssl s_client -connect wikipedia.org:443 | openssl x509 -noout -subject -issuersubject= /serialNumber=sOrr2rKpMVP70Z6E9BT5reY008SJEdYv/C=US/O=*.wikipedia.org/OU=GT03314600/OU=See www.rapidssl.com/resources/cps (c)11/OU=Domain Control Validated - RapidSSL(R)/CN=*.wikipedia.orgissuer= /C=US/O=GeoTrust, Inc./CN=RapidSSL CA

You can see that the certificate was issued for servers matching *.wikipedia.org bythe RapidSSL CA.

An HTTPS Example

Assuming you have a web server with acertificate issued by a well known CA, you can make a secure request with code assimple this:

URL url = new URL("https://wikipedia.org");URLConnection urlConnection = url.openConnection();InputStream in = urlConnection.getInputStream();copyInputStreamToOutputStream(in, System.out);

Yes, it really can be that simple. If you want to tailor the HTTP request, you can cast toan HttpURLConnection. The Android documentation forHttpURLConnection has further examples about how to deal with requestand response headers, posting content, managing cookies, using proxies, caching responses,and so on. But in terms of the details for verifying certificates and hostnames, the Androidframework takes care of it for you through these APIs.This is where you want to be if at all possible. That said, below are some other considerations.

Common Problems Verifying Server Certificates

Suppose instead of receiving the content from getInputStream(), it throws an exception:

javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:374) at libcore.net.http.HttpConnection.setupSecureSocket(HttpConnection.java:209) at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.makeSslConnection(HttpsURLConnectionImpl.java:478) at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.connect(HttpsURLConnectionImpl.java:433) at libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:290) at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:240) at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:282) at libcore.net.http.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:177) at libcore.net.http.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:271)

This can happen for several reasons, including:

1. The CA that issued the server certificate was unknown
2. The server certificate wasn't signed by a CA, but was self signed
3. The server configuration is missing an intermediate CA

The following sections discuss how to address these problems while keeping yourconnection to the server secure.

Unknown certificate authority

In this case, the SSLHandshakeException occursbecause you have a CA that isn't trusted by the system. It could be becauseyou have a certificate from a new CA that isn't yet trusted by Android or your app isrunning on an older version without the CA. More often a CA is unknown because it isn't apublic CA, but a private one issued by an organization such as a government, corporation,or education institution for their own use.

Fortunately, you can teach HttpsURLConnectionto trust a specific set of CAs. The procedurecan be a little convoluted, so below is an example that takes a specific CA froman InputStream, uses it to create a KeyStore,which is then used to create and initialize aTrustManager. A TrustManager is what the systemuses to validate certificates from the serverand—by creating one from a KeyStore with one or more CAs—thosewill be the only CAs trusted by that TrustManager.

Given the new TrustManager,the example initializes a new SSLContext which providesan SSLSocketFactory you can use to override the defaultSSLSocketFactory fromHttpsURLConnection. This way theconnection will use your CAs for certificate validation.

Here is the example infull using an organizational CA from the University of Washington:

// Load CAs from an InputStream// (could be from a resource or ByteArrayInputStream or ...)CertificateFactory cf = CertificateFactory.getInstance("X.509");// From https://www.washington.edu/itconnect/security/ca/load-der.crtInputStream caInput = new BufferedInputStream(new FileInputStream("load-der.crt"));Certificate ca;try { ca = cf.generateCertificate(caInput); System.out.println("ca=" + ((X509Certificate) ca).getSubjectDN());} finally { caInput.close();}// Create a KeyStore containing our trusted CAsString keyStoreType = KeyStore.getDefaultType();KeyStore keyStore = KeyStore.getInstance(keyStoreType);keyStore.load(null, null);keyStore.setCertificateEntry("ca", ca);// Create a TrustManager that trusts the CAs in our KeyStoreString tmfAlgorithm = TrustManagerFactory.getDefaultAlgorithm();TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlgorithm);tmf.init(keyStore);// Create an SSLContext that uses our TrustManagerSSLContext context = SSLContext.getInstance("TLS");context.init(null, tmf.getTrustManagers(), null);// Tell the URLConnection to use a SocketFactory from our SSLContextURL url = new URL("https://certs.cac.washington.edu/CAtest/");HttpsURLConnection urlConnection = (HttpsURLConnection)url.openConnection();urlConnection.setSSLSocketFactory(context.getSocketFactory());InputStream in = urlConnection.getInputStream();copyInputStreamToOutputStream(in, System.out);

With a custom TrustManager that knows about your CAs,the system is able to validatethat your server certificate come from a trusted issuer.

Caution:Many web sites describe a poor alternative solution which is to install aTrustManager that does nothing. If you do this you might as well notbe encrypting your communication, because anyone can attack your users at a public Wi-Fi hotspotby using DNS tricks to send your users'traffic through a proxy of their own that pretends to be your server. The attacker can thenrecord passwords and other personal data. This works because the attacker can generate acertificate and—without a TrustManager that actuallyvalidates that the certificate comes from a trustedsource—your app could be talking to anyone. So don't do this, not even temporarily. You canalways make your app trust the issuer of the server's certificate, so just do it.

Self-signed server certificate

The second case of SSLHandshakeException isdue to a self-signed certificate, which means the server is behaving as its own CA.This is similar to an unknown certificate authority, so you can use thesame approach from the previous section.

You can create your own TrustManager,this time trusting the server certificate directly. This has all of thedownsides discussed earlier of tying your app directly to a certificate, but can be donesecurely. However, you should be careful to make sure your self-signed certificate has areasonably strong key. As of 2012, a 2048-bit RSA signature with an exponent of 65537 expiringyearly is acceptable. When rotating keys, you should check for recommendations from anauthority (such as NIST) about what is acceptable.

Missing intermediate certificate authority

The third case of SSLHandshakeExceptionoccurs due to a missing intermediate CA. Most publicCAs don't sign server certificates directly. Instead, they use their main CA certificate,referred to as the root CA, to sign intermediate CAs. They do this so the root CA can be storedoffline to reduce risk of compromise. However, operating systems like Android typicallytrust only root CAs directly, which leaves a short gap of trust between the servercertificate—signed by the intermediate CA—and the certificate verifier,which knows the root CA. To solvethis, the server doesn't send the client only it's certificate during the SSL handshake, buta chain of certificates from the server CA through any intermediates necessary to reach atrusted root CA.

To see what this looks like in practice, here's the mail.google.com certificatechain as viewed by the openssls_client command:

$ openssl s_client -connect mail.google.com:443---Certificate chain 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com i:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA 1 s:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority---

This shows that the server sends a certificate for mail.google.comissued by the Thawte SGC CA, which is an intermediate CA, and a second certificatefor the Thawte SGC CA issued by a Verisign CA, which is the primary CA that'strusted by Android.

However, it is not uncommon to configure a server to not include the necessaryintermediate CA. For example, here is a server that can cause an error in Android browsers andexceptions in Android apps:

$ openssl s_client -connect egov.uscis.gov:443---Certificate chain 0 s:/C=US/ST=District Of Columbia/L=Washington/O=U.S. Department of Homeland Security/OU=United States Citizenship and Immigration Services/OU=Terms of use at www.verisign.com/rpa (c)05/CN=egov.uscis.gov i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 International Server CA - G3---

What is interesting to note here is that visiting this server in most desktop browsersdoes not cause an error like a completely unknown CA or self-signed server certificate wouldcause. This is because most desktop browsers cache trusted intermediate CAs over time. Oncea browser has visited and learned about an intermediate CA from one site, it won'tneed to have the intermediate CA included in the certificate chain the next time.

Some sites do this intentionally for secondary web servers used to serve resources. Forexample, they might have their main HTML page served by a server with a full certificatechain, but have servers for resources such as images, CSS, or JavaScript not include theCA, presumably to save bandwidth. Unfortunately, sometimes these servers might be providinga web service you are trying to call from your Android app, which is not as forgiving.

There are two approaches to solve this issue:

• Configure the server to include the intermediate CA in the server chain. Most CAs provide documentation on how to do this for all common web servers. This is the only approach if you need the site to work with default Android browsers at least through Android 4.2.
• Or, treat the intermediate CA like any other unknown CA, and create a TrustManager to trust it directly, as done in the previous two sections.

Common Problems with Hostname Verification

As mentioned at the beginning of this article,there are two key parts to verifying an SSL connection. The firstis to verify the certificate is from a trusted source, which was the focus of the previoussection. The focus of this section is the second part: making sure the server you aretalking to presents the right certificate. When it doesn't, you'll typically see an errorlike this:

java.io.IOException: Hostname 'example.com' was not verified at libcore.net.http.HttpConnection.verifySecureSocketHostname(HttpConnection.java:223) at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.connect(HttpsURLConnectionImpl.java:446) at libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:290) at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:240) at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:282) at libcore.net.http.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:177) at libcore.net.http.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:271)

One reason this can happen is due to a server configuration error. The server isconfigured with a certificate that does not have a subject or subject alternative name fieldsthat match the server you are trying to reach. It is possible to have one certificate be usedwith many different servers. For example, looking at the google.com certificate withopenssl s_client -connect google.com:443 | openssl x509 -text you can see that a subjectthat supports *.google.com but also subject alternative names for *.youtube.com,*.android.com, and others. The error occurs only when the server name youare connecting to isn't listed by the certificate as acceptable.

Unfortunately this can happen for another reason as well: virtual hosting. When sharing aserver for more than one hostname with HTTP, the web server can tell from the HTTP/1.1 requestwhich target hostname the client is looking for. Unfortunately this is complicated withHTTPS, because the server has to know which certificate to return before it sees the HTTPrequest. To address this problem, newer versions of SSL, specifically TLSv.1.0 and later,support Server Name Indication(SNI), which allows the SSL client to specify the intendedhostname to the server so the proper certificate can be returned.

Fortunately, HttpsURLConnection supportsSNI since Android 2.3. Unfortunately, ApacheHTTP Client does not, which is one of the many reasons we discourage its use. One workaroundif you need to support Android 2.2 (and older) or Apache HTTP Client is to set up an alternativevirtual host on a unique port so that it's unambiguous which server certificate to return.

The more drastic alternative is to replace HostnameVerifierwith one that uses not thehostname of your virtual host, but the one returned by the server by default.

Caution: Replacing HostnameVerifiercan be very dangerous if the other virtual host isnot under your control, because a man-in-the-middle attack could direct traffic to anotherserver without your knowledge.

If you are still sure you want to override hostname verification, here is an examplethat replaces the verifier for a single URLConnectionwith one that still verifies that the hostname is at least on expected by the app:

// Create an HostnameVerifier that hardwires the expected hostname.// Note that is different than the URL's hostname:// example.com versus example.orgHostnameVerifier hostnameVerifier = new HostnameVerifier() { @Override public boolean verify(String hostname, SSLSession session) { HostnameVerifier hv = HttpsURLConnection.getDefaultHostnameVerifier(); return hv.verify("example.com", session); }};// Tell the URLConnection to use our HostnameVerifierURL url = new URL("https://example.org/");HttpsURLConnection urlConnection = (HttpsURLConnection)url.openConnection();urlConnection.setHostnameVerifier(hostnameVerifier);InputStream in = urlConnection.getInputStream();copyInputStreamToOutputStream(in, System.out);

But remember, if you find yourself replacing hostname verification, especiallydue to virtual hosting, it's still very dangerous if the other virtual host isnot under your control and you should find an alternative hosting arrangementthat avoids this issue.

Warnings About Using SSLSocket Directly

So far, the examples have focused on HTTPS using HttpsURLConnection.Sometimes apps need to use SSL separate from HTTP. For example, an email app might use SSL variantsof SMTP, POP3, or IMAP. In those cases, the app would want to use SSLSocketdirectly, much the same way that HttpsURLConnection does internally.

The techniques described sofar to deal with certificate verification issues also apply to SSLSocket.In fact, when using a custom TrustManager, what is passed toHttpsURLConnection is an SSLSocketFactory.So if you need to use a custom TrustManager with anSSLSocket, followthe same steps and use that SSLSocketFactory to create yourSSLSocket.

Caution:SSLSocket does not perform hostname verification. It isup the your app to do its own hostname verification, preferably by calling getDefaultHostnameVerifier() with the expected hostname. Furtherbeware that HostnameVerifier.verify()doesn't throw an exception on error but instead returns a boolean result that you mustexplicitly check.

Here is an example showing how you can do this. It shows that when connecting togmail.com port 443 without SNI support, you'll receive a certificate formail.google.com. This is expected in this case, so check to make sure thatthe certificate is indeed for mail.google.com:

// Open SSLSocket directly to gmail.comSocketFactory sf = SSLSocketFactory.getDefault();SSLSocket socket = (SSLSocket) sf.createSocket("gmail.com", 443);HostnameVerifier hv = HttpsURLConnection.getDefaultHostnameVerifier();SSLSession s = socket.getSession();// Verify that the certicate hostname is for mail.google.com// This is due to lack of SNI support in the current SSLSocket.if (!hv.verify("mail.google.com", s)) { throw new SSLHandshakeException("Expected mail.google.com, " "found " + s.getPeerPrincipal());}// At this point SSLSocket performed certificate verificaiton and// we have performed hostname verification, so it is safe to proceed.// ... use socket ...socket.close();

Blacklisting

SSL relies heavily on CAs to issue certificates to only the properly verified ownersof servers and domains. In rare cases, CAs are either tricked or, in the case of Comodo or DigiNotar, breached,resulting in the certificates for a hostname to be issued tosomeone other than the owner of the server or domain.

In order to mitigate this risk, Android has the ability to blacklist certain certificates or evenwhole CAs. While this list was historically built into the operating system, starting inAndroid 4.2 this list can be remotely updated to deal with future compromises.

Pinning

An app can further protect itself from fraudulently issued certificates by atechnique known as pinning. This is basically using the example provided in the unknown CA caseabove to restrict an app's trusted CAs to a small set known to be used by the app's servers. Thisprevents the compromise of one of the other 100+ CAs in the system from resulting in a breach ofthe apps secure channel.

Client Certificates

This article has focused on the user of SSL to secure communications with servers. SSL alsosupports the notion of client certificates that allow the server to validate the identity of aclient. While beyond the scope of this article, the techniques involved are similar to specifyinga custom TrustManager.See the discussion about creating a custom KeyManager in the documentation forHttpsURLConnection.

Nogotofail: A Network Traffic Security Testing Tool

Nogotofail is a tool gives you an easy way to confirm that your apps are safe against known TLS/SSL vulnerabilities and misconfigurations. It's an automated, powerful, and scalable tool for testing network security issues on any device whose network traffic could be made to go through it.

Nogotofail is useful for three main use cases:

• Finding bugs and vulnerabilities.
• Verifying fixes and watching for regressions.
• Understanding what applications and devices are generating what traffic.

Nogotofail works for Android, iOS, Linux, Windows, Chrome OS, OSX, in fact any device you use to connect to the Internet. There’s an easy-to-use client to configure the settings and get notifications on Android and Linux, as well as the attack engine itself which can be deployed as a router, VPN server, or proxy.

You can access the tool at the Nogotofail open source project.