Intercepting Go TLS Connections with Wireshark

Posted on Friday May 14, 2021

I wrote previously about how I like to use mitmproxy for debugging HTTP services. This is a continued exploration of debugging network services, in particular focused around inspecting TLS encrypted traffic that your application is sending and receiving.

Transport Layer Security is a fundamental building block of modern secure communications on the Internet, and increasingly the software we write is expected to be a fluent speaker of TLS. While this brings security benefits for users, it also increases the complexity of understanding what our software is doing because when we try to use tools like Wireshark or tcpdump to inspect network traffic, all we see is encrypted data. Let’s see what a regular HTTP request looks like in Wireshark:

$ curl http://www.benburwell.com

Here, we can see the HTTP request and response. But what happens when we make the request over TLS?

$ curl https://www.benburwell.com

Here all we see are some TLS packets with embedded “encrypted application data.” We can see that a connection is being made, but we can”t inspect the raw HTTP request or response as we’d like to.

But all is not lost! There is a way for Wireshark to decrypt TLS connections and show you dissected application protocol packets, it just requires a little configuration. To understand how this works, we first need to understand a little bit about TLS.

How decrypting TLS in Wireshark works

TLS encrypts data within a session using a “master secret,” a symmetric encryption key that is established by using a key exchange protocol. So in order for Wireshark to be able to decrypt and dissect TLS packets, we need some way to tell it the master secret for the session.

The master secret is agreed upon using a cryptographic protocol when the TLS connection is established. The exact implementation varies, but in general the client and the server use some clever math to derive a value that is known at both ends and yet is never directly sent over the wire, such that it is computationally expensive for intermediate observers to derive the secret for themselves. We won’t get into the specifics, but one important detail for later is that this exchange involves the client sending the server a large random number in plain text, before the encrypted stream begins.

Conveniently, many TLS client libraries support the use of a key log file, which does pretty much exactly what it sounds like: when the SSLKEYLOGFILE environment variable is set, the library writes the key needed to decrypt the traffic each time it establishes a TLS connection. Originally, this was implemented in Mozilla’s (at the time Netscape’s) Network Security Services library, so you might also see it referred to as a “NSS Key Log File.” Let’s give this a try!

$ SSLKEYLOGFILE=/tmp/keys curl -s https://www.benburwell.com >/dev/null
$ cat /tmp/keys
CLIENT_RANDOM 40b1a54e6b38f7accb90e1f5162534b8628389f4257e39f614a3ca28514db2c7 3121d2812c459996b072165c2ece4a1c85687d7073de06be0e1c16bf4a862fbe26a8cba24db1a4a0a9684fb19ad52f97

(Note that SSLKEYLOGFILE support was only enabled by default in curl 7.58, so if this isn’t working for you, check which version of curl you have).

This line in the key log means that for the TLS connection that was initiated with the CLIENT_RANDOM of 40b1..., the master secret is 3121.... So now we just need to tell Wireshark about this. Let’s start a new capture and make another request:

Now, we can right-click on the “Transport Layer Security” layer and select Protocol Preferences -> (Pre)-Master-Secret log filename... and enter the path to our SSLKEYLOGFILE, /tmp/keys, and something magical happens:

Now, when Wireshark encounters a TLS handshake, it can extract the random value sent by the client and consult the key log file to discover a matching CLIENT_RANDOM line and use the corresponding session master secret to decrypt the data sent over the connection. So in addition to seeing the TLS details as before, we can also see the decrypted HTTP requests!

Configuring Go to use a TLS Key File

Go doesn’t support the SSLKEYLOGFILE environment variable directly, but it does have a different mechanism to achieve the same result. The crypto/tls.Config struct has a KeyLogWriter field:

// KeyLogWriter optionally specifies a destination for TLS master secrets
// in NSS key log format that can be used to allow external programs
// such as Wireshark to decrypt TLS connections.
KeyLogWriter io.Writer

In typical Go fashion, I/O has been abstracted to an io.Writer interface. Since we can use an *os.File to satisfy this interface, all we need to do to produce a file containing the TLS secrets is to open a file and pass that through the tls.Config.KeyLogWriter:

package main

import (
  "crypto/tls"
  "net/http"
  "os"
)

func main() {
  f, err := os.OpenFile("/tmp/keys", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600)
  if err != nil {
    panic(err)
  }
  defer f.Close()

  client := &http.Client{
    Transport: &http.Transport{
      TLSClientConfig: &tls.Config{
        KeyLogWriter: f,
      },
    },
  }

  client.Get("https://www.benburwell.com")
}

Building and running our program results in CLIENT_RANDOM lines being appended to our /tmp/keys file and picked up by Wireshark, which in turn is able to decrypt the messages being sent by our program:

In practice, for decrypting HTTP traffic for debugging, I find mitmproxy to be faster and easier, since it doesn’t require changes to the program. However, sometimes it’s preferable to look at the actual bytes on the wire, which is where using a key log file with Wireshark might be a better approach.

Additionally, there are plenty of protocols other than HTTP that use TLS connections, and where proxying isn’t an option. For example, I’ve used a key log file with Wireshark to debug a Go program that was making an IMAP connection to a mail server. Because of the way Go’s libraries tend to be layered, the code to do this was very similar to the HTTP example above; I just needed to use my custom tls.Config when constructing an IMAP client instead of a HTTP client.