This isn’t a particularly researched or necessarily best approach at capturing http traffic. It’s a quick and dirty approach that I was able to use to trace API calls to debug why the response wasn’t what I expected from the request.

First, install tcpdump on the target container. I did so by connecting to the live container.

docker exec -it my_app sh
apt install tcpdump

Then, install Tshark on the host. You could probably use WireShark but I was running this on a remote shell already so terminal-based was easier.

apt install tshark

Lastly, pipe tcpdump from the container to t-shark on the host. Keep that shell running while you invoke your app and you can observe the payload data of any API calls that the app makes. The data is in Base64 and can be easily unpacked.

docker exec -it my_app -- tcpdump -i eth0 -w - | tshark -T fields -e tcp.payload -r -

It’s important to note that it will still be encrypted if HTTPS is being used. I tweaked my app to hit http://example.com for regular plaintext HTTP. I didn’t care about the (404) response here, I just wanted to see the request. This is insecure and if you need to debug with truly sensitive data then it’s better to target an internal IP in your network.

So what was my issue?

As for my issue, it turned out that one of the header values was misconfigured with a spurious newline. This was interpreted as end of HTTP headers as per spec, effectively mangling the rest of the request. Always check your values for whitespaces and control characters.

A shout-out to Tim Downey for his article on capturing network traffic. It gave me a huge step in the right direction.