Using tcpdump to see HTTP requests and responses

While developing web-applications (and also non-web applications that use HTTP as universally understood protocol for inter-program communication), I often need to see what’s really happening on the wire.

There are many tools to do this. Modern browsers have either built-in functionality or some extentions to show in a nice interface what browser sends and what server gives back. But I am old-fashioned. And not always I have a browser that submits requests, that could be another application. So I use tcpdump, which is always close at hand (yes, I am not developing on Windows).

I can’t say it’s easy to use, but it’s not difficult either. You don’t have to understand whole man page, just some bits from it. In short, use

$ sudo tcpdump -s 0 -A port 80

Explanations and other options follow.

To see what’s going on in the world of HTTP, you usually need to watch port 80, so use port 80 expression. It happened to me couple of times already that I forgot that I’m debugging my application, that runs in some container on port 8080 instead. That would mean port 8080. This form would reveal both requests and responses. If you only need requests, make it dst port 80. Of course, there is also src port 80 form of it. If your client and server are on different hosts, you might achieve different results depending on which host you run tcpdump, and you could add src host x.x.x.x or dst host x.x.x.x expressions to limit what you see.

HTTP is a text-based protocol, so I would rather see characters than digits. For that, use -A flag. Sometimes I want to se both, so I use -X instead. And, by default, tcpdump will not show you full packets, so use -s 0 to disable that.

Lastly, sometimes tcpdump will bind to a wrong interface, so you need to specify correct one with -i flag. Typically that would be one of ethX on Linux, or enY on Mac OS X. Consult output of /sbin/ifconfig.

And, even after lastly, but that should have been on front, you need to have super-user priviledge  to use tcpdump. In other words, use sudo tcpdump, or become root user (I didn’t suggest that).

Now some examples. On my Mac I need to specify interface to bind to, so that’s en0 if I’m on a wire:

$ sudo tcpdump -s0 -A -i en0 port 80

or en1 on AirPort, or gif0 if I use VPN to another network. If you’re testing server on the same box, you should use lo0 (and maybe different port as well, like 8080).

To debug strange characters, charset or line end problems, I use -X instead of -A, and here I’m only looking at requests (don’t forget correct -i flag for your situation):

$ sudo tcpdump -s0 -X dst port 80

Leave it runnig for a while, and you’ll see how many http connections you’ve never thought about your box does! To limit output to show conversations with only one server, do this (remember to add correct -i if needed):

$ sudo tcpdump -s0 -A host X.X.X.X and port 80

You could use host name instead of IP address:

$ sudo tcpdump -s0 -A host wordpress.com and port 80

and if you do that while typing in your blog entry, you might even get a link to apply for a new job :)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s