Charles a Web Debugging Tool, HTTP Monitor, Reverse Proxy tool might turn out to be the magic piece of code I was looking for several times already (including some Apache plugins) for testing web applications. Ok – I got my Axis TCPMON – so what do I need this you ask?

Actually it enables a developer to view all of the HTTP traffic between their machine and the Internet. This includes requests, responses and the HTTP headers (which contain the cookies and caching information) – ok there's tools like Axis TCPMON already out for doing such things.

But Charles can also simulate modem speeds by effectively throttling your bandwidth and introducing latency, so that you can experience an entire website as a modem user might. This is especially useful when doing black-box tests and tuning for larger web applications and java applets (you know that super-slow trading application makes you not wanna buy any stocks because the application just sucks?).

Other nice and handy features include:

• Cookies are shown in the HTTP headers, so you can see exactly what cookies you are sending and receiving.
• Download Stats: Assets loaded from an HTML page are recorded so that you can see how many images etc are loaded by a page, and where from
• Mirror all responses to disk, recording your session (good for archiving your tests)
• Blacklist sites so that requests are blocked.
• See the results of caching by seeing cached responses (304 Not Modified), and requests containing last modified dates (IfModifiedSince).
• Disable caching by removing cache related headers from requests and responses as they pass through Charles, ensuring that you are always requesting the latest file.
• See whether a cache has served your request by looking for cache-hit HTTP headers.
• View the encrypted HTTPS communications.
• Reveal unexpected requests, such as typos and 404s.
• Throttle your web connection to a specified bytes/second speed, and millisecond latency. This enables you to simulate modem conditions on a high speed internet connection
• Spoof DNS name to ip mappings so that you can test a domain name before it has gone live. Very useful for testing your virtual hosting.
• Export to CSV all of the summary data captured by Charles for analysis and reporting in Excel
• Reverse proxy creates ports on the localhost that act as regular HTTP servers, but forward all requests to a specified web server – great stuff – I just have to make up some good ideas about this feature
• SSL debugging enables you to view requests and responses in plain text even when communicating with an SSL secured web server.
• HTTP/1.1 support – understands some HTTP/1.1 specific responses, and forces no keep-alives

Charles is shareware.

Bill pointed me to an open source java HTTP/S proxy that can help you also in your debugging, Muffin has no reverse proxy functionnality nor throttling. The SSL debugging is already in the CVS version – later contributions might add these.