Issue Details (XML | Word | Printable)

Key: LP-1
Type: Bug
Status: Closed
Resolution: Fixed
Priority: Blocker
Assignee: Adam Fisk
Reporter: Adam Fisk
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
LittleProxy

FireFox, others can't decode chunked transfer encoding from proxy

Created: 24/Oct/09 09:27 PM   Updated: 07/Nov/09 05:18 PM   Resolved: 07/Nov/09 05:18 PM
Component/s: None
Affects Version/s: None
Fix Version/s: 0.2

Time Tracking:
Not Specified


 Description  « Hide
When the proxy receives responses from external servers over a certain size, it has to break those responses apart to avoid holding the entire response body in memory. The proxy switches to use Transfer-Encoding: chunked. While the chunked encoding from the proxy appears to be handled perfectly, FireFox and likely other browser cannot read it successfully. The encoding works on Chrome, however.

This currently applies to files over 81,920 bytes. Interestingly, Flash also appears to handle the encoding fine, as we have yet to see it affect videos.

To reproduce this, perform the following:

1) Start LittleProxy (should be documented on the Wiki)
2) Configure FireFox to point to the proxy IP address and port -- use 127.0.0.1/8080
3) Open a browser page to any file over the above mentioned size, for example: http://i.i.com.com/cnwk.1d/i/bto/20091023/sandberg.jpg

You should see that with the proxy, the image doesn't appear. If you reconfigure FF to not use the proxy, the image should appear.

Note this was tested with FireFox 3.0.14 on OSX.

Adam Fisk added a comment - 01/Nov/09 06:17 PM
I've added a test for this case at:

src/test/java/org/littleshoot/proxy/HttpProxyTest.java

The test downloads the above file: http://i.i.com.com/cnwk.1d/i/bto/20091023/sandberg.jpg both directly and through the proxy. In the proxy case, it decodes the chunked Transfer-Encoding the proxy adds, and it then verifies the resulting message body is identical to the message body in the direct download, non-chunked case. It is.

So, at least according to the test, Netty chunked encoding is fine. The question is, what's different about the way the test is decoding the chunked encoding and the way FireFox is decoding it?

Adam Fisk added a comment - 05/Nov/09 06:40 PM
Stephen Haberman on the Netty user list took the time to figure out the issue here:

Netty correctly adds the Transfer-Encoding: chunked header in the response. It also correctly performs the chunked encoding itself. Chunked encoding, however, is only supported in HTTP 1.1. If the response was a HTTP 1.0 response, the browser wouldn't even look for chunked encoding, and would just choke when reading the response body.

I submitted a patch to Netty to fix the problem there. I also fixed the LittleProxy code to solve it in the case where Netty gives the wrong response (as it always will until the patch is accepted and released).

Thanks again Stephen!