HttpClient Cookie Path Validation

I noticed some unusual behavior while using HttpClient 4 (version is important since it's a complete rewrite/namespace change from version 3) to log into a website.  The login was failing and I saw the following message in the log (I've changed names to protect the innocent).

2011-02-25 16:41:27,154 [pool-1-thread-2] WARN  org.apache.http.client.protocol.ResponseProcessCookies[126] - Cookie rejected: "[version: 0][name: tracker][value: +][domain: .rawr.com][path: /splash][expiry: Thu Feb 25 16:41:27 MST 2010]". Illegal path attribute "/splash". Path of origin: "/login.php"

In this case the server is setting a cookie with an invalid path and HttpClient is properly rejecting the cookie.  According to the RFC spec, a server must set the cookie path using the URL path, or an ancestor path.  For example, when accessing http://foo.com/a/b/, the server may set a cookie with path /a/b/, /a, or /, but not /a/c.  But since most all browsers (I only checked Firefox) ignore the spec and accept the cookie, I need HttpClient to do the same.

It's important to note that I'm using the browser compatibility cookie policy, but apparently it is missing some compatibility:

client.getParams().setParameter(ClientPNames.COOKIE_POLICY,  CookiePolicy.BROWSER_COMPATIBILITY);

Diving into the HttpClient code reveals that the cookie is being rejected by BasicPathHandler.java.  This class is registered by BrowserCompatSpec.java (the implementation for CookiePolicy.BROWSER_COMPATIBILITY).  In order to accept the cookie, it was necessary to subclass BrowserCompatSpec.java and and overwrite the existing path handler with an implementation that performs no validation:

HtmlUnitBrowserCompatCookieSpec() {
     super();
     
     final BasicPathHandler pathHandler = new BasicPathHandler() {
      @Override
      public void validate(final Cookie cookie, final CookieOrigin origin) throws MalformedCookieException {
       // nothing, browsers seem not to perform any validation
      }
     };
     registerAttribHandler(ClientCookie.PATH_ATTR, pathHandler);
    }

Credit goes to HtmlUnit for the solution.

Retries with the Spring HTTP Invoker

I've been doing work on a project that uses Spring Remoting with the HTTP Invoker.  It makes remoting a piece of cake, and since it's HTTP, it's almost trivial to introduce a load balancer for HA/Scalability.  I didn't realize that it was supporting retries until I saw this in the logs:

2011-02-07 19:59:37,541 [http-8080-2] INFO  org.apache.commons.httpclient.HttpMethodDirector[439] - I/O exception (java.net.ConnectException) caught when processing request: Connection refused
2011-02-07 19:59:37,541 [http-8080-2] INFO  org.apache.commons.httpclient.HttpMethodDirector[445] - Retrying request

I was a bit surprised to discover this, since it is not documented, and I was about to implement my own retry logic.  The retry support actually has nothing to do with Spring Remoting, as it is built into HttpClient, which is used by the CommonsHttpInvokerRequestExecutor.  By default, the HttpClient code will attempt up to three retries. The retry code is on line 433 of HttpMethodDirector.java (v 1.34 2005/01/14 19:40:39 olegk). The code is smart about what type of exceptions it will retry. The most typical exception that would get retried is an IOException (e.g. java.net.ConnectException). The retry code does not involve any delays, which if it did would be a big reason to disable it. But the code can block up to the connect timeout, under certain conditions, for example if the server is under heavy load.  This of course translates to a potential 3*timeout delay.  If you would like to suppress retries, or change the retry value, here's how to do it. Note, you will need to create the invoker programmatically to do this.

HttpInvokerProxyFactoryBean fb = new HttpInvokerProxyFactoryBean();

CommonsHttpInvokerRequestExecutor client = new CommonsHttpInvokerRequestExecutor();

client.getHttpClient().getParams().getDefaults().setParameter(HttpMethodParams.RETRY_HANDLER, new  DefaultHttpMethodRetryHandler(0, false));

fb.setHttpInvokerRequestExecutor(client);

The first argument to DefaultHttpMethodRetryHandler is the retry value.  You could also configure the invoker in the Spring config, and then get a reference via the Spring context.  Alternatively, you can use SimpleHttpInvokerRequestExecutor instead of CommonsHttpInvokerRequestExecutor. The former uses the java.net.HttpURLConnection, with no retry support, while the latter uses Apache HttpClient.

This is a good time to set your connection and read timeouts

client.getHttpConnectionManager().getParams().setConnectionTimeout(10000);
client.setReadTimeout(20000);

There, that's it. I hope. Note: I didn't exactly test this, since retries seem to be a good thing for me, however, it seems like it should work.