File upload with HttpComponents Client 4.0 (successor of Commons HttpClient 3.x)

Background: I’m working on an application that should be able to integrate with various issue trackers (JIRA, Bugzilla, etc.) First I tried using the latest stable branch of Apache Commons HttpClient 3.1 but soon faced problems as JIRA (running on Tomcat) was not able to extract all parameters I was sending with “multipart/form-data” encoding. I’ve found similar problems reports on the Internet but no solution. So, I’ve decided to try the latest HttpComponents Client(4.0-beta2, while core library was 4.0-beta3 at the time of writing) which came with new problems but my form submission and file upload worked at the end. I summarize my experience here.

As soon as I downloaded HttpComponents Client I realized why they changed the name from Commons HttpClient. This is a completely new API! If you already have code that relies on Commons HttpClient you can not easily switch jar file and make some minor changes. Fortunatelly, HttpComponents use different packages so you should be able to keep your HttpClient code and have both versions coexist in the same application without conflicts. However, I can’t resist to mention that HttpComponents API looks to me a bit over-engineered and so low level that it hurts. (But maybe it’s just me being spoiled after a month on Groovy.)

HttpComponents Client still doesn’t have an appropriate documentation and I could not find a single example of file upload that worked for me. So, after mingling my existing code and code from HttpComponents test cases, I’ve finally made a first version that was able to upload file and here’s somewhat simplified version of it:

public void testUpload() throws Exception {
 HttpClient httpclient = new DefaultHttpClient();
 HttpPost httppost = new HttpPost(myUploadUrl);

 MultipartEntity reqEntity = new MultipartEntity(
  HttpMultipartMode.BROWSER_COMPATIBLE);

 reqEntity.addPart("string_field",
  new StringBody("field value"));

 FileBody bin = new FileBody(
  new File("/foo/bar/test.png"));
 reqEntity.addPart("attachment_field", bin );

 httppost.setEntity(reqEntity);

 System.out.println("executing request " + httppost.getRequestLine());
 HttpResponse response = httpclient.execute(httppost);
 HttpEntity resEntity = response.getEntity();

 if (resEntity != null) {
  String page = EntityUtils.toString(resEntity);
  System.out.println("PAGE :" + page);
 }
}

I omit package imports in the example above but I’m sure you’ll manage without it. Notice use ofHttpMultipartMode.BROWSER_COMPATIBLE parameter in line 6. It was crucial for me as upload didn’t work when I used default constructor for MultipartEntity (that initializes entity for the strict multipart mode). With hope that code above is all what’s needed, I made a small change (replacing lines 11-13 above) to upload image from byte array instead from file:

reqEntity.addPart("attachment_field",
  new InputStreamBody(
   new ByteArrayInputStream(imageBytes),
   "image/png", "test.png"));

Surprisingly, this change broke my upload! Again, documentation and googling on this topic didn’t give me any clue on what could be wrong. So, I plunged into HttpComponents source code and found that the major different between the InputStreamBody and FileBody was that methodgetContentLength() of the former was returning -1 (which is somewhat logical but again…). As I already had image as a byte array, content length was known to me. So, I extended InputStreamBody class as follows:

class InputStreamKnownSizeBody extends InputStreamBody {
 private int lenght;

 public InputStreamKnownSizeBody(
   final InputStream in, final int lenght,
   final String mimeType, final String filename) {
  super(in, mimeType, filename);
  this.lenght = lenght;
 }

 @Override
 public long getContentLength() {
  return this.lenght;
 }
}

Now I could use my InputStreamKnownSizeBody class to upload any in-memory file:

reqEntity.addPart("attachment_field",
  new InputStreamKnownSizeBody(
   new ByteArrayInputStream(imageBytes),
   imageBytes.length, "image/png", "test.png"));

At the end, although I’m not delighted with complexities of HttpComponents, it worked for me at the end and I resolved file upload issues I experienced with Commons HttpClient. I hope this post will be of help to someone.

At least few people were struggling with requirement for the MIME library (see bellow discussion). I’ve personally switched to the 4.1 Alpha 2 (the latest version at the time of writing) which removes dependency on org.apache.james.mime4j and comes with ownhttpmime-4.1-xxx.jar library.