HTML5-powered Ajax file uploads

This post was originally published on the New Bamboo blog, before New Bamboo joined thoughtbot in London.

This information is out of date. See the followup article on FormData.

Introduction

File uploads have traditionally had very bad usability on the web. The standard solution was uploading files as part of a form, leaving the user to just wait until the process was done. We could offer barely any feedback of what was going on.

Several options appeared to make the process more bearable for the user. Some alternatives were client-based, such as using some Flash-powered element like SWFUpload. Other alternatives laid more on the side of the server, like leveraging NGINX’s mod_uploadprogress with a pinch of Ajax. However, there was still the question of why there was no solution that avoided proprietary technology, required minimal hassle OR was free of far-fetched hacks.

Until now.

An important part of Panda, our video encoding service, is file uploads. Our users need to upload large videos and we offer an HTTP interface and a Javascript widget to do exactly that. However, our widget is based in Flash, and we would always rather offer solutions based on webstandards. Therefore, we set out to see how the upcoming HTML5 specification could help us. Specifically the features contained in two specs: the File API and XMLHttpRequest Level 2. I’ll explain here how to make effective use of them. If you’re interested in seeing the end result checkout the panda_uploader plugin on Github

Compatibility and detection

At the time of writing, this technique only works in the latest Webkit and Gecko browsers. Therefore you will still need to fall back to other methods if you wish to support Internet Explorer, Opera or others.

The best way to know if the client browser supports this feature (or any other feature for that matter) is object detection. In short, use Javascript to check whether elements of the API exist or not. In this case, the code is the following:

function supportAjaxUploadWithProgress() {
    return supportFileAPI() &amp;&amp; supportAjaxUploadProgressEvents();

    function supportFileAPI() {
        var fi = document.createElement('INPUT');
        fi.type = 'file';
        return 'files' in fi;
    };

    function supportAjaxUploadProgressEvents() {
        var xhr = new XMLHttpRequest();
        return !! (xhr &amp;&amp; ('upload' in xhr) &amp;&amp; ('onprogress' in xhr.upload));
    };
}

If the function supportAjaxUploadWithProgress() returns true, you are good to go. If not, you’ll have to revert to a different upload technique.

Simplest Ajax upload

The frontend code is pretty simple initially. The spec establishes that file input fields have a files property that gives us access to some attributes of the file, like so:

<input id="the-file" name="file" type="file" />

var fileInput = document.getElementById('the-file');
console.log(fileInput.files); // A FileList with all selected files

Note that fileInput.files is a FileList because the HTML5 spec allows for file inputs to select multiple files. Let’s keep it simple though, assuming only one file will be selected. The general case is easy to infer from there.

Once the user selects a file, the list will be filled up with actual file objects:

var file = fileInput.files[0];
console.log(file.fileName); // "my-holiday-photo.jpg"
console.log(file.size); // 1282632
console.log(file.type); // image/jpeg

You can use this file object as an argument to the XMLHttpRequest.send() call, to send the file asynchronously over to the server:

var xhr = new XMLHttpRequest();
xhr.open('POST', '/upload/uri', true);
xhr.send(file); // Simple!

Feedback events

But we still don’t have any feedback on how the upload process is going. Let’s make the example complete with a simple progress indicator. This will show the progress on the debug console:

var fileInput = document.getElementById('the-file');
var file = fileInput.files[0];

var xhr = new XMLHttpRequest();
xhr.upload.addEventListener('progress', onprogressHandler, false);
xhr.open('POST', '/upload/uri', true);
xhr.send(file); // Simple!

function onprogressHandler(evt) {
    var percent = evt.loaded/evt.total*100;
    console.log('Upload progress: ' + percent + '%');
}

Please note that the event is not set on the xhr object itself, but on xhr.upload. If you get this wrong, you’ll be notified of the progress of the response from the server after the request is complete, rather than the upload preceding it.

Additionally, there are some other events that can be useful:

xhr.upload.onloadstart: the upload begins
xhr.upload.onload: the upload ends successfully
xhr.upload.onerror: the upload ends in error
xhr.upload.onabort: the upload has been aborted by the user

And of course we’ll have to use our old friend xhr.onreadystaterequest to read the response from the server.

Reading the raw data

There’s still one problem though, which is not evident from just reading the code above. When a file is uploaded using this method, the browser will send a request whose content will be just the file contents as raw data. This means that your server-side framework won’t be able to make it available to you as per normal.

For example, in PHP normally you would use the $_FILES global variable; in Rails the params method; in Django request.POST dictionary. None of these would work with this technique. It also means you won’t be given the name of the file, which is normally available using the mentioned accessors.

But don’t worry, both things can be solved easily. First, the file name: you can pass it to the server on the very same upload request by putting it in a request header. For example:

xhr.setRequestHeader("X-File-Name", file.name);

If you do this before calling send(), you will be able to read the file name form the headers on the server side. Popular methods include:

PHP: $_SERVER['HTTP_X_FILE_NAME']
Rails: request.env['HTTP_X_FILE_NAME']
Django: request.META['HTTP_X_FILE_NAME']

Now there’s the matter of how to read the file itself. Again, this will depend on your server side technology:

PHP: file_get_contents("php://input")
Rails: request.env['rack.input']
Django: request.raw_post_data

Being a good netizen

Another small detail is that, by default, the request is sent without a mimetype. Depending on your setup, this may not be a problem for you, but it’s recommended that you set it. This is so third party applications can interoperate better with your code. And makes you a good netizen!.

For this, there’s only one line you need to add to the code above:

xhr.setRequestHeader("Content-Type", "application/octet-stream");

Firefox!

Hang on, there’s one last thing! Firefox implements the above from version 3.5, but there are a couple of pitfalls you have to be aware of.

Set events BEFORE opening the XmlHttp connection

Specially the “progress” event; otherwise Firefox won’t fire it. This means that you have to do as follows:

xhr.upload.addEventListener('progress', onprogressHandler, false);
xhr.open('POST', '/upload/uri', true);

Instead of the following, “wrong” code (for Firefox anyway!):

xhr.open('POST', '/upload/uri', true);
xhr.upload.addEventListener('progress', onprogressHandler, false);

On Firefox 3.5, two names change

On Firefox 3.5, instead of xhr.send() you have to do xhr.sendAsBinary(). Also, instead of file.name, the name of the file is stored in file.fileName. Fortunately, Firefox 3.6 adopts the W3C convention and these changes don’t apply.

You can still support both conventions in you code. For the file name do the following:

filename = file.name || file.fileName;

And for the send() call, do is:

if ('getAsBinary' in file) {
  // Firefox 3.5
  xhr.sendAsBinary(file.getAsBinary());
}
else {
  // W3C-blessed interface
  xhr.send(file);
}

A complete example

We have published a complete example of how this works. It’s on GitHub and you are invited to play with it:

Example ajax upload

This uses Sinatra with a piece of Rack middleware that we have also made available separately, called Rack::RawUpload. This middleware will convert a raw upload into a normal request. This way, you won’t need to fiddle around with the file name and its contents.

Sign up to receive a weekly recap from thoughtbot