Uploading large files with AJAX and jQuery

In classic web forms where the entire form is submitted to its action target and the entire page is refreshed, files can be easily included as part of the form payload when the method is POST and the encoding type is multipart/form-data.

Classic forms are fine but often with modern web-apps it is preferable to submit form data using AJAX. HTML5 offers some new interfaces to assist with this – FormData and FileReader.

A recap on small file uploads

I’ve blogged about FormData and how to use it for AJAX form submissions. Using FormData to upload files can be described as a classic form submission but with AJAX – the payload data and sequence of events is very similar. On AJAX call the file is read from the system and sent as part of the payload. The data cannot be viewed before the file(s) are sent to the server and if the file(s) are too big you won’t know until the server tells you, if you don’t have a timeout.

Also read Uploading files with AJAX and jQuery using FormData API, which covers a effortless way of converting a classic form submit into an AJAX form submit.

FileReader allows for splitting huge files into chunks and uploading them sequentially

FileReader interface – think big

Now I want to talk about the FileReader interface and its strengths.

FileReader gives you the benefit of accessing the full file data before it is submitted. It is possible to parse the data for format integrity, and if the files are huge, they can be split and sent in a series of submissions and finally assembled on the server. Verify that your targeted browsers support FileReader but at this point support is widely available and Internet Explorer is deprecated.

Use of FileReader requires a bit of coding. First, how to read file data. FileReader has a few methods but for the scope of this article I’m going to highlight FileReader.readAsDataURL() which reads the data into a Data URI of format data:*/*;base64,Base64EncodedString

The other half is handling the data once it is read. To do this, we must listen for the load event by attaching an event handler to FileReader.onLoad. Following is a small concept example.

var reader = new FileReader();

reader.addEventListener('load', function(e) {
  // e.target.result contains the encoded data payload
});

reader.readAsDataURL(file);

One more thing to discuss before a full example is the file itself. FileReader doesn’t have direct access to the local system – its methods receive File objects. For the scope of this article I’m going to highlight extracting a File object from the HTML file input element. An example follows.

<input type="file" id="file-input"/>

// fileInput is a File object when a file has been selected
fileInput = document.getElementById('file-input').files[0];

Putting it all together

Now that we’ve seen all of the pieces involved in putting FileReader together, I’ll show a full example of how to split a large file into chunks and upload. I will not show the server-side as many strategies could be used for recombining the data.

<form id="my-form">
  <input type="file" id="file-input"/>
</form>

var chunk_size = 1024 * 1024; // 1mb
var reader = new FileReader();

jQuery('#my-form').submit(function(e) {
  // we handle the form submission ourself
  e.preventDefault();

  // acquire File object from file input field
  fileInput = document
    .getElementById('file-input')
    .files[0];

  // begin upload process
  uploadFile(fileInput);
});

function uploadFile(fileInput) {
  _uploadChunk(fileInput, 0, chunk_size);  
}

function _uploadChunk(file, offset, range) {
  // if no more chunks, send EOF
  if(offset >= file.size) {
    jQuery.post('http://example.com/action', {
      filename: file.name,
      eof: true
    });
    return;
  }

  // prepare reader with an event listener
  reader.addEventListener('load', function(e) {
    var filename = file.name;
    var index = offset / chunk_size;
    var data = e.target.result.split(';base64,')[1];

    // build payload with indexed chunk to be sent
    var payload = {
      filename: filename,
      index: index,
      data: data,
    };

    // send payload, and buffer next chunk to be uploaded
    jQuery.post('http://example.com/action',
      payload,
      function() {
        _uploadChunk(file, offset + range, chunk_size);
      }
    );
  }, {once: true} ); // register as a once handler!

  // chunk and read file data
  var chunk = file.slice(offset, offset + range);
  reader.readAsDataURL(chunk);
}

How I proxied a file upload through a Next JS middleware