Chunk File Upload Doc

chunk_file_doc.md

File Chunk Upload

To support large file upload we need to segregate the file to different chunks and we will be sending each chunk in a single request to the server. Once they recieve the complete chunk, we will combine the chunk to get the actual file.

Following are the steps we followed to implement the feature.

• Since we are using jquery already, we make use of Jquery Chunk upload library (https://github.com/blueimp/jQuery-File-Upload/wiki/Chunked-file-uploads) which is already being supported byHyraxUploader.

  Following are the changes we did in the front end. we provide option to enable the chunk upload to the hyraxUploader function

  # app/assets/javascripts/hyku/groups/uploader.js
  
  Blacklight.onLoad(function() {
    var options = {maxFileSize: 10737418240, maxChunkSize: 10000000, maxRetries: 7, retryTimeout: 5000};
    $('#fileupload').hyraxUploader(options);
    $('#fileupload').bind('fileuploadfail', function (e, data) {
      var filename = data.originalFiles[0].name;
      $.ajax({
        type: 'GET',
        dataType: 'JSON',
        url: '/fail_uploads/delete_file',
        data:{
          file_upload: {
            filename: filename
          }
        }
      });
    });
  });

  We are also adding a call back to delete the file incase when the chunk file upload got disconnected between the process.

• Secondly, we need to create our own controller to support chunk upload and combine the file once the file is uploaded completely/

    # Overriding the UploadsController from Hyrax.
    def create
      # Replace all the non characters to underscore except - and  .
      file_name = params[:files].first.original_filename.gsub(/[^a-zA-Z0-9\-.]/, '_')
      @upload = Hyrax::UploadedFile.where(file: file_name, user_id: current_user.id, file_status: 0).first
      if @upload.nil?
        @upload = Hyrax::UploadedFile.create(file: params[:files].first, user_id: current_user.id)
      elsif @upload.file_status.zero?
        path = @upload.file.path
        File.open(path, "ab") { |f| f.write(params[:files].first.read) }
      end
    end

  Once we receive the first chunk of the file in the controller, we will be creating a record in the table with the file_name, status and the current_user fields and the chunk file will be saved to a path. In the next request we check if the chunk is from the same file and based on that we append it with the previous record that we have created.