Uploading Images via APIs in the Open Event Server

APIs help us to send and receive data in some particular data format that can then be used individually or integrated with a frontend UI. In our case, the entire API server is used to manage all the requests from the frontend and send back the necessary response. Usually, the application is to send simple form data which is then stored into the backend database and a valid jsonapi response is shown. However other than normal text, url, datetime data one very important data is media files, in our case event images, user images, slides, etc. In this blog, we will particularly deal with how we can upload images in the server using API.

Sending Data

Firstly, we need to decide how do we send the data in the post request of the API. So we are sending a base64 encoded string representing the image along with the image extension appended to it, for example, data:image/png;base64,iVBORw0KGgoAAAANS. This is a widely used format for showing images over the web. So when we send a POST request we send a json encoded body like:

{
    "data": "data:image/png;base64,iVBORw0KGgoAAAANS"
}

Converting Base64 Data to Image

There are 2 parts of the data in the string that we receive. The first part basically tells us the format of the image(.gif in this case) and string encoding(base64 in this case), the second part gives us the encoded string. So, from the first part, we extract the file extension for the image file to be created. We use uuid.uuid4() for a random filename.

filename = '{}.{}'.format(str(uuid.uuid4()).data.split(";")[0].split("/")[1])

Now to write the base64 encoded string as an image file, we first need to get only the encoded string part from the data and then decode it. We use string decode function of python for the decoding purpose. Then we write the data to the image file and save it.

file.write(data.split(",")[1].decode("base64")

API Response

Finally using whatever logic you are using for serving your static files, you generate the static file path for the image saved. And then create another json encoded response which returns you the url for the saved image in the server.

{
    "url": "https://xyz.storage.com/asd/fgh/hjk/1233456.png"
}

And there you have your image uploaded and served.

File upload progress in a Node app using Socket.io

If you look at the webapp generator, you’ll see that there is an option to upload a zip file containing event data. We wanted to give visual cue to the user when he is uploading to see how much file has uploaded.

We are uploading the file, and giving the generate start command via socket.io events instead of POST requests here.

To observe file upload progress on socket (when sending file using a Buffer), there is an awesome node module available called socketio-upload-progress.

In our webapp you can see we implemented it on the frontend here in the form.js and here in the backend in app.js

Basically on the backend you should add the socketio-file-upload module as a middleware to express

var siofu = require("socketio-file-upload");
var app = express()
    .use(siofu.router)
    .listen(8000);

After a socket is opened, set up the upload directory and start listening for uploads

io.on("connection", function(socket){
    var uploader = new siofu();
    uploader.dir = "/path/to/save/uploads";
    uploader.listen(socket);
});

On the frontend, we’ll listen for an input change on an file input type element whose id is siofu_upload

var socket = io.connect();
var uploader = new SocketIOFileUpload(socket);
uploader.listenOnInput(document.getElementById("siofu_input"));

One thing to note here is that, if you observe percentage of upload on frontend, it’ll give you false values. The correct values of how much data is actually transferred can be found in the backend. So observe progress in backend, and send percentage to frontend using the same socket.

  uploader.on('progress', function(event) {
    console.log(event.file.bytesLoaded / event.file.size)
    socket.emit('upload.progress', {
      percentage:(event.file.bytesLoaded / event.file.size) * 100
    })
});

 

Handling file uploading in HTML5 and Express

The Open Event Webapp Generator has a pure HTML front end form, and Express based backend.

 

We needed an option where users can upload their own JSON files to create a schedule page for it.

On the HTML form that’s easy. As you can see here we simple add input tags with type=”file”


<input type="file" name="sponsorfile" id="sponsorfile">

In our express app, we need to use multer to be able to handle file uploads.

We create a middleware called uploadedFiles, and pass the middleware to the get() block

 

var express = require('express');
var multer = require('multer');
var app = express();

var upload = multer({dest: 'uploads/'});

var uploadedFiles = upload.fields([
  {name: 'speakerfile', maxCount: 1},
  {name: 'sessionfile', maxCount: 1},
  {name: 'trackfile', maxCount: 1},
  {name: 'sponsorfile', maxCount: 1},
  {name: 'eventfile', maxCount: 1},
  {name: 'locationfile', maxCount: 1}
]);

app.post('/live', uploadedFiles, function(req, res) {
         // req.files has the files
         // req.body has the POST body params
  });
});

Now we can access the files inside req.files (that will have a path to the temporary location of the file, you’ll have to use some filesystem module to read the files)