Handling Zip Upload in Open Event Webapp

In Open Event Webapp, we use Socket.IO library for real-time communication between client and the server. To be able to generate a site for an event, the user has to first upload the file to the generator. There are a lot of node libraries like multer and formidable which exists for this purpose. But they don’t support display of real-time stats regarding the progress of the uploaded file. To solve this issue, the project used socket-io-file-upload library which uploads the file to the Node.js server using Socket.IO and show live percentage denoting how much of the zip has been uploaded to the server.

It was working quite well until we discovered a major problem with the library. It didn’t support canceling the upload. If we clicked on the cancel button, it stopped showing the progress on the front end but actually continued to upload the file to the server on the back end. We had only two choices: Either to refresh the page or just wait for the existing zip file to upload completely. The former is a really bad solution and the latter result in wastage of time and bandwidth.

On investigating the issue, the problem was with how the library is designed. This was not a fault in our code or the library either. It was just that the library didn’t support this functionality.  After thoroughly searching for the solutions, we came across this library named socket-io-stream which met our requirements. It supports bidirectional binary data transfer with Stream API through Socket.IO. We can also cancel the upload in between.

For streaming between server and client, we will send stream instances first. To receive streams, we just wrap socket with socket-io-stream and then listen for any events as usual.

Client Side:

function uploadFile(file) {
  // Calculating size of the file and creating a stream to be sent to the server

  var size = (file.size/(1024*1024)).toString().substring(0, 3);
  var stream = ss.createStream();

  /* Creating a read stream and initializing variable to keep track of the size of the file uploaded so far */

  var blobStream = ss.createBlobReadStream(file);
  var fileUploadSize = 0;

  // Tracking upload progress and updating the variables

  blobStream.on('data', function(chunk) {
    fileUploadSize += chunk.length;
    var percent = (fileUploadSize / file.size * 100);

    /* isCancelling is a boolean which stores the status of the zip upload.That is, whether it is live or canceled */

      if (isCancelling) {
           var msg = 'File upload was cancelled by user';
           stream.destroy();
           socket.emit('cancelled', msg);
           isCancelling = false;
           console.log(msg);
       }

       updateUploadProgress(Math.floor(percent));
       if (percent === 100) {
           uploadFinished = true;
           enableGenerateButton(true);
       }
     });

  // Sending the stream to the server and transferring read data to it

   ss(socket).emit('file', stream, {size: file.size, name: file.name});
   blobStream.pipe(stream);
}

Server Side:

// Importing the library
const ss = require('socket.io-stream');

// Listening to the stream sent from the client
ss(socket).on('file', function(stream, file) {
  generator.startZipUpload(socket.connId);

  // Declare a filename and write the incoming data to it
  var filename = path.join(__dirname, '..', 'uploads/connection-' + id.toString()) + '/upload.zip';

  stream.pipe(fs.createWriteStream(filename));
});

That’s all we need to do to stream data from the client to the server. Head over here for more information regarding the library.

Diving into the codebase of the Open Event Front-end Project

This post aims to help any new contributor to get acquainted with the code base of the Open Event Front-end project.

The open event front-end is primarily the front-end for the Open Event API server. The project provides the functionality of signing up, add, update and view events details and many other functions to the organisers, speakers and attendees of the event which can be concert, conference, summit or a meetup. The open event front-end project is built on a JavaScript web application framework Ember.js. Ember uses a library Ember data for managing the data in the application and for communicating with server API via endpoints. Ember is a battery included framework which means that it creates all the boilerplate code required to set up the project and create a working web application which can be modified according to our needs.

The open event front-end is primarily the front-end for the Open Event API server. The project provides the functionality of signing up, add, update and view events details and many other functions to the organisers, speakers and attendees of the event which can be concert, conference, summit or a meetup. The open event front-end project is built on a JavaScript web application framework Ember.js. Ember uses a library Ember data for managing the data in the application and for communicating with server API via endpoints. Ember is a battery included framework which means that it creates all the boilerplate code required to set up the project and create a working web application which can be modified according to our needs.

For example: When I created a new project using the command:

$ ember new open-event-frontend

It created a new project open-event-frontend with all the boilerplate code required to set up the project which can be seen below.

create .editorconfig
create .ember-cli
create .eslintrc.js
create .travis.yml
create .watchmanconfig
create README.md
create app/app.js
create app/components/.gitkeep
Installing app
create app/controllers/.gitkeep
create app/helpers/.gitkeep
create app/index.html
create app/models/.gitkeep
create app/resolver.js
create app/router.js
create app/routes/.gitkeep
create app/styles/app.css
create app/templates/application.hbs
create app/templates/components/.gitkeep
create config/environment.js
create config/targets.js
create ember-cli-build.js
create .gitignore
create package.json
create public/crossdomain.xml
create public/robots.txt
create testem.js
create tests/.eslintrc.js
create tests/helpers/destroy-app.js
create tests/helpers/module-for-acceptance.js
create tests/helpers/resolver.js
create tests/helpers/start-app.js
create tests/index.html
create tests/integration/.gitkeep
create tests/test-helper.js
create tests/unit/.gitkeep
create vendor/.gitkeep
NPM: Installed dependencies
Successfully initialized git.

Now if we go inside the project directory we can see the following files and folders have been generated by the Ember-CLI which is nothing but a toolkit to create, develop and build ember application. Ember has a runtime resolver which automatically resolves the code if it’s placed at the conventional location which ember knows.

➜ ~ cd open-event-frontend
➜ open-event-frontend git:(master) ls
app  config  ember-cli-build.js  node_modules  package.json  public  README.md  testem.js  tests  vendor

What do these files and folders contain and what is their role in reference to the project, “Open event front-end”?

Directory structure of open event frontend project

Fig 1: Directory structure of the Open Event Front-end project

Let’s take a look at the folders and files Ember CLI generates.

App: This is the heart of the project. It is responsible for deciding how our application will look and work. This is the place where folders and files for models, components, routes, templates and styles are stored. The majority of our application’s code is written in this folder. The adapter directory contains the application.js file which is responsible for authorising all outgoing API requests. Since our application is an event application so it has various screens which display the list of events, sessions, schedule, speakers, functionality to add, delete, modify the event details etc. Most of the screens have some features in common which can be reused everywhere inside the application which is kept in components directory. The controllers directory contains the files which will be responsible for the front-end behaviour which is not related to the models of our application. For example actions like sharing the event details. The data of the event is fetched by API but when the share functionality has to work is decided by the controller. Models directory helps in mapping the format of data i.e. in which format the data is fetched from the API. As we interact with the application, it moves through many different states/screens using different URLs. The routes directory decides which template will be rendered and which model will be loaded to display the data in the current template. The services directory contains the services which kept on running in the application like authentication service to check whether the current user has logged in or not, translation service to translate the details into some other language other than English which is supported by the app if required. The styles directory contains .scss files for styling of layouts. The template directory contains the layouts for the user interface of the application in form of Handlebar Templates which contains the HTML code which will be rendered on the screen when the template will be called via routes. The utils directory contains all the utility files like the format of date and time, demographic details, event type, type of topics etc.   

Directory structure of App folder

Fig 2: Directory structure of the app folder

Config: It contains files where we can we configure the settings of our application. The environment.js file contains the details like application name and API endpoints and other details required to configure the application. The deploy.js file contains the configurational details about the deployment.

Directory structure of config folder

Fig 3: Directory structure of config folder

Mirage: It contains the files which are required to mock the API server. Since we need to fetch the data from the server so whenever we make a request to get and post the data, mirage will respond to that data. It also contains the file which helps in seeding fake data for the application. Here mocking means creating objects that simulate the behaviour of real objects. The default.js application handles the seeding the fake data but currently, we don’t require that hence it’s empty. The config.js file has the method which sets the API host and namespace of the application.

Directory structure of mirage folder

Fig 4: Directory structure of mirage folder

Node_modules: As we know that Ember is built with Node and uses various node.js modules for functioning. So this directory contains all the files generated from the running the npm package manager which we run at starting while installing and setting up the project on the local device. The package.json file contains all the file maintains the list of npm dependencies for our app.

Public: This directory contains all the assets such as images and fonts related to the application. In the case of our application, to add images apart from the ones that are fetched from the API, they have to be placed here.

Directory structure of public folder

Fig 5: Directory structure of public folder

Translations: Since our application provides various languages support so this directory contains the files to translate the messages.

Directory structure of translations folder

Fig 6: Directory structure of translations folder

Tests: This is the best feature which Ember has provided. We don’t have to generate test they are automatically generated by the Ember-CLI which can be modified according to our requirements. Ember generates three types of test which are Acceptance, Integration and Unit tests. Other than that Ember offers testing for helpers, controllers, routes and models. Test runner of Ember-CLI testem is configured in testem.js. All three types of test have been used in our application.

Directory structure of test folder

Fig 7: Directory structure of test folder

ember-cli-build.js: This file contains the detail how Ember CLI should build our app. In the case of our project, it contains all the metadata such as app imports and other details to build the app.   

Vendor: This directory usually contains all the front-end dependencies (such as JavaScript or CSS) which are not managed by Bower go (which is a package manager for managing components that contain HTML, CSS, JavaScript, fonts). But in the case of our project, we have only kept .gitkeep inside this since we have not included any dependency which Bower doesn’t maintain.

To summarise, this post explains about the content and the role of different files and folders in the directory structure of the Open Event Front-end Project which may help a new contributor to get started by getting a basic understanding of the project.

Creating functional components in Ember JS in Open Event Front-end

In the Open Event Front-end using Ember JS we are creating a functional ticket ordering component which lets users select the amount of tickets and update the total & subtotal price accordingly. How are we doing it?

Creating a component

Components in ember are modular widgets which are used to define a single HTML element, as an independent element. Components can be reused without any dependencies across the project. This provides us a modular architecture to define a component once and reuse it across the project.

For creating a new component use:

ember g component ticket-component

In ember all components must have at least one hyphen in their name. This is an Ember convention, but it is an important one as it ensures there are no naming collisions with future HTML elements. After execution of the above command three new files are generated in the project under:

  • app/components: Contains the logic of the component.
  • app/templates: Contains the view of the component.
  • tests/integration/components: Contains all tests related to component.

Passing an array of ticket detail to template

We can pass data from APIs to the templates using ember-data, but for simplicity we are passing an array of dummy data to get it working. To pass the array to the template we need to add the following JSON to the application controller under controllers/application.js in the application.

tickets:[
  {
    description: 'Discounted ticket for all community members',
    date: 'Mon, May 22',
    price: 40.50,
    name: 'Community Ticket',
    type: 'paid',
    id: 1,
    quantity: 10,
    orderQuantity: 0,
    min: 0,
    max: 5
  }
]

For passing the data to the component template we need to pass the above object to the template using the following syntax in the application.hbs route.

{{public/ticket-component tickets=tickets}}

Rendering ticket details

We iterate over the tickets array passed by application.hbs using each handlebars helper, and we are also using semantic ui dropdown.

For computing subtotals we are using a ember-math-helpers which can be installed using

ember install ember-math-helpers

The semantic ui dropdown is used for selecting the number of orders. This allows us to bind a hidden field for storing the selected value from the dropdown.

{{#ui-dropdown class='compact selection' forceSelection=false}}
 {{input type='hidden' id=(concat ticket.id '_quantity') value=ticket.orderQuantity}}
{{/ui-dropdown}}

For downloading semantic-ui to the project we use:

ember install semantic-ui-ember

Computing subtotal using ember-math-helpers using the mult function which multiplies the price of tickets and the number of order selected from the dropdown.

<td id='{{ticket.id}}_subtotal' class="ui right aligned">
  $ {{number-format (mult ticket.orderQuantity ticket.price)}}
</td>

Computing total price of ordered tickets

To compute total price we are using computed properties which lets us observe properties in the component and compute other properties which are dependent on it. We add a computed property for the orderQuantity property associated with each of the tickets and update the total accordingly.

total: computed('tickets.@each.orderQuantity', function() {
  let sum = 0.0;
  this.get('tickets').forEach(ticket => {
    sum += (ticket.price * ticket.orderQuantity);
  });
  return sum;
})

This ensures an update of the total if any changes are made to the orderQuantity property of any ticket.

 

Recyclerview in Open Event Android: A great upgradation to ListView

Recently, I was fixing a trivial bug in the Open Event Android app, an interesting thing caught my attention. Most of the lists in the app where actually not implemented using the traditional ListView, but instead used something called RecyclerView. The beautiful transitions that make this app the beauty it is, is due to the amazing and easy to code Recycler Views used in it.

A screen shot showcasing the simplicity of the FOSSASIA Open event android app.

 

List View is one among the various views that Android provides. In a nutshell, it is literally a group of generic views in Android. Why then use an entire new and more complex view, if all that it offers is a number of views displayed together? Well, it certainly has certain add-ons to the most generic set of views like Image and TextViews in Android.

I encourage you to think about what different could the ListView be as compared to multiple declarations of TextView. The thing with multiple declarations of a Text or Image View is that they aren’t “scrollable”, that is, if you want numerous Views in a single activity, without List View, you will have to fit them on one screen, else make another activity. So, if you want a scrollable view with same basic view items, List View is the way to go about it in Android.

The next question that may arise is:

How does a Listview work?

Google defines Adapters as “a bridge between an AdapterView and the underlying data for that view”. List Views use Adapters that retrieve data from Arrays, Lists and other data structures and queries, plugs them into our List View and thus, creates a beautiful looking List in no time. In Spite of all this, List View certainly lags in various fields that deal with performance, memory and compactness issues. As a great improvement Recycler View was introduced.

Using Recyclerview you are not restricted to vertical scrollable views. It has the ability to provide horizontal, vertical, grid and staggered layouts among others, with the help of different layout managers:-

recyclerView.setLayoutManager(new StaggeredGridLayoutManager(2,StaggeredGridLayoutManager.VERTICAL));
recyclerView.setLayoutManager(new LinearLayoutManager(RecyclerView_Activity.this,LinearLayoutManager.HORIZONTAL, false));

RecyclerView.ItemDecoration class allows us to add special drawings to our view. Thus, we can add dividers, borders and much more using this class.

public class DividerItemDecoration extends RecyclerView.ItemDecoration {
 
     private Drawable mDivider;
 
     public DividerItemDecoration(Drawable divider) {
         mDivider = divider;
     }

Adding animation used to be a devil of a task in Listview days, however with the release of Recyclerview, it has become a lot more easy. RecyclerView.ItemAnimator class helps in animating items while the adapters are being changed.

Next, we have something related to the performance:-

In a List View, you might have come across the term, ‘Viewholder’, but if you haven’t, that’s okay too. However, in Recyclerview, using View Holders have been made mandatory and for a good reason. The class goes by the name RecyclerView.ViewHolder.

What exactly are View Holders?
The clean dividers and minimal design in the Recyclerview makes up for a subtle UI.

In a broader sense, it is used to provide information about the identity of an itemView and it’s position within a Recyclerview. Without View Holders we have an overhead of calling the function findViewById() for every item in our Recyclerview! This upgradation from the typical findViewById() to View Holders awards us with better and more smooth scrolling through the list.

How does Recyclerview work?

List View is an ancestor to Recyclerview. Recyclerview has everything that List View offers to us (except for a few things like setchoicemode(), which I might take up in my upcoming blog posts), and much more.

A great achievement that Android achieved through Recyclerview has been it’s ability to “recycle” views on the way. In laymen terms, if you scroll through a Recyclerview, the views that go away from the screen during scrolling are the same views that come into the screen. The only difference is that the data in the views leaving the screen is changed to the data of the views entering your screen and so they are presented as new views. In this way, a Recyclerview helps saving memory for sure as compared to List Views.

There are still a lot of other things to learn about Recycler and List Views and you find more resources here.

 

Implementing Google Maps in Open Event ember Front-end

In the deployment of the Open Event Front-end on eventyay.com we want to show an event’s location on a map using Google Maps. Since Eventyay.com requires Google map to be used at several web pages, we can use Google map as an ember component for reusability.

How did we do it? The following walks you through the entire process.

The first step is to: Install g-map addon-

  • ember g-map, which is an addon, is available on ember add-on library which can be found at emberobserver.com. So first import that library using ember cli command
ember install ember-g-map
  • This command adds an ember-g-map module in our node_modules and also updated package.json file of our app now our ember-g-map modules contains the following directories. The following three are required to be highlighted that serves our purpose-
  • Addon/ – this directory contains all the implementation logic which acts as a prototype for the g-map component
  • App/- this directory basically exports a copy of g-map from our Addon/ directory to the web page where it is required. The export includes the logic implement inside the Addon/ directory which can later be imported by the parent app for use.Package.json – this file holds various metadata relevant to the ember-g-map

Generate Component-

  • The next step is to generate a component event-map (we can name our component anything we want but be sure to add a hyphen between the name). The component can be generated with the help of the following command of ember cli-
ember g component public/event-map

This will add three files in our app –

event-map.js located at app/component/public/event-map.js

event-map.hbs located at app/template/component/public/event-map.hbs

event-map-test.js located at tests/integration/component/public/event-map-test.js

Define Component-

Now let us use these files to add a google map component in our app-

  • First of all, we have to tell index.hbs about our new component that we are going to add. So we add our component {{public/event-map}} in our index.hbs file which is located at app/templates/public/index.hbs And we fetch latitude, longitude and location name from model and store in the event object and pass it to our component so that this information is available in the component.
<div class="location">
<h1 id="getting-here">{{t 'Getting Here'}}</h1>
 {{public/event-map event=model.event}}
</div>
  • The event-map.js file will be used to add ui grid class to assign a class to the div of the component. This will alleviate the redundancy regarding the creation of an extra div to enclose the component and then giving it the ui grid class.
import Ember from 'ember';
 
const { Component } = Ember;
 
export default Component.extend({
 classNames: ['ui', 'stackable', 'grid']
});
  • In our event-map.hbs we have with us the latitude, longitude, and locationName available and we are good to use our g-map addon. So in our event-map.hbs we can directly add g-map and g-map-marker but make sure that we pass lat and lang attributes to it. We can use custom options like backgroundColor, draggable etc. but here we have used street view and gestureHandling. gestureHandling is set as “cooperative” so that for smaller screen device,  zooming is possible only with two fingers so that scrolling would become easy.
<div class="eight wide column event-map">
 {{#g-map lat=event.latitude lng=event.longitude 
 zoom=12 gestureHandling='cooperative' 
 streetView='StreetViewPanorama' as |context|}}
 {{g-map-marker context lat=event.latitude lng=event.longitude}}
 {{/g-map}}
</div>
<div class="eight wide column address">
 <h1>{{t 'Address'}}</h1>
 <p>{{event.locationName}}</p>
</div>
  • Next we have to decide the size of aur google map canvas on which our map is to be displayed. We define the size of our canvas in the public-event.scss file which is located under styles/pages/public-event-scss.
.event-map > .g-map {
 height: 100%;
 width: 100%;
}
 
.event-map > .g-map > .g-map-canvas {
 height: 300px;
}
  • Lastly, we have to modify our event-map-test.js file. Here we perform integration test for our component. We create a dummy object with latitude, longitude, and locationName in it and pass it to our component and then check if it renders correctly or not.
import { test } from 'ember-qunit';

import moduleForComponent from 'open-event-frontend/tests/helpers/component-helper';

import hbs from 'htmlbars-inline-precompile';

moduleForComponent('public/event-map', 'Integration | Component | public/event map');

let event = Object.create({ latitude: 37.7833, longitude: -122.4167, locationName: 'Sample event location address' });

test('it renders', function(assert) {

  this.set('event', event);

  this.render(hbs `{{public/event-map event=event}}`);

  assert.equal(this.$('.address p').text(), 'Sample event location address');

});

Now use ember s command in our ember cli and visit localhost:4200

Then open a event at the end we see our Google map integrated.

Use ember test –server command to check if all our tests (integration, acceptance, etc.) are passed.

 

Find out more at – ember-g-map

Quick-filter for filtering of events

With a myriad of events present, it might get difficult for users to search for a specific event. So, giving the user a tool to search according to whatever clue they have to search for an event makes it easier for them. Search tool based on location, date (or day) or event’s name or rather a combination of any of these will answer most of the problems to this issue.

To create such a tool we implement a quick-filter component.

An Ember Component is a view that is completely isolated. Properties accessed in its templates got to the view object and actions are targeted at the view object. There is no access to the surrounding context or outer controller, all contextual information must be passed in.

Now, let’s create a component ‘quick-filter’

This command will create 3 files

  1. Quick-filter.js

A JS file is used mainly to run client side JavaScript code on a webpage. The quick-filter component encapsulates certain snippets of Handlebar templates that can be reused in our code. If we need to customize the behaviour of our component we define a subclass of Ember.Component in the app/components

import Ember from ’ember’;
const { Component } = Ember;
export default Component.extend({
tagName   : ‘footer’
classNames: [‘ui’, ‘action’, ‘input’, ‘fluid’]
});

 

  1. Quick-filter-test.js

This is where we check whether our component is compatible with other components of the system or not. Here, for now, we are just making sure if our component renders or not, by checking the presence of ‘All Dates’.

import { test } from ’ember-qunit’;
import moduleForComponent from ‘open-event-frontend/tests/helpers/component-helper’;
import hbs from ‘htmlbars-inline-precompile’;

moduleForComponent(‘quick-filter’, ‘Integration | Component | quick-filter’);

test(‘it renders’, function(assert) {
this.render(hbs`{{quick-filter}}`);
assert.ok(this.$().html().trim().includes(‘Search’));
});

 

  1. Quick-filter.hbs

Here we design our component. We have used semantic UI elements for designing. Specifically speaking we have used

  • ui-action-input
  • ui-dropdown
  • ui-blue-button-large

Here we have used semantics fluid class to make the component take width of its container.

{{input type=‘text’ placeholder=(t ‘Search for events’)}}
{{#unless device.isMobile }}
 {{input type=‘text’ placeholder=(t ‘Location’)}}
 {{#ui-dropdown class=‘search selection’ selected=filterDate forceSelection=false}}
   {{input type=‘hidden’ id=‘filter_date’ value=filterDate}}
   <i class=“dropdown icon”></i>
   <div class=“default text”>{{t ‘All Dates’}}</div>
   <div class=”menu”>
     <div class=”item” data-value=”all-dates”>{{t ‘All Dates’}}</div>
     <div class=“item” data-value=“today”>{{t ‘Today’}}</div>
     <div class=“item” data-value=“tomorrow”>{{t ‘Tomorrow’}}</div>
     <div class=”item” data-value=”this-week”>{{t ‘This Week’}}</div>
     <div class=“item” data-value=“this-weekend”>{{t ‘This Weekend’}}</div>
     <div class=“item” data-value=“next-week”>{{t ‘Next Week’}}</div>
     <div class=”item” data-value=”this-month”>{{t ‘This Month’}}</div>
   </div>
 {{/ui-dropdown}}
{{/unless}}
<button class=“ui blue button large” type=“button”>{{t ‘Search’}}</button>

Now, our component is ready, and the only part remaining is to place it in our application. We place it in app/templates/index.hbs

<div class=“ui container”>
 {{quick-filter}}
 <h2>{{t ‘Upcoming Events’}}</h2>
 <div class=”ui stackable three column grid”>
   {{#each model as |event|}}
     {{event-card event=event shareEvent=(action ‘shareEvent’)}}
   {{/each}}
 </div>

Now our filter component is up and running.

Generating the Google IO Open Event Android App

The main aim of FOSSASIA Open Event Android App is to give an event organiser the ability to generate the app through a single click by providing the necessary json and binary files. As of late the Android application was tested on Google IO 2017 event. The sample files can be seen here. The data with respect to the event was taken from this site (https://events.google.com/io/). What was astonishing about this application is the simplicity with which we can make an event specific application by giving the vital assets required (json and binary files).

What was needed for generating the Google IO 2017 app?

For generating the app we had to provide the following files:

  1. images folder containing the necessary images of speaker, the logo of the event etc.
  2. event json file which has all the event specific information like the name of the event, the schedule of the event, the description of the event etc.
  3. forms json file having session and speaker form data.
  4. meta json file having the root url of the event.
  5. microlocations json file having all the locations where the events are going to happen.
  6. session_types json file consisting data of all the type of session which will occur in the vent.
  7. sessions json file consisting session specific data like the title of the session, start time and end time of session, which track that session belongs to etc.
  8. speakers json file consisting of speaker specific data like the name of the speaker, image of the speaker, social links of the speaker etc.
  9. sponsers json file consisting list of all sponsers of the event.
  10. tracks json file consisting of tracks specific data.
  11. config.json file which consists of the api url, app name.

After providing the required information we go to this site (http://droidgen.eventyay.com/) and the first thing this site asks us is the email id. Then we upload the required files mentioned above in a zip folder and we have a apk which we can test it out on our Android phone.

How did the Google IO sample app look like?

The files for the sample event can be found over here:

Folder Link:

https://github.com/fossasia/open-event/tree/master/sample/GoogleIO17

Zip File Link:

https://github.com/fossasia/open-event/blob/master/sample/GoogleIO17.zip

What were the issues found in the sample app?

There were certain issues which we observed on testing the app with the Google IO event:

  1. The theme of the app remains the same no matter which event it is. It is important to give the event organiser the ability to customise the theme of the app.
  2. The support for local speaker images needs to be provided as we want to give the event organiser an option to include the images locally or not.
  3. The background of the logo needs to be changed because in certain logos, the dark background causes visibility problems.
  4. Certain information in the app like the event information is hard-coded and needs to be taken from the assets folder instead of strings.xml.

Resources

Automatic Imports of Events to Open Event from online event sites with Query Server and Event Collect

One goal for the next version of the Open Event project is to allow an automatic import of events from various event listing sites. We will implement this using Open Event Import APIs and two additional modules: Query Server and Event Collect. The idea is to run the modules as micro-services or as stand-alone solutions.

Query Server
The query server is, as the name suggests, a query processor. As we are moving towards an API-centric approach for the server, query-server also has API endpoints (v1). Using this API you can get the data from the server in the mentioned format. The API itself is quite intuitive.

API to get data from query-server

GET /api/v1/search/<search-engine>/query=query&format=format

Sample Response Header

 Cache-Control: no-cache
 Connection: keep-alive
 Content-Length: 1395
 Content-Type: application/xml; charset=utf-8
 Date: Wed, 24 May 2017 08:33:42 GMT
 Server: Werkzeug/0.12.1 Python/2.7.13
 Via: 1.1 vegur

The server is built in Flask. The GitHub repository of the server contains a simple Bootstrap front-end, which is used as a testing ground for results. The query string calls the search engine result scraper scraper.py that is based on the scraper at searss. This scraper takes search engine, presently Google, Bing, DuckDuckGo and Yahoo as additional input and searches on that search engine. The output from the scraper, which can be in XML or in JSON depending on the API parameters is returned, while the search query is stored into MongoDB database with the query string indexing. This is done keeping in mind the capabilities to be added in order to use Kibana analyzing tools.

The frontend prettifies results with the help of PrismJS. The query-server will be used for initial listing of events from different search engines. This will be accessed through the following API.

The query server app can be accessed on heroku.

➢ api/list​: To provide with an initial list of events (titles and links) to be displayed on Open Event search results.

When an event is searched on Open Event, the query is passed on to query-server where a search is made by calling scraper.py with appending some details for better event hunting. Recent developments with Google include their event search feature. In the Google search app, event searches take over when Google detects that a user is looking for an event.

The feed from the scraper is parsed for events inside query server to generate a list containing Event Titles and Links. Each event in this list is then searched for in the database to check if it exists already. We will be using elastic search to achieve fuzzy searching for events in Open Event database as elastic search is planned for the API to be used.

One example of what we wish to achieve by implementing this type of search in the database follows. The user may search for

-Google Cloud Event Delhi
-Google Event, Delhi
-Google Cloud, Delhi
-google cloud delhi
-Google Cloud Onboard Delhi
-Google Delhi Cloud event

All these searches should match with “Google Cloud Onboard Event, Delhi” with good accuracy. After removing duplicates and events which already exist in the database from this list have been deleted, each event is rendered on search frontend of Open Event as a separate event. The user can click on any of these event, which will make a call to event collect.

Event Collect

The event collect project is developed as a separate module which has two parts

● Site specific scrapers
In its present state, event collect has scrapers for eventbrite and ticket-leap which, given a query, scrape eventbrite (and ticket-leap respectively) search results and downloads JSON files of each event using Loklak‘s API.
The scrapers can be developed in any form or any number of scrapers/scraping tools can be added as long as they are in alignment with the Open Event Import API’s data format. Writing tests for these against the concurrent API formats will take care of this. This part will be covered by using a json-validator​ to check against a pre-generated schema.

● REST APIs
The scrapers are exposed through a set of APIs, which will include, but not limited to,
➢ api/fetch-event : ​to scrape any event given the link and compose the data in a predefined JSON format which will be generated based on Open Event Import API. When this function is called on an event link, scrapers are invoked which collect event data such as event, meta, forms etc. This data will be validated against the generated JSON schema. The scraped JSON and directory structure for media files:
➢ api/export : to export all the JSON data containing event information into Open Event Server. As and when the scraping is complete, the data will be added into Open Event’s database as a new event.

How the Import works

The following graphic shows how the import works.




Let’s dive into the workflow. So as the diagram illustrates, the ‘search​’ functionality makes a call to api/list API endpoint provided by query-server which returns with events’ ‘Title’ and ‘Event Link’ from the parsed XML/JSON feed. This list is displayed as Open Event’s search results. Now the results having been displayed, the user can click on any of the events. When the user clicks on any event, the event is searched for in Open Event’s database. Two things happen now:

  • The event page loads if the event is found.
  • If the event does not already exist in the database, clicking on any event will

➢ Insert this event’s title and link in the database and get the event_id

➢ Make a call to api/fetch-event in event-collect which then invokes a site-specific scraper to fetch data about the event the user has chosen

➢ When the data is scraped, it is imported into Open Event database using the previously generated event_id. The page will be loaded using jquery ajax ​as and when the scraping is done.​When the imports are done, the search page refreshes with the new results. The Open Event Orga Server exposes a well documented REST API that can be used by external services to access the data.

Creating nested routes in Open Event Front-end and Navigating them with Tabs via semantic UI – Ember Integration

Semantic UI is a modern development framework which helps build responsive and aesthetically beautiful layouts. While it is a really powerful framework in itself, it additionally offers seamless integrations with some of the other open source frameworks including ember js.

Open Event Front-end is a project of FOSSASIA organisation, which was created with the aim of decoupling the front end and the back end for the open event orga server. It is primarily based on ember JS and uses semantic UI for it’s UI.

Here we will be making a nested route /events/ with /events/live/, events/draft, events/past , events/import as it’s subroutes.

To get started with it, we simply use the ember CLI to generate the routes

$ ember generate route events

Then we go on to generate the successive sub routes as follows

$ ember generate route events/live
$ ember generate route events/past
$ ember generate route events/draft
$ ember generate route events/import

The router.js file should be looking like this now.

this.route('events', function() {
    this.route('live');
    this.route('draft');
    this.route('past');
    this.route('import');
  });

This means that our routes and sub routes are in place. Since we used the ember CLI to generate these routes, the template files for them would have generated automatically. Now these routes exist and we need to write the data in the templates of these routes which will get displayed to the end user.

Since the routes are nested, the content of the parent route can be made available to all the children routes via the outlet in ember js.

Next, we go to the template file of events/ route which is at templates/events.hbs And write the following code to create a menu and use ember integration of semantic UI link-to to link the tabs of the menu with the corresponding correct route. It will take care of selecting the appropriate data for the corresponding route and display it in the correct tab via the outlet

<.div class="row">
  <.div class="sixteen wide column">
    <.div class="ui fluid pointing secondary menu">
      {{#link-to 'events.live' class='item'}}
        {{t 'Live'}}
      {{/link-to}}
      {{#link-to 'events.draft' class='item'}}
        {{t 'Draft'}}
      {{/link-to}}
      {{#link-to 'events.past' class='item'}}
        {{t 'Past'}}
      {{/link-to}}
      {{#link-to 'events.import' class='item'}}
        {{t 'Import'}}
      {{/link-to}}
    <./div>
  <./div>
<./div>
<.div class="ui segment">
  {{outlet}}
<./div>

So finally, we start filling in the data for each of these routes. Let’s fill some dummy data at templates/events/live.hbs

<.div class="row">
  <.div class="sixteen wide column">
    <.table class="ui tablet stackable very basic table">
      <.thead>
        <.tr>
          <.th>{{t 'Name'}}<./th>
          <.th>{{t 'Date'}}<./th>
          <.th>{{t 'Roles'}}<./th>
          <.th>{{t 'Sessions'}}<./th>
          <.th>{{t 'Speakers'}}<./th>
          <.th>{{t 'Tickets'}}<./th>
          <.th>{{t 'Public URL'}}<./th>
          <.th><./th>
        <./tr>
      <./thead>
      <.tbody>
        <.tr>
          <.td>
            <.div class="ui header weight-400">
              <.img src="http://placehold.it/200x200" alt="Event logo" class="ui image">
              Sample Event
            <./div>
          <./td>
          <.td>
            March 18, 2016 - 09:30 AM
            <.br>(to)<.br>
            March 20, 2016 - 05:30 PM
          <./td>
          <.td>
            <.div class="ui ordered list">
              <.div class="item">sample@gmail.com ({{t 'Organizer'}})<./div>
              <.div class="item">sample2@gmail.com ({{t 'Manager'}})<./div>
            <./div>
          <./td>
          <.td>
            <.div class="ui list">
              <.div class="item">{{t 'Drafts'}}: 0<./div>
              <.div class="item">{{t 'Submitted'}}: 0<./div>
              <.div class="item">{{t 'Accepted'}}: 0<./div>
              <.div class="item">{{t 'Confirmed'}}: 0<./div>
              <.div class="item">{{t 'Pending'}}: 0<./div>
              <.div class="item">{{t 'Rejected'}}: 0<./div>
            <./div>
          <./td>
          <.td>
            2
          <./td>
          <.td>
            <.div class="ui bulleted list">
              <.div class="item">{{t 'Premium'}} (12/100)<./div>
              <.div class="item">{{t 'VIP'}} (10/15)<./div>
              <.div class="item">{{t 'Normal'}} (100/200)<./div>
              <.div class="item">{{t 'Free'}} (100/500)<./div>
            <./div>
          <./td>
          <.td>
            <.div class="ui link list">
              <.a class="item" target="_blank" rel="noopener" href="http://nextgen.eventyay.com/e/ecc2001a">
                http://nextgen.eventyay.com/e/ecc2001a
              <./a>
            <./div>
          <./td>
          <.td class="center aligned">
            <.div class="ui vertical compact basic buttons">
              {{#ui-popup content=(t 'Edit event details') class='ui icon button'}}
                <.i class="edit icon"><./i>
              {{/ui-popup}}
              {{#ui-popup content=(t 'View event details') class='ui icon button'}}
                <.i class="unhide icon"><./i>
              {{/ui-popup}}
              {{#ui-popup content=(t 'Delete event') class='ui icon button'}}
                <.i class="trash outline icon"><./i>
              {{/ui-popup}}
            <./div>
          <./td>
        <./tr>
      <./tbody>
    <./table>
  <./div>
<./div>

 Similarly we can fill the required data for each of the routes.And this is it, our nested route is ready. Here is a screenshot what you might expect.

Screenshot highlighting the tabs

Resources

Sorting language-translation in Open Event Server project using Jinja 2 dictsort.

Working on the Open Event Server project an issue about arranging language-translation listing in alphabetical order came up. To solve this issue of language listing arrangement i.e. #2817, I found the ‘d0_dictsort’ function in jinja2 to sort dictionaries. It is a defined in jinja2.filters. Python dicts are unsorted and in our web application we at times may want to order them by either their key or value. So this function comes handy.

This is what the function looks like:

do_dictsort(value, case_sensitive=False, by='key')

We can write them in three ways as:

{% for record in my_dictionary|dictsort %}
    case insensitive and sort the dict by key

{% for record in my_dictionary|dicsort(true) %}
    case sensitive and sort the dict by key

{% for record in my_dictionary|dictsort(false, 'value') %}
    sort the dict by value, normally sorted and case insensitive
  1.       The first way is easily understood that dict has been sorted by key not taking case into consideration. It is just in the same way written as dictsort(false).
  2.       Second way is basically the first being case sensitive. dictsort(true) here tells us that case is sensitive.
  3.      Third way is dictsort(false,’value’). The first parameter defines that case insensitive while second parameter defines that it is sorted by ‘value’.

The issues was to sort translation selector for the page in alphabetical order. The languages were stored in a dictionary which to change in order, I found this function very easy and useful.

Basically what we had was:

This is how the function was used in the code for the sort. Like this:

<ul class="dropdown-menu lang-list">
   {% for code in all-languages|dictsort(false,'value') %}
       <li><a  href="#" style="#969191" class="translate" id="{{ code[0] }}">{{  all_languages[code[0]] }}<>a><li>
    {% endfor %}
<ul>


Here:
{{ all_languages }} is the list which contained the languages like French, English, etc., which could be accessed with its global language code. code here(index for all_languages) is a tuple of {‘global_language_code’,’language’} (An example would be (‘fr’,’French’), so code[0] gave me the language_code.

Finally, the result:

This is one of the simple ways to sort your dictionaries.