Choosing Semantic UI over Bootstrap for the Open Event Front-end

There are many design frameworks. In this blogpost, I am going to list out some pros of Semantic UI and why it is better suited for the Open Event project than Bootstrap or other frameworks.

Components:

The very main reason why we chose Semantic UI in Open Event frontend is the number of components it offers, such as step, segment, rails, cards, lists, headers, dimmers, dropdown, and form validation. We can use many of these components right away, which helps us to speed up development.

Installation:

Semantic UI also provides very good integrations with the majority of technologies available out there. Bootstrap, however, can be integrated anywhere with CDN’s.

Following are the technologies that Semantic UI can be integrated with:

  • As a NPM package
  • React
  • Angular
  • Ember JS

Also Semantic UI has CDN’s. Thus Semantic UI has easeful installation. The documentation on the official site provides a good view of installation.

Documentation:  

In regards to the documentation too, Semantic UI is more suitable for us. The official site has a neat and clean documentation for each component. In the case of Bootstrap, the documentation is also good but personally, for me, the Semantic UI’s documentation is easier to read understand.

Designed with completely em:

em is a unit in CSS which ensures that components are scaled according to the different device sizes.Every component is made by using em and rem so that the webpage doesn’t get distorted whenever screen size changes.

Concise and Expressive:  

In English, it’s much easier to say “There are three tall men” than “There is a tall man, a tall man, and a tall man”. This means that in Bootstrap, to achieve a thing (say a navbar), we need to have 2-3 nested classes of “navbar”, ”navbar-default”, etc. whereas with Semantic UI, it can be done with single one.

Code appearance:

In the case of Semantic UI, the code looks much cleaner since the classes created are of a short length and cause less confusion. Following code indicates how clean things can be done with Semantic UI.

class="ui secondary menu"> class="active item"> Home class="item"> Messages class="item"> Friends
class="right menu">
class="item">
class="ui icon input"> type="text" placeholder="Search..."> class="search link icon">
</div> <a class="ui item"> Logout </a> </div> </div>

                    Fig. Semantic UI

<nav class="navbar navbar-default">
class="container-fluid">
class="navbar-header"> class="navbar-brand" href="#">Brand
<!-- Collect the nav links, forms, and other content for toggling -->
class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
class="navbar-form navbar-left">
class="form-group"> type="text" class="form-control" placeholder="Search">
<button type="submit" class="btn btn-default">Submit</button> </form> <ul class="nav navbar-nav navbar-right"> <li><a href="#">Link</a></li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Dropdown <span class="caret"></span></a> <ul class="dropdown-menu"> <li><a href="#">Action</a></li> <li><a href="#">Another action</a></li> <li><a href="#">Something else here</a></li> <li role="separator" class="divider"></li> <li><a href="#">Separated link</a></li> </ul> </li> </ul> </div><!-- /.navbar-collapse --> </div><!-- /.container-fluid --> </nav>

                      Fig. Bootstrap

Looks:

In terms of looks, it is well said that “User makes user experience” i.e the user experience depends more on the user viewing the website rather than the developer. Still, according to me, every Bootstrap site looks similar whereas a site built with Semantic provides a different, crisp look. Unlike other sites, a site built with Semantic looks beautiful.

Redesigning infinitely:

Creating a site in Semantic means you never have to rewrite your codebase from scratch. Redesigning means retooling your UI toolkit, adjusting UI definitions, not creating entirely new HTML layouts.

Conclusion:

There are many reasons why Semantic UI is a bit more suitable, powerful than Bootstrap for the Open Event project. Still, the choice of the framework depends on the purpose of your website or what you want to build. The official documentation of Semantic-UI can be found out at https://semantic-ui.com.

Connecting Social Apps with Open Event Orga API Server

Orga API Server serves the organizer with many features but there was need of one feature which will allow us to provide Organizer an option to connect with their social media audience directly from API server. This will also allow the Orga Server users to share their experience for different events on their social media platforms.

For this feature, some of the social media platforms added are:

  • FaceBook
  • Twitter
  • Instagram
  • Google+

    Before connecting with these social media platforms, we have to implement and test The Auth Tool – OAuth 2.0

Without going on introductory introduction to OAuth 2.0, Let’s focus on its implementation in Orga API Server

OAuth Roles

OAuth defines four roles:

  • Resource Owner
  • Client
  • Resource Server
  • Authorization Server

Let’s look at the responsibility of these roles when you connect your social apps with API server

> Resource Owner: User/Organizer ( You )

You as User or Organizer connection your accounts with API server are the resource owners who authorize API server to access your account. During authorization, you provide us access to read your account details like Name, Email, Profile photo, etc.

> Resource / Authorization Server: Social Apps

The resource server here is your social platforms/apps where you have your account registered. These Apps provide us limited access to fetch details of your account once you authorize our application to do so. They make sure the token we provide match with the authorization provided before through your account.

> Client: Orga API Server

Orga API Server acts as the client to access your account details. Before it may do so, it must be authorized by the user, and the authorization must be validated by the API.

The process to add

A simple work plan to follow:

  1. Understanding how OAuth is implemented.
  2. Test OAuth implementation on all 4 social medias.
  3. After Necessary correction. Make sure we have all views(routes) to connect these 4 social medias.
  4. Implementing the same feature on the template file.
  5. Make sure these connect buttons are shown only when Admin has registered its client credentials in Settings.
  6. Creating a view to unlink your social media account.

Understanding how OAuth is implemented.

Current Implementation of Oauth is very simple and interesting on API server. We have Oauth helper classes which provide all necessary endpoints and different methods to get the job done.

Test OAuth implementation on all 4 social medias.

Now we can work on testing on the callbacks of all 4 social apps. We have callback defined in views/util_routes.py For this, I picked up the auth OAuth URLs and called them directly on my browsers and testing their callback. Now on callback, those methods required some change to save user data on database thus connecting their accounts with API server. This lead to changes in update_user_details and on callback methods.

def update_user_details(first_name=None,
                        last_name=None,
                        facebook_link=None,
                        twitter_link=None,
                        file_url=None,
                        instagram=None,
                        google=None):

Make sure we have all views(routes) to connect these 4 social medias

This has to be done on views/users/profile.py Addition of one method

@profile.route('/google_connect/', methods=('GET', 'POST'))
def google_connect():
        ....
        ....
    return redirect(gp_auth_url)

and testing, correction on other 3 methods

Implementing the same feature on template file.

Updating gentelella/users/settings/pages/applications.html to add changes required to add this feature. This included ability to show URLs of connected accounts and functioning connect and disconnect button

Make sure these connect buttons are shown only when Admin has registered its client credentials in Settings.

    fb = get_settings()['fb_client_id'] != None and get_settings()['fb_client_secret'] != None
        ....
        ....
        ...

The addition of such snippet provides data to the template to decide whether to show those fields or not. It will not make any sense if there is no application created to connect those accounts by Admin.

Creating a view to unlink your social media account.

utils_routes.route('/unlink-social/<social>')
def unlink_social(social):
    if login.current_user is not None and login.current_user.is_authenticated:
        ...
        ...

A method is created to unlink the connected accounts so that users can anytime disconnect their accounts from API server.

Where to connect?

Settings > Applications

 

How it Works (GIF below )

Using FastAdapter in Open Event Organizer Android Project

RecyclerView is an important graphical UI component in any android application. Android provides RecyclerView.Adapter class which manages all the functionality of RecyclerView. I don’t know why but android people have kept this class in a very abstract form with only basic functionalities implemented by default. On the plus side it opens many doors for custom adapters with new functionalities for example, sticky headers, scroll indicator, drag and drop actions on items, multiview types items etc. A developer should be able to make an adapter of his need by extending RecyclerView.Adapter. There are many custom adapters developers have shared which comes with built in functionalities. FastAdapter is one of them which comes with all the good functionalities built in and also it is very easy to use. I just got to use this in the Open Event Organizer Android App of which the core feature is Attendees Check In. We have used FastAdapter library to show attendees list which needs many features which are absent in plane RecyclerView.Adapter. FastAdapter is built in such way that there are many different ways of using it on developer’s need. I have found a simplest way which I will be sharing here. The first part is extending the item model to inherit AbstractItem.

public class Attendee extends AbstractItem<Attendee, AttendeeViewHolder> {
  @PrimaryKey
  private long id;
  ...
  ...

  @Override
  public long getIdentifier() {
      return id;
  }

  @Override
  public int getType() {
      return 0;
  }

  @Override
  public int getLayoutRes() {
      return R.layout.attendee_layout;
  }

  @Override
  public AttendeeViewHolder getViewHolder(View view) {
      return new AttendeeViewHolder(DataBindingUtil.bind(view));
  }

  @Override
  public void bindView(AttendeeViewHolder holder, List<Object> list) {
      super.bindView(holder, list);
      holder.bindAttendee(this);
  }

  @Override
  public void unbindView(AttendeeViewHolder holder) {
      super.unbindView(holder);
      holder.unbindAttendee();
  }
}

The methods are pretty obvious by name. Implement these methods accordingly. You may notice that we have used Databinding here to bind data to views but it is not necessary. Also you will have to create your ViewHolder for adapter. You can either use RecyclerView.ViewHolder or you can just create a custom one by inheriting it as per your need. Once this part is over you are half done as most of the things are been taken care in model itself. Now we will be writing code for adapter and setting it to your RecyclerView.

FastItemAdapter<Attendee> fastItemAdapter = new FastItemAdapter<>();
fastItemAdapter.setHasStableIds(true);
...
// functionalities related code
...
recyclerView.setAdapter(fastItemAdapter);

Initialize FastItemAdapter which will be our main adapter handling all the direct functions related to the RecyclerView. Set up some boolean constants according to the project need. In our project we have Attendee model which has id as a primary field. FastItemAdapter can take advantage of distinct field of the model called as identifier . Hence it is set true as Attendee model has id field. But you should be careful about setting it to True as then you must have implemented getIdentifier in the model to return correct field which will be used as an identifier by our adapter. And the adapter is good to set to the RecyclerView.

Now we got to decide which functionalities we will be implementing to our RecyclerView. In our case we needed: 1. Search filter for attendees, 2. Sticky header for attendees groups arranged alphabetically and 3. On click listener for attendee item.

FastItemAdapter has ItemFilter adapter wrapped inside which manages all the filtering stuff. Filtering logic can be set using it.

fastItemAdapter.getItemFilter().withFilterPredicate(this::shallFilter);

Where shallFilter is method which takes attendee object and returns boolean whether to filter the item or not. And after this you can use FastItemAdapter’s filter method to filter the items. For sticky headers you need to implement StickyRecyclerHeadersAdapter extending AbstractAdapter. In this class you will have to implement your filter logic in getHeaderId method. This must return an unique id for items of the same group.

@Override
public long getHeaderId(int position) {
   IItem item = getItem(position);
   if(item instanceof Attendee && ((Attendee)item).getFirstName() != null)
       return ((Attendee) item).getFirstName().toUpperCase().charAt(0);
   return -1;
}

Like in this case we have grouped attendees alphabetically hence just returning initial character’s ASCII value will do good. You can modify this method according to your need. For other unimplemented methods just keep their default return values. With this you will also have to implement onCreateHeaderViewHolder and onBindHeaderViewHolder methods to bind view and data to the header layout. Once this is done you are ready to set sticky headers to your RecyclerView with following code:

stickyHeaderAdapter = new StickyHeaderAdapter();
final HeaderAdapter headerAdapter = new HeaderAdapter();
recyclerView.setAdapter(stickyHeaderAdapter.wrap((headerAdapter.wrap(fastItemAdapter))));

final StickyRecyclerHeadersDecoration decoration = new StickyRecyclerHeadersDecoration(stickyHeaderAdapter);
recyclerView.addItemDecoration(decoration);
adapterDataObserver = new RecyclerView.AdapterDataObserver() {
   @Override
   public void onChanged() {
       decoration.invalidateHeaders();
   }
};
stickyHeaderAdapter.registerAdapterDataObserver(adapterDataObserver);

For click listener, the code is similar to the RecyclerView.Adapter’s one.

fastItemAdapter.withOnClickListener(new FastAdapter.OnClickListener<Item>() {
  @Override
  public boolean onClick(View v, IAdapter<Item> adapter, Item item, int position) {
     // your on click logic
   return true;
  }
});

With this now you have successfully implemented FastItemAdapter to your RecyclerView. Although there are some important points to be taken care of. If you are using filter in your application then you will have to modify your updateItem logic. As when filter is applied to the adapter its items list is filtered. And if you are updating the item using its position from original list it then it will result in exception or updating the wrong item. So you will have to change the position to the one in filtered list. For example the updateAttendee method from Organizer App code looks like this:

public void updateAttendee(int position, Attendee attendee) {
   position = fastItemAdapter.getAdapterPosition(attendee);
   fastItemAdapter.getItemFilter().set(position, attendee);
}

Data Scraping with Selenium

There still exists websites without any APIs. Scraping data from such sites can be very time-consuming and manual. I created samples for Open Event App generator. One of the samples that I created was for All Hands Hawaii 2016. This site didn’t have any API to enable easy data scraping.

How do we find out if a website is using an API or not?

Using Google Chrome, go to View → Developer → Developer Tools. Under the Network →XHR look for API endpoint with a bit of Hit and Trial method. (XHR stands for XMLHttpRequest)

However, what if there is no API being used in the site? How would you scrape data in that case? Will you now manually click onto every hyperlink on the site and visit every page to get the data by manually copying and pasting it? Could there be someone doing that manual job for you? Or better could there be “something” doing that job for you? Yes, It’s selenium.

Selenium Web Browser Automation

Selenium is a tool that automates the task of browsing through the internet. Although, technically it is used for web testing purposes but there is no restriction to it’s utility.

Let’s get started with basics of Selenium:-

INSTALLATION:
  1. Run the following command pip install selenium (Quick Tip: It is advised to use virtualenv)
  2. Selenium requires drivers to run. Different browsers use different drivers. Choose an appropriate driver for your browser. some common drivers are shown below (Source)-

Chrome: https://sites.google.com/a/chromium.org/chromedriver/downloads

Firefox: https://github.com/mozilla/geckodriver/releases

Safari: https://webkit.org/blog/6900/webdriver-support-in-safari-10/

BASIC FUNCTIONALITIES-SELENIUM:

Visit a page ( using the get() ):

 driver.get(urlofpage)

Navigating to various elements on the visited/current webpage:

  1. BY ID: 
    WebElement element = driver.findElement(By.id(“ui_elementid”));
  2. BY CLASS NAME:
    List<WebElement> cheeses = driver.findElements(By.className(“cheese”));
  3. BY TAG NAME:
    WebElement tag = driver.findElement(By.tagName(“tag_name”));
  4. BY CSS: WebElement cs = driver.findElement(By.cssSelector(“#”));
  5. BY LINK TEXT:
    WebElement cheese = driver.findElement(By.linkText(“blog”));

    If the element href is something like urlofpage?q=blog.

  6. BY XPATH:
  7.  List<WebElement> xp = driver.findElements(By.xpath(“//input”))

 

We can also use JavaScript along with Selenium. This might be a helpful thread for the same.

Another really good link for the same is http://www.marinamele.com/selenium-tutorial-web-scraping-with-selenium-and-python

Testing User Interactions with Espresso

Espresso is a testing framework which provides the facility to write the tests for user interactions and unitary tests. Since the release of its version 2 it is now a part of Android Testing Support Library.

The android apps we build at FOSSASIA follow rigorous testing methods. See this simple UI test  in the Phimp.me app using espresso to check if button and bottom navigation are displayed in an activity. You can also find our other tests related to API and databases in the Open Event Android App.

In this blog we learn how to add this facility to your app and write a test for a simple app that takes the name of from the user and prints it on the other screen on button click.

Adding espresso support

  • Install android support repository if not already present. You do it by following Tools -> Android -> SDK Manager
Tools you need to download for testing
  • Add the following dependencies to your app’s build.gradle file
dependencies {
    androidTestCompile 'com.android.support.test:runner:0.5'
    androidTestCompile 'com.android.support.test:rules:0.5'
    androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2.2'

}
  • Specify the test instrumentation runner in default config
android {

    defaultConfig {

        // ....

        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"

    }

}

Before we begin with writing our tests knowing some basic components will help in understanding the code better. Writing tests with espresso is easy as its construction is similar to English language.

The three major components are 

  • ViewActions        : Allows you to interact with views
  • ViewAssertions   : Allows you to assert the state of a view.
  • ViewMatchers     : Allows you to locate a view in the current view hierarchy.

Suppose we want to test if text is displayed in the view, we can do it by

onView(withId(R.id.textView))                              //ViewMatcher

 .perform(click())                                         //ViewAction

 .check(matches(isDisplayed()));                           //ViewAssertion

Example

Consider an app which takes a name from the user and displays it on the next screen on clicking the button.

To perform this kind of test we will write

//Locate the view with id "name" and type the text "Natalie"

onView(withId(R.id.name)).perform(typeText("Natalie"));

//Locate the view with id "next" and click on it

onView(withId(R.id.next)).perform(click());

//Locate the view with id "new_name" and check its text is equal with "Natalie"

onView(withId(R.id.new_name)).check(matches(withText("Natalie")));

You can run tests by right clicking on the class and selecting the “run test” option. If the interaction is not as expected then the message will be displayed.

Up until now unit test were in main focus but as we move towards the more complex apps where user interaction plays an essential role, UI testing becomes equally necessary.

References:

ButterKnife for Open Event Android

Open Event Android, by FOSSASIA has been coded in a very clean and systematic manner. One of the great things used in it is the ButterKnife library. It has made increased the readability and understand ability of the app’s code.

ButterKnife, developed and maintained by Jake Wharton(Square Inc.) is an Android library designed to make Java code less complex and efficient. The library operates with the use of annotations and hence, binds classes to the relevant annotations in Java and do the job.

First off, here’s the very first thing that you want to do before starting to use ButterKnife — Adding it’s dependency in the build.gradle file of your project.

compile 'com.jakewharton:butterknife:8.6.0
annotationProcessor 'com.jakewharton:butterknife-compiler:8.6.0'

At FOSSASIA, we have extensively used this amazing library to simplify the code for our Open Event Android application. I hope this blog will help newcomers in understanding our code base in a better way.

We have the famous @Bindview annotation by ButterKnife to bind views to Android components without having to call R.FindViewById() method for every view. The following is a code sample from Open Event Android about the same.

@BindView(R.id.toolbar) Toolbar toolbar;

@BindView(R.id.nav_view) NavigationView navigationView;

@BindView(R.id.progress) ProgressBar downloadProgress;

@BindView(R.id.layout_main) CoordinatorLayout mainFrame;

@BindView(R.id.drawer) DrawerLayout drawerLayout;

@BindView(R.id.appbar) AppBarLayout appBarLayout;

Similarly, the ButterKnife library also handles events in an Android application and has annotations like @onClick, @onLongClick among others. So while using ButterKnife for an event, choose the appropriate event and the corresponding annotation followed by the Java method that is to be executed. Here is an example for the same.

@OnClick(R.id.submit)

public void submit(View view) {

// TODO submit data to server...

}

The ButterKnife library also helps to bind resources in your project.

It simplifies the View holder pattern inside a list adapter as illustrated in the following example.

public class MyAdapter extends BaseAdapter {

@Override public View getView(int position, View view, ViewGroup parent) {

   ViewHolder holder;

   if (view != null) {

     holder = (ViewHolder) view.getTag();

   } else {

     view = inflater.inflate(R.layout.whatever, parent, false);

     holder = new ViewHolder(view);

     view.setTag(holder);

   }

   holder.name.setText("John Doe");

   // etc...

   return view;

 }

static class ViewHolder {

   @BindView(R.id.title) TextView name;

   @BindView(R.id.job_title) TextView jobTitle;

   public ViewHolder(View view) {

     ButterKnife.bind(this, view);

   }

 }

}

The above code sample has been taken from http://jakewharton.github.io/butterknife/ and you can visit the site for further in-depth knowledge on ButterKnife.

How ButterKnife works?

I will be talking a bit about the working of the ButterKnife library. Firstly, I would like to introduce annotations concept in Java briefly before proceeding.

“In the Java, computer programming language, an annotation is a form of syntactic metadata that can be added to Javasource code. Classes, methods, variables, parameters and packages may be annotated.” As it is rightly said, Annotations can be of various uses like Getting information for the compiler, Compile-time and deployment-time processing, Runtime processing. However, please note that annotations can NOT modify and edit the existing classes. They can simply make more classes.

With this, I’m gonna continue on the working of ButterKnife. From the above discussion it must be clear now that ButterKnife is not changing any of our classes. It makes new classes to handle the code.

ButterKnife goes through all the files in your project and identifies the ButterKnife annotations used, if any. It then creates new classes for every annotation encountered, based on the annotations used. These new classes contain the general methods that would have been used in a project without ButterKnife. When we finally call ButterKnife.inject(this) , the inject()of all the new classes made by ButterKnife are called which then perform the desired function during execution.

Finally, try to go through the complete documentation for Butterknife and Java annotations for more information. 

Using Flexbox for Responsive Layout in Open Event Webapp

Recently, I tackled the issue of alignment of different buttons and input bar in the Open Event Webapp. The major challenge was to create a responsive design which adapts well on all the platforms: desktop, tablets and mobile. Doing it using grid layout provided by Bootstrap was rather tough and complicated and I solved the problem using the flexbox.

What is Flexbox?

Flexible Boxes, also called as Flexbox, is a new layout mode introduced in CSS3. The best part of using flexbox is that it ensures that the elements behave in a predictable manner when the page layout accommodates different screen sizes and display devices. In other words, it helps us in making a responsive design in a simpler manner. It is an alternative to block elements being floated and manipulated using media queries. Using Flexbox, a container and its children can be arranged in any direction: up, down or left and right. Size is flexible and elements inside the flexbox can grow or shrink to occupy unused space or prevent overflow respectively.

The difference with grid layouts?

There is a slight difference between flexbox and grid layouts which makes one suitable for creating a fully complete layout and the other not so much.

Ideally, grids (provided by Bootstrap for example) are used for creating an entire layout. Flexbox is suited for styling separate containers such as navbars for example.

flex_terms.png

                                 Structure of Flexbox Container

How did I solve it?

First I defined a container which acted as a wrapper for all the buttons and the search bar. Now, to convert this container into an actual flexbox, we have to add display: flex property to it.

.container {
 display: flex;
}

Did the issue got solved? It doesn’t look so. The container enclosed in the red border is the flexbox.

flexnogrow.png

As we can see from the picture itself, the elements are quite close to each other and a lot of space is wasted. Fortunately, flexbox provides us a handy property called flex-grow which deals with this type of problem. It defines the ability for a flex item to grow if necessary. That it, it tells how much amount of the available space inside the flex container the item is allowed to take. If all the items have flex-grow set to 1, then the remaining space in the container would be equally distributed to all the items. In a similar way, if an item has flex-grow set to 2, then it would occupy twice the amount of available space when compared to the other items.

So, I applied the flex-grow:1 on the items.

.list-btn {
   flex-grow: 1;
}

.search-filter {
   flex-grow: 1;
}
.starred-btn {
   flex-grow: 1;
}

.calendar-btn {
   flex-grow: 1;
}

How does it look now?

flexbox_initial.png

Much better. But is the problem solved now? No. There is one more thing that we haven’t checked yet. Yes. It’s the responsiveness. We haven’t yet checked how it displays on tablets and mobiles yet? Let’s test and check and what we see.

mobilenowrap.png

Ok. Something is not right. The items are squeezing together. That doesn’t look good. We don’t want the elements to wrap on a single line. What we actually want is that the items would stack on top of each other when the screen size is reduced. That would look much more neat and tidy.

To change this default behavior of wrapping of items, we use the flex-wrap property on the container. Specifying flex-wrap: wrap does the trick and the items wrap as needed.

mobilewrap2.pngmobilewrap1.png

 

.container {
   display: flex;
   flex-wrap: wrap;
}

The result looks much better now. The items wrap on a single line only when there is enough space available. Otherwise, they wrap up onto multiple lines from top to bottom.

Flexbox is a great tool for creating custom layouts for separate containers. Apart from the properties discussed here, there are a plethora of other options which can be used to customize the behavior of flexbox and the items contained inside it.  Check out the links below for more information!

Resources:

Intro to concurrency and Refactoring Open Event Android using RxJava

Functional reactive programming seems to have taken the whole development world by storm. It’s one of the hottest thing even after 2 years of constant traction in the communities of several programming languages, where different implementations of the specifications defined by Rx or Reactive Extensions have changed the paradigm of programming for many professional and enthusiast developers.

RxJava is no exception, not only has it been widely adopted by Android and Java developers unanimously, but also received attention of well known and top developers of both communities. The reason of its success is the fluent API with heavy toolset it provides from the Functional Programming paradigm and its ease and natural ability to handle concurrency on different levels based on the type of operation being performed, i.e., computations, I/O, etc. It basically takes away the several constraints and concurrency related checklists developers had to maintain while working with thread management. So, now, developers can’t make an excuse for using database operations on the Main Thread because offloading it on another thread is hard.

So, in this blog post, I will be detailing the process of converting the existing synchronous code of your app into a performant reactive code without breaking the structure of your project, like we did in Open Event Android (Github Repo). Before starting, I have assumed that you know how to add RxJava dependency to your project as it is covered in many other blog posts and the documentation is also very clear. Secondly, you should also add RxAndroid dependency as it contains the Scheduler needed to work on Android’s Main Thread. So, Let’s start.

Current State

Currently, our code loads the queries from database synchronously on Main Thread using the SQLiteDatabase for an Android application. This is how it looks like –

As we can see, we are directly returning the loaded results to the caller. This is called synchronous call, meaning the caller will block till the function is returned, and can’t move further to do anything else. It basically waits for the function to return, which may take hundreds of milliseconds to seconds based on the function it performs.

New Android version crash the applications that perform Network interactions on the main thread but no such restriction for disk based operations is there, making it hard to enforce best performance practices. Before RxJava, there were interfaces made for different kinds of objects, passed in as parameters of the db request function, which created a new thread and performed operations and when completed, returned back the results to the main thread using the postOnUiThread method, so that the views could update themselves. The interface implementations passed are called callbacks because they call a particular function that you provide back when the asynchronous operation is completed. Even the calling of callback function is delegated on the implementor and may result in undesired effects. The query done in this fashion is called an asynchronous query because the execution of this takes place in parallel with main thread and is not synchronised with the main thread. It may take up forever to complete, complete even before the main thread moved on to next operation or even return when the main thread was completed and done waiting for it and destroyed. This will result in a weird crash even when the application was closed, because the returned function will try to update the views which are not even there.

Problems like these made Android Devs lazy and compromise with the performance of their application. Not anymore! RxJava is here to solve half of our problems. You see, RxJava does provide a solution to achieve effortless concurrency but does not ensure thread safety, memory contention, race conditions, deadlocks and other concurrency related issues for you. These you must code up for yourself.

So, after the introduction of Rx and its dire need in Android projects, we will move on to a basic procedure to convert any synchronous code to asynchronous call using RxJava Observable.

Let’s subscribe

The Observable class in RxJava is the most used and standard stream class you will use. Observable handles a stream of object and passes them as they arrive to the Subscriber attached to it. As you may guess, for a stream of data that arrives in a non deterministic fashion ( we don’t know when it will arrive ), we require an asynchronous query, and this is where RxJava excels at. You can configure an Observable to wait for result in one thread so that main thread doesn’t block and deliver result on another thread. You can either create a new thread or use certain pre configured schedulers for basic type of operations :

  1. Schedulers.newThread() : Creates a new thread for each request
  2. Schedulers.io() : For I/O bound work like a network call, database access
  3. Schedulers.computation() : For heavy computations
  4. AndroidSchedulers.mainThread() : For returning to UI thread of Android ( Present in RxAndroid )

There are other types of Schedulers like Schedulers.trampoline(), etc that are used for other purposes like testing, but the above ones are most commonly used ones and we’ll be using Schedulers.computation() for loading the SQLite query on the thread from Computation Thread Pool and AndroidSchedulers.mainThread() for delivering the result on UI thread.

Using Computation instead of I/O because I/O uses unbounded executor, meaning it continues adding threads to the thread pool, which isn’t good. So, we use computation instead. You can create your own bounded executor and pass it as a scheduler

The basic operation of passing an object to a subscriber is :

Observable.just(getEventDetails())
        .subscribe(new Consumer<Event>() {
          @Override
          public void accept(@NonNull Event event) throws Exception {
              Log.d("EVENT", event.toString());
          }
      });

 

Using lambda notation, we get a terse form of the same :

Observable.just(getEventDetails())
 	.subscribe(event -> Log.d("EVENT", event.toString()));

We’ll be using lambda notations from now on.

In the above example, we are just loading and passing the Event object to the subscriber below who logs it. But this is not asynchronous, everything gets executed on main thread. The above code is equivalent to :

Event event = getEventDetails();
Log.d("EVENT", event.toString());

 

So why use it, you say? Well, we can still get a lot of goodies from functional programming this way. For example,

String track = "Android";

Observable.fromIterable(getSessionList())
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .toList()
    .subscribe(titles -> Log.d("Titles", titles.toString()));

 

What this code does is, take a list of sessions and emit each session at a time, filter out the ones which don’t have Android as their track name, take out their titles and puts them in a list and gives it to subscriber.

Now imagine doing it in plain Java. Create a list of string, loop through each session, check track, push title to that list and this much when this example is the most basic of use cases of RxJava.

But how to achieve concurrency. If the there are 10000 sessions, this code will take huge time even if sessions are in memory and not loaded from database. So we will listen to these list events on computation thread.

Observable.fromIterable(getSessionList())
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .subscribeOn(Schedulers.computation())
    .subscribe(titles -> adapter.setItems(titles));

 

That’s it. Now each filtering and mapping and converging to a list is done on another thread.

If you want to listen to each session one at a time, and not all at once when it is completed, you can remove toList() operator

But now, our app will crash! Because when we deliver the result to subscriber, we are still on computation thread, so we need to come back to Main Thread because Android Views are not thread safe, meaning they cannot be accessed from any thread other than UI thread. So in order to do that, we just use observeOn() operator :

Observable.fromIterable(getSessionList())
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .toList()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(titles -> adapter.setItems(titles));

 

Still, our code has a critical problem, the mapping and filtering takes place on background thread, but the loading of session list still takes place on UI thread because it is loaded first and then passed to Observable

Observable methods like just, from, fromIterable, etc all take object from the current thread, meaning passing the object to these functions will not occur on the Scheduler you have supplied. This is very basic programming concept that language parses rightmost parameter first but usually is misunderstood in terms of Rx programming.

So, what do we do? We use fromCallable which waits till the containing function returns and then operates on it

Observable.fromCallable(this::getSessionList)
    .flatMapIterable(sessions -> sessions)
    .filter(session -> session.getTrack().getName().equals(track))
    .map(Session::getTitle)
    .toList()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(titles -> adapter.setItems(titles));

 

We’re done! We have changed our synchronous database call to an asynchronous call.

Another use case is when you just have to do an operation asynchronously and not return anything, then fromCallable won’t work as it expects some return value to operate on, instead use Completable

Completable.fromAction(this::clearDatabase)
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(() -> {
        // Completed
        showToast("Success");
    });

 

Note that here we use method reference to call a function, you can just pass in a lambda or Action implementation to do some in place work like this

Completable.fromAction(() -> {
    doSomeStuff();
    // ...
    doOtherStuff(); 
}).subscribeOn(Schedulers.computation())
  .observeOn(AndroidSchedulers.mainThread())
  .subscribe(() -> {
      // Completed
      showToast("Success");
  });

 

Now, you can wrap all your slow methods into observable or completable without changing any code structure and your code will look like this :

On parting note, a trick to avoid repeated subscribeOn observeOn :

private <T> ObservableTransformer<T, T> applySchedulers() {
    return upstream -> upstream.subscribeOn(Schedulers.computation())
                               .observeOn(AndroidSchedulers.mainThread());
}

 

Create this function and just call compose on each Observable and call the function inside that, passing the transformer., like it is shown in the picture above

That’s it for now. Have a happy and lag free day!

Handling date-time in the Open Event Project

Handling date time in code becomes little tricky when the project is used internationally because then there comes additional term Timezone. Timezone is a property of a location which needs to be considered while comparing that time with the time in another location. For example – there are two villages A and B. One day Ram from village A calls his friend Shyam in village B at 8:00 am to wish “good morning”. But Shyam receives Ram’s call at 6pm on same day and he replies “good evening”. That means village A’s timezone is 10 hrs  behind village B’s timezone. So here we need some reference timezone about which all other timezones can be declared. This is where UTC (Coordinated Universal Time) comes into play. UTC is reference timezone by which all the timezones are declared. For example – Indian timezone is 5 hrs and 30 mins ahead of UTC which is denoted as UTC+05:30. In languages, these timezones are declared in date time library using constants such as ‘Asia/Kolkata’ which is Indian Standard Time. I will be talking about working with  date time in python in this blog. In the FOSSASIA’s Open Event project since it is event management system, handling date-time with the timezone is one of the important tasks.

Here is the relevant code:

>>> import datetime
>>> import pytz
>>> now = datetime.datetime.now()

datetime.datetime.now()  returns naive datetime as of the time setting of the machine on which the code is running. Naive date means it doesn’t contain any info about the timezone. It just contains some number values of year, month, hours etc. So just by looking at naive date we cannot actually understand the time. There comes aware datetime which contains timezone info.

>> now
datetime.datetime(2017, 5, 12, 21, 46, 16, 909983)
>>> now.isoformat()
'2017-05-12T21:46:16.909983'
>>> aware_now = pytz.timezone('Asia/Kolkata').localize(now)
>>> aware_now
datetime.datetime(2017, 5, 12, 21, 46, 16, 909983, tzinfo=<DstTzInfo 'Asia/Kolkata' IST+5:30:00 STD>

Pytz provides timezone object which takes string argument for timezone which has localize method which adds timezone info to the datetime object. Hence now aware datetime has timezone info too. Now if we print the time.

>>> aware_now.isoformat()
'2017-05-12T21:46:16.909983+05:30

We get the +05:30 extra string at the end which gives timezone info. +05:30 means the timezone is 5 hrs and 30 mins ahead of UTC timezone. The comparison between datetimes can be made between naive-naive and aware-aware. If we try to compare between naive and aware,

>>> now < aware_now
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't compare offset-naive and offset-aware datetimes

>>> now2 = datetime.datetime.now()
>>> now2
datetime.datetime(2017, 5, 15, 9, 44, 25, 990666)
>>> now < now2
True
>>> aware_now.tzinfo
<DstTzInfo 'Asia/Kolkata' IST+5:30:00 STD>

tzinfo carries timezone info of the datetime object. We can make aware date to unaware by method replacing tzinfo to None.

>>> unaware_now = aware_now.replace(tzinfo=None)
>>> unaware_now.isoformat()
'2017-05-12T21:46:16.909983'

Formating datetime is done by mostly these two methods. One of which takes string format in which the result is required and another returns datetime in iso format.

>>> now.strftime('%Y-%m-%dT%H:%M:%S%z')
'2017-05-12T21:46:16'
>>> aware_now.strftime('%Y-%m-%dT%H:%M:%S%z')
'2017-05-12T21:46:16+0530'

>>> now.time().isoformat()
'21:46:16.909983'
>>> now.date().isoformat()
'2017-05-12'

>>> now_in_brazil_east = datetime.datetime.now(pytz.timezone('Brazil/East'))
>>> now_in_brazil_east.isoformat()
'2017-05-15T06:49:51.311012-03:00'

We can pass timezone argument to the now method to get current time in the passed timezone. But the care must be taken as this will use the timezone setting of the machine on which code is running to calculate the time at the supplied timezone.

Application

In the FOSSASIA’s Open Event Project, date-time is taken from the user along with the timezone like in one of the example shown below.

Date Time Getter Open Event
Date Time Getter in Create Event Step 1 Open Event Front End

This is part of the event creation page where user has to provide date-time along with the timezone choice. At back-end the date-time and timezone are stored separately in the database. The event model looks like

class Event(db.Model):
 """Event object table"""
 ...
 start_time = db.Column(db.DateTime, nullable=False)
 end_time = db.Column(db.DateTime, nullable=False)
 timezone = db.Column(db.String, nullable=False, default="UTC")
 ...

The comparison between the stored date-time info with the real time cannot be done directly. Since the timezones of the both times need to be considered along with the date-time values while comparison. Likewise there is one case in the project code where tickets are filtered based on the start time.

def get_sales_open_tickets(event_id, event_timezone='UTC'):
  tickets = Ticket.query.filter(Ticket.event_id == event_id).filter(
      Ticket.sales_start <= datetime.datetime.now(pytz.timezone(event_timezone)).replace(tzinfo=None)).filter(
      Ticket.sales_end >= datetime.datetime.now(pytz.timezone(event_timezone)).replace(tzinfo=None))
  …

In this case, first current time is found out using timezone method in the timezone which is stored as a separate data field in the database. Since comparison cannot be done between aware and naive date-time. Hence once current date-time is found out in the user’s timezone, it is made naive using replace method which makes the aware date-time into naive again. Hence can be compared with the naive date-time stored already.

Debugging Using Stetho in Open Event Android

The Open Event Android project helps event organizers to generator Apps (apk format) for their events/conferences by providing api end point or zip generated using Open Event server. In this android app data is fetched from the internet using network calls to the Open Event server and the data of the event is stored in a database. It is difficult to debug an app with so many network calls and database connectivity. A way to approach this is using  Stetho which is very helpful tool for debugging an app which deals with network calls and database.  

Stetho is Facebook’s open source project works as debug bridge for android applications which gives powerful Chrome Developers Tools for debugging android applications using Chrome desktop browser.

What can you do using stetho ?

  • Analyze network traffic
  • Inspect elements(layouts/views)  
  • View SQLite database in table format
  • Run queries on SQLite database
  • View shared preference and edit it
  • Manipulate android app from command line

Setup

1. Add Gradle dependency

To add stetho in your app add ‘com.facebook.stetho:stetho:1.5.0’ dependency in your app  module’s build.gradle file. This dependency is strictly required.

dependencies{
    compile 'com.facebook.stetho:stetho:1.5.0'
}

For network inspection add one of the following dependency according to which you will be using

'com.facebook.stetho:stetho-okhttp:1.5.0'
'com.facebook.stetho:stetho-okhttp3:1.5.0'
'com.facebook.stetho:stetho-urlconnection:1.5.0'

2. Initialize stetho

Initialize stetho in class MyApplication which extends Application class by overriding  onCreate() method. Make sure you have added MyAppication in manifest.

public class MyApplication extends Application {
    public void onCreate() {
        super.onCreate()
        Stetho.initializeWithDefaults(this);
    }
}

Stetho.initializeWithDefaults(this) initializes stetho with defaults configuration(without network inspection and more). It will be able to debug database.

Manifest file

<Application   android:name=”.MyApplication    …   />

For enabling network inspection add StethoInterceptor  in OkHttpClient

new OkHttpClient.Builder()
    .addNetworkInterceptor(new StethoInterceptor())
    .build();

Using Chrome Developer Tools

1. Open Inspect window

Run stetho initialized app on device or emulator then start Google chrome and type chrome://inspect. You will see you device with your app’s package name. Click on “inspect”2. Network Inspection

Go to Network tab. It will show all network calls. With almost all info like url(path), method, returned status code, returned data type, size, time taken etc.

You can also see preview of image and preview, response of returned json data by clicking on Name.

3. SQLite Database

Go to “Resources”  tab and select “Web SQL”. You will see database file(.db). By clicking on database file you will see all tables in that database file. And by clicking on table name you will see data in row-column format for that table.

4. Run queries on SQLite database

Same as above go to “Resources”  tab and select “Web SQL”. You will see database file(.db). By clicking on database file you will see console on right side, where you can run queries on SQLite database. Example,

SELECT * FROM tracks ;

5. Shared Preferences Inspection

Go to “Resources”  tab and select “Local Storage”. You will show all files that your app used to save key-value pairs in shared preference and by clicking on file you will see all key-value pairs.

6. Element(View/Layout) Inspection

Go to “Elements” tab. You will see top layout/view in view hierarchy. By clicking it you will see child layout/view of that layout/view. On hover on layout/view you view will be inspected in your device/emulator.

 

In this blog the most important have been put forward, but there are still  some nice stuff available,like:

  • An integration with JavaScript Console : Enables JavaScript code execution that can interact with the application
  • Dumpapp  : It allows an integration higher than the Developer Tools, enabling the development of custom plugins.

By now, you must have realized that stetho can significantly improve your debugging experience. To learn more about stetho, refer to