Filtering List with Search Manager in Connfa Android App

It is a good practice to provide the facility to filter lists in Android apps to improve the user experience. It often becomes very unpleasing to scroll through the entire list when you want to reach a certain data point. Recently I modified Connfa app to read the list of speakers from the Open Event Format. In this blog I describe how to add filtering facility in lists with Search Manager.

First, we declare the search menu so that the widget appears in it.

<?xml version="1.0" encoding="utf-8"?>
<menu xmlns:android="http://schemas.android.com/apk/res/android">
    <item android:id="@+id/search"
        android:title="Search"
        android:icon="@drawable/search"
        android:showAsAction="collapseActionView ifRoom"
        android:actionViewClass="android.widget.SearchView" />
</menu>

In above menu item the collapseActionView attribute allows your SearchView to expand to take up the whole action bar and collapse back down into a normal action bar item when not in use. Now we create the SearchableConfiguration which defines how SearchView behaves.

<?xml version="1.0" encoding="utf-8"?>
<searchable
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:label="@string/app_name"
    android:hint="Search friend">
</searchable>

Also add this to the activity that will be used with <meta-data> tag in the manifest file. Then associate searchable configuration with the SearchView in the activity class

@Override
public boolean onCreateOptionsMenu(Menu menu) {
    MenuInflater inflater = getMenuInflater();
    inflater.inflate(R.menu.search_menu, menu);

    SearchManager searchManager = (SearchManager)
                            getSystemService(Context.SEARCH_SERVICE);
    searchMenuItem = menu.findItem(R.id.search);
    searchView = (SearchView) searchMenuItem.getActionView();

    searchView.setSearchableInfo(searchManager.
                            getSearchableInfo(getComponentName()));
    searchView.setSubmitButtonEnabled(true);
    searchView.setOnQueryTextListener(this);

    return true;
}

Implement SearchView.OnQueryTextListener in activity, need to override two new methods now

@Override
public boolean onQueryTextSubmit(String searchText) {
  
  return true;
}

@Override
public boolean onQueryTextChange(String searchedText) {

   if (mSpeakersAdapter != null) {
       lastSearchRequest = searchedText;
       mSpeakersAdapter.getFilter().filter(searchedText);
   }
   return true;
}

Find the complete implementation here. In the end it will look like this,

 

References

Android Search View documentation – https://developer.android.com/reference/android/widget/SearchView.html

Using Data Access Object to Store Information

We often need to store the information received from the network to retrieve that later. Although we can store and read data directly but by using data access object to store information enables us to do data operations without exposing details of the database. Using data access object is also a best practice in software engineering. Recently I modified Connfa app to store the data received in Open Event format. In this blog, I describe how to use data access object.

The goal is to abstract and encapsulate all access to the data and provide an interface. This is called Data Access Object pattern. In a nutshell, the DAO “knows” which data source (that could be a database, a flat file or even a WebService) to connect to and is specific for this data source. It makes no difference in applications when it accesses a relational database or parses xml files (using a DAO). The DAO is usually able to create an instance of a data object (“to read data”) and also to persist data (“to save data”) to the datasource.

Consider the example from Connfa app in which get the tracks from API and store them in SQL database. We use DAO to create a layer between model and database. Where AbstractEntityDAO is an abstract class which have the functions to perform CRUD operation. We extend it to implement them in our DAO model. Here is TrackDAO structure,

public class TrackDao extends AbstractEntityDAO<Track, Long> {

    public static final String TABLE_NAME = "table_track";

    @Override
    protected String getSearchCondition() {
        return "_id=?";
    }
    
    ...
}

Find the complete class to see the detailed methods to implement search conditions, get key columns, create instance etc.  here.

Here is a general method to get the data from the database. Where getFacade() for the given layer element, this method returns the requested facade object to represent the passed in layer element.

public List<ClassToSave> getAllSafe() {
   ILAPIDBFacade facade = getFacade();
   try {
       facade.open();
       return getAll();

   } finally {
       facade.close();
   }
}

Now we can create an instance to use these methods instead of directly using SQL operations. This functions gets the data and sort it accordingly.

private TrackDao mTrackDao;
 public List<Track> getTracks() {
   List<Track> tracks = mTrackDao.getAllSafe();
   Collections.sort(tracks, new Comparator<Track>() {
       @Override
       public int compare(Track track, Track track2) {
           return Double.compare(track.getOrder(), track2.getOrder());
       }
   });
   return tracks;
}

References:

 

How User Event Roles relationship is handled in Open Event Server

Users and Events are the most important part of FOSSASIA‘s Open Event Server. Through the advent and upgradation of the project, the way of implementing user event roles has gone through a lot many changes. When the open event organizer server was first decoupled to serve as an API server, the user event roles like all other models was decided to be served as a separate API to provide a data layer above the database for making changes in the entries. Whenever a new role invite was accepted, a POST request was made to the User Events Roles table to insert the new entry. Whenever there was a change in the role of an user for a particular event, a PATCH request was made. Permissions were made such that a user could insert only his/her user id and not someone else’s entry.

def before_create_object(self, data, view_kwargs):
        """
        method to create object before post
        :param data:
        :param view_kwargs:
        :return:
        """
        if view_kwargs.get('event_id'):
            event = safe_query(self, Event, 'id', view_kwargs['event_id'], 'event_id')
            data['event_id'] = event.id

        elif view_kwargs.get('event_identifier'):
            event = safe_query(self, Event, 'identifier', view_kwargs['event_identifier'], 'event_identifier')
            data['event_id'] = event.id
        email = safe_query(self, User, 'id', data['user'], 'user_id').email
        invite = self.session.query(RoleInvite).filter_by(email=email).filter_by(role_id=data['role'])\
                .filter_by(event_id=data['event_id']).one_or_none()
        if not invite:
            raise ObjectNotFound({'parameter': 'invite'}, "Object: not found")

    def after_create_object(self, obj, data, view_kwargs):
        """
        method to create object after post
        :param data:
        :param view_kwargs:
        :return:
        """
        email = safe_query(self, User, 'id', data['user'], 'user_id').email
        invite = self.session.query(RoleInvite).filter_by(email=email).filter_by(role_id=data['role'])\
                .filter_by(event_id=data['event_id']).one_or_none()
        if invite:
            invite.status = "accepted"
            save_to_db(invite)
        else:
            raise ObjectNotFound({'parameter': 'invite'}, "Object: not found")


Initially what we did was when a POST request was sent to the User Event Roles API endpoint, we would first check whether a role invite from the organizer exists for that particular combination of user, event and role. If it existed, only then we would make an entry to the database. Else we would raise an “Object: not found” error. After the entry was made in the database, we would update the role_invites table to change the status for the role_invite.

Later it was decided that we need not make a separate API endpoint. Since API endpoints are all user accessible and may cause some problem with permissions, it was decided that the user event roles would be handled entirely through the model instead of a separate API. Also, the workflow wasn’t very clear for an user. So we decided on a workflow where the role_invites table is first updated with the particular status and after the update has been made, we make an entry to the user_event_roles table with the data that we get from the role_invites table.

When a role invite is accepted, sqlalchemy add() and commit() is used to insert a new entry into the table. When a role is changed for a particular user, we make a query, update the values and save it back into the table. So the entire process is handled in the data layer level rather than the API level.

The code implementation is as follows:

def before_update_object(self, role_invite, data, view_kwargs):
        """
        Method to edit object
        :param role_invite:
        :param data:
        :param view_kwargs:
        :return:
        """
        user = User.query.filter_by(email=role_invite.email).first()
        if user:
            if not has_access('is_user_itself', id=user.id):
                raise UnprocessableEntity({'source': ''}, "Only users can edit their own status")
        if not user and not has_access('is_organizer', event_id=role_invite.event_id):
            raise UnprocessableEntity({'source': ''}, "User not registered")
        if not has_access('is_organizer', event_id=role_invite.event_id) and (len(data.keys())>1 or 'status' not in data):
            raise UnprocessableEntity({'source': ''}, "You can only change your status")

    def after_update_object(self, role_invite, data, view_kwargs):
        user = User.query.filter_by(email=role_invite.email).first()
        if 'status' in data and data['status'] == 'accepted':
            role = Role.query.filter_by(name=role_invite.role_name).first()
            event = Event.query.filter_by(id=role_invite.event_id).first()
            uer = UsersEventsRoles.query.filter_by(user=user).filter_by(event=event).filter_by(role=role).first()
            if not uer:
                uer = UsersEventsRoles(user, event, role)
                save_to_db(uer, 'Role Invite accepted')


In the above code, there are two main functions –
before_update_object which gets executed before the entry in the role_invites table is updated, and after_update_object which gets executed after.

In the before_update_object, we verify that the user is accepting or rejecting his own role invite and not someone else’s role invite. Also, we ensure that the user is allowed to only update the status of the role invite and not any other sensitive data like the role_name or email. If the user tried to edit any other field except status, then an error is shown to him/her. However if the user has organizer access, then he/she can edit the other fields of the role_invites table as well. The has_access() helper permission function helps us ensure the permission checks.

In the after_update_object we make the entry to the user event roles table. In the after_update_object from the role_invite parameter we can get the exact values of the newly updated row in the table. We use the data of this role invite to find the user, event and role associated with this role. Then we create a UsersEventsRoles object with user, event and role as parameters for the constructor. Then we use save_to_db helper function to save the new entry to the database. The save_to_db function uses the session.add() and session.commit() functions of flask-sqlalchemy to add the new entry directly to the database.

Thus, we maintain the flow of the user event roles relationship. All the database entries and operation related to users-events-roles table remains encapsulated from the client user so that they can use the various API features without thinking about the complications of the implementations.

 

Reference:

Persistence Layer in Open Event Organizer Android App

Open Event Organizer is an Event Managing Android App with the core features of Attendee Check In by QR Code Scan and Data Sync with the Open Event API Server. As an event can be large, so the app will be dealing with a large amount of a data. Hence to avoid repetitive network requests for fetching the data, the app maintains a local database containing all the required data and the database is synced with the server. Android provides android.database.sqlite package which contains the API needed to use the database on the Android. But it is really not a good practice to use the sqlite queries everywhere in the app. So there comes a persistence layer. A persistence layer works between the database and the business logic. Open Event Organizer uses Raizlabs’s DbFlow, an ORM based Android Database Library for the same. I will be talking about its implementation through the app in this blog.

First of all, you declare the base class of the database which is used to create the database by Android for the app. You declare all the base constants here. The class looks like:

@Database(
   name = OrgaDatabase.NAME,
   version = OrgaDatabase.VERSION,
   ...
)
public class OrgaDatabase {
   public static final String NAME = "orga_database";
   public static final int VERSION = 2;
   ...
}

OrgaDatabase.java
app/src/main/java/org/fossasia/openevent/app/data/db/configuration/OrgaDatabase.java

Initialise the database in the Application class using FlowManager provided by the library. Choose the Application class to do this to ensure that the library finds the generated code in the DbFlow.

FlowManager.init(
   new FlowConfig.Builder(context)
       .addDatabaseConfig(
           new DatabaseConfig.Builder(OrgaDatabase.class)
           ...
           .build()
       )
       .build());

OrgaApplication.java
app/src/main/java/org/fossasia/openevent/app/OrgaApplication.java

The database is created now. For tables creation, DbFlow uses model classes which must be annotated using the annotations provided by the library. The basic annotations are – @Table, @PrimaryKey, @Column, @ForeignKey etc.

For example, the Attendee class in the app looks like:

@Table(database = OrgaDatabase.class)
public class Attendee ... {

   @PrimaryKey
   public long id;

   @Column
   public boolean checkedIn;
   ...
   ...
   @ForeignKey(
       onDelete = ForeignKeyAction.CASCADE,
       onUpdate = ForeignKeyAction.CASCADE)
   public Order order;
   ...
}

Attendee.java
app/src/main/java/org/fossasia/openevent/app/data/models/Attendee.java

This will create a table named attendee with the columns and relationships annotated. Now comes the part of accessing data from the database. Open Event App uses RxJava’s support to the DbFlow library which enables async data accessing. The getItems method from DataBaseRepository looks like:

public <T> Observable<T> getItems(Class<T> typeClass, SQLOperator... conditions) {
   return RXSQLite.rx(SQLite.select()
       .from(typeClass)
       .where(conditions))
       .queryList()
       .flattenAsObservable(items -> items);
}

 

The method returns an observable emitting the items from the result. For data saving, the method looks like:

DatabaseDefinition database = FlowManager.getDatabase(OrgaDatabase.class);
FastStoreModelTransaction<T> transaction = FastStoreModelTransaction
   .insertBuilder(FlowManager.getModelAdapter(itemClass))
   .addAll(items)
   .build();
database.executeTransaction(transaction);

 

And for updating data, the method looks like:

ModelAdapter<T> modelAdapter = FlowManager.getModelAdapter(classType);
modelAdapter.update(item);

DatabaseRepository.java
app/src/main/java/org/fossasia/openevent/app/data/db/DatabaseRepository.java

DbFlow provides DirectModelNotifier which is used to get notified of the database change anywhere in the app. Open Event App uses PublishSubjects to send notifications on database change event. The implementation of the DatabaseChangeListener in the app looks like:

public class DatabaseChangeListener<T> ... {
   private PublishSubject<ModelChange<T>> publishSubject = PublishSubject.create();
   private DirectModelNotifier.ModelChangedListener<T> modelModelChangedListener;
   ...
   public void startListening() {
       modelModelChangedListener = new DirectModelNotifier.ModelChangedListener<T>() {
           @Override
           public void onTableChanged(@Nullable Class<?> aClass, @NonNull BaseModel.Action action) {
               // No action to be taken
           }
           @Override
           public void onModelChanged(@NonNull T model, @NonNull BaseModel.Action action) {
               publishSubject.onNext(new ModelChange<>(model, action));
           }
       };
       DirectModelNotifier.get().registerForModelChanges(classType, modelModelChangedListener);
   }
   ...
}

DatabaseChangeListener.java
app/src/main/java/org/fossasia/openevent/app/data/db/DatabaseChangeListener.java

The class is used in the app to get notified of the data change and to update the required local data fields using data from item emitted by the publishSubject of the class. This is used in the app where same data is accessed at more than one places. For example, There are two fragments – AttendeesFragment and AttendeeCheckInFragment from which an attendee’s check in status is toggled. So when the status is toggled from AttendeeCheckInFragment, the change must be updated in the AttendeesFragment’s attendees list. This is carried out using DatabaseChangeListener in the AttendeesPresenter which provides attendees list to the AttendeesFragment. And on the change in the attendee’s check in status, AttendeePresenter’s attendeeListener listens for the change and update the attendee in the list accordingly.

Links:
1. Raizlabs’s DbFlow , an ORM Android Database Library Github Repo Link
2. DbFlow documentation
3. Android database managing API android.database.sqlite

Ticket Ordering or Positioning (back-end)

One of the many feature requests that we got for our open event organizer server or the eventyay website is ticket ordering. The event organizers wanted to show the tickets in a particular order in the website and wanted to control the ordering of the ticket. This was a common request by many and also an important enhancement. There were two main things to deal with when ticket ordering was concerned. Firstly, how do we store the position of the ticket in the set of tickets. Secondly, we needed to give an UI in the event creation/edit wizard to control the order or position of a ticket. In this blog, I will talk about how we store the position of the tickets in the backend and use it to show in our public page of the event.

Continue reading “Ticket Ordering or Positioning (back-end)”

Doing a table join in Android without using rawQuery

The Open Event Android App, downloads data from the API (about events, sessions speakers etc), and saves them locally in an SQLite database, so that the app can work even without internet connection.

Since there are multiple entities like Sessions, Speakers, Events etc, and each Session has ids of speakers, and id of it’s venue etc, we often need to use JOIN queries to join data from two tables.

 

Android has some really nice SQLite helper classes and methods. And the ones I like the most are the SQLiteDatabase.query, SQLiteDatabase.update, SQLiteDatabase.insert ones, because they take away quite a bit of pain for typing out SQL commands by hand.

But unfortunately, if you have to use a JOIN, then usually you have to go and use the SQLiteDatabase.rawQuery method and end up having to type your commands by hand.

But but but, if the two tables you are joining do not have any common column names (actually it is good design to have them so – by having all column names prefixed by tablename_ maybe), then you can hack the usual SQLiteDatabase.query() method to get a JOINed query.

Now ideally, to get the Session where speaker_id was 1, a nice looking SQL query should be like this –

SELECT * FROM speaker INNER JOIN session
ON speaker_id = session_speaker_id
WHERE speaker_id = 1

Which, in android, can be done like this –

String rawQuery = "SELECT * FROM " + SpeakerTable.TABLE_NAME + " INNER JOIN " + SessionTable.TABLE_NAME
        + " ON " + SessionTable.EXP_ID + " = " + SpeakerTable.ID
        + " WHERE " + SessionTable.ID + " = " +  id;
Cursor c = db.rawQuery(
        rawQuery,
        null
);

But of course, because of SQLite’s backward compatible support of the primitive way of querying, we turn that command into

SELECT *
FROM session, speaker
WHERE speaker_id = session_speaker_id AND speaker_id = 1

Now this we can write by hacking the terminology used by the #query() method –

Cursor c = db.query(
        SessionTable.TABLE_NAME + " , " + SpeakerTable.TABLE_NAME,
        Utils.concat(SessionTable.PROJECTION, SpeakerTable.PROJECTION),
        SessionTable.EXP_ID + " = " + SpeakerTable.ID + " AND " + SpeakerTable.ID + " = " +  id,
        null,
        null,
        null,
        null
);

To explain a bit, the first argument String tableName can take table1, table2 as well safely, The second argument takes a String array of column names, I concatenated the two projections of the two classes. and finally, put by WHERE clause into the String selection argument.

You can see the code for all database operations in the android app here  https://github.com/fossasia/open-event-android/blob/master/android/app/src/main/java/org/fossasia/openevent/dbutils/DatabaseOperations.java

Implementing Admin Trash in Open Event

So last week I had the task of implementing a trash system for the Admin. It was observed that sometimes a user may delete an item and then realize that the item needs to be restores. Thus a trash system works well in this case. Presently the items that are being moved to the trash are:

  • Deleted Users
  • Deleted Events
  • Deleted Sessions

So it works like this. I added a column in_trash to the tables User, Event and Sessions to mark whether the item is in the trash or not

in_trash = db.Column(db.Boolean, default=False)

So depending on whether the value is True or False the item will be in the trash of the admin. Thus for a normal user on deleting an event, user or session a message would flash that the item is deleted and the item would not be shown in the table list of the user. However it would not be deleted from the database.

trash4.png

trash5.png

Thus for the user the item is deleted. The item’s in_trash property is set to True and it gets moved to the trash. The items are displayed in the “Deleted Items” section of the Admin panel

trash1trash2trash3

The items deleted are displayed in the trash and as soon as they deleted in the trash they are deleted from the database permanently. A message will flash for the Admin when it is deleted

trash11

trash10.png

Thus the trash is implemented. 🙂

Two more things are left:

  • To restore items from trash
  • To automatically delete the items in trash after an inactivity of 30 days

This will soon be implemented 🙂

Open-Event Permissions System and integrating it with decorators

All the large scale applications require a permissions system. Thus we also implemented a permissions system in our open-event organization server. It consists of certain pre-decided roles:

  1. Super-Admin
  2. Admin
  3. Organizer
  4. Co organizer
  5. Track organizer
  6. Anonymous user

Now we had to decide the permissions which each role would have. Hence we created a documentation regarding what URLs can be accessed by each role. We developed a list of services which the roles could use their permissions to access:

  1. Tracks
  2. Microlocations
  3. Speakers
  4. Sessions
  5. Sponsors

Thus the final step was to implement the permissions system to the appropriate views or URLs. Here comes the power of Flask decorators . I created a individual decorators @is_organizer, @is_admin, @is_super_admin etc… to check the respective roles. I created one main decorator @can_access to see whether the role can access the particular URL or view function

decorator

So in the above decorator I have simply take in the url and check whether it has ‘create’, ‘edit’ or ‘delete’ words in it. Depending on that the control goes in the particular IF statement. Now once it is decided what operation is being performed it checks what service is being accessed by the user. For example: if the operation is edit then it will check whether the service being edited is an event, session, sponsor etc…

Similar checks are performed by each operation. A check is performed of the request.url to see whether the string for that service is present in it. After it knows what service is being accessed its just a matter of using the CRUD functions of user table to check if the role accessing the resource has the requested permission using the functions:

  1. user.can_create()
  2. user.can_read()
  3. user.can_update()
  4. user.can_delete()

After this its just a matter of adding the decorator to each of the view functions and the system is implemented.  🙂