KISS Datatable

Recenlty I’ve faced a problem with sorting columns in Datatable.

What is Datatable?

Datatable is a plug-in for Jquery library. It provides a lot of features like pagination, quick search or multi-column ordering. Besides, you can develop easily Bootstrap or Foundation ui css styles. There are also more other option but It doesn’t make sense to list it here, because you can visit their site and you can read clearly documentation. On Datatable website you can see a lot of examples. First of them shows how to improve your ordinary table to awesome and rich of features table. One function changes everything, It’s fantastic!  

$('#myTable').DataTable();

Returning to my problem which I’ve faced, as I told it was problem related to sorting column in table.

I know sorting is a trivial thing. I hope that everyone knows it 🙂 Sorting by a date is also implemented in a datatable library. So everything is clear when we don’t change date format to human readable format. I mean something like this ‘3 hours ago’, ‘1 year ago’.

When Open Event team tested how datatable manages ordering columns in that format it didn’t work. It’s quite hard to sort by that format, So I’ve invented an idea. Surely you are wondering what i’ve invented. I’ve postponed my minds about sort by this values. It can direct to overwork. When I thought about it, weird ideas came to my mind, a lots of conditions, If statements… Therefore I’ve resigned from this. I’ve used KISS principle. KISS means ‘keep it simple stupid’. I like it!

Therefore that sorting is implemented on frontend side. I’ve decided not to display human readable date format at the beginning. I find  all dates which have format “YYYY-MM-DD HH:mm” then I replace that format to human readable format. So it’s very quick and comfortable, and doesn’t require a lot conditions to write. Of course I’ve tried to implement it in Datatable library. I suppose that it would  require more effort than it’s now.

Below You can see great function which changes a date in frontend side but does not change data in a datatable. So sorting process takes place in a datatable using format  “YYYY-MM-DD HH:mm” but user see human readable format. Isn’t it awesome?!

function change_column_time_to_humanize_format(datatable_name, column_id) {
  $(datatable_name).each(function( key, value ) {
    $(value).children().each(function( key1, value2 ) {
       if(key1 === column_id ){
          var current_value = $(value2).text().slice(0, 16);
          var changed_value = moment(current_value, "YYYY-MM-DD hh:mm").fromNow()
          var isValid = moment(current_value, "YYYY-MM-DD HH:mm", true).isValid()
          if (changed_value !== current_value && isValid === true){
              $(value2).text(changed_value)
          }
      }
  });
});

 

Python code examples

I’ve met many weird examples of  behaviour in python language while working on Open Event project. Today I’d like to share some examples with you. I think this knowledge is necessary, if you’d like to increase a  bit your knowledge in python area.

Simple adding one element to python list:

def foo(value, x=[]):
  x.append(value)
  return x

>>> print(foo(1))
>>> print(foo(2))
>>> print(foo(3, []))
>>> print(foo(4))

OUTPUT

[1] 
[1, 2] 
[3]
[1, 2, 4]

First output is obvious, but second not exactly. Let me explain it, It happens because x(empty list) argument is only evaluated once, So on every call foo(), we modify that list, appending a value to it. Finally we have [1,2, 4] output. I recommend to avoid mutable params as default.

Another example:

Do you know which type it is?

>>> print(type([ el for el in range(10)]))
>>> print(type({ el for el in range(10)}))
>>> print(type(( el for el in range(10))))

Again first and second type are obvious <class ‘list’>, <class ‘set’>. You may  think that last one should return type tuple but it returns a generator <class ‘generator’>.

Example:

Do you think that below code returns an exception?

list= [1,2,3,4,5]
>>> print(list [8:])

If you think that above expression returns index error you’re wrong. It returns empty list [].

Example funny boolean operators

>>> 'c' == ('c' or 'b')
True
>>> 'd' == ('a' or 'd')
False
>>> 'c' == ('c' and 'b')
False 
>>> 'd' == ('a' and 'd')
True

You can think that that OR and AND operators are broken.

You have to know how python interpreter behaves while looking for OR and AND operators.

So OR Expression takes the first statement and checks if it is true. If the first statement is true, then Python returns object’s value without checking second value. If first statement is false interpreter checks second value and returns that value.

AND operator checks if first statement is false, the whole statement has to be false. So it returns first value, but if first statement is true it checks second statement and returns second value.

Below i will show you how it works

>>> 'c' == ('c' or 'b')
>>> 'c' == 'c'
True
>>> 'd' == ('a' or 'd')
>>> 'd' == 'a'
False
>>> 'c' == ('c' and 'b')
>>> 'c' == 'b'
False 
>>> 'd' == ('a' and 'd')
>>> 'd' == 'd'
True

I hope that i have explained you how the python interpreter checks OR and AND operators. So know above examples should be more understandable.

Twitter Oauth

Oauth_logo.svg.png

What is Oauth?

It’s an open protocol which allows to secure an authorization in a simple and standard method from web, mobile and desktop applications.Facebook, Google Twitter, Github and more web services use this protocol to authenticate user. Using Oauth is very convenient, because it delegates user authentication to the service which host user account. It allows us to get resources from another web service without giving any login or password. If you have a service and want to prepare a authentication via Twitter, the best solution is to use OAuth. Recently Open Event team met a problem in an user profile page. We’d like to automatically fill information about user. Of course, to solve it we use Oauth protocol, to authenticate with Twitter After a three-steps authentication we can get name and profile picture.If you need another information from Twitter profile like recent tweets or followers’ list. You have to visit Twitter API site to see more samples of resource which you can get

How do Open event team implement communication between Orga-server and Twitter?

All services have a very similar flow. Below i will show you how it looks in our case.

Before starting you need to create your own twitter app. You can create app in Twitter apps site https://apps.twitter.com/. If  create an app you will see a CONSUMER KEY and CONSUMER SECRET KEY which shouldn’t be human-readable, so remember not to share these keys.

Below example shows how to get basic information about twitter profile

We use oauth2 python library https://github.com/joestump/python-oauth2

consumer = oauth2.Consumer(key=TwitterOAuth.get_client_id(),

                          secret=TwitterOAuth.get_client_secret())

client = oauth2.Client(consumer)

TwitterOAuth.get_client_id() CONSUMER KEY

TwitterOAuth.get_client_secret()  – CONSUMER SECRET KEY

Then we send GET request to request_token endpoint to get oauth_token

client.request('https://api.twitter.com/oauth/request_token', "GET")
Response: oauth_token=Z6eEdO8MOmk394WozF5oKyuAv855l4Mlqo7hhlSLik&
oauth_token_secret=Kd75W4OQfb2oJTV0vzGzeXftVAwgMnEK9MumzYcM&
oauth_callback_confirmed=true

Next step is to redirect user to Twitter Authentication site

twitter-oauth.png

You can see in an url a redirect_uri. So after sign in Client will get a callback from Twitter with oauth_verifier and oauth_token params

https://api.twitter.com/oauth/authenticate?oauth_token=RcYGqAAAAAAAwdbbAAABVoM1UMo&oauth_token_secret=wZfpPpCugAmdF3AuohEnvBTRxdCmllxu&oauth_callback_confirmed=true&redirect_uri=http://open-event-dev.herokuapp.com/tCallback

The last step is to get an access token. If we have an oauth_verifier and an oauth_token it’s pretty easy

def get_access_token(self, oauth_verifier, oauth_token):

   consumer = self.get_consumer()

   client = oauth2.Client(consumer)

   return client.request(

       self.TW_ACCESS_TOKEN_URI + 'oauth_verifier=' + oauth_verifier + 
       "&oauth_token=" + oauth_token, "POST")

Where TW_ACCESS_TOKEN_URI is https://api.twitter.com/oauth/access_token

Final step is to get our user information

resp, content = client.request("https://api.twitter.com/1.1/users/show.json?
                               screen_name=" + access_token["screen_name"] +
                               "&user_id=" + access_token["user_id"] , "GET")

user_info = json.loads(content)

In an user_info variable you can get a profile picture or a profile name.

Summarizing, oauth protocol is very secure and easy to use by developer. At the beginning an oauth flow can seem to be a little hard to  understand but if you spend some time trying tp understand it, everything becomes easier.  And it’s secured. because you don’t need to store a login or a password, and an access token has an expired time. This is the main feature of Oauth protocol.

GET and POST requests

If you wonder how to get or update page resource, you have to read this article.

It’s trivial if you have basic knowledge about HTTP protocol. I’d like to get you little involved to this subject.

So GET and POST are most useful methods in HTTP protocol.

What is HTTP?

Hypertext transfer protocol – allow us to communicate between client and server side. In Open Event project we use web browser as client and for now we use Heroku for server side.

Difference between GET and POST methods

GET – it allows to get data from specified resources

POST – it allows to submit new data to specified resources for example by html form.

GET samples:

For example we use it to get details about event

curl http://open-event-dev.herokuapp.com/api/v2/events/95

Response from server:

Of course you can use this for another needs, If you are a poker player I suppose that you’d like to know how many percentage you have on hand.

curl http://www.propokertools.com/simulations/show?g=he&s=generic&b&d&h1=AA&h2=KK&h3&h4&h5&h6&_

POST samples:

curl -X POST https://example.com/resource.cgi

You can often find this action in a contact page or in a login page.

How does request look in python?

We use Requests library to communication between client and server side. It’s very readable for developers. You can find great documentation  and a lot of code samples on their website. It’s very important to see how it works.

>>> r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
>>> r.status_code
200

I know that samples are very important, but take a look how Requests library fulfils our requirements in 100%. We have decided to use it because we would like to communicate between android app generator and orga server application. We have needed to send request with params(email, app_name, and api of event url) by post method to android generator resource. It executes the process of sending an email – a package of android application to a provided email address.

data = {
    "email": login.current_user.email,
    "app_name": self.app_name,
    "endpoint": request.url_root + "api/v2/events/" + str(self.event.id)
}
r = requests.post(self.app_link, json=data)

 

Programmer principles

As programmers we develop our programming skills and learn something every single day. We write code and solve many troubles. But is our aim to simply write code? I am sure it is not. I think writing code just for doing it is not interesting, and it’s definitely not Open Event team’s objective. Personally, I like reading code like a poem. We should always try to eliminate bad practises and ugly code. There are a few principles how to do it. Let me share them with you now.

SOLID principle

SOLID  is a mnemonic acronym introduced by Michael Feathers, and it simply means five basic principles of object oriented programming. These principles, when applied together, make it more likely that a programmer will create a system that is easy to maintain and extend over time. They are guidelines that can be applied while working on software to remove code smells by causing the programmer to refactor the software’s source code.  It is also a part of an overall strategy of agile. So, here they are:

S – Single responsibility principle

This principle means that there should never be more than one reason for a class to change.

In other words, a class should have only one potential change in a software’s specification. You should not add everything into your class. The best practise here is to check if the logic you are introducing should be in this class or not. Responsibility is the heart of this principle, so to rephrase there should never be more than one responsibility per class. Use layers for a help. And try to divide big classes into smaller ones.

O – Open/closed principle

Software entities like classes, module and functions should be open for extension, but closed for modification.

All of them should be private by default.

To make an object behaving differently without modifying it use abstractions, or place behavior(responsibility) in derivative classes. If properties of the abstracted class need to be compared or organized together, another abstraction should handle this. This is the basis of the “keep all object variables private” argument.

L – Liskov substitution principle

Functions that use pointers or references to base classes have to be able to use objects of derived classes without knowing/alerting the correctness of a program

A great example you can find here. If you are using a method defined at a base class upon an abstracted class, the function must be implemented properly on the subtype class. A great example provided here http://williamdurand.fr/2013/07/30/from-stupid-to-solid-code/  you can find below.

“ A rectangle is a plane figure with four right angles. It has a width, and a height. Now, take a look at the following pseudo-code:

rect = new Rectangle();

rect.width  = 10;
rect.height = 20;

assert 10 == rect.width
assert 20 == rect.height

We simply set a width and a height on a Rectangle instance, and then we assert that both properties are correct. So far, so good.

Now we can improve our definition by saying that a rectangle with four sides of equal length is called a square. A square is a rectangle so we can create aSquare class that extends the Rectangle one, and replace the first line above by the one below:

rect = new Square();

According to the definition of a square, its width is equal to its height. Can you spot the problem? The first assertion will fail because we had to change the behavior of the setters in the Square class to fit the definition “

I – Interface segregation principle

Many client-specific interfaces are better than one general-purpose interface.

Implementing methods that you don’t use is not recommended in this way. The idea here is to keep your components focused and try to minimize the dependencies between them. Enforcing that principle gives you low coupling, and high cohesion.

D – Dependency inversion principle

This means that “one should depends upon abstractions, do not depend upon concretions”

Interfaces should depend on other interfaces. Don’t add concrete classes to method signatures of an interface. However, use interfaces in your class methods.

So, we can also say that rather than working with classes that are tight coupled, use interfaces. This reduces dependency on implementation specifics and makes code more reusable.

Why SOLID?

I hope all of you understand the importance of using SOLID principles in your everyday code practise. Finally, let me underline again the main arguments why you should starting following them now. The most important thing is that thanks to them you can create easy to maintain software, then you can reuse your code, and finally it helps you to test easier. Do you need anymore to be  persuaded  to do it? I think it’s that’s crucial advantages and they are enough.

Source:

https://pl.wikipedia.org/wiki/SOLID_(programowanie_obiektowe)

https://scotch.io/bar-talk/s-o-l-i-d-the-first-five-principles-of-object-oriented-design

http://williamdurand.fr/2013/07/30/from-stupid-to-solid-code/

http://www.codeproject.com/Articles/60845/The-S-O-L-I-D-Object-Oriented-Programming-OOP-Prin

Can solving lint bugs be interesting?

Today I am going to present you how we’ve changed monotonous solving bugs into motivating process.

PEP

Most developers need to improve their code quality. To do  that they can use style guide for e.g for Python code (PEP). PEP contains an index of all Python Enhancement Proposals.

Below you can find which logs PEP returned in a command line.

Do you think that this logs’ presentation is  good enough to interest a developer? Will he solve these  thousands of bugs?

Undoubtedly, there are much information about errors and warnings so PEP returns long logs. But developer can not even know how to start solving bugs. And even if she/he finally starts, after each commit he/she needs to run that script again to check if quantity of bugs are increased or decreased. It seems to be endless, exhausting and very monotonous.  Nobody is encouraged to do it.

logi.png

Quality monitoring

Open Event team wants to increase our productivity and code quality. Therefore we use a tool which allow us to check code style, security, duplication complexity and test coverage on every commit. That tool is Codacy and it fulfils our requirements in 100%. It is very helpful because it adds comments to pull requests and enables developer quickly find where a bug is located. It’s very comfortable, because you don’t need to check issues in above awful logs results. Take a look how it looks in Codacy.

-DO NOT MERGE  Ticketing Flow by niranjan94 · Pull Request  1927 · fossasia open event orga server.png

Isn’t it clear? Of course that it’s. Codacy shows in which line issue ocurres and which type of issue it’s.

Awesome statistics dashboard

I’d like to give an answer how you can engage your team to solve issues and make this process more interesting. On the main page codacy tool welcomes you with great statistics about your project.

open event orga server   Codacy   Dashboard

You can see number of issues, category like code complexity, code style, compatibility, documentation, error prone, performance, security and unused code. That params show in which stage of code quality your project is. I think that every developer’s aim is to have the highest code quality and increasing these statistics. But if project has many issues, developer sees only a few changes in project charts.

Define Goals

Recently I’ve discovered how you can motivate yourself more. You can define a goal which you’d like achive. It can be goal of category or goal of file. For example Open Event team has defined goal for a specific file to achieve. If you define small separate goals, you can quicker see the results of your work.

open event orga server_2   Codacy   Goals

On the left sidebar you can find a item which is named “Goals”. In this area you can easily add your projects goals. Everything is user friendly so you shouldn’t have a problem  to create own goals.

How can you get an access to Instagram API?

unnamed.png

First of all you need to know that Instagram API uses OAuth 2.0 protocol. OAuth 2.0 provides a specific authorization flow for web apps, desktop apps and mobile apps. Instagram requires authentication before getting information from their API, but don’t be afraid it’s very simple.

Pre Requirements:

  • Created account in Instagram
  • Registered Client(You can create your own client here)

Instagram Developer Documentation.png

Requirements:

CLIENT_ID -79e1a142dbeabd57a3308c52ad43e31d
CLIENT_SECRET -34a6834081c44c20bd11e0a112a6adg1
REDIRECT_URI - http://127.0.0.1:8001/iCallback

You can get above information from https://www.instagram.com/developer/clients/manage/

CODE - You need to open page https://api.instagram.com/oauth/authorize/?client_id=CLIENT-ID&redirect_uri=REDIRECT-URI&response_type=code
https://api.instagram.com/oauth/authorize/?client_id=79e1a142dbeabd57a3308c52ad43e31d&redirect_uri=http://127.0.0.1:8001/iCallback&response_type=code

You will be redirected to

http://your-redirect-uri?code=CODE

In my case it looks like this:

http://127.0.0.1:8001/iCallback?code=2e122f3d76e8125b8b4982f16ed623c2

Now we have all information to get access token!

 curl -F 'client_id=CLIENT_ID'  
      -F 'client_secret=CLIENT_SECRET' 
      -F 'grant_type=authorization_code'  
      -F 'redirect_uri=REDIRECT_URI' 
      -F 'code=CODE'  https://api.instagram.com/oauth/access_token

if everything is ok you should receive

 { "access_token": "fb2e77d.47a0479900504cb3ab4a1f626d174d2d",
   "user": { "id": "1574083",
   "username": "rafal_kowalski",
   "full_name": "Rafal Kowalski",
   "profile_picture": "..." } } 

In Open Event we used it to get all media from instagram – to use it  as for example a background in an event details’ page

curl 'https://api.instagram.com/v1/users/self/media/recent/?access_token=ACCESS_TOKEN'

 

How can you add a bug?

It’s very simple to start testing, You don’t need any special experience in testing area.To start testing in Open Event project you need to open our web application http://open-event.herokuapp.com/ and you can start.Isn’t it easy? So You should focus on finding as many bugs as possible to provide your users with perfectly working software. If you find a bug you need to describe many details

How can you report a bug to Open Event?

Go to Github page issues, click new issue(green button).

Screen Shot 2016-07-01 at 21.03.45.png

Our Requirements:

Good description – If you found a bug you have to reproduce it, so you have nice background to describe it very well. It’s important because, good issue’s description saves developers time. They don’t need to ask announcer about details of bug. The best description which tester can add is how developer can simply achieve a bug step by step.

Logs – description is sometimes not enough so you need to attach logs which are generated from our server(It’s nice if you have access, if you don’t have ask me)

Pictures – it’s helpful, because we can quickly find where bug is located

Labels – You need to assign Screen Shot 2016-07-01 at 21.26.17.png label to issue

That’s all!

 

How do we work? Agile

It’s not typical team. We don’t meet face-to-face each other on daily basis, one trouble is that we have to work from different time zones, and we have different operating system, the way of working, or even culture. But we are FOSSASIA Open Event team of six amazing developers so it does not discourage us to achieve our goals.

But even if we experience all these things we have to learn how to work together successfully. We are trying, and I think it is becoming better and better every day, the Agile methodology.

So first of all, before coding started, we had prepared user stories in issues. Every issue had been very well documented, and we had divided each issue to smaller ones. It brings us a lot of benefits, because it doesn’t matter where you work. If sub issue is empty you can take it. Due to the fact that issue is very well documented you don’t need to even ask someone what to do. This case saves a lot of time. I know that writing clear issues is very boring and It could seemed not to be beneficial, but it’s. We are still trying to improve our performance and we are looking for new opportunities where and how we can improve our work, without lost quality of our development process.

Then, we have one week sprint (milestones). It helps us to control our work and we know where are we, what have we done, and what is still to do. We know if our speed is enough or we have to move faster. And it also enables us to see our progress.Screen Shot 2016-06-25 at 11.27.08.png

Moreover, we have daily scrums. We answer three following questions – what did you do yesterday, what do you plan to do today and what is preventing you from achieving your goals. Furthermore, we follow continues integration system. We push code to our common Open Event repository on daily basis. It helps us to control our code and fix bugs quickly. I am sure we will continue along this path, and successfully finish to code an amazing Open Event web app.

Migrations make us crazy!

Our Open Event team is not small, so problems with migrations occur very often. I’d like to describe how to solve these common problems and avoid Contributors frustration. Because more of us didn’t know how to solve this problems at the beginning so we wasted a lot of time to find a source of problem.

The most common mistake is that we forget run migration on Heroku. Developer is sometimes surprised because something works for him but he forgets to run migration on server. These small mistakes lead to huge problems and at that time our app throws a lots of bugs related to database. We can often see “Internal server error”.Screen Shot 2016-06-17 at 22.55.25.png So if developer changes table he has to run migration!

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db migrate

But above command Quite often doesn’t solve our problem, because We face other problems while updating DB for example

alembic.util.exc.CommandError: Multiple head revisions are present for given argument ‘head’; please specify a specific target revision, ‘<branchname>@head’ to narrow to a specific head, or ‘heads’ for all heads

this problem is caused by two developers which push code to Github with migrations without merging two heads to achieve one head.

So to solve this problem you only have to know ids of two heads

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db heads

e38935822969 (head)
f55fde3d62b1 (head)

Then you have to merge it

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db merge e38935822969 f55fde3d62b1

Generating /vagrant/migrations/versions/ecb671d1eb4b_.py … done

Upgrade DB

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db upgrade

INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 00ea66754d06 -> d84f976530e1, empty message
INFO [alembic.runtime.migration] Running upgrade d84f976530e1 -> 1b3e4f5f56bd, empty message
INFO [alembic.runtime.migration] Running upgrade 1b3e4f5f56bd -> e38935822969, empty message
INFO [alembic.runtime.migration] Running upgrade e38935822969, f55fde3d62b1 -> ecb671d1eb4b, empty message

And finally run migrations

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db migrate
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.ddl.postgresql] Detected sequence named ‘role_id_seq’ as owned by integer column ‘role(id)’, assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named ‘microlocation_id_seq’ as owned by integer column ‘microlocation(id)’, assuming SERIAL and omitting….