Steemer App update - faster and cleaner with fewer api calls

\n\n

Repository

https://github.com/hispeedimagins/steem

---

steemer is and will remain the first native android app which is and will be forever completely free for the people to use.

Why free?

Because @steemit needs it. A quick way to interact with the blockchain. One that will not eat into the users rewards.

---

New Features

Room Database

In the past I have been using sqllite and doing most of the crud myself. Room is an open source project by Google which takes care of most of the abstraction and the crud leaving us with no more boring and tedious work. Score for humans. This update adds Room db to handle most of the articles leaving us to do the real work.

To implement Room in your app you have to follow simple steps.

First you add it to your build gradle files then add entities for it. You can follow along by looking at the commits.
Entities as you may have guessed are what tell Room of which class to store in a db and what name to give it. You have to annote it with

@Entity(tableName = "follow_tables",indices = [Index(value = ["uniqueName"],unique = true)])

the index tells Room to make the field called _uniqueName unique and also to store it as an index for quick access.

Step two is where we now add the Room db singleton class. This keeps everything sane and in order. The DAO's tell room what query to execute and when. You will also need type converters. Type converters tell Room how to store data which cannot be easily stored in the db. You usually convert it to string or whatever. Some important queries will be discussed later.

Now we make repo classes for each DAO. These repo classes give us an abstraction over the DAO and the room class so our main classes do not have to deal with them. They also have the asynctasks to execute queries on the db as Room does not allow queries on the main thread by default.

After all of this we make View model classes which use the Repo classes giving us a clean api. Now this will seem counter intuitive at first. The viewmodels are under the android architecture classes. These classes are what should be used to access and hold data because they survive lifecycle changes. They do not leak and are handled by the system.

After all of this is when we begin to implement the viemodels classes. To create a viewmodel you execute this

vm = ViewModelProviders.of(this).get(ArticleRoomVM::class.java)

ArticleRoomVM is substituted by your class name. So you do not create it with your context but the system does it for us and handles it without causing problems.
Next you observe on the data you need, observers pause when the system pauses the app and are safe from sudden changes in app operations.

vm?.getLastDbKey()?.observe(this,android.arch.lifecycle.Observer {
})

Any change in the value of last db key will be reflected here without me doing any work. So if it is saved by some other class in the background I get an update here. That is cooler thing about observers.

problems faced

Implementing this at first seemed simple. All I had to do was save all the articles in the db and observer the changes then load it into the adapter. This is where the first hurdle presents itself. Since the user may scroll down, causing the app to load earlier articles in an ascending order. Which is fine.

Now the user closes the app, then if it is started again, the older data is showed first, not the new one, which the user expects. Now there is no order to our data and it shows the new articles at the end. The user will not scroll that much just to see the new ones.

The second hurdle is that fetching ALL the data from the db then loading it into the adapter will cause an oom as the app expands.

The solution

One would assume that we delete the db on each startup, but that would defeat the purpose of a db an cause a delay.

For the first problem we use this from the DAO

@Query("SELECT * from widget_holder where myDbKey > :dbkey ORDER BY myDbKey ASC")
fun getPagedList(dbkey:Int): DataSource.Factory<Integer, FeedArticleDataHolder.FeedArticleHolder>

I has been simplified for this. The dbkey parameter tells the db to only fetch articles after that db key. Here dbkey is the internal key of the data. For getting the last key you use

"@Query("SELECT myDbKey FROM widget_holder WHERE myDbKey = (SELECT MAX(myDbKey) FROM article_holder)")
fun getLastDbId(): LiveData"

Now when our app starts we do not load the data from the start of the db but from the end of it. Saving us time and having articles ready for blazing fast access.

Note : The get last key has to be one executed once, even if observing the data or you will see nothing as last will keep updating and moving ahead as data is added.

Though this solves both problems, to make the app quicker we go a step further.

Instead of fetching all the data at once we will fetch it in portions of 20 per page size. All of this implemented by the inbuilt support for paged lists. This saves us more time and space. Though you can skip based on your apps complexity. The first solution can suffice.

commits

1. Add room to gradle and make entities
2. Add DAO's and the Room DB singleton
3. Room repo classes
4. View model classes
5. Implement viewmodels for articles
6. Model for widget data

---

Open an article via DB

Now since we saved everything in the database. We can put it to some good use. When an article is to be opened we check if a db id is passed. If it is then we load it via our database. If not then we load it normally via network calls.
We use our premade articlevm and stick it here. Creating it and observring on the single item.

We also do not load the comments if opening via the db as it will reduce network calls. The user can refresh and load the comments manually.

commit

open via db

---

PagedRecyclerView

This reduces the memory consumption by calling on more data from the db when needed and optimizes the speed.
It uses

AsyncPagedListDiffer(this, article_DIFF_CALLBACK)

instead of a list. To add items we do not add to its underlying list but pass the paged list via submit list. The calling for more data is handled by it automatically. The Diff_callback decides if the article is old or new.

Finding this implementation took time under all the tutorials as they pass the diff via the normal viewholder. But this method is used for people who have other restraints and have to perform it in this manner. This is more of a raw type of method.

The tutorial on android dev site is here

commits

Paged recyclerview adapter

---

Widgets use Room as well

Widgets would take a long time to load the network data which would then again disappear when the system needed to free up resources. This was slow and made it less usable. To stop this we use the Widget repo to save all articles and then load via them using the same method as articles do with the lastdbkey

The only problem would be that I would have to add another sorting field in the articles db which would get cluttered with widget data. Now Room also has a limitation that it cannot create multiple tables via the same class.

The workaround was to cope the articles class and to make separate repos for accessing the widget db. So for writing we would have to convert an article to a widget item BUT reading an item can be done via the article class so most of the code gets reused. This was a neat trick learnt while developing. I assumed it would work as it works with json and it is working perfectly.

So now the widget when waiting for data to arrive from the network first loading old data from the db using the last key, then saves the new data, then reads it back as an article and reverses the list. So the user sees the lates data and the old one.

Commits

Widgets save to db and load via it

---

Followers join Room

The followers and following data was handled at the time to make it work. But with room the whole implementation was reworked. First, on each startup followers and following data would be loaded from the network then examined via an algorithm what was added/removed and then the same was added to the databases. This was cumbersome and caused too many network calls, slowing the app and causing network strain.

The first solution was to not use any heavy logic simple check the number of followers/following reported by steem and match the count to our db. If the count changes we modify, else we do nothing. This may cause a slight amount of stale data in the condition where you follow A and unfollow B. This is rare and to be handled manually later.

For now the new working is to first check if the total number has changed, if yes then we delete the database and then we load the new follower/following data. We also show a new ui while doing that.

While make this I had to keep the followers/following class same, hence one database because of Room's limitation. Which meant making one indexed coloumn unique and append -follower/-following to each name, which makes sure everything is unique and working fine.

commits

1. Check for change in number then delete
2. handle ui update callbacks
3. The ui

---

Misc changes

1. SharedPreferences singleton class to make life easier across the app.
2. Code cleanup
3. While processing articles return via interface if present to save in db
4. function and interface addtions
5. Code cleanup and interface changes

---

Comments

Forgot to add comments before pushing so here they are in one huge update
comments

---

This update may not bring many "features" as such but greatly improves the app with the right practices and also reduces the load on steems resources with less network calls. Hope you enjoy it.

---

Since this app was not open source when I started it, I did not comment much. Only after I have put it on Github have I started adding comments.

---

Want to contact me?

Discord server for steemer - https://discord.gg/GkNZCGu
My discord username - hispeedimagins#6619
email - hispeedimagins@gmail.com

---

GitHub Account\nhttps://github.com/hispeedimagins