Firestore - how to structure a feed and follow system

I've seen your question a little later but I will also try to provide you the best database structure I can think of. So hope you'll find this answer useful.

I'm thinking of a schema that has there three top-level collections for users, users that a user is following and posts:

Firestore-root
   |
   --- users (collection)
   |     |
   |     --- uid (documents)
   |          |
   |          --- name: "User Name"
   |          |
   |          --- email: "[email protected]"
   |
   --- following (collection)
   |      |
   |      --- uid (document)
   |           |
   |           --- userFollowing (collection)
   |                 |
   |                 --- uid (documents)
   |                 |
   |                 --- uid (documents)
   |
   --- posts (collection)
         |
         --- uid (documents)
              |
              --- userPosts (collection)
                    |
                    --- postId (documents)
                    |     |
                    |     --- title: "Post Title"
                    |     |
                    |     --- date: September 03, 2018 at 6:16:58 PM UTC+3
                    |
                    --- postId (documents)
                          |
                          --- title: "Post Title"
                          |
                          --- date: September 03, 2018 at 6:16:58 PM UTC+3

if someone have 10,000 followers than a new post was added to all of the 10,000 follower's Timeline.

That will be no problem at all because this is the reason the collections are ment in Firestore. According to the official documentation of modeling a Cloud Firestore database:

Cloud Firestore is optimized for storing large collections of small documents.

This is the reason I have added userFollowing as a collection and not as a simple object/map that can hold other objects. Remember, the maximum size of a document according to the official documentation regarding limits and quota is 1 MiB (1,048,576 bytes). In the case of collection, there is no limitation regarding the number of documents beneath a collection. In fact, for this kind of structure is Firestore optimized for.

So having those 10,000 followers in this manner, will work perfectly fine. Furthermore, you can query the database in such a manner that will be no need to copy anything anywhere.

As you can see, the database is pretty much denormalized allowing you to query it very simple. Let's take some example but before let's create a connection to the database and get the uid of the user using the following lines of code:

FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
String uid = FirebaseAuth.getInstance().getCurrentUser().getUid();

If you want to query the database to get all the users a user is following, you can use a get() call on the following reference:

CollectionReference userFollowingRef = rootRef.collection("following/" + uid + "/userFollowing");

So in this way, you can get all user objects a user is following. Having their uid's you can simply get all their posts.

Let's say you want to get on your timeline the latest three posts of every user. The key for solving this problem, when using very large data sets is to load the data in smaller chunks. I have explained in my answer from this post a recommended way in which you can paginate queries by combining query cursors with the limit() method. I also recommend you take a look at this video for a better understanding. So to get the latest three posts of every user, you should consider using this solution. So first you need to get the first 15 user objects that you are following and then based on their uid, to get their latest three posts. To get the latest three posts of a single user, please use the following query:

Query query = rootRef.collection("posts/" + uid + "/userPosts").orderBy("date", Query.Direction.DESCENDING)).limit(3);

As you are scrolling down, load other 15 user objects and get their latest three posts and so on. Beside the date you can also add other properties to your post object, like the number of likes, comments, shares and so on.

If someone have large amount of posts than every new follower received all of those posts in his Timeline.

No way. There is no need to do something like this. I have already explained above why.

Edit May 20, 2019:

Another solution to optimize the operation in which the user should see all the recent posts of everyone he follow, is to store the posts that the user should see in a document for that user.

So if we take an example, let's say facebook, you'll need to have a document containing the facebook feed for each user. However, if there is too much data that a single document can hold (1 Mib), you need to put that data in a collection, as explained above.

There have two situations

1. Users in your app have a small number of followers.

2. Users in your app have a large number of followers. If we are going to store whole followers in a single array in a single document in firestore. Then it will hit the firestore limit of 1 MiB per document.

1. In the first situation, each user must keep a document which stores the followers' list in a single document in a single array. By using arrayUnion() and arrayRemove() it is possible to efficiently manage followers list. And when you are going to post something in your timeline you must add the list of followers in post document.

   And use query given below to fetch posts

   postCollectionRef.whereArrayContains("followers", userUid).orderBy("date");
   

2. In the second situation, you just need to break user following document based on the size or count of followers array. After reaching the size of the array into a fixed size the next follower's id must add into the next document. And the first document must keep the field "hasNext", which stores a boolean value. When adding a new post you must duplicate post document and each document consist of followers list that breaks earlier. And we can make the same query which is given above to fetch documents.