Get a count of total documents with MongoDB when using limit

I am interested in optimizing a "pagination" solution I'm working on with MongoDB. My problem is straight forward. I usually limit the number of documents returned using the limit() functionality. This forces me to issue a redundant query without the limit() function in order for me to also capture the total number of documents in the query so I can pass to that to the client letting them know they'll have to issue an additional request(s) to retrieve the rest of the documents.

Is there a way to condense this into 1 query? Get the total number of documents but at the same time only retrieve a subset using limit()? Is there a different way to think about this problem than I am approaching it?

Solution 1:

Mongodb 3.4 has introduced $facet aggregation

which processes multiple aggregation pipelines within a single stage on the same set of input documents.

Using $facet and $group you can find documents with $limit and can get total count.

You can use below aggregation in mongodb 3.4

db.collection.aggregate([
  { "$facet": {
    "totalData": [
      { "$match": { }},
      { "$skip": 10 },
      { "$limit": 10 }
    ],
    "totalCount": [
      { "$group": {
        "_id": null,
        "count": { "$sum": 1 }
      }}
    ]
  }}
])

Even you can use $count aggregation which has been introduced in mongodb 3.6.

You can use below aggregation in mongodb 3.6

db.collection.aggregate([
  { "$facet": {
    "totalData": [
      { "$match": { }},
      { "$skip": 10 },
      { "$limit": 10 }
    ],
    "totalCount": [
      { "$count": "count" }
    ]
  }}
])

Solution 2:

No, there is no other way. Two queries - one for count - one with limit. Or you have to use a different database. Apache Solr for instance works like you want. Every query there is limited and returns totalCount.

Solution 3:

MongoDB allows you to use cursor.count() even when you pass limit() or skip().

Lets say you have a db.collection with 10 items.

You can do:

async function getQuery() {
  let query = await db.collection.find({}).skip(5).limit(5); // returns last 5 items in db
  let countTotal = await query.count() // returns 10-- will not take `skip` or `limit` into consideration
  let countWithConstraints = await query.count(true) // returns 5 -- will take into consideration `skip` and `limit`
  return { query, countTotal } 
}

Solution 4:

Times have changed, and I believe you can achieve what the OP is asking by using aggregation with $sort, $group and $project. For my system, I needed to also grab some user info from my users collection. Hopefully this can answer any questions around that as well. Below is an aggregation pipe. The last three objects (sort, group and project) are what handle getting the total count, then providing pagination capabilities.

db.posts.aggregate([
  { $match: { public: true },
  { $lookup: {
    from: 'users',
    localField: 'userId',
    foreignField: 'userId',
    as: 'userInfo'
  } },
  { $project: {
    postId: 1,
    title: 1,
    description: 1
    updated: 1,
    userInfo: {
      $let: {
        vars: {
          firstUser: {
            $arrayElemAt: ['$userInfo', 0]
          }
        },
        in: {
          username: '$$firstUser.username'
        }
      }
    }
  } },
  { $sort: { updated: -1 } },
  { $group: {
    _id: null,
    postCount: { $sum: 1 },
    posts: {
      $push: '$$ROOT'
    }
  } },
  { $project: {
    _id: 0,
    postCount: 1,
    posts: {
      $slice: [
        '$posts',
        currentPage ? (currentPage - 1) * RESULTS_PER_PAGE : 0,
        RESULTS_PER_PAGE
      ]
    }
  } }
])