Conducting MongoDB Query Performance Analysis Simplified 101

One of the common challenges that every growing business faces are the ability to efficiently handle the exponentially growing data. Apart from the Traditional Relational Databases, organizations are now using Document-oriented Open-source NoSQL Databases. There are several NoSQL databases out there, but MongoDB is the most commonly used, and it is available both as a Cloud Service and for Deployment on Self-Managed Systems.

In this article, you will gain information about MongoDB Query Performance Analysis. You will also gain a holistic understanding of MongoDB, its key features, conducting MongoDB Query Performance Analysis, and the ways of optimizing Query Performance. Read along to find out in-depth information about undergoing MongoDB Query Performance Analysis.

What is MongoDB?

MongoDB is a NoSQL database that was developed by MongoDB Inc. and is schema-free. It was designed and created using C++ and JavaScript, allowing for higher connectivity. It uses a collection of documents and has an option for creating schemas. It doesn’t follow the same structure as a traditional database wherein the data is stored in the form of rows.  Dive into the use cases of MongoDB and why MongoDB is primarily used.

Key Features of MongoDB

Main features of MongoDB which make it unique are:

• High Performance
• Scalability
• Availability
• Flexibility

1) High Performance

Data operations on MongoDB are fast and easy because of their NoSQL nature. Data can be quickly stored, manipulated, and retrieved without any compromise on data integrity. MongoDB’s Replication enables you to backup & recover data from different servers.

2) Scalability

In the Big Data era, MongoDB data can be distributed across a cluster of machines quickly and equally, free of bulkiness. The scalability of MongoDB allows it to handle a growing amount of data capably. Sharding is a process in MongoDB used to horizontally scale the data across multiple servers when the size of the data increases.

3) Availability

Data is highly available with MongoDB as it makes multiple copies of the same data and sends copies of data across different servers. In case any server fails, data can be retrieved from another server without delay.

4) Flexibility

MongoDB can easily be combined with different database management systems, including both MongoDB SQL and NoSQL types. The document-oriented structure makes MongoDB schema dynamically flexible, and different types of data can be easily stored and manipulated.

How to Conduct MongoDB Query Performance Analysis?

Conducting MongoDB Query Performance Analysis can be difficult if you don’t know which aspects should be measured. Fortunately, MongoDB provides various tools, including a very useful tool for conducting MongoDB query performance analysis, i.e., explain (“executionStats”). This tool provides some general measurements, such as the number of documents examined and the execution time, which can be used for statistical analysis.

The cursor.explain(“executionStats”) and the db.collection.explain(“executionStats”) methods provide statistics about the performance of a query. These statistics can be useful in measuring if and how a query uses an index.

For further information, you can visit db.collection.explain() for details. For an understanding of performance differences between MongoDB vs MySQL databases, see a detailed guide that will provide insights into indexing, scalability, and query optimization for each platform.

Conducting MongoDB Query Performance Analysis

MongoDB Query Performance Analysis can be conducted in two different cases.

• MongoDB Query Performance Analysis: Query with No Index
• MongoDB Query Performance Analysis: Query with Index

Lets us consider a collection named inventory with the following documents:

{ "_id" : 1, "item" : "f1", type: "food", quantity: 500 }
{ "_id" : 2, "item" : "f2", type: "food", quantity: 100 }
{ "_id" : 3, "item" : "p1", type: "paper", quantity: 200 }
{ "_id" : 4, "item" : "p2", type: "paper", quantity: 150 }
{ "_id" : 5, "item" : "f3", type: "food", quantity: 300 }
{ "_id" : 6, "item" : "t1", type: "toys", quantity: 500 }
{ "_id" : 7, "item" : "a1", type: "apparel", quantity: 250 }
{ "_id" : 8, "item" : "a2", type: "apparel", quantity: 400 }
{ "_id" : 9, "item" : "t2", type: "toys", quantity: 50 }
{ "_id" : 10, "item" : "f4", type: "food", quantity: 75 }

1) Query with No Index

The following query returns Documents with quantity values ranging from 100 to 200, inclusive:

db.inventory.find( { quantity: { $gte: 100, $lte: 200 } } )

The following Documents are returned by the above query:

{ "_id" : 2, "item" : "f2", "type" : "food", "quantity" : 100 }
{ "_id" : 3, "item" : "p1", "type" : "paper", "quantity" : 200 }
{ "_id" : 4, "item" : "p2", "type" : "paper", "quantity" : 150 }

To view the query plan selected, you can chain the cursor.explain(“executionStats”) cursor method to the end of the find command:

db.inventory.find(
   { quantity: { $gte: 100, $lte: 200 } }
).explain("executionStats")

The explain() returns the following results:

{
   "queryPlanner" : {
         "plannerVersion" : 1,
         ...
         "winningPlan" : {
            "stage" : "COLLSCAN",
            ...
         }
   },
   "executionStats" : {
      "executionSuccess" : true,
      "nReturned" : 3,
      "executionTimeMillis" : 0,
      "totalKeysExamined" : 0,
      "totalDocsExamined" : 10,
      "executionStages" : {
         "stage" : "COLLSCAN",
         ...
      },
      ...
   },
   ...
}

• COLLSCAN is displayed by queryPlanner.winningPlan.stage to indicate a Collection scan. Collection scans indicate that the mongod had to scan the entire Collection Document by Document to identify the results. This is a generally expensive operation and can result in slow queries.
• The value 3 in executionStats.nReturned indicates that the query matches and returns three documents.
• The value 0 in executionStats.totalKeysExamined indicates that this query does not use an index.
• The value 10 displayed by executionStats.totalDocsExamined indicates that MongoDB had to scan all the documents in the collection i.e, 10 to find the three matching documents.

The difference between the number of matching documents and the number of examined documents indicates that the query could benefit from the use of an Index to improve efficiency.

2) Query with Index

You can add an Index on the quantity field to support the query on the quantity field.

You can use the explain() method to view the query plan statistics:

db.inventory.createIndex( { quantity: 1 } )

db.inventory.find(
   { quantity: { $gte: 100, $lte: 200 } }
).explain("executionStats")

It will give the following result.

{
   "queryPlanner" : {
         "plannerVersion" : 1,
         ...
         "winningPlan" : {
               "stage" : "FETCH",
               "inputStage" : {
                  "stage" : "IXSCAN",
                  "keyPattern" : {
                     "quantity" : 1
                  },
                  ...
               }
         },
         "rejectedPlans" : [ ]
   },
   "executionStats" : {
         "executionSuccess" : true,
         "nReturned" : 3,
         "executionTimeMillis" : 0,
         "totalKeysExamined" : 3,
         "totalDocsExamined" : 3,
         "executionStages" : {
            ...
         },
         ...
   },
   ...
}

• IXSCAN is displayed by queryPlanner.winningPlan.inputStage.stage to indicate index use.
• The value 3 in executionStats.nReturned indicates that the query matches and returns three Documents.
• MongoDB scanned three index entries, as indicated by the value 3 in executionStats.totalKeysExamined. The number of keys examined corresponds to the number of documents returned, indicating that the mongod only needed to examine index keys to return the results. The mongod didn’t have to scan all of the documents, and only the three that matched had to be pulled into memory. As a result, the query is highly efficient.
• executionStats.totalDocsExamined displays 3 which indicates that MongoDB scanned three documents.

Without the index, the query would scan the entire Collection of ten Documents to return 3 matching Documents. The query would also have to scan the entire content of each Document, potentially storing them in memory. As a result, the query operation becomes costly and potentially slow.

MongoDB Query Performance Analysis: How to Optimize?

MongoDB Query performance can be optimized in the following ways.

• Create Indexes to Support Queries
• Limit the Number of Query Results to Reduce Network Demand
• Use Projections to Return Only Necessary Data
• Use $hint to Select a Particular Index
• Use the Increment Operator to Perform Operations Server-Side

1) Create Indexes to Support Queries

You can create Indexes for commonly used queries. If a query searches multiple fields, a Compound Index can be created. You should prefer creating Indexes because scanning an Index takes significantly less time than scanning a Collection. The Index structures are smaller than the document references, and they store references in chronological order.

For example: Suppose you have a Collection named posts that contain blog posts and if you regularly run queries that sort on the author_name field, you can optimize the query by creating an Index on the author_name field:

db.posts.createIndex( { author_name : 1 } )

Indexes also improve the efficiency of queries that sort on a specific field on a regular basis.

For example: If you run queries that sort on the timestamp field on a regular basis, you can optimize the query by creating an Index on the timestamp field.

• Creating this Index:

db.posts.createIndex( { timestamp : 1 } )

• Optimizes this query:

db.posts.find().sort( { timestamp : -1 } )

The direction of a single-key Index is irrelevant because MongoDB can read Indexes in both ascending and descending order.

2) Limit the Number of Query Results to Reduce Network Demand

Cursors in MongoDB return results in groups of multiple documents. If you know how many results you want, you can use the limit( ) method to reduce the demand for network resources.

This is typically used in combination with sort operations. For example, if you only need 10 results from your query to the posts collection, you can issue the following command:

db.posts.find().sort( { timestamp : -1 } ).limit(10)

3) Use Projections to Return Only Necessary Data

When you only need a subset of fields from a Document, you can improve performance by returning only the fields you require:

For example: If you only need the timestamp, title, author, and abstract fields in your query to the posts collection, you can issue the following command:

db.posts.find( {}, { timestamp : 1 , title : 1 , author : 1 , abstract : 1} ).sort( { timestamp : -1 } )

4) Use $hint to Select a Particular Index

In most cases, the Query Optimizer chooses the best Index for a given operation. You can, however, use the hint() method to persuade MongoDB to use a specific Index. You can also use hint() to aid performance testing or on queries that require you to select a field or fields that are included in multiple Indexes.

5) Use the Increment Operator to Perform Operations Server-Side

To increment or decrement values in Documents, you can use MongoDB’s $inc operator. As an alternative to selecting a Document, making simple changes in the client, and then writing the entire Document to the server, the operator increments the value of the field on the server.

The $inc operator can also aid in the avoidance of race conditions, which occur when two application instances are queried for a document, manually increment a field, and save the entire document back at the same time.

Conclusion

In this article, you have learned about MongoDB Query Performance Analysis. This article also provided detailed information on MongoDB, its key features, conducting MongoDB Query Performance Analysis, and the ways of optimizing Query Performance. Read further on MongoDB Replica Set Configuration, MongoDB Compass Windows Installation, and MongoDB Count Method to discover more detailed concepts on MongoDB.

Hevo Data, a No-code Data Pipeline, provides you with a consistent and reliable solution to manage data transfer between 150+ data sources(including 60+ Free Sources) and a wide variety of Desired Destinations with a few clicks & enrich your data to make it analysis-ready.

Want to give Hevo a try? Sign Up for a 14-day free trial and experience the feature-rich Hevo suite firsthand. You may also have a look at the amazing Hevo price, which will help you select the best plan for your requirements.

Share your experience of understanding MongoDB Query Performance Analysis in the comment section below! We would love to hear your thoughts.

FAQs