Jupyter Notebook MongoDB Integration: 2 Easy Methods

Since the requirements of enterprises keep changing, MongoDB has become an ideal database for companies to match the dynamically transforming data. It supports flexible schema and manages workload with horizontal scalability.

MongoDB’s concurrent nature can also easily handle multiple users while providing adequate data availability and real-time analytics. However, for further analysis and visualization, you should establish Jupyter Notebook MongoDB integration. You can edit, share, and collaborate through Jupyter Notebook. It enables users to take advantage of Python libraries for in-depth data exploration and gain essential insights.

In this article, you will learn about Jupyter Notebook MongoDB Integration and the process of connecting Jupyter to MongoDB via Python and Apache Spark.

Prerequisites

• Basic understanding of Python, database management, and integration.

What is Jupyter Notebook?

The Jupyter Notebook is a web-based computing application that helps users to clean, analyze and visualize data efficiently. With Jupyter Notebook, you also create and share documents containing live code, equations, etc.

Initially, Project Jupyter developed and maintained the Jupyter Notebook, but it is now supported financially by NumFOCUS, a public charity in the United States that supports open-source projects.

It is a free, open-source learning environment, especially for data scientists, to build machine learning workflows and perform data analytics to gain meaningful insights.

The notebook also encourages good knowledge management by allowing users to document and chronologically explain segments of their code. 

Key Features of Jupyter Notebook

• Basic Workflow: The workflow of the Jupyter Notebook is identical to a standard IPython session with an additional feature. Jupyter Notebook enables users to work on computational problems by breaking them down into small parts to enhance organizing ideas and ensure the previous component works accurately. If any calculations or analysis take time to run, you can interrupt it using the Kernel and Interrupt option.
• Plotting: Jupyter Notebook can dynamically display plots that are the outputs of the executed code segments. It helps visualize data via various graphs like scatter plots, error charts, box plots, bar charts, histograms, and more. The %matplotlib magic functions provide an inline option to render the plot of the executed cell without calling the show() function.
• Security: When any notebook opens, Jupyter stores a signature of each authorized notebook to prevent the execution of untrusted code. The server cross-checks whether the notebook opened has a stored signature or not. If there is no match, the HTML and JavaScript output will not be displayed until the cells are re-executed. Only notebooks the user has executed are trusted, and only these notebook documents will be displayed correctly by the Jupyter Notebook.

What is MongoDB?

MongoDB is a document-based NoSQL database with high scalability and flexibility to efficiently query, analyze, and index data.

Its simple and user-friendly document model ensures quick learning and usage for developers. MongoDB dynamically stores data via JSON-like documents, meaning you can change the data structure and fields of the document over time.

The model architecture, Ad hoc queries, and real-time aggregation offer users powerful methods to retrieve and analyze data. MongoDB is a free and equally distributed database with horizontal scaling and geographic allocation. 

Key Features of MongoDB

• Data Availability: In MongoDB, data replication occurs by deploying many servers for disaster retrieval, providing better availability and stability, and making data accessible all the time to users. The main server accepts all the written queries and executes them across secondary nodes to replicate data.
• Sharding: Sharding is the process of splitting large datasets across various well-distributed collections (shards) to provide better execution and database performance. MongoDB’s sharding allowed greater horizontal scalability, meaning that each shard in every cluster is a part of the dataset functioning as an individual database. The collections of these shards form a single, unified, comprehensive database that can manage the requirements of a growing application with zero downtime.
• Load Balancing: For optimal load balancing, MongoDB via replication and sharding provides horizontal scalability. The platform manages numerous concurrent queries for the same data with the help of locking protocols and concurrency management to ensure proper data consistency.  
• Real-time Analytics: Designing the database schema in advance without knowing the commands the users will run is impossible. MongoDB uses an ad hoc query, a short-lived query whose value relies on a variable to optimize performance. Every time the ad hoc query is executed, the results vary according to the variable, making MongoDB a flexible schema database for businesses that need effective real-time analytics.

Jupyter Notebook MongoDB Integration

Jupyter Notebook is a platform where Python programmers can work on multiple datasets, collaborate with others, and document their coding process.

For retrieving data and performing statistical calculations, you need to connect the Jupyter Notebook to a flexible database like MongoDB.

Jupyter provides users with the flexibility to handle data, and MongoDB is a suitable fit to keep up with this dynamicity. Since MongoDB uses JSON-like documents, it has a flexible schema, thereby helping users to work with different types of data for analysis. 

There are two methods for Jupyter MongoDB connection — Python and Apache Spark. 

Jupyter Notebook MongoDB Connection Using Python

Follow the below-given steps to establish a MongoDB Jupyter Notebook connection with the help of Python:

• Start the MongoDB server by executing the mongod command in the command prompt.
• In another command prompt, start the mongo shell by running the mongo command.

• Launch Jupyter Notebook and create a new file.
• Install the PyMongo module to connect the Jupyter Notebook and MongoDB localhost:

pip install pymongo

• Import the PyMongo module and run the cell.
• Connect MongoDB by executing the below command.

client = MongoClient(“localhost”, 27017) 

• In the next cell, execute this command to fetch any collection from the MongoDB database.

DB = client[‘collection_name’]

Kudos! your Jupyter Notebook MongoDB integration is completed.

Jupyter Notebook MongoDB Connection Using Apache Spark

Apache Spark is an open-source analytic engine for large-scale processing that you can use to leverage MongoDB data in Jupyter notebooks via the assistance of MongoDB Spark Connector and PySpark.

Follow these below-given steps to establish a Jupyter Notebook MongoDB connection using Apache Spark:

Building the required environment

Build an environment of a MongoDB cluster, JupyterLab, and an Apache Spark deployment of one master with two worker nodes. To build this environment from scratch, follow the given instructions correctly:

• Git clone the RWaltersMA/mongo-spark-jupyter repository from GitHub.
• To build the docker images, run sh build.sh and for the environment, run sh run.sh command.
• Open the command prompt and run the mongosh command to open the mongo shell.
• Run the Jupyter Lab. To check if it’s running, navigate to http://localhost:8888.
• Verify if the Spark master is running as well. To check it, navigate to http://localhost:8080.

Creating Connection via PySpark

The MongoDB Connector for Apache Spark supports Scala, Java, R, and Python. In this article, we will use Python and the PySpark library. Once your environment is built and up, you can perform these instructions to extract data from MongoDB:

• Create a new Python notebook in the Jupyter Notebook.
• With PySpark, create a Resilient Distributed Dataset (RDDs).
• Run these commands to configure the Spark connector to use the local MongoDB. 

from pyspark.sql import SparkSession
spark = SparkSession.\
builder.\
appName("pyspark-notebook2").\
master("spark://spark-master:7077").\
config("spark.executor.memory", "1g").\
config("spark.mongodb.input.uri","mongodb://mongo1:27017,mongo2:27018,mongo3:27019/Stocks.Source?replicaSet=rs0").\
config("spark.mongodb.output.uri","mongodb://mongo1:27017,mongo2:27018,mongo3:27019/Stocks.Source?replicaSet=rs0").\
config("spark.jars.packages", "org.mongodb.spark:mongo-spark-connector_2.12:3.0.0").\
getOrCreate()

• Load the MongoDB data into the data frame by running the below command:

df = spark.read.format("mongo").load()

Now you can use the df to work with the data in MongoDB in Jupyter Notebook.

Congratulation your Jupyter Notebook MongoDB integration is established.

Conclusion

This article covered Jupyter Notebook, MongoDB, and their integration to enhance data analysis. MongoDB, a flexible NoSQL database, supports large-scale applications with features like sharding and replication, enabling efficient load balancing, real-time analysis, and concurrent data operations. Integrating MongoDB with Jupyter allows seamless data analysis, prediction, and visualization for valuable insights.

To get a holistic view of your business, consolidating data from various sources like Marketing, Sales, and CRM is essential. Tools like Hevo Data can automate data integration, transformation, and loading to a Cloud Data Warehouse, saving engineering time and effort. With Hevo, you can effortlessly make your data analysis ready and focus on driving business growth. Sign up for a 14-day free trial and experience the feature-rich Hevo suite firsthand.

FAQs