Jupyter Notebook MongoDB Integration: 2 Easy Methods

Since the requirement of enterprises keeps changing, MongoDB has become an ideal database for companies to match the dynamically transforming data. It supports flexible schema and manages workload with horizontal scalability.

MongoDB’s concurrent nature can also easily handle multiple users while providing adequate data availability and real-time analytics.

However, for further analysis and visualization, you should establish Jupyter Notebook MongoDB integration.

You can edit, share and collaborate through Jupyter Notebook. It enables users to take advantage of Python libraries for in-depth data exploration to gain essential insights.

In this article, you will learn about Jupyter Notebook MongoDB Integration and the process of connecting Jupyter to MongoDB via Python and Apache Spark.

Prerequisites

• Basic understanding of Python, database management, and integration.

What is Jupyter Notebook?

The Jupyter Notebook is a web-based computing application that helps users to clean, analyze and visualize data efficiently. With Jupyter Notebook, you also create and share documents containing live code, equations, etc.

Initially, Project Jupyter developed and maintained the Jupyter Notebook, but it is now supported financially by NumFOCUS, a public charity in the United States that supports open-source projects.

It is a free, open-source learning environment, especially for data scientists, to build machine learning workflows and perform data analytics to gain meaningful insights.

The notebook also encourages good knowledge management by allowing users to document and chronologically explain segments of their code. 

Key Features of Jupyter Notebook

• Basic Workflow: The workflow of the Jupyter Notebook is identical to a standard IPython session with an additional feature. Jupyter Notebook enables users to work on computational problems by breaking them down into small parts to enhance organizing ideas and ensure the previous component works accurately. If any calculations or analysis take time to run, you can interrupt it using the Kernel and Interrupt option.
• Plotting: Jupyter Notebook can dynamically display plots that are the outputs of the executed code segments. It helps visualize data via various graphs like scatter plots, error charts, box plots, bar charts, histograms, and more. The %matplotlib magic functions provide an inline option to render the plot of the executed cell without calling the show() function.
• Security: When any notebook opens, Jupyter stores a signature of each authorized notebook to prevent the execution of untrusted code. The server cross-checks whether the notebook opened has a stored signature or not. If there is no match, the HTML and JavaScript output will not be displayed until the cells are re-executed. Only notebooks the user has executed are trusted, and only these notebook documents will be displayed correctly by the Jupyter Notebook.

What is MongoDB?

MongoDB is a document-based NoSQL database with high scalability and flexibility to efficiently query, analyze, and index data.

Its simple and user-friendly document model ensures quick learning and usage for developers. MongoDB dynamically stores data via JSON-like documents, meaning you can change the data structure and fields of the document over time.

The model architecture, Ad hoc queries, and real-time aggregation offer users powerful methods to retrieve and analyze data. MongoDB is a free and equally distributed database with horizontal scaling and geographic allocation. 

Key Features of MongoDB

• Data Availability: In MongoDB, data replication occurs by deploying many servers for disaster retrieval, providing better availability and stability, and making data accessible all the time to users. The main server accepts all the written queries and executes them across secondary nodes to replicate data.
• Sharding: Sharding is the process of splitting large datasets across various well-distributed collections (shards) to provide better execution and database performance. MongoDB’s sharding allowed greater horizontal scalability, meaning that each shard in every cluster is a part of the dataset functioning as an individual database. The collections of these shards form a single, unified, comprehensive database that can manage the requirements of a growing application with zero downtime.
• Load Balancing: For optimal load balancing, MongoDB via replication and sharding provides horizontal scalability. The platform manages numerous concurrent queries for the same data with the help of locking protocols and concurrency management to ensure proper data consistency.  
• Real-time Analytics: Designing the database schema in advance without knowing the commands the users will run is impossible. MongoDB uses an ad hoc query, a short-lived query whose value relies on a variable to optimize performance. Every time the ad hoc query is executed, the results vary according to the variable, making MongoDB a flexible schema database for businesses that need effective real-time analytics.

Jupyter Notebook MongoDB Integration

Jupyter Notebook is a platform where Python programmers can work on multiple datasets, collaborate with others, and document their coding process.

For retrieving data and performing statistical calculations, you need to connect the Jupyter Notebook to a flexible database like MongoDB.

Jupyter provides users with the flexibility to handle data, and MongoDB is a suitable fit to keep up with this dynamicity. Since MongoDB uses JSON-like documents, it has a flexible schema, thereby helping users to work with different types of data for analysis. 

There are two methods for Jupyter MongoDB connection — Python and Apache Spark. 

Jupyter Notebook MongoDB Connection Using Python

Follow the below-given steps to establish a MongoDB Jupyter Notebook connection with the help of Python:

• Start the MongoDB server by executing the mongod command in the command prompt.
• In another command prompt, start the mongo shell by running the mongo command.

• Launch Jupyter Notebook and create a new file.
• Install the PyMongo module to connect the Jupyter Notebook and MongoDB localhost:

pip install pymongo

• Import the PyMongo module and run the cell.
• Connect MongoDB by executing the below command.

client = MongoClient(“localhost”, 27017) 

• In the next cell, execute this command to fetch any collection from the MongoDB database.

DB = client[‘collection_name’]

Kudos! your Jupyter Notebook MongoDB integration is completed.

Jupyter Notebook MongoDB Connection Using Apache Spark

Apache Spark is an open-source analytic engine for large-scale processing that you can use to leverage MongoDB data in Jupyter notebooks via the assistance of MongoDB Spark Connector and PySpark.

Follow these below-given steps to establish a Jupyter Notebook MongoDB connection using Apache Spark:

Building the required environment

Build an environment of a MongoDB cluster, JupyterLab, and an Apache Spark deployment of one master with two worker nodes. To build this environment from scratch, follow the given instructions correctly:

• Git clone the RWaltersMA/mongo-spark-jupyter repository from GitHub.
• To build the docker images, run sh build.sh and for the environment, run sh run.sh command.
• Open the command prompt and run the mongosh command to open the mongo shell.
• Run the Jupyter Lab. To check if it’s running, navigate to http://localhost:8888.
• Verify if the Spark master is running as well. To check it, navigate to http://localhost:8080.

Creating Connection via PySpark

The MongoDB Connector for Apache Spark supports Scala, Java, R, and Python. In this article, we will use Python and the PySpark library. Once your environment is built and up, you can perform these instructions to extract data from MongoDB:

• Create a new Python notebook in the Jupyter Notebook.
• With PySpark, create a Resilient Distributed Dataset (RDDs).
• Run these commands to configure the Spark connector to use the local MongoDB. 

from pyspark.sql import SparkSession
spark = SparkSession.\
builder.\
appName("pyspark-notebook2").\
master("spark://spark-master:7077").\
config("spark.executor.memory", "1g").\
config("spark.mongodb.input.uri","mongodb://mongo1:27017,mongo2:27018,mongo3:27019/Stocks.Source?replicaSet=rs0").\
config("spark.mongodb.output.uri","mongodb://mongo1:27017,mongo2:27018,mongo3:27019/Stocks.Source?replicaSet=rs0").\
config("spark.jars.packages", "org.mongodb.spark:mongo-spark-connector_2.12:3.0.0").\
getOrCreate()

• Load the MongoDB data into the data frame by running the below command:

df = spark.read.format("mongo").load()

Now you can use the df to work with the data in MongoDB in Jupyter Notebook.

Congratulation your Jupyter Notebook MongoDB integration is established.

Conclusion

In this article, you learned about the Jupyter Notebook, MongoDB, and how to perform Jupyter MongoDB notebook integration.

MongoDB is a NoSQL flexible database that provides optimal load balancing for large-scale applications via its horizontal scalability features like sharding and data replication. Its concurring handles all the write and read operations while providing real-time analysis for enterprise applications.

You can establish Jupyter Notebook MongoDB integration to further analyze your MongoDB data. Using Jupyter, you can perform analysis, predict and visualize data easily to gain meaningful insights.

Don’t forget to drop your comment or suggestion in the comment section below on Jupyter MongoDB Notebook Connection.

Apart from MongoDB, you would be using several applications and databases across your business for Marketing, Accounting, Sales, Customer Relationship Management, etc. To get a complete overview of your business performance, it is important to consolidate data from all these sources.

To achieve this you need to assign a portion of your Engineering Bandwidth to Integrate Data from all sources, Clean & Transform it, and finally, Load it to a Cloud Data Warehouse or a destination of your choice for further Business Analytics. All of these challenges can be comfortably solved by a Cloud-Based ETL tool such as Hevo Data.

Visit our Website to Explore Hevo

Hevo Data, a No-code Data Pipeline can seamlessly transfer data from a vast sea of 150+ sources such as MongoDB & MongoDB Atlas to a Data Warehouse or a Destination of your choice. It is a reliable, completely automated, and secure service that doesn’t require you to write any code!  

If you are using MongoDB as your NoSQL Database Management System and searching for a no-fuss alternative to Manual Data Integration, then Hevo can effortlessly automate this for you. Hevo, with its strong integration with 150+ sources(Including 50+ Free Sources), allows you to not only export & load data but also transform & enrich your data & make it analysis-ready in a jiffy.

Want to take Hevo for a ride? Sign Up for a 14-day free trial and simplify your Data Integration process. Do check out the pricing details to understand which plan fulfills all your business needs.

Tell us about your experience of learning about the Jupyter MongoDB notebook Connection! Share your thoughts with us in the comments section below.