It is commonplace for modern applications to combine a data-driven backend with scalable cloud hosting to provide consistent availability and reliability such that end users can access applications in a seamless and predictable manner.
This article is going to introduce you to how to configure Heroku, a popular Platform-as-a-Service (PaaS), and MongoDB, a NoSQL document Database using Python. You will be taken on a step-by-step journey on how this can be achieved. Read along to know more about Heroku MongoDB Python integration.
Table of Contents
- What is Heroku?
- What is MongoDB?
- How to Work with Heroku MongoDB Python Integration?
What is Heroku?
Heroku is a Cloud service that can be defined as a container-based Cloud data Platform-as-a-Service. Heroku’s popularity amongst Developers has increased rapidly in recent times as it is fully managed and simple to use, therefore making it easier for developers to deploy, manage, and scale their applications on the platform to reach a target audience.
Since Heroku is fully managed, you do not have to worry about the maintenance of servers, hardware, or infrastructure, rather, you can focus on the building, management, and deployment of your apps using modern tools, workflows, polyglot, and other provided services by Heroku to increase your productivity and ultimately create great high-performance applications that will be accepted in the market space. Heroku supports different languages such as Node.Js, Ruby, Java, PHP, Python, Go, Scala, Clojure, and any language that runs on Linux with Heroku via a third-party build pack for developers to make their choice from.
Finally, because of Heroku’s simple setup, it is an ideal tool for businesses with limited budgets or individuals or organizations that are just getting to try out the various opportunities found in the Cloud.
What is MongoDB?
MongoDB uses agile methodologies with a flexible schema approach, thereby making it a go-to for developers for building and deployment of applications. It can also be described as an open-source document-oriented NoSQL Database Management platform which is an alternative to traditional relational databases as it makes use of collections and documents as opposed to using tables and rows as found in traditional databases making MongoDB useful when working with large sets of distributed data.
MongoDB is a tool that can manage document-oriented information by using JSON-like format to store and retrieve information and this format directly maps to native objects in most modern programming languages.
MongoDB offers drivers for all the major programming languages and allows developers to start building applications immediately without having to configure a database first. MongoDB also makes it easy to store structured and unstructured data and it can handle a high volume of data, scaling them both vertically and horizontally.
How to Work with Heroku MongoDB Python Integration?
To be able to pair Heroku with MongoDB to build and deploy fully managed cloud applications effectively, you will have to use MongoDB Atlas as it integrates easily with Heroku applications. In subsequent sections, you will be introduced to MongoDB Atlas, shown steps of setting up clusters using Atlas, given the process involved in configuring Heroku to connect to the clusters created on Atlas using config vars, and connecting your applications to MongoDB Atlas cluster through Heroku config var values.
But first, it is assumed that you are conversant with MongoDB and have written applications that use MongoDB. It is also believed that you are also familiar with Heroku and know how to deploy applications plus have the Heroku CLI installed, have Git on your system, and know your way around it.
Follow the sections below to set up your Heroku MongoDB Python integration:
- Brief Introduction to MongoDB Atlas Cluster
- Heroku MongoDB Python: Setting up MongoDB Atlas Cluster
- Heroku MongoDB Python: Using Config Vars to Configure Heroku to Connect With Atlas Clusters
- Heroku MongoDB Python: Connecting Apps to MongoDB Atlas Clusters Using Heroku Config Var Values
Brief Introduction to MongoDB Atlas Cluster
As stated in the previous section, MongoDB Atlas integrates easily with Heroku applications so you will be introduced to it and shown steps in setting up an Atlas Cluster which will be used to set up a connection string to a Heroku config variable. If you are familiar with MongoDB, using MongoDB Atlas will be a natural fit.
MongoDB Atlas is a fully-managed cloud database service for MongoDB clusters in the cloud. MongoDB Atlas automates infrastructure positioning, setup, and deployment so you can access your database when needed. The clusters are made up of geographically distributed database instances, and it is easy to control who has access to your database as it uses a unique Virtual Private Cloud (VPC) to ensure network isolation.
MongoDB Atlas’ other features include automated backup, auto-scaling, multi-AZ fault tolerance, a range of both Management/Analytics tools.
Read along to set up the MongoDB Atlas and Heroku MongoDB Python connection.
Simplify MongoDB ETL & Data Analysis with Hevo’s No-code Data Pipeline
Hevo Data, a No-code Data Pipeline, helps load data from any data source such as Databases, SaaS applications, Cloud Storage, SDK,s, and Streaming Services and simplifies the ETL process. It supports MongoDB and other 100+ Data Sources including 40+ Free Sources. It is a 3-step process by just selecting the data source, providing valid credentials, and choosing the destination.
Hevo loads the data onto the desired Data Warehouse/destination in real-time and enriches the data and transforms it into an analysis-ready form without having to write a single line of code. Its completely automated pipeline, fault-tolerant, and scalable architecture ensure that the data is handled in a secure, consistent manner with zero data loss and supports different forms of data. The solutions provided are consistent and work with different BI tools as well.GET STARTED WITH HEVO FOR FREE
Check out why Hevo is the Best:
- Secure: Hevo has a fault-tolerant architecture that ensures that the data is handled securely and consistently with zero data loss.
- Schema Management: Hevo takes away the tedious task of schema management & automatically detects the schema of incoming data and maps it to the destination schema.
- Minimal Learning: Hevo, with its simple and interactive UI, is extremely simple for new customers to work on and perform operations.
- Hevo Is Built To Scale: As the number of sources and the volume of your data grows, Hevo scales horizontally, handling millions of records per minute with very little latency.
- Incremental Data Load: Hevo allows the transfer of data that has been modified in real-time. This ensures efficient utilization of bandwidth on both ends.
- Live Support: The Hevo team is available round the clock to extend exceptional support to its customers through chat, email, and support calls.
- Live Monitoring: Hevo allows you to monitor the data flow and check where your data is at a particular point in time.
Simplify your Data Analysis with Hevo today!SIGN UP HERE FOR A 14-DAY FREE TRIAL!
Heroku MongoDB Python: Setting up MongoDB Atlas Cluster
Let’s get started with the Heroku MongoDB Python connection. To set up your Atlas Cluster, do the following:
- Create an Atlas account: This is done by registering for an Atlas account using your email address or your Google account here. If you already created a MongoDB account using your mail address, simply sign in to your account instead.
- Create your Organization and Project: Before you go any further, Atlas will prompt you to create an organization and project where your cluster will be deployed.
- Deploy your first cluster: Select from the range of cluster options depending on your needs. The options include Shared Clusters which is free, Dedicated Clusters for $0.08/hr, and Dedicated Multi-Region Clusters for $0.13/hr. Click on Create a Cluster and choose options for your clusters as explained below:
- Cloud Provider and Region: this involves you choosing where you want your cluster to be deployed to and you must select the closest region available to your application if possible, choose the same region to minimize latency.
- Cluster Tier: this involves the available resources found for each cluster option. The resources include RAM, Storage, vCPU, and the base price.
- Additional Settings: depending on the tier chosen, additional settings might be required of you such as the MongoDB version you wish to deploy, backup option, etc.
- Cluster Name: this is the name you will give to the cluster and it can not be changed once given. Click on the Create Cluster button after specifying your options.
- Create a Database User for Your Cluster: Before users can access clusters on Atlas, Atlas will require clients to authenticate as MongoDB database users, therefore, you will have to create a database user for your cluster by doing the following:
- Navigate to the Database Access Section in the left-hand navigation bar.
- Click on Create a new Database User.
- On the prompt that appears, choose an authentication method such as username, password and select the database privileges the database user will enjoy like Atlas admin.
- Click the Add User button to create the cluster’s first database user.
- Grant Authorized IP Addresses Access to Your Cluster: Choose the IP addresses that will be allowed to access the cluster. You can set the cluster to allow access from anywhere or you can identify the exact IP addresses where your application will be hosted and set the IP address or ranges that should be granted access.
Heroku MongoDB Python: Using Config Vars to Configure Heroku to Connect With Atlas Clusters
After setting up your Atlas cluster, the next thing to do is to create an application-level config var that will hold your cluster’s connection string. This application set up links Heroku to the MongoDB Atlas cluster using the config vars found within your application.
To achieve this, do the following:
- Log into Heroku CLI using Heroku login.
- Clone your app from MongoDB Atlas using Github that will be deployed to Heroku.
- Create the Heroku App.
After creating the app, you will need to get your Atlas cluster connection string. This can be gotten by going to your cluster dashboard and doing the following:
- Click on the Connect button.
- Choose the Connect your application option.
- The connection string needed to connect to your cluster will be visible at this point. Simply copy the connection string and edit it by replacing <password> with your own database user’s password and <dbname> with the name of the dataset your application will use.
The next step in the process is to set a MONGODB_URI config var as this is used to store the connection string in a Heroku config variable. To set up the connection string to the config var MONGODB_URI, you can set a command such as:
heroku config:set MONGODB_URI="mongodb+srv://yourUsername:yourPassword@yourClusterName.n9z04.mongodb.net/sample_mflix?retryWrites=true&w=majority"
or simply do the following steps:
- In Heroku, go-to Apps.
- Click on the name of the app you created previously.
- Select Settings.
- Next, go to the Config Vars section and click on Reveal Config Vars.
- On the prompt, add your config var.
Heroku MongoDB Python: Connecting Apps to MongoDB Atlas Clusters Using Heroku Config Var Values
Config vars can be used on your application’s code as environment variables and accessing these variables is dependent on the language used. Various languages can be used such as Java, Ruby, Node.Js but for this article, Python will be used.
You will use PyMongo which is a client containing tools that make it easier to work with MongoDB from Python. MongoDB’s syntax is logical, at least for fundamental query operations, because it transfers well to Object-Oriented Programming. The connection string is sent to the pymongo.MongoClient() constructor when connecting to MongoDB. To load the .env file into the environment, we use load_dotenv() and include it in the connection string. We must first open the JSON file before we can import it into MongoDB. Because MongoDB expects data to be document objects, no transformation is required.
In PyMongo, there are 2 methods for inserting a single document or a list of documents: insert_one() and insert_many(). PyMongo’s syntax is fairly similar to MongoDB’s original syntax. To use PyMongo to access the MongoDB Atlas database, you will need to install the required dependencies. You can do so by running the commands below in a terminal:
$ pip install python-dotenv $ pip install pymongo $ pip install dnspython
In this article, you were introduced to the managed Cloud hosting service – Heroku, you were also introduced to MongoDB and shown how to set up a MongoDB Atlas cluster. The article then looked at how to link Heroku and MongoDB using Python and environment variables. Finally, it explained how Heroku MongoDB Python connection can be done through MongoDB Atlas, setting up a config var, and MongoDB_URI.
However, as a Developer, extracting complex data from a diverse set of data sources to your MongoDB Database can seem to be quite challenging. This is where a simpler alternative like Hevo can save your day! Hevo Data is a No-Code Data Pipeline that offers a faster way to move data from 100+ Data Sources such as MongoDB and other 40+ Free Sources, into your Data Warehouse to be visualized in a BI tool. Hevo is fully automated and hence does not require you to code.VISIT OUR WEBSITE TO EXPLORE HEVO
Want to take Hevo for a spin?
Share your experience with Heroku MongoDB Python Integration in the comments section below!