Tornado MongoDB Deployment: 9 Easy Steps

In this article, we will show MongoDB integration with Tornado projects. Integrating Tornado MongoDB can be tricky. But, Python async driver for MongoDB, Motor, comes with built-in support for Tornado. The Motor makes it simple to use MongoDB in Tornado regardless of the type of server you are building.

MongoDB is a widely used Non-Relational database and Tornado is a popular asynchronous Python web server. The Motor is a full-featured, async, and non-blocking MongoDB driver for Python Tornado. The Motor is an async driver whose name is extracted from “MOngo + TORnado“.

Apart from Motor, there are other methods too to connect MongoDB from a Tornado app. But, they require a tradeoff. Either use PyMongo which does not give advantages of an async web server or use AsyncMongo which provides an asynchronous library for accessing MongoDB and is built on the tornado but lacks key features of Motor.

What is MongoDB?

MongoDB is an Open-Source, Non-structured, Document-Oriented Database System. MongoDB is a leading NoSQL, Cross-Platform database system. SQL Databases are running from very early days but NoSQL database like MongoDB was first released on February 11, 2009, by MongoDB Inc. Its last stable release MongoDB 5.0.5 is released on 6 December 2021. It is written in different programming languages like JavaScript, Python, Java, C++, C, Perl, etc.

MongoDB is based on the concept of collection and document. A database is a container for all collections. A collection in MongoDB is similar to a table in SQL. And a document in MongoDB is similar to a row or record in SQL. MongoDB stores schema-less data in form of  BSON (Binary JSON) documents as shown in the above image. This documented-oriented approach makes MongoDB very flexible to operate and offers a richer experience with modern programming techniques.

To install MongoDB click here.

Features of MongoDB

The main features of MongoDB are:

• High-performance NoSQL Database Management System
• Documented-oriented storage in BSON format
• High availability due to multiple copies of the same data across servers.
• Have no complex joins
• Rich Queries
• Sharding to distributing big data across multiple machines
• Flexibility due to its schemaless feature
• Horizontally scalable

What is Tornado?

Tornado is a scalable, Python web framework and asynchronous networking library. It was originally developed at FriendFeed, later acquired by Facebook in 2009. Tornado is now open-sourced and written in Python. It runs on all main operating systems like Linux, Mac OS X, and Windows.

By using non-blocking network I/O, Tornado solves C10K issues. The tornado can easily create JSON(JavaScript Object Notation) APIs. Tornado is ideal for building high-performance apps and thousands of concurrent users.

To Install Tornado click here.

Features of Tornado

The main features of Tornado are:

High-Performance

Asynchronous programming of Tornado makes code faster to execute which leads to high performance. Asynchronous programming is a way of concurrent programming in which the process releases the CPU during the waiting time, so the CPU is free for other processes which are ready to continue.

C10K

The tornado can scale to tens of thousands of open connections which makes it ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user.

Minimal and Lightweight

Tornado is a very lightweight and minimal framework. Tornado is not heavy like Django, it is just right for its purpose.

Tornado MongoDB Deployment Steps

In this section, we will see step by step guide to integrating MongoDB with Tornado projects using Motor. Each step is explained with its python codes and outputs which we have run in the Jupyter notebook.

• Tornado MongoDB: Prerequisites
• Tornado MongoDB: Connection String
• Tornado MongoDB: Implementing nest_asyncio
• Tornado MongoDB: Tornado Application Startup Sequence
• Tornado MongoDB: Creating a Motor Client to connect with MongoDB
• Tornado MongoDB: Creating/Getting a Database
• Tornado MongoDB: Creating/Getting a Collection
• Tornado MongoDB: Inserting Document
• Tornado MongoDB: Querying Document

Tornado MongoDB: Prerequisites

Install all the below-mentioned mandatory prerequisites before proceeding further.

• Python 3.6 or above
• Motor: Motor is MongoDB driver for Python Tornado. The motor can be installed using pip:

$ pip install motor

• Tornado: Tornado is a Python web framework.  The Tornado can be installed using pip:

$ pip install tornado

• Nest_asyncio: Nest_asyncio patch asyncio to allow nested event loops. Nest_asyncio can be installed using pip:

$ pip install nest_asyncio

• Pymongo: Python driver for MongoDB. Pymongo can be installed using pip:

$ pip install pymongo

• Dnspython: DNS toolkit to connect to DB cluster.  Dnspython can be installed using pip:

$ pip install dnspython

• A MongoDB Atlas cluster: MongoDB Atlas provides an easy way to host and manage your data in the cloud. Here is the “Get Started with Atlas” guide. Follow the instructions in this guide to create your account and MongoDB cluster. Save your username, password, and connection string to use it further.

• Download MongoDB Compass on your local system using this link and then install the same.

Tornado MongoDB: Connection String

After installing all the dependencies, first, we need to know about the connection string. The connection string can be different based on the connection needs to be done with a cluster machine or standalone machine.

• The connection string for MongoDB Compass for the cluster is extracted from MongoDB Atlas.

Image Snapshot of the connection string for MongoDB database cluster

‘mongodb+srv://<UserName>:<Password>@<clustername>.gvtft.mongodb.net/test’

Edit the string with your user name and password while using.

• The connection string for MongoDB Compass for the Standalone machine

'mongodb://localhost:27017’

Tornado MongoDB: Implementing nest_asyncio

We are going to use tornado.ioloop for the I/O event loop for non-blocking sockets which sometimes gives runtime errors as below:

“RuntimeError: This event loop is already running”.

This problem occurs when the event loop is already running then it’s impossible to run tasks and wait for results. So, the solution is to implement nest_asyncio as below:

import nest_asyncio
nest_asyncio.apply()

By design, asyncio does not allow its event loop to be nested and patches asyncio to allow nested use of asyncio. run and loop.run_until_complete.

Tornado MongoDB: Tornado Application Startup Sequence

To start a tornado web application, create one or more RequestHandler subclasses and an application object which routes all incoming requests to handlers. And the main() function is used to start the server. 

Start the code with import statements and then implement the Tornado web application code.

import motor.motor_tornadoimport tornado.ioloopimport tornado.web
class MainHandler(tornado.web.RequestHandler):
    def get(self):
        db = self.settings['db']

def make_app():    return tornado.web.Application([        (r"/", MainHandler),    ])
if __name__ == "__main__":    app = make_app()

Tornado MongoDB: Creating a Motor Client to connect with MongoDB

Motor Client represents a MongoDB process or cluster of processes. We need to explicitly create one of the client objects and connect it to running MongoDB cluster or standalone MongoDB. This Motor Client is further used for the application.

async def get_server_info():
    #conn_str = "<your MongoDB Atlas connection string>"
    conn_str = "mongodb+srv://<UserName>:<Password>@cluster0.gvtft.mongodb.net/test"    #To connect to standalone DB use this string    #conn_str = "mongodb://localhost:27017"
    # set a 5-second connection timeout
    client = motor.motor_tornado.MotorClient(conn_str, serverSelectionTimeoutMS=5000)
    try:
        print("Client Server info: " ,await client.server_info())
    except Exception:
        print("Unable to connect to the server.")
    return client    

client = tornado.ioloop.IOLoop.current().run_sync(get_server_info)

Output:

Client Server info:  {'version': '5.0.6', 'gitVersion': '212a8dbb47f07427dae194a9c75baec1d81d9259', 'modules': ['enterprise'], 'allocator': 'tcmalloc', 'javascriptEngine': 'mozjs', 'sysInfo': 'deprecated', 'versionArray': [5, 0, 6, 0], 'openssl': {'running': 'OpenSSL 1.0.1e-fips 11 Feb 2013', 'compiled': 'OpenSSL 1.0.1e-fips 11 Feb 2013'}, 'buildEnvironment': {'distmod': 'rhel70', 'distarch': 'x86_64', 'cc': '/opt/mongodbtoolchain/v3/bin/gcc: gcc (GCC) 8.5.0', 'ccflags': '-Werror -include mongo/platform/basic.h -fasynchronous-unwind-tables -ggdb -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -fno-omit-frame-pointer -fno-strict-aliasing -O2 -march=sandybridge -mtune=generic -mprefer-vector-width=128 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable -Wno-missing-braces -fstack-protector-strong -Wa,--nocompress-debug-sections -fno-builtin-memcmp', 'cxx': '/opt/mongodbtoolchain/v3/bin/g++: g++ (GCC) 8.5.0', 'cxxflags': '-Woverloaded-virtual -Wno-maybe-uninitialized -fsized-deallocation -std=c++17', 'linkflags': '-Wl,--fatal-warnings -pthread -Wl,-z,now -fuse-ld=gold -fstack-protector-strong -Wl,--no-threads -Wl,--build-id -Wl,--hash-style=gnu -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro -Wl,--compress-debug-sections=none -Wl,-rpath,/usr/lib64/perl5/CORE -Wl,-z,origin -Wl,--enable-new-dtags', 'target_arch': 'x86_64', 'target_os': 'linux', 'cppdefines': 'SAFEINT_USE_INTRINSICS 0 PCRE_STATIC NDEBUG _XOPEN_SOURCE 700 _GNU_SOURCE _FORTIFY_SOURCE 2 BOOST_THREAD_VERSION 5 BOOST_THREAD_USES_DATETIME BOOST_SYSTEM_NO_DEPRECATED BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS BOOST_ENABLE_ASSERT_DEBUG_HANDLER BOOST_LOG_NO_SHORTHAND_NAMES BOOST_LOG_USE_NATIVE_SYSLOG BOOST_LOG_WITHOUT_THREAD_ATTR ABSL_FORCE_ALIGNED_ACCESS'}, 'bits': 64, 'debug': False, 'maxBsonObjectSize': 16777216, 'storageEngines': ['devnull', 'ephemeralForTest', 'inMemory', 'queryable_wt', 'wiredTiger'], 'ok': 1.0, '$clusterTime': {'clusterTime': Timestamp(1645553843, 1), 'signature': {'hash': b'ax9dn+Fnx04xc2x14xcex00&Kxfahxd6x82xc8pL', 'keyId': 7037217873416159233}}, 'operationTime': Timestamp(1645553843, 1)}

Client-Server Connection is done successfully!!

Tornado MongoDB: Creating/Getting a Database

Multiple independent databases can be accessed with a single instance of MongoDB. From an open motor client, we can create or connect to the particular database using the below Python code.

Code:

db = client.test_database
print("Database created: ",db)

Output:

Database created:  MotorDatabase(Database(MongoClient(host=['cluster0-shard-00-01.gvtft.mongodb.net:27017', 'cluster0-shard-00-00.gvtft.mongodb.net:27017', 'cluster0-shard-00-02.gvtft.mongodb.net:27017'], document_class=dict, tz_aware=False, connect=False, authsource='admin', replicaset='atlas-x9rwqe-shard-0', ssl=True, serverselectiontimeoutms=5000, driver=DriverInfo(name='Motor', version='2.3.0', platform='Tornado 6.1')), 'new_database'))

The database is created successfully!!

Tornado MongoDB: Creating/Getting a Collection

A Collection is a group of documents stored in a database just like tables in the relational database. Code for creating or getting an existing collection is shown below:

collection = db.new_collectionprint("Collection created: ",collection )

Output:

Collection created: MotorCollection(Collection(Database(MongoClient(host=['cluster0-shard-00-01.gvtft.mongodb.net:27017', 'cluster0-shard-00-00.gvtft.mongodb.net:27017', 'cluster0-shard-00-02.gvtft.mongodb.net:27017'], document_class=dict, tz_aware=False, connect=False, authsource='admin', replicaset='atlas-x9rwqe-shard-0', ssl=True, serverselectiontimeoutms=5000, driver=DriverInfo(name='Motor', version='2.3.0', platform='Tornado 6.1')), 'new_database'), 'new_collection'))

The collection is created successfully!!

Tornado MongoDB: Inserting Document

Motor insert MongoDB documents with Python dictionaries. To store a single document in MongoDB, call insert_one() in an await expression, and to insert many documents at once call insert_many() in an await expression.

To create documents in a dictionary format to insert in new_collection:

item_1 = {
"id" : 1,
"item_name" : "Cricket Bat",
"max_discount" : "10%",
"price" : 990,
"category" : "Sports"
}

item_2 = {
"id" : 2,
"item_name" : "Hockey Stick",
"category" : "Sports",
"price" : 550,
"item_description" : "Hockey Playing Stick"
}

item_3 = {
"id" : 3,
"item_name" : "Electric Kettle",
"max_discount" : "10%",
"price" : 1990,
"category" : "Kitchen Appliance"
}

item_4 = {
"id" : 4,
"item_name" : "Cheese",
"category" : "Food",
"price" : 55,
"item_description" : "Mozarella Cheese"
}

Code to insert multiple documents using insert_many():

async def do_insert():
    result = await db.new_collection.insert_many(
        [item_1,item_2, item_3,item_4])
    print('Inserted %d documents' % (len(result.inserted_ids),))

tornado.ioloop.IOLoop.current().run_sync(do_insert)

Output:

Inserted 4 documents

Documents are inserted successfully!!

Tornado MongoDB: Querying Document

To query for a set of documents use find(). E.g. We need to find items with prices less than 551. Use the below code to run a query.

async def do_find():
    c = db.new_collection
    async for document in c.find({'price': {'$lt': 551}}):
        print(document)    

tornado.ioloop.IOLoop.current().run_sync(do_find)

Output:

{'_id': ObjectId('62152ec5b0370e4e556fdcf5'), 'id': 2, 'item_name': 'Hockey Stick', 'category': 'Sports', 'price': 550, 'item_description': 'Hockey Playing Stick'}
{'_id': ObjectId('62152ec5b0370e4e556fdcf7'), 'id': 4, 'item_name': 'Cheese', 'category': 'Food', 'price': 55, 'item_description': 'Mozarella Cheese'}

Two items with ‘id’=2 and ‘id’ = 4 come as output because their price is less than 551.

Querying is done successfully!!

Many such operations like finding a particular document, counting documents, reading, updating, or deleting a document can be easily done using python code. To learn coding details of these operations click here.

From the above outputs and results, we can see that Tornado MongoDB deployment is done successfully!!

MongoDB Atlas Cluster

Let’s see the result of Tornado MongoDB deployment on the GUI platform of the MongoDB Atlas cluster. Below is the snapshot of MongoDB Atlas Cluster showing Database creation, collection creation, and data insertion done successfully.

We can see from the image all 4 documents were inserted successfully in the new_collection of new_database.

MongoDB Compass

We can see all the updates done in the database standalone or cluster on our local system as well using MongoDB Compass which we have installed in the prerequisite step.

Based on the connection string MongoDB Compass can be connected to a standalone MongoDB or cluster. Connect to MongoDB cluster through MongoDB Compass using the connection string as shown in the snapshot below

After successful connection, we can see new_database, new_collection, and 4 documents inserted successfully in the snapshot of MongoDB Compass below.

Conclusion

Learning something new is always looks tricky at first glance. But with the right knowledge, training, and hands-on experience, we can learn new technology easily. Just, try your hands on the Tornado web application with MongoDB. Soon, you will find it as easy to implement as a relational database.

Just follow the step-by-step deployment procedure as shown in the article to get the desired results.

To meet the growing storage and computing needs of data, you will need to invest some of your engineering bandwidth in integrating data from all sources, cleaning and transforming it, and finally loading it into a Cloud Data Warehouse for further Business Analytics. A Cloud-Based ETL tool, such as Hevo Data, can efficiently handle all of these challenges.

Visit our Website to Explore Hevo

Hevo Data, a No-code Data Pipeline provides you with a consistent and reliable solution to manage data transfer between a variety of sources like MongoDB and a wide variety of Desired Destinations, with a few clicks. Hevo Data with its strong integration with 100+ sources (including 40+ free sources) allows you to not only export data from your desired data sources & load it to the destination of your choice, but also transform & enrich your data to make it analysis-ready so that you can focus on your key business needs and perform insightful analysis using BI tools.

Want to take Hevo for a spin? Sign Up for a 14-day free trial and experience the feature-rich Hevo suite first hand. You can also have a look at the unbeatable pricing that will help you choose the right plan for your business needs.

Share with us your experience of Tornado MongoDB Deployment in the comments below!