MongoImport- MongoDB Database Tools

Your green leaf emblem database system features a built-in functionality called mongoimport to import files like JSON, CSV, and TSV. It is a CLI utility. It is often used to bring in data from other MongoDB databases or third-party export tools. Oftentimes, application development within teams requires the exchange of files. And this is where mongo import comes in handy.

Mongoimport command plays a significant role in managing MongoDB database. In this tutorial, we’ll show you how to import all three file types to your MongoDB database instance in a few short steps. So, without any extra bother, let’s go right into it.

Prerequisites

All users are required to have MongoDB database tools installed on their system before using the mongoimport command. 

If you are a new user, you can explore our easy-to-follow instructions on installing MongoDB GUI for your platform.

• Windows (MongoDB Compass)
• macOS (MongoDB Compass)
• Ubuntu (MongoDB Compass)

What is MongoDB?

MongoDB is a non-relational (NoSQL) database program.

If you work with big data, you know that fitting diverse data into a rigid relational model is a pain in the neck. SQL databases or Relational Databases (RDBMS) store information in rows and columns with a pre-defined schema that is not quite fit for storing large data volumes. 

Non-relational or NoSQL databases like MongoDB contain schemas that are dynamic, so developers can change them “on the fly.” MongoDB has emerged as a superior option to SQL databases with an acute focus on better scaling and fast queries. Because of such dynamic schema architecture, MongoDB allows for frequent application changes and makes programming simpler for developers.

For developers MongoDB is a plus since it provides official support for all the popular languages— C, C++, C#, and .Net, Go, Java, Node.js, Perl, PHP, Python, Motor, Ruby, Scala, Swift, Mongoid. This allows developers to work with their favorite languages hence leading to faster development time and fewer bugs.

To learn more about MongoDB, and its contrast from SQL Databases, visit our helpful guide here- MongoDB vs SQL Databases: 4 Comprehensive Aspects. 

For more information on the essential use-cases of MongoDB, visit our other guide on Best 7 Real-World MongoDB Use Cases.

Key Features of MongoDB

• Fast Queries: Queries in MongoDB run significantly faster (as much as 100 times) than in an average Relational Database. This is because data in SQL databases is normalized, and queries for a single object or entity require you to join data from multiple tables, hence slower operations.
• Handle Large Unstructured Data: MongoDB can magically handle large volumes of unstructured data owing to its document data model, which stores all related data together within a single document. MongoDB also allows you to query in a different manner that is more sensitive to your workload.
• Horizontal Scaling: One major aspect where most SQL databases fall short is horizontal scaling. They either support it in an ad-hoc way or only on relatively immature technologies. In contrast, MongoDB supports horizontal scaling, which means you get to add cheaper, commodity servers whenever you need to.
• Sharding: MongoDB allows you to store large data by distributing it to several servers connected to your application. If one server can’t handle the size of the data, it gets shared to another server through a process called auto-sharding.
• Easy for Developers: MongoDB can map its data structures to those of programming languages. This lessens the hassle and time required for developers to learn new languages, configure and store data in MongoDB.

What is mongoimport and Where is it Used?

The mongoimport command is used to import your content from an extended JSON, CSV, or TSV export created by mongoexport. It also supports restoring or importing data from other third-party export tools.

This command is of immense help when it comes to managing your MongoDB database. It’s super-fast and multi-threaded, more so than any custom script you might write to do your import operation. The mongoimport command can be combined with other MongoDB command-line tools, such as jq for JSON manipulation, csvkit for CSV manipulation, or even curl for dynamically downloading data files from servers on the internet. 

Syntax

The mongoimport command has the following syntax:

mongoimport <options> <connection-string> <file>

In an extended format, it looks like this:

mongoimport --host (host name) -u (name of user) -p (password of user) --authenticationDatabase (admin database used for authentication)   --db (Name of database) --collection (name of collection) --drop --file /name_of_file (Name of file which was we have used to import into collection)

In the upcoming sections, we’ll elaborate more on the variations you can do with the syntax to import different file types like JSON, CSV, or TSV. 

Applications

No matter the type of web application that your team builds, there are countless occasions where you want to reference some data or import a certain collection that already exists in your database. All of these activities involve using the MongoDB mongoimport command to retrieve information from a bunch of JSON or CSV files. When your customers run your web applications, MongoDB can be coupled with your web apps to run as a backend source of information and mongoimport can come in handy on such occasions.  

When compared to CSV or TSV file formats, JSON is preferred since it is both a hierarchical data format, similar to MongoDB documents, and it is explicit about the sorts of data it encodes. With that information in place, we won’t advise you to convert your data into JSON forms every time before importing it with mongoimport; instead, take a close look at it and decide accordingly whether or not you want to restructure it.

MongoDB mongoimport Command Considerations

As a precautionary note, avoid using mongoimport and mongoexport for entire instance production backups. This is because mongoimport and mongoexport commands can not consistently retain all rich BSON data types, and JSON only represents a subset of the kinds provided by BSON. For this type of backup functionality, we recommend you to instead use mongodump and mongorestore.

To clarify how mongoexport and mongoimport command uses strict mode representation to retain information, here’s an example of an insert operation in the mongo shell that uses the shell mode representation for the BSON types “data_date” and “data_numberlong”:

use test
db.traffic.insert( { _id: 1, volume: NumberLong('2980000'), date: new Date() } )

The argument passed to data_numberlong has to be quoted to avoid potential loss of accuracy. Now, when we use mongoexport to export the data:

mongoexport --db test --collection traffic --out traffic.json

The exported data pops out in a strict mode representation to preserve type information:

{ "_id" : 1, "volume" : { "$numberLong" : "2980000" }, "date" : { "$date" : "2014-03-13T13:47:42.483-0400" } }

How to Import Data into MongoDB Using mongoimport

As mentioned previously, the mongoimport command is used to import JSON, CSV, or TSV files. For a clear content structure and easy navigation, we’ve listed steps to import data from JSON, CSV, or TSV files into three different parts. 

• Part 1: Importing JSON Files.
• Part 2: Importing CSV Files.
• Part 3: Importing TSV Files.

Before proceeding to these sections, you must first ensure to connect mongoimport Windows, macOS, or Ubuntu utility to your MongoDB instance. While there are several ways to connect mongoimport to your MongoDB database, we recommend you to use the –uri option, like this:

mongoimport --uri 'mongodb+srv://mycluster-ABCDE.azure.mongodb.net/test?retryWrites=true&w=majority' 
   --username='MYUSERNAME' 
   --password='SECRETPASSWORD'

Note: If you are using MongoDB Atlas Deployment, you can find your Atlas connection string when you visit Database Deployments > Connect. 

If you aren’t using Atlas Deployment, you’ll have to generate your own URI. If you are connecting to a single server, your URI will look like this- mongodb://your.server.host.name:port/, and if you are running a replicaset and connecting to multiple hostnames your URI would look like this- mongodb://username:password@host1:port,host2:port/?replicaSet=replicasetname.

For more information do check out the documentation page here. 

Part 1: Importing JSON Files

To import JSON file(s) from a collection, use the code below:

mongoimport --db DB_Name --collection Collection_Name --type=json --
file Name-of-file-to-import

Where, 

• DB_Name represents the name of the database that contains the collection Collection_Name.
• type specifies the file type JSON (Optional field).
• Name-of-file-to-import represents the name and path of the JSON file to be imported/restored.

In a concise way, you can also write the above code as:

mongoimport -d DB_NAME -c COLLECTION_name --file Name-of-file-to-import

In case, you would like to drop any existing collection with the same name as the one you’re trying to create/import, you can inform the same to mongoimport command using the –drop flag.

mongoimport -d DB_NAME -c COLLECTION_name --drop --file Name-of-file-to-import

If you want to change your host or port number while importing JSON data, you can do so using the following MongoDB mongoimport command. 

mongoimport --host 123.123.123.1 --port 1234 -d DB_NAME -c COLLECTION_name --file Name-of-file-to-import

By default, mongoimport connects to a running mongo on port 27017 on localhost. 

Part 2: Importing CSV Files

You can use the mongoimport command to import CSV files into a collection using the headerline option. Headerline option notifies the mongoimport command of the first line; to be not imported as a document since it contains field names instead of data.

To import a collection from a CSV file, use the code as mentioned below:

mongoimport --db DB_Name --collection Collection_Name --type=csv --
headerline --file=Name-of-file-to-import

Where,

• DB_Name represents the name of the database that contains the collection Collection_Name.
• type specifies the file type CSV (Optional field).
• headerline details the mongoimport command to take the 1st record of CSV file(s) as field names. 
• Name-of-file-to-import represents the name and path of the CSV file to be imported/restored.

Importing CSV Files Lacking Header

In the event your CSV file doesn’t have a header row, then you must notify the mongoimport command of the names of each column included in your file. There are two ways of doing so:

• Specify Field Types
• Use a Field File

Specify Field Types

One way is to write field names on the command-line using the –fields option. This can prove cumbersome if your CSV file contains lots of columns, as can be seen from the command below:

mongoimport 
   --collection=Collection_Name 
   --file=Name-of-file-to-import 
   --type=csv 
   --fields="username,”"identifier","one time password","recovery password","recovery account","one time code","recovery code","user id","first name","last name","birth year","gender",”department”,”company”,”country”

Use a Field File

The other method is to notify the mongoimport command on field columns using a separate .txt file. Here we’ve created a file called “Field_file.txt” containing all column names present in our CSV file.

username
identifier
one time password
recovery password
recovery account
one time code
recovery code
user id
first name
last name
birth year
gender
department
company
country

After producing such a file, you can run the following command which will use your .txt file (in our example “Field_file.txt”) to extract information on these column names as field names in MongoDB:

mongoimport 
   --collection= Collection_Name
   --file=Name-of-file-to-import 
   --type=csv 
   --fieldFile=Field_file.txt

Part 3: Importing TSV Files

TSV files are conceptually the same when compared with CSV file formats. Whether you use mongoimport Windows utility or another one, as a result, you may import TSV files using the same technique as described for CSV files. 

There’s only one minor change to keep in mind- instead of using the –type=csv, you can change it and use the –type=tsv option to tell mongoimport of the new format.

mongoimport Command Options

The mongoimport command can be appended with options that help developers explore and achieve more. While an extensive list of supported options can be found on the MongoDB mongoimport documentation page, in this blog we’ve featured only the important ones:

Option	Description
–help	Returns information on the options and use of mongoimport.
–host=<hostname><:port>, -h=<hostname><:port>	Used to specify the hostname to which mongoimport should connect. By default, this is set to localhost:27017.
–ignoreBlanks	Ignore fields or columns with empty values.
–drop	Modifies the import process so that the target instance drops the collection before importing the data from the input.
–stopOnError	Forces mongoimport to halt insert operation at the first error instead of continuing.
–mode=<insert|upsert|merge|delete>	Tells how mongoimport should handle existing documents in the database that match documents in the import file.

And this brings us to the end of mongoimport Windows, macOS, Ubuntu command utility. 

Take a peek at a few more interesting blogs to pique your interest:

• Install MongoDB Python on Windows: 5 Easy Steps
• Conducting MongoDB Query Performance Analysis Simplified 101
• How to Migrate Relational Database to MongoDB?: Made Easy

Conclusion

We hope this blog piece sparked your interest and served you with in-depth information on one of the powerful MongoDB Package Component tools that is provided natively- the mongoimport command. Using this guide, we do know for sure that you will be able to successfully use the mongoimport command for your own use of web applications and use-cases, effortlessly.

If your business actively uses MongoDB for developing and hosting your web applications, there’s a high chance you would want to have a secure backup for your MongoDB database. While exporting your MongoDB databases manually can get unwieldy and not so fun, a better and faster method is to use a Cloud-based Data Warehouse for your data storage and transformation needs. A simple and speedy solution like Hevo Data helps you in that. 

What makes Hevo the best option?

Hevo is a No-Code ETL Pipeline builder. It lets you migrate your data from your MongoDB database (and 100+ Data Sources) to any Data Warehouse of your choice like Amazon Redshift, Snowflake, Google BigQuery, or Firebolt within minutes with just a few clicks. Any individual or team, even from a non-data team can set up a Data Pipeline from their Database or SaaS Application into their Data Warehouse in a jiffy and start loading their data. 

Visit our Website to Explore Hevo

Why not give Hevo a try? Sign Up here for a 14-day free trial and experience the feature-rich Hevo suite first hand. You can also check our pricing and make a decision on your best-suited plan. 

Share your opinions about learning the procedure for using mongoimport for MongoDB in the comments area below. We’d like to hear your thoughts and ideas.