Transferring a file with utmost security is essential to today’s data-driven world. If the file goes to the wrong hand, a slight mistake may lead to a security breach. And it causes many consequences for the company.
There are many methods for securing this process. File Transfer using SFTP is beneficial as it is encrypted with protocols like SSH. And, that ensures tight security. This blog will provide a brief overview of the SFTP protocol and the important prerequisites to set up this connection.
Read along to find out how you can set up your File Transfer securely using SFTP for your organization. Let’s get started!
Table of Contents
Before setting up the File Transfer using SFTP process, the following prerequisites are important to be addressed:
- Access to Local and remote device
- A terminal for creating an SSH connection
- Basic working knowledge about file system and Unix commands
Introduction to SFTP
SFTP is the acronym for Secure File Transfer Protocol and is a File Protocol that allows users to transfer files or other data between computers or over the web securely. It is built on top of FTP and includes Secure Shell (SSH) security components.
SSH is a cryptographic component of internet security, and SFTP transfers files using SSH and FTP. The client needs to be authenticated by a server that protects the data against middle-man attacks.
SFTP is used when there is a need to transfer some sensitive information securely over the web or between computers. For example, trade data might not be covered by any data privacy rule, but it can be devastating if it falls into the wrong hands. So a business may want to opt for SFTP to transmit the files securely.
The working of SFTP is shown in the below figure.
To learn more about SFTP, click this link.
Hevo Data, a No-code Data Pipeline helps to load data from any data source such as Databases, SaaS applications, Cloud Storage, SDK,s, and Streaming Services and simplifies the ETL process. It supports FTP, along with 100+ data sources (including 30+ free data sources), and is a 3-step process by just selecting the data source, providing valid credentials, and choosing the destination. Hevo not only loads the data onto the desired Data Warehouse/destination but also enriches the data and transforms it into an analysis-ready form without having to write a single line of code.
Its completely automated pipeline offers data to be delivered in real-time without any loss from source to destination. Its fault-tolerant and scalable architecture ensure that the data is handled in a secure, consistent manner with zero data loss and supports different forms of data. The solutions provided are consistent and work with different BI tools as well.
Get Started with Hevo for Free
Check out why Hevo is the Best:
Sign up here for a 14-Day Free Trial!
- Secure: Hevo has a fault-tolerant architecture that ensures that the data is handled in a secure, consistent manner with zero data loss.
- Schema Management: Hevo takes away the tedious task of schema management & automatically detects the schema of incoming data and maps it to the destination schema.
- Minimal Learning: Hevo, with its simple and interactive UI, is extremely simple for new customers to work on and perform operations.
- Hevo Is Built To Scale: As the number of sources and the volume of your data grows, Hevo scales horizontally, handling millions of records per minute with very little latency.
- Incremental Data Load: Hevo allows the transfer of data that has been modified in real-time. This ensures efficient utilization of bandwidth on both ends.
- Live Support: The Hevo team is available round the clock to extend exceptional support to its customers through chat, email, and support calls.
- Live Monitoring: Hevo allows you to monitor the data flow and check where your data is at a particular point in time.
Steps to Set Up File Transfer using SFTP
Setting up File Transfer using SFTP is a straightforward process and generally, the connection can be established by following 2 methods namely by using the Username and Password method and by using SSH keys for password-less FTP login. This method of setting up File Transfer using SFTP uses the latter method and follows the steps as shown below:
Download the Whitepaper on Automating Data Integration
Learn the key benefits of automating Data Integration
Step 1: Generating SSH Keys
The first step in setting up File Transfer using SFTP by using SSH keys is to generate these SSH keys. You can do this by following the below steps:
- Open the Linux/Unix terminal in Mac or command prompt in Windows or any terminal in Linux machine as shown below.
- To set up an RSA token, execute the below command on the terminal.
ssh-keygen -t rsa
- Once the above command is successful, a few questions will pop up. Ideally, just press enter for default as shown below.
Enter file in which to save the key (/home/tautvydas/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
- Keeping the passphrase blank will enable the user to log in to the remote server without any password. However, if you would like more security, set up some passphrases, and every time you log in to a remote server, you will be asked for a password. You will end up with something similar to the below figure.
- Please note that there will be two keys generated (Public and Private). One is id_rsa, and the other is id_rsa.pub. The private key is – id_rsa, and it should be in your local machine, whereas id_rsa.pub is the Public key, and it needs to be transferred to the remote server.
Step 2: Copying SSH Keys to a Remote Server
The next step in File Transfer using SFTP is to copy the keys generated onto a remote server. There are several authentication and authorization policies to be taken care of in this step as shown below:
- Now that you have generated an RSA key pair place the public key to the remote server.
- To copy the public file, execute the below command on the local machine:
- ssh-copy-id user@server_ip
- Here the user is the username for the remote server, and the server_ip is the IP address for the remote server.
- Once you provide the above information, you will see a warning message like:
The authenticity of host 'Server's IP address' can't be established.
RSA key fingerprint is ...
Are you sure you want to continue connecting (yes/no)?
- Type “Yes” in the command line and hit enter to continue. You will then popped up with another message saying:
Warning: Permanently added 'SERVER IP' (RSA) to the list of known hosts.
- Type the password of the remote server user, and then click enter. You will be then greeted with another message saying:
Now try logging into the machine, with "ssh 'user@serverip'", and check in:
to make sure we haven't added extra keys that you weren't expecting.
- Now to login to the remote machine every time, use the below command:
- As we have added the RSA token, it will not ask for a password unless you have set some passphrase at the time of RSA token generation.
Step 3: Initiating an SFTP Connection
The next step in File Transfer using SFTP is to initiate the SFTP connection. This too is a straightforward process and can be done by following the below steps:
- Open the command terminal and execute the below command:
- Here the user is of the remote server and server_ip is the IP address of the remote server.
- After the successful connection, you will see the sftp prompt.
Step 4: Transferring Files from Remote Servers to Local Systems
The next step in File Transfer using SFTP is to transfer files from Remote Servers to Local Systems. You can do this by following the below steps:
- To transfer files from remote server to local system using SFTP, use the following command:
Syntax - get /path/to/remote/server/file.txt
Example - get /etc/config.txt
- Once you execute the above command in the sftp prompt, you will see the file is getting copied into the local machine.
- To download multiple files from remote server, execute the below command:
Syntax - mget /path/to/remote/server/*.txt
Example - mget /etc/*.txt
Step 5: Transferring Files from Local Systems to Remote Servers
The next step in File Transfer using SFTP is to transfer files from Local Systems to Remote Servers.
- Open the command terminal, and execute the below command to copy the file from local machine to remote server.
Syntax - get file.txt /path/to/remote/server/directory
Example - get file.txt /etc/config/
- To move the file from local machine to a remote server, use the following command.
Syntax - put /path/to/local/file/abc.txt /path/to/remote/directory
Example - put /home/user/abc.txt /etc/config
- To transfer multiple files from local to remote, use the following command.
mput /home/abc/*.txt /etc/config
This article gave a step-by-step guide on File Transfer using SFTP. It also gave a brief overview of SFTP and highlighted some important prerequisites companies must take care of before setting up this process. Overall, File Transfer using SFTP plays an important role in the transfer of data among multiple data sources and can be an asset to your organization.
Visit our Website to Explore Hevo
In case you want to integrate data from data sources like FTP into your desired Database/destination and seamlessly visualize it in a BI tool of your choice, then Hevo Data is the right choice for you! It will help simplify the ETL and management process of both the data sources and destinations.
Want to take Hevo for a spin? Sign Up for a 14-day free trial and experience the feature-rich Hevo suite first hand. You can also have a look at the unbeatable pricing that will help you choose the right plan for your business needs.
Share your experience of learning about File Transfer using SFTP in the comments section below.