Batch Processing in Spring Boot Simplified 101

Manisha Jena • Last Modified: October 26th, 2023

As data collection grows, organizations rely on bulk processing to effectively handle large amounts of data. It includes both automated and complex processing of large datasets without the need for user interaction. As a result, organizations use batch processing frameworks to streamline their overall workflow and handle billions of events every day.

Spring Boot is a Java web framework that is open source and based on microservices. The Spring Boot framework uses prebuilt code within its codebase to create a completely configurable and fully production-ready environment.

In this article, you will gain information about Batch Processing in Spring Boot. You will also gain a holistic understanding of Batch Processing, its key features, the Spring boot framework, its key features, and the need for Spring Boot Batch configuration. It also provides a step-by-step guide to configuring the Spring Boot in Batch Processing and also the creation of an application using Batch Processing in Spring Boot.

Table of Contents

What is Batch Processing?

Herman Hollerith, an American inventor who invented the first tabulating machine, used the Batch Processing method for the first time. This device, which could count and sort data stored on punched cards, was the forerunner of the modern computer. The cards and the data on them could then be collected and processed in batches. With this innovation, large amounts of data could be processed more quickly and accurately than with manual entry methods.

Batch processing is a fast way to complete a large number of iterative data jobs. In simple terms, it is a method for consistently processing large amounts of data. When sufficient computing resources are available, the batch method allows you to process data with little to no user interaction.

After you’ve collected and saved your data, you can process it using the batch processing method during an event known as a “batch window.” It provides an efficient workflow layout by prioritizing processing tasks and completing data jobs only when necessary.

In the diagram below, the Batch Reference Architecture is depicted in a simplified form. A Job, which is made up of several Steps, usually encapsulates a Batch Operation. Each Step usually only has one ItemReader, ItemProcessor, and ItemWriter. A JobLauncher executes a job, and metadata about configured and executed jobs are saved in a JobRepository. It defines the basic concepts and terminology used by Spring Batch Parallel Processing.

Batch processing in Spring Boot: Spring Batch| Hevo Data
Image Source

Key Features of Batch Processing

Batch Processing has become popular due to its numerous benefits for enterprise data management. It has several advantages for businesses:

  • Efficiency: Batch Processing allows a business to process jobs when computing or other resources are readily available. Non-urgent tasks can be scheduled as batch processes, while time-sensitive tasks can be prioritized. Batch systems can also run in the background to alleviate processor strain.
  • Simplicity: Batch Processing isn’t much complex as it does not require any special hardware or system support when compared to Stream Processing. It requires less maintenance for data input.
  • Faster Business Intelligence: Batch Processing allows businesses to process large volumes of data quickly, resulting in faster and more efficient Business Intelligence. Many records can be processed at once in batch processing, which reduces processing time and ensures that data is delivered on time. Furthermore, because multiple jobs can be handled concurrently, business intelligence is now available faster than ever before.
  • Improved Data Quality: Batch processing reduces the likelihood of errors by automating most or all components of a processing job and minimizing user interaction. To achieve a higher level of data quality, precision and accuracy are improved.

What is Spring Boot?

Batch processing in Spring Boot: Spring Boot| Hevo Data
Image Source

Spring Boot is a framework that extends the Spring framework. SpringBoot simplifies the Spring framework by supplying pluggable dependencies such as Spring Kafka, Spring Web Services, Spring Security, and others.

Spring Boot is gaining popularity because it uses Java as its programming language and allows developers to quickly build enterprise-grade applications with minimal configuration.

Key Features of Spring Boot

  • Spring Boot applications are auto configurable, which means that they can be configured via a list of dependencies or simply by a properties file. For example, if you list MySQL as a dependency, the SpringBoot application will start with MySQL connector included, which allows you to work seamlessly.
  • Spring Boot application can run on standalone mode, i.e., on one’s machine/laptop. Users don’t need a separate webserver to deploy the application.
  • Spring Boot provides many options while configuring, and hence if you include JPA in your Pom.xml, it will automatically configure a memory database, hibernate entity, and sample data source. This feature makes SpringBoot an Opinionated framework.
  • SpringBoot Framework reduces the overall development time and increases the team’s productivity by avoiding boilerplate code, configurations, and many more.
  • Spring Boot framework makes it easy for developers to create and test Java-based applications by providing a default setup for unit and integration tests.
Replicate Data in Minutes Using Hevo’s No-Code Data Pipeline

Hevo Data, a Fully-managed Data Pipeline platform, can help you automate, simplify & enrich your data replication process in a few clicks. With Hevo’s wide variety of connectors and blazing-fast Data Pipelines, you can extract & load data from 150+ Data Sources (including 40+ Free Sources) straight into your Data Warehouse or any Databases.

To further streamline and prepare your data for analysis, you can process and enrich raw granular data using Hevo’s robust & built-in Transformation Layer without writing a single line of code!


Hevo is the fastest, easiest, and most reliable data replication platform that will save your engineering bandwidth and time multifold. Try our 14-day full access free trial today to experience an entirely automated hassle-free Data Replication!

What is the Need for Spring Boot Batch Configuration?

Spring boot batch processing is the automated processing of large amounts of data without the need for human intervention thus making it simpler and easier. Further, the Spring boot batch generates a job that includes steps. Each stage will include tasks such as reading, processing, and writing. The spring boot batch chunk aids in the configuration of the execution.

Spring Boot Batch includes reusable functions such as logging/tracing, transaction management, job processing statistics, job restart, skip, and resource management that are necessary when processing large volumes of records. It also provides more advanced technical services and features that, through optimization and partitioning techniques, will enable extremely high-volume and high-performance batch jobs. Simple as well as complex, high-volume batch jobs can use the framework to process large amounts of data in a highly scalable manner.

How to Configure the Spring Boot in Batch Processing?

You can follow the step-by-step configuration guide to configure the Spring Boot in Batch processing. In this case, it showcases a simple example of how Batch processing in Spring Boot works.

1) Batch Processing in Spring Boot: Pom.xml

Spring boot batch and database dependencies are listed in the pom.xml file. A database is required by the spring boot batch to store batch-related information. Spring boot batch dependencies will provide all required classes for batch execution. 

All jars relevant to spring boot batch will be included in the dependency spring-boot-starter-batch. The spring boot application will include the h2 database driver due to the h2 database dependency. You can take the following reference:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi=""
		<relativePath/> <!-- lookup parent from repository -->
	<description>Demo project for Spring Boot</description>

2) Batch Processing in Spring Boot: Spring Boot Main class

The default spring boot main class is used to begin the spring boot batch. There are no additional annotations or configurations on the Main class. You can consider the below class.

package com.yawintutor;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

public class SpringBootBatch2Application {

	public static void main(String[] args) {, args);

3) Batch Processing in Spring Boot: Application properties

In the configuration, two more application properties should be added. 

  • The database url, which will be used to connect to the database, will be included in the first property. 
  • The spring boot batch’s second property will enable it to create batch tables while the application is running.

4) Batch Processing in Spring Boot: Start Spring Boot Application

After completing all of the preceding considerations and changes, you can now launch the spring boot application. It will now function normally. However, if there are any database errors, they should be resolved before proceeding.

5) Batch Processing in Spring Boot: ItemReader Implementation

The ItemReader interface is used to define the reader class. This class should include the data reading code. The spring boot batch will read the data using the read method.

You can go through the following code snippet for defining the reader class.

package com.yawintutor;

import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;

public class MyCustomReader implements ItemReader<String>{

	private String[] stringArray = { "Zero", "One", "Two", "Three", "Four", "Five" };

	private int index = 0;

	public String read() throws Exception, UnexpectedInputException,
			ParseException, NonTransientResourceException {
		if (index >= stringArray.length) {
			return null;
		String data = index + " " + stringArray[index];
		System.out.println("MyCustomReader    : Reading data    : "+ data);
		return data;


6) Batch Processing in Spring Boot: ItemProcessor Implementation

You can define the data processing class using the ItemProcessor interface. This class should contain the data processing code. To process the data, the Spring Boot batch will invoke the process method.

You can refer to the following data processing class in the given code snippet.

package com.yawintutor;

import org.springframework.batch.item.ItemProcessor;

public class MyCustomProcessor implements ItemProcessor<String, String> {

	public String process(String data) throws Exception {
		System.out.println("MyCustomProcessor : Processing data : "+data);
		data = data.toUpperCase();
		return data;


7) Batch Processing in Spring Boot: ItemWriter Implementation

The ItemWriter interface is used to define the writer class. This class should contain the code for writing the data after it has been processed. The spring boot batch will write the data using the write method.

You can follow the given writer class in the code snippet.

package com.yawintutor;

import java.util.List;

import org.springframework.batch.item.ItemWriter;

public class MyCustomWriter implements ItemWriter<String> {

	public void write(List<? extends String> list) throws Exception {
		for (String data : list) {
			System.out.println("MyCustomWriter    : Writing data    : " + data);
		System.out.println("MyCustomWriter    : Writing data    : completed");

8) Batch Processing in Spring Boot: Spring Boot Batch Configurations

The spring boot batch configuration file specifies the batch job and batch steps. The JobBuilderFactory class generates a batch task. The StepBuilderFactory class generates a batch step. The batch steps will be executed by the batch job. Batch jobs such as ItemReader, ItemProcessor, and ItemWriter will be defined in the batch step. The spring boot batch configuration defines how the batch should be run.

You can refer to the following code snippet:

package com.yawintutor;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

public class BatchConfig {
	public JobBuilderFactory jobBuilderFactory;

	public StepBuilderFactory stepBuilderFactory;

	public Job createJob() {
		return jobBuilderFactory.get("MyJob")
				.incrementer(new RunIdIncrementer())
	public Step createStep() {
		return stepBuilderFactory.get("MyStep")
				.<String, String> chunk(1)
				.reader(new MyCustomReader())
				.processor(new MyCustomProcessor())
				.writer(new MyCustomWriter())

9) Batch Processing in Spring Boot: Spring Boot Batch Schedulers

The spring boot batch schedulers are executed automatically, invoking the spring boot batch tasks. The JobLauncher class will run the spring boot batch. A spring boot scheduler is used in this example to launch the spring boot batch at regular intervals.

package com.yawintutor;

import java.text.SimpleDateFormat;
import java.util.Calendar;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.annotation.Scheduled;

public class SchedulerConfig {

	JobLauncher jobLauncher;

	Job job;

	SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.S");

	@Scheduled(fixedDelay = 5000, initialDelay = 5000)
	public void scheduleByFixedRate() throws Exception {
		System.out.println("Batch job starting");
		JobParameters jobParameters = new JobParametersBuilder()
				.addString("time", format.format(Calendar.getInstance().getTime())).toJobParameters();, jobParameters);
		System.out.println("Batch job executed successfullyn");

10) Batch Processing in Spring Boot: Start Spring Boot Application

The spring boot batch configuration is complete. Now, you can start the spring boot batch application. The scheduler will begin the spring boot batch and complete all of the steps and tasks. The log will look something as given below. Logs to read, process, and write tasks are displayed in the log. This process is repeated several times until the data is ready for processing. When the data is finished, the batch will stop.

2021-07-22 23:00:55.897  INFO 39139 --- [           main]      : Job: [FlowJob: [name=MyJob]] launched with the following parameters: [{, time=2021-07-22 16:22:57.293}]
2021-07-22 23:00:55.927  INFO 39139 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [MyStep]
MyCustomReader    : Reading data    : 0 Zero
MyCustomProcessor : Processing data : 0 Zero
MyCustomWriter    : Writing data    : 0 ZERO
MyCustomWriter    : Writing data    : completed
MyCustomReader    : Reading data    : 1 One
MyCustomProcessor : Processing data : 1 One
MyCustomWriter    : Writing data    : 1 ONE
MyCustomWriter    : Writing data    : completed
MyCustomReader    : Reading data    : 2 Two
MyCustomProcessor : Processing data : 2 Two
MyCustomWriter    : Writing data    : 2 TWO
MyCustomWriter    : Writing data    : completed
MyCustomReader    : Reading data    : 3 Three
MyCustomProcessor : Processing data : 3 Three
MyCustomWriter    : Writing data    : 3 THREE
MyCustomWriter    : Writing data    : completed
MyCustomReader    : Reading data    : 4 Four
MyCustomProcessor : Processing data : 4 Four
MyCustomWriter    : Writing data    : 4 FOUR
MyCustomWriter    : Writing data    : completed
MyCustomReader    : Reading data    : 5 Five
MyCustomProcessor : Processing data : 5 Five
MyCustomWriter    : Writing data    : 5 FIVE
MyCustomWriter    : Writing data    : completed
2021-07-22 23:00:55.954  INFO 39139 --- [           main] o.s.batch.core.step.AbstractStep         : Step: [MyStep] executed in 27ms
2021-07-22 23:00:55.958  INFO 39139 --- [           main]      : Job: [FlowJob: [name=MyJob]] completed with the following parameters: [{, time=2021-07-22 16:22:57.293}] and the following status: [COMPLETED] in 42ms
What Makes Hevo’s Data Processing Best-In-Class

Providing a high-quality ETL solution can be a difficult task if you have a large volume of data. Hevo’s automated platform empowers you with everything you need to have a smooth Data Collection, Processing, and Aggregation experience.

Our platform has the following in store for you!

  • Fully Managed: Hevo requires no management and maintenance as it is a fully automated platform.
  • Data Transformation: Hevo provides a simple interface to perfect, modify, and enrich the data you want to transfer.
  • Faster Insight Generation: Hevo offers near real-time data replication so you have access to real-time insight generation and faster decision making. 
  • Schema Management: Hevo can automatically detect the schema of the incoming data and map it to the destination schema.
  • Scalable Infrastructure: Hevo has in-built integrations for 150+ Data sources (with 40+ free sources) that can help you scale your data infrastructure as required.
  • Live Support: Hevo team is available round the clock to extend exceptional support to its customers through chat, email, and support calls.

BONUS PROJECT: Let’s Create a Basic Application Using Batch Processing in Spring Boot

We’ll create a job that reads a CSV file, transforms it with a custom processor, and stores the final results in an in-memory database.

The jobs will be based on importing data from a Coffee list. The steps to be undergone to create the application using Batch Processing in Spring Boot are as follows:

Step 1: Batch Processing in Spring Boot: Maven Dependencies

To create a basic batch-driven application using Spring Boot, you need to first add the spring-boot-starter-batch to your pom.xml file.

You can go through the following code snippet:


You also need to add the org.hsqldb dependency. For this, you can go through the following code snippet:


Step 2: Batch Processing in Spring Boot: Defining a Spring Batch Job

  • First, you need to define the entry point of your application. For this, you can follow the below code snippet.
public class SpringBootBatchProcessingApplication {
    public static void main(String[] args) {, args);
  • Define the application configuration properties in the src/main/resources/ file.

This file is a flat CSV file. Hence, Spring can handle it without any additional modifications. 

It contains the location of your input coffee list. And each line in the list contains different characteristics of the coffee such as brand, origin, etc. The format of the list:

Blue Mountain,Jamaica,Fruity
  • Now, you need to add an SQL script to create a table named “coffee” to store the data. The name of the SQL script is schema-all.sql.
CREATE TABLE coffee  (
    brand VARCHAR(20),
    origin VARCHAR(20),
    characteristics VARCHAR(30)

Now, Spring Boot will automatically run the script whenever you begin.

  • Further, you will create a domain class to hold the items of Coffee. Here, the Coffee class has three properties: brand, origin, and characteristics.
public class Coffee {
    private String brand;
    private String origin;
    private String characteristics;
    public Coffee(String brand, String origin, String characteristics) {
        this.brand = brand;
        this.origin = origin;
        this.characteristics = characteristics;
    // getters and setters

Step 3: Job Configuration

  • You begin with a standard Spring @Configuration class in this step. 
  • After that, you can annotate your class with @EnableBatchProcessing. This annotation provides access to a variety of useful beans that help with jobs, saving time. It will also give you access to some useful factories that will come in handy when creating job configurations and job steps. 
  • You can include a reference to the previously declared file.input property in the final section of the initial configuration.
public class BatchConfiguration {

    public JobBuilderFactory jobBuilderFactory;

    public StepBuilderFactory stepBuilderFactory;
    private String fileInput;
    // ...
  • You need to define a reader bean in the configuration. You can take the following reference, where the reader bean searches for a file named coffee-list.csv and parses each line into an object named “Coffee”.
public FlatFileItemReader reader() {
    return new FlatFileItemReaderBuilder().name("coffeeItemReader")
      .resource(new ClassPathResource(fileInput))
      .names(new String[] { "brand", "origin", "characteristics" })
      .fieldSetMapper(new BeanWrapperFieldSetMapper() {{
public JdbcBatchItemWriter writer(DataSource dataSource) {
    return new JdbcBatchItemWriterBuilder()
      .itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
      .sql("INSERT INTO coffee (brand, origin, characteristics) VALUES (:brand, :origin, :characteristics)")
  • Now, you need to add the actual job steps and configuration as given below.
public Job importUserJob(JobCompletionNotificationListener listener, Step step1) {
    return jobBuilderFactory.get("importUserJob")
      .incrementer(new RunIdIncrementer())
public Step step1(JdbcBatchItemWriter writer) {
    return stepBuilderFactory.get("step1")
      .<Coffee, Coffee> chunk(10)
public CoffeeItemProcessor processor() {
    return new CoffeeItemProcessor();

The step is first configured to write up to ten records at a time using the chunk(10) declaration. The coffee data can then be read using the reader bean, which is configured using the reader method. Following that, you send each coffee item to a custom processor where you can apply some custom business logic. Finally, you use the writer to add each coffee item to the database.

The job definition is contained in importUserJob. It contains an id generated by the RunIDIncrementer class. You’ve also added a JobCompletionNotificationListener to be notified when the job gets completed.

Step 4: The Custom Coffee Processor

The custom processor defined in the previous step looks something like this.

public class CoffeeItemProcessor implements ItemProcessor<Coffee, Coffee> {

    private static final Logger LOGGER = LoggerFactory.getLogger(CoffeeItemProcessor.class);

    public Coffee process(final Coffee coffee) throws Exception {
        String brand = coffee.getBrand().toUpperCase();
        String origin = coffee.getOrigin().toUpperCase();
        String chracteristics = coffee.getCharacteristics().toUpperCase();

        Coffee transformedCoffee = new Coffee(brand, origin, chracteristics);"Converting ( {} ) into ( {} )", coffee, transformedCoffee);

        return transformedCoffee;

The ItemProcessor interface allows you to apply specific business logic during job execution. The CoffeeItemProcessor class is defined, which accepts a Coffee object as input and converts all of its properties to uppercase.

Step 5: Job Completion

You can write a JobCompletionNotificationListermer that can provide some feedback when the job is finished. You can refer to the following code snippet.

public void afterJob(JobExecution jobExecution) {
    if (jobExecution.getStatus() == BatchStatus.COMPLETED) {"!!! JOB FINISHED! Time to verify the results");

        String query = "SELECT brand, origin, characteristics FROM coffee";
        jdbcTemplate.query(query, (rs, row) -> new Coffee(rs.getString(1), rs.getString(2), rs.getString(3)))
          .forEach(coffee ->"Found < {} > in the database.", coffee));

Step 6: Running the Job

After completing everything, now you can run the job. Your job ran successfully and each of the coffee items is stored in the database.

17:41:16.336 [main] INFO  c.b.b.JobCompletionNotificationListener -
  !!! JOB FINISHED! Time to verify the results
17:41:16.336 [main] INFO  c.b.b.JobCompletionNotificationListener -
  Found < Coffee [brand=BLUE MOUNTAIN, origin=JAMAICA, characteristics=FRUITY] > in the database.
17:41:16.337 [main] INFO  c.b.b.JobCompletionNotificationListener -
  Found < Coffee [brand=LAVAZZA, origin=COLOMBIA, characteristics=STRONG] > in the database.
17:41:16.337 [main] INFO  c.b.b.JobCompletionNotificationListener -
  Found < Coffee [brand=FOLGERS, origin=AMERICA, characteristics=SMOKEY] > in the database.


In this article, you have learned about Batch Processing in Spring Boot. This article also provided information on Batch Processing, its key features, the Spring boot framework, its key features, and the need for Spring Boot Batch configuration. It also provides a step-by-step guide to configuring the Spring Boot in Batch Processing and also the creation of an application using Batch Processing in Spring Boot.

Hevo Data, a No-code Data Pipeline provides you with a consistent and reliable solution to manage data transfer between a variety of sources and a wide variety of Desired Destinations with a few clicks.

Visit our Website to Explore Hevo

Hevo Data with its strong integration with 150+ Data Sources (including 40+ Free Sources) allows you to not only export data from your desired data sources & load it to the destination of your choice but also transform & enrich your data to make it analysis-ready. Hevo also allows the integration of data from non-native sources using Hevo’s in-built REST API & Webhooks Connector. You can then focus on your key business needs and perform insightful analysis using BI tools. 

Want to give Hevo a try? Sign Up for a 14-day free trial and experience the feature-rich Hevo suite first hand. You may also have a look at the amazing Hevo price, which will assist you in selecting the best plan for your requirements.

Share your experience of understanding Spring Boot with Spring Batch in the comment section below! We would love to hear your thoughts.

No-code Data Pipeline for your Data Warehouse