2. What are Batch Applications ?
● Execution of a series of programs ("jobs")
without manual intervention
● Conducting a group of computer tasks at
one time
Batch Applications
3. The input data is pre-selected through
scripts, command-line parameters or Job-
control language
Batch Applications
4. ● No user intervention
● No UI or minimal UI
● Need not be high
available
● Single machine based
processing
● Larger Input data
● System latency is not
an issue
● Transactions run for
long time
Batch Processing vs Online Processing
● Interacting with System
● Command-line based
UI or GUI interface
● High availability
● Distributed processing
● Smaller Input data
● System response time
is critical
● Short transaction time
6. ● Shift the time of job processing to when the
computing resources are less busy. (e.g., Nightly,
Weekly or Monthly Jobs)
● Avoids idling the computing resources with minute-
by-minute manual intervention and supervision.
● Manage large volume of transactions
● Long running application
Benefits of Batch Processing
7. ● Payroll Processing
● Bank monthly statements
● Credit card companies processing bills
● Fraud detection system
● Extract Transform Load Data processing
● Business Intelligence or Social Intelligence
Reporting
● Predictive or Risk analysis systems
Examples Of Batch Processing
9. ● File-driven applications are driven by
records or values retrieved from a file
● Database-driven applications are driven
by rows or values retrieved from the
database
● Message-driven applications are driven by
messages retrieved from a message
queue
Batch App Categories
10. ● Normal processing in a batch window
during off-line
● Concurrent batch & on-line processing
● Parallel processing of many different batch
runs or jobs at the same time
● Partitioning (i.e. processing of many
instances of the same job at the same
time)
Processing Options
11. ● Simple Batch Repeat
● Automatic Retry After Failure
● Commit Batch Process Periodically: chunk processing
● Asynchronous Chunk Processing: parallel processing within
a chunk.
● Copy File to File in a Batch
● Massively Parallel Batch Processing
● Manual Restart After Failure
● Sequential Processing of Dependent Steps
● Partial Processing: skip records (e.g. on rollback).
● Whole-Batch Transaction
● Scheduled Processing
● Non-Sequential Processing of Steps (Conditional
Branching)
Batch App Use Cases
13. Programming Model to Develop Batch
Application in Java/JEE
Batch Programming Model
14. Developing Batch Application Java/JEE
● Java Batch (JSR-352)
● Spring Batch
● Websphere App Server Batch Support
Batch Programming Model
15. ● Java Batch (JSR 352)
● First time addresses developing Batch
Applications in Java
● Part of JEE 7, Scheduled to release third
quarter of 2013
● IBM & VMware are the Spec lead
JSR-352
16. ● Spring Batch is an open source module
from Spring Framework
● Spring Batch framework started in 2007
● JSR-352 is influenced by Spring Batch
Spring Batch
19. A Job is an instance that encapsulates an entire batch process. A job is
typically put together using a Job Specification Language and consists of
multiple steps. The Job Specification Language is implemented with XML
and is referred as "Job XML".
A Step is a domain object that encapsulates an independent, sequential
phase of a job. A step contains all of the information necessary to define
and control the actual batch processing.
JobOperator provides an interface to manage all aspects of job processing,
including operational commands, such as start, restart, and stop, as well as
job repository commands, such as retrieval of job and step executions.
Batch Processing
20. JobRepository holds information about jobs current
running and jobs that run in the past. JobOperator
provides access to this repository.
Reader-Processor-Writer pattern is the primary pattern
and is called as Chunk-oriented processing.
In this, ItemReader reads one item at a time,
ItemProcessor processes the item based upon the
business logic, such as calculate account balance and
hands it to ItemWriter for aggregation. Once the 'chunk'
number of items are aggregated, they are written out,
and the transaction is committed.
Batch Processing
28. The Big Data and Apache Hadoop
ecosystem the future for the Batch
Application looks good.
Integration of Spring Batch to Hadoop
framework
Batch Application Future