More than Just Lines on a Map: Best Practices for U.S Bike Routes
Collaborate 2012 - Elastic cloud is your data management solution ready
1. Elastic Cloud: Is Your Data Management Solution Ready?
Ganesan Rathinam
Chain-Sys Corporation
Austin Davis
Chain-Sys Corporation
Introduction
With Elastic Cloud a reality for E business Suite, there is a need for a flexible approach to data management which
works along with Oracle’s open interfaces for integration as well as traditional bulk data loading needs. This
presentation will propose a flexible approach which can work across a corporate network or on the elastic cloud
framework.
The objectives are to give an overview of architectural approaches for traditional bulk data loading, present usage
of web services to achieve integration and share strategies for data maintenance and cleansing.
Flexible Computing
Economics and Convenience demand flexible computing environment/resources. Before the advent of
virtualization, IT departments had to buy the best and powerful (expensive) hardware that would meet the peak
demand of the Business. Multi-tier computing and load balancing helps a lot to meet such peak loads. Much of the
computing power that is in the enterprise stays idle and with minimal usage for long periods of time. (under
utilization). Years ago in the batch job oriented mainframes era, that machine was worked to its full capacity round
the clock. When you had spare capacity on that mainframe, you’d rent it to run batch jobs from other
organizations.
Virtualization
Here multiple server machines are pooled together and the combined CPU power, RAM and disk capacities are
made available for use by the Virtualization software. In Oracle virtualizations, you’d install Oracle VM Server on
the raw machines and a single console control is made available by installing Oracle VM Manager. Server pool or
pools can be created. The big pool can be used to create a one big massive virtual computing machine or you can
create multiple guest machines with their own guest operating systems and networking capabilities. Virtualization
is different from hosting servers in a data center.
Cloud (Private and Public)
Computing in such virtualized environments has another name: Cloud Computing. When all the nitty gritty details
of virtualization and network management are done by in house IT staff, it is called a Private Cloud. When such
management is done by a third party vendor, it is a Public Cloud. Amazon’s EC2 is an example of a public cloud.
Oracle Public Cloud is another example. (cloud.oracle.com). I can host an application and make it available over the
COLLABORATE 12 Page | 1
2. web to customers. I can claim that the application is on the cloud, whether there is cloud computing going on
behind the scenes or not. Anyways the end users are not given the task of managing servers and operating systems
and they can think that they are working off the cloud (virtually speaking). Cloud is buzzword today. People love to
ride it. Oracle E-Business Suite on Demand would not be strictly falling under cloud computing. Oracle provides
hosting and other Application Management Services.
Traditional EBS Data Load Process
Input data has to be brought into an Oracle Staging table using PL/SQL or SQL*Loader Programs. Custom PLSQL
programs have to be written to get the data from the Staging table to hit the Oracle E-Business Suite API’s and/or
Interface tables. The custom program has to be registered as a concurrent program and the actual executables
need to be migrated from Development to other instances manually. The program has to be invoked manually and
maintained. Errors are sometimes returned in the return parameter, sometimes in a table. The error handling need
to be handled in the programs and error correction is done offline on the Error Tables using SQL update scripts.
Traditional EBS Data Interface Process
Data Interfacing can be offline batch oriented, asynchronous near to real time oriented or online real time
transaction oriented. Batch data extraction and/or loading can be accomplished traditionally with
extraction/conversion programs as described in previous section. Writing/developing programs has always been
time consuming, costly and resource intensive. Real time transaction handling is the domain of technologies,
architectures and languages such as Oracle Service Oriented Architecture (SOA), Oracle Business Process Extension
Language (BPEL), Oracle Process Integration Pack (PIPs), Oracle Application Integration Architecture (AIA), Oracle
Enterprise Service Bus. Oracle iRepository has all the list of public API’s supported by Oracle in their EBS R12
applications ( http://irep.oracle.com or EBS Responsibility: Integration Repository). In house
experts/architects/programmers or external consultants are called in to execute such interface solutions. Please
refer http://www.oracle.com/fr/products/consulting/018856.pdf for information on PIP.
Impact of EBS in Cloud Environment on Data Management
Oracle E-Business suite Application Management Services are provided by Oracle Corporation or other third party
companies that provide Oracle EBS on the cloud. Companies started implementing ERPs in the nineties, moving
away from their custom operational softwares. They wanted to take advantage of BUY vs BUILD, industry best
practices that come built into the ERP solutions, and to reduce the cost of maintaining applications. The next step
has been to give away the responsibilities of maintaining ERP, server hardware and networking equipment. All the
while companies have been keen on concentrating on their core businesses rather than big budget IT projects.
Now Data Loading and Data Interfacing for Oracle E-Business Suite: Do companies want to architect, develop and
maintain such programs or get them as a product or rent them off the cloud? When a product can be bought,
building it would not make much sense, unless specific needs of business prevent such a move.
Customers running enterprise applications off the cloud, want enterprise quality data load and data interface tools
to work with their enterprise operational applications. They are acutely aware of the dynamics and challenges of
developing such programs/tools and wisely choose best of class tools, preferably running in the cloud.
The same applies to customers running enterprise applications in house.
COLLABORATE 12 Page | 2
3. Data Load, Interface Tools for EBS Data Management
There are two kinds of data loading tools: The first kind more or less mimics keystrokes into an Oracle E-Business
Suite form and thereby is like a fast secretary typing into the EBS forms. Many vendors offer loading through the
front end GUI of Oracle EBS. The excel macros pump data into the various fields and hit SUBMIT for each record.
The second acts not on the front end, but interacts more like a well written conversion program.
People used to book conference hall for 3 or 4 days each month and a bank of 20 to 25 pc machines to run
dataloader to load master or transactional data into the EBS instance. That used to be time and labor intensive
even with the EBS instance sitting in the local LAN. The issue is, “Enterprise Applications need Enterprise quality
data load tools”.
EBS Data Load Tools in the Cloud Environment
How about a hub and spoke model? A “NOT FRONT END” loading engine with templates (adapters) for the various
API’s and Interface tables, kept at the hub. And pointing spokes (connectivity) to various Oracle EBS instances, on
the cloud or otherwise. With a powerful server at the hub and with multithreading capabilities, all data loading
should happen in minutes and hours instead of days together.
Another attribute required of an enterprise quality load solution: “No babysitting”. The very idea of Computer was
to take away work from us, humans. The computer is the one that is supposed to slog. We’d intervene. Well at the
right time with right inputs based on classy reports. The boss can be productive only if the secretary does her job
well, gives a good report and highlights areas needing attention and feedback.
Characteristics of right data management tool for EBS in Cloud or otherwise
The data load tool should have connect and load capability. It should be deployed in a server and be accessible
from anywhere and should be able to load date into any one of many Oracle E-Business Suite instances. This is the
“Hub and Spoke” model. Data load should be available and used as a Service. Having multiple EBS instances should
not lead to maintenance effort multiplied. The software should scale to loading of million+ records without having
the need of manual intervention. Error handling should be robust and handled in a uniform and structured fashion.
While doing data load or data interface the tool should not violate any of Oracle EBS’ security measures.
Capabilities should exist for an administrator to trace who loaded this transaction and at what time (Audit Trail).
The centralized data loading tool should be self documenting the processes and provide control across the
organization with respect to data loading or interfacing. The system should have Data Extraction as an In-Built
Solution. Easy capabilities for Data Cleansing, Transformation, Pre-validations should be provided. Templates for
loading/interfacing have to be available.
The data load/interfacing solution has to be a thin client solution. The enterprise job cannot be entrusted to the
desktop machine. Moreover when the tasks are accomplished at a centralized server, the organization has more
control and accountability for the operations performed. Keyboard emulation and screen filling solutions are so
fraught with the limitations on throughput and the vagaries of the desktop machine. E-Business Suite screens are
designed for data or transaction capture from users. If that method is stretched for automating data load, it cannot
prevent EBS dialog box error or warning messages from appearing. Once such a message occurs the automation
comes to a grinding halt until someone remedies/corrects the situation. Similar is the case with excel macro driven
tools. The tool should provide easy drag and drop data mapping.
COLLABORATE 12 Page | 3
4. When people deploy applications on the private or public cloud, they expect the tools that go along also to follow
suit. That is for the ease of maintenance or zero maintenance from the end users point of view. Commercial
reasons such as renting the software tool as against purchasing also favors the tools to be cloud based or cloud
compatible. Especially suited are data migration into R12 projects which typically run for three to twelve months
for renting a data migration tool.
The tool should be future proof in ways such as support for Oracle Fusion Applications. The cost advantage of using
such tools as against in house written conversion programs is that when you upgrade to newer Oracle versions, the
vendor would keep pace and provide the software upgrade and there is no need for rewriting or modifying
conversion programs.
Conclusion
We have identified the direction in which data loading and data interfacing should be moving for best ROI, less
pain and more productivity. The qualities expected in the solutions have been enumerated.
COLLABORATE 12 Page | 4