Three Ways to Implement Distributed Processing Within the Oracle Database

Organizations frequently decide to implement a new technology stack instead of fully exploring the capabilities and features of their existing system. This is especially true with Oracle Database, which has a powerful scheduler, advanced queues and the ability to be used with IBM JMS MQ. With these advanced features and capabilities, organizations can develop a distributed processing system in Oracle Database that saves time and money. Instead of building a whole new infrastructure of multiple servers with Java processes or Kubernetes clusters, organizations can enhance the use of existing system resources and expertise by staying with Oracle Database.

Companies that conduct simple to medium complex computations, including online transaction processing (OLTP) and online analytical processing (OLAP) workloads, will likely benefit from this approach. Distributed processing in the Oracle Database can be used in transaction batch balancing, calculating financial stress scenario profits and losses, and simple to medium complex computation needs to quicken overall processing time. When executed correctly, this level of multitasking will enhance the use of existing system resources and expertise and prevent companies from going through frustrations with new tech stack development, such as new specialist hiring, unplanned expenditures, prolonged application deliveries, postproduction issues and new maintenance costs. 

How to Implement Distributed Processing in the Oracle Database

The real question is: Why spend money and valuable employee time implementing new technology stacks when an Oracle Database offers efficient and effective multitasking? Using an existing Oracle Database allows organizations to maximize the use of existing hardware, software and staff knowledge. It also reduces time to market because there is little to no learning curve for staff already familiar with Oracle. Plus, getting up and running can be faster and easier than dealing with new technology. Employees have three options when implementing distributed processing using an Oracle Advanced Queue or an IBM JMS MQ framework:

1. Option 1: A queue-monitoring Oracle scheduler job creates a new scheduler task for each piece of work with a maximum allowed configured number of tasks. This option allows only a certain number of tasks to run in parallel simultaneously, which limits database resource usage and prevents a negative impact on other applications and processes running simultaneously. Also, each task process dies after the work is completed. Then, another brand-new process begins to complete another piece of work (see Figure 1).

2. Option 2: Here, a queue-monitoring job and a minimum number of scheduler tasks run continuously in parallel. These monitor and minimum-configured parallel scheduler tasks run as soon as the system is up, just like with many container processes or servers that run Java processes. Then, if a monitoring job sees a queue length surpassing a certain threshold or a piece of work sits in the queue for too long, the monitor brings additional scheduler tasks up to a defined maximum threshold. After the additional work is completed, the added tasks are killed by the monitor, and the minimum allowed number of tasks continuously runs, which allows automatic horizontal scaling (see Figure 2).

3. Option 3: With this popular option, organizations always run a fixed number of scheduler tasks. As requests come in, the running tasks process them and then wait for new work. With this option, there is no increase in system load, and no additional framework is needed to scale up or down. This is a simple setup as opposed to having another tech stack with containers or servers running Java processes (see Figure 3).

No matter which option they choose, organizations realize many benefits from using distributed processing within the Oracle Database, including data independence, optimized resources, scalability and high availability. Amazon demonstrates what can be achieved with this type of setup. By leveraging distributed processing within Oracle Databases, Amazon handles massive transaction volumes and geographically diverse user bases. According to Thomson Data, more than 76,000 companies from various sectors use Oracle Database, including services, fraud detection, order creation and validation, and finance. Finance company JPMorgan Chase relies on the Oracle Database to improve and streamline various financial operations, including treasury, trade and commerce.

Leveraging Distributed Processing in the Oracle Database

Oracle Database is heavily used in the financial sector, helping companies balance transaction batches in parallel as terminals continue to close the batches. It also promotes stress testing, profit and loss calculations, intraday pricing updates and collateral processing. In the services sector, there are many cases in which Oracle Database is used to break down large service quotes into smaller requests and process requests in parallel to speed up overall response time.

One area in which distributed processing in Oracle particularly shines is transaction batch balancing in card processing. With distributed processing, companies can balance batches in parallel to speed up the processing time and then create overnight funding files to prevent delays in issuing customer payments. This system may also improve fraud detection and regulatory compliance and better ensure that service level agreement (SLA) requirements are met.

Companies can also use distributed processing in the Oracle Database for transaction processing. With this approach, each batch of transactions is self-contained and can be processed independently. Also, a maximum limit of parallel tasks can be configured based on available system resources. For example, normal operations can start with a medium configuration that can be updated to a maximum level under high load conditions. A monitoring process can increase tasks to the configured limit as the load increases and remove them as the load subsides, providing companies with automatic horizontal scaling.

Why Choose Distributed Processing in Oracle over a New Tech Stack?

In addition to its ease of use and efficiency, using distributed processing in the Oracle Database has another significant advantage over creating a new tech stack – it can usually be implemented and maintained by an existing team of Oracle Database administrators, developers or both. That means companies do not need to hire new staff, a common expense with new tech stacks. Oracle Database provides the required functionality so companies don’t have to purchase new queueing and caching mechanisms, hardware or software for internal applications. External applications/clients can use IBM JMS MQ to send work to Oracle Database. 

Dealing with Oracle Database Performance Challenges

Distributed processing in the Oracle Database offers many positives, but several challenges can arise and prevent companies from receiving maximum benefits. Five of the most common challenges include insufficient system resources, process deadlocks/bottlenecks, load balancing/distribution, horizontal scaling and vertical scaling. Actions companies can take to mitigate these risks include:

• Identify how complex computing needs to be done. Before implementing distributed processing in the Oracle Database, it’s crucial for organizations to ask questions, such as:
• Does the database have enough capacity to implement distributed processing?
• Does the existing database team have enough knowledge to accomplish the task?
• Can the company’s existing team implement task orchestration in the Oracle Database?
• Should the new system rely on external or internal caching, or both?
• Will different applications use the same infrastructure and reuse some or all results from previous processing?
• Configure a minimum number of processes to run. It’s essential to base process minimums on the average daily load and nothing else. That means organizations should ignore request bursts because these will be effectively processed over time and won’t have any real business impact.
• Configure the maximum number of processes to run. To determine process maximums, it is vital for organizations to stress the system under very high load conditions, such as three or five times the normal load. The goal here is for other critical workloads on the database not to be negatively impacted during the test. Conducting this exercise will ensure that all applications perform optimally under maximum load.
• Thoroughly analyze the distribution of data per task. It is important for the data to be independent to avoid the possibility of creating deadlocks.
• Thoroughly analyze the distribution of applications. In multihost database applications, it is imperative for the applications to be distributed correctly. That means the same type of tasks run on the same host so that interconnected data transfer is minimal. Proper distribution will boost database performance.
• Consistently review the system metrics. Conducting regular reviews helps identify if the database needs to be scaled horizontally or vertically and whether all applications are performing optimally.

Companies that take the time to prepare for possible challenges can better overcome them and receive the most benefits from the system. 

Tapping into Oracle’s Power and Efficiency

More organizations, especially those in financial sectors like card processing and clearing house management, are discovering they don’t need to spend money and valuable time building a new technology stack like containers or hosting a running Java service for simple to medium complexity tasks. These tasks are efficiently handled in the Oracle Database, provided the database has enough capacity to implement distributed processing and the existing database team has sufficient implementation knowledge (see Figure 4). When these two requirements are met, organizations can realize extensive benefits – from cost savings and easier management and maintenance to knowledge upscaling and the avoidance of multipoint failures.