Five Cool Use Cases for the Spring Component of the SOA Suite 11g

1) http://www.slideshare.net/gschmutz/five-cool-use-cases-for-the-spring-component-in-oracle-soa-suite


2) https://blogs.oracle.com/rammenon/entry/spring_framework_samples

Performance Tuning :- Oracle SOA Suite 11g

When installed out-of-the-box, Oracle SOA Suite 11g performs adequately for a development environment. In this post, some key areas you need to focus on to improve your performance.

Light and heavy load tests were conducted on both synchronous and asynchronous Mediator services (not BPEL yet), and the recommendations are documented below. 

In summary:

·                     Upgrading to PS3 addresses memory instability issues

·                     Size your JVM appropriately

·                     Moving from Sun JDK to JRockit results in a 32% performance improvement

·                     Increasing the Mediator worker threads results in a 30% performance improvement for async services

·                     Changing the audit level from Development to Production results in a 46% performance improvement

·                     Changing the audit level from Production to Off results in a further 61% performance improvement

·                     Tuning the audit configuration causes Production and Off audit levels to perform equally the same

·                     Implementing parallel processing of routing rules may improve performance of Mediator anywhere from 4% to 509%

·                     It is recommended using parallel garbage collection

1. Use SOA Suite above 11.1.1.4

Upgrade to SOA Suite PS3 (11.1.1.4) or above. This resolves an enormous amount of memory issues.

2. Determine the size needed for your Java Heap Space

If you are running AIA Foundation Pack, then realistically you will need to set your Java Heap Space from 6GB to 10GB, depending on the amount of EBOs you are loading and the amount of composites you have. If you are running just SOA Suite, then you could manage with 2GB to 4GB.

1. Start up your SOA server, and wait until all composites are loaded2. Log in to the WebLogic Admin Console (at http://soaserver:7001/console)
3. Navigate to Home --> Servers --> soa_server1 --> Monitoring --> Performance
4. Click on "Garbage Collect"
5. Inspect the Heap Free Current
6. Though every environment is different, increase your heap space until you at least have 2 GB available

The Java Heap Space is configured in$MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:

PORT_MEM_ARGS="-Xms6144m -Xmx6144m ...

3. Set your PermSize (for Sun JDK only)

It is recommended a min and max of either 1GB or 1.5GB for your PermSize.

The PermSize is configured in $MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:

PORT_MEM_ARGS="... -XX:PermSize=1024m -XX:MaxPermSize=1024m ...

4. Set your Nursery Size (for JRockit only)
Generally speaking, the Nursery Size should be around 30% of your Java Heap Space.

The Nursery Size is configured in $MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.shas follows:

PORT_MEM_ARGS="... -Xns2048m ...

5. Use parallel garbage collection (for Sun JDK only)

Garbage collection (and some other recommended settings) is configured in$MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:

PORT_MEM_ARGS="... -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+ExplicitGCInvokesConcurrent -XX:-TraceClassUnloading ...

6. Use JRockit instead of Sun JDK

Performance testing has shown an average of 32% performance improvement when using JRockit versus Sun JDK.

7. Modify and tune the Audit Level

Adjusting the Audit Level and tuning it is necessary.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> Common Properties --> More SOA Infra Advanced Configuration Properties...
3. Click on "Audit Config"
4. Set the following values and click on "Apply" afterwards

AuditConfig/compositeInstanceStateEnabled = false    <-- for="" it="" must="" set="" span="" this="" to="" work="" you="">

AuditConfig/level = Production
AuditConfig/policies/Element_0/isActive = false
AuditConfig/policies/Element_0/name = Immediate
AuditConfig/policies/Element_1/isActive = true
AuditConfig/policies/Element_1/name = Deferred
AuditConfig/policies/Element_1/properties/Element_0/name = maxThreads
AuditConfig/policies/Element_1/properties/Element_0/value = 10
AuditConfig/policies/Element_1/properties/Element_1/name = flushDelay
AuditConfig/policies/Element_1/properties/Element_1/value = 5000
AuditConfig/policies/Element_1/properties/Element_2/name = batchSize
AuditConfig/policies/Element_1/properties/Element_3/value = 100

Performance tests have shown that changing from Development to Production results in an average of 46% performance improvement (without tuning the audit settings).

Performance tests have shown that changing from Production to Off results in an average of  61% performance improvement (without tuning the audit settings). 

Performance tests have shown that if you apply the audit tuning described above, both Production and Off perform the same, effectively translating to a 61% performance improvement of the Production audit level.

8. Modify Mediator Service Engine properties (for Mediator only)

Manipulating the threads will have a direct impact on the ability of the server to process asynchronous Mediator services.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> Mediator Properties
3. Edit the settings as follows:

Metrics Level:                                        Disabled
Parallel Worker Threads:                        12              <-- br="" experiment="" increasing="" this="" value="">Parallel Maximum Rows Retrieved:         600            <-- 50x="" above="" br="" set="" setting="" the="" to="">Parallel Locker Sleep Thread:                 1                <-- for="" o:p="" parallel="" reduces="" routing="" rules="" waits="">

Performance tests have shown that changing from 4 worker threads and  200 max rows to 12 worker threads and 600 rows results in an average of 30% performance improvement for asynchronous Mediator services and 0% performance improvement for synchronous Mediator services.

9. Reduce soa-infra Log Levels

Unless required, reduce the SOA Suite Log Levels to error.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> Logs --> Log Configuration
3. Set all Log Levels to ERROR:1 (SEVERE)
4. This must be repeated for all soa_server managed servers in your cluster 

Performance tests have shown that reducing this from NOTIFICATION:1 (INFO) to ERROR:1 (SEVERE) results in an average of  7% performance improvement.

10. Perform Database Tuning

These are the database parameters I used in a large scale implementation. However, remember that they are based on the database AWR reports, so you may not want to just go ahead and apply them blindly. They are here for your reference and have resulted in some improvement in our environment.

AUDIT_TRAIL            = NONE
DB_FILES               = 1024
DISK_ASYNCH_IO         = TRUE 
FILESYSTEMIO_OPTIONS   = SETALL
JOB_QUEUE_PROCESSES    = 10
LOG_BUFFER             = 209715200 
NLS_SORT               = BINARY
SESSION_CACHED_CURSORS = 500
PROCESSES              = 1500
SESSION_MAX_OPEN_FILES = 50
UNDO_MANAGEMENT        = AUTO
PLSQL_CODE_TYPE        = NATIVE
MEMORY_TARGET          = 0
SGA_TARGET             = 6g 
PGA_AGGREGATE_TARGET   = 2g 
TRACE_ENABLED          = FALSE

11. Modify BPEL Process Manager Properties (for BPEL only)

Manipulating the threads will have a direct impact on the ability of the server to process asynchronous BPEL processes.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> BPEL Properties
3. Edit the settings as follows:

Dispatcher System Threads:       10            <-- 10="" 2="" at="" br="" default="" from="" increase="" least="" of="" the="" to="" value="">Dispatcher Invoke Threads:         20            <-- br="" can="" depends="" handle="" if="" load="" targets="" the="" your="">Dispatcher Engine Threads:        30            <-- above="" br="" exceed="" not="" of="" should="" sum="" the="" two="" values="">Payload Validation:                    Disabled

Honestly, these numbers will vary based on the types of process designs you have.


Applicable Versions:

·                     Oracle SOA Suite 11g (11.1.1.4) or above

RestfulService using MyEclipse

http://www.myeclipseide.com/documentation/quickstarts/webservices_rest/

Important OSB Tips

http://anindyabhattacharjee.blogspot.com/2011/12/difference-in-retrieving-value-from-osb.html

http://svgonugu.wordpress.com/2011/06/15/common-mistakes-in-osb-message-flow/

Good article to handle empty elements while summation.


http://stackoverflow.com/questions/3563510/how-to-handel-empty-elements-during-doing-sum-of-elements-using-xslt

OSB Threading Model

Threading Model in OSB is explained with different scenarios:

Basic Thread Model

OSB goes to great lengths to avoid holding on to threads.  Lets start by looking at how how OSB deals with a simple request/response routing to a business service in a route node.

Most Business Services are implemented by OSB in two parts.  The first part uses the request thread to send the request to the target.  In the diagram this is represented by the thread T1.  After sending the request to the target (the Business Service in our diagram) the request thread is released back to whatever pool it came from.  A multiplexor (muxer) is used to wait for the response.  When the response is received the muxer hands off the response to a new thread that is used to execute the response pipeline, this is represented in the diagram by T2.

OSB allows you to assign different Work Managers and hence different thread pools to each Proxy Service and Business Service.  In our example we have the “Proxy Service Work Manager” assigned to the Proxy Service and the “Business Service Work Manager” assigned to the Business Service.  Note that the Business Service Work Manager is only used to assign the thread to process the response, it is never used to process the request.

This architecture means that while waiting for a response from a business service there are no threads in use, which makes for better scalability in terms of thread usage.

First Observation

Note that if the Proxy and the Business Service both use the same Work Manager then there is potential for starvation.  For example:

• Request Pipeline makes a blocking callout, say to perform a database read.
• Business Service response tries to allocate a thread from thread pool but all threads are blocked in the database read.
• New requests arrive and contend with responses arriving for the available threads.

Similar problems can occur if the response pipeline blocks for some reason, maybe a database update for example.

Solution

The solution to this is to make sure that the Proxy and Business Service use different Work Managers so that they do not contend with each other for threads.

Do Nothing Route Thread Model

So what happens if there is no route node?  In this case OSB just echoes the Request message as a Response message, but what happens to the threads?  OSB still uses a separate thread for the response, but in this case the Work Manager used is the Default Work Manager.

So this is really a special case of the Basic Thread Model discussed above, except that the response pipeline will always execute on the Default Work Manager.

Proxy Chaining Thread Model

So what happens when the route node is actually calling a Proxy Service rather than a Business Service, does the second Proxy Service use its own Thread or does it re-use the thread of the original Request Pipeline?

Well as you can see from the diagram when a route node calls another proxy service then the original Work Manager is used for both request pipelines.  Similarly the response pipeline uses the Work Manager associated with the ultimate Business Service invoked via a Route Node.  This actually fits in with the earlier description I gave about Business Services and by extension Route Nodes they “… uses the request thread to send the request to the target”.

Call Out Threading Model

So what happens when you make a Service Callout to a Business Service from within a pipeline.  The documentation says that “The pipeline processor will block the thread until the response arrives asynchronously” when using a Service Callout.  What this means is that the target Business Service is called using the pipeline thread but the response is also handled by the pipeline thread.  This implies that the pipeline thread blocks waiting for a response.  It is the handling of this response that behaves in an unexpected way.

When a Business Service is called via a Service Callout, the calling thread is suspended after sending the request, but unlike the Route Node case the thread is not released, it waits for the response.  The muxer uses the Business Service Work Manager to allocate a thread to process the response, but in this case processing the response means getting the response and notifying the blocked pipeline thread that the response is available.  The original pipeline thread can then continue to process the response.

Second Observation

This leads to an unfortunate wrinkle.  If the Business Service is using the same Work Manager as the Pipeline then it is possible for starvation or a deadlock to occur.  The scenario is as follows:

• Pipeline makes a Callout and the thread is suspended but still allocated
• Multiple Pipeline instances using the same Work Manager are in this state (common for a system under load)
• Response comes back but all Work Manager threads are allocated to blocked pipelines.
• Response cannot be processed and so pipeline threads never unblock – deadlock!

Solution

The solution to this is to make sure that any Business Services used by a Callout in a pipeline use a different Work Manager to the pipeline itself.

Summary

• Request Pipeline
• Executes on Proxy Work Manager (WM) Thread so limited by setting of that WM.  If no WM specified then uses WLS default WM.
• Route Node
• Request sent using Proxy WM Thread
• Proxy WM Thread is released before getting response
• Muxer is used to handle response
• Muxer hands off response to Business Service (BS) WM
• Response Pipeline
• Executes on Routed Business Service WM Thread so limited by setting of that WM.  If no WM specified then uses WLS default WM.
• No Route Node (Echo functionality)
• Proxy WM thread released
• New thread from the default WM used for response pipeline
• Service Callout
• Request sent using proxy pipeline thread
• Proxy thread is suspended (not released) until the response comes back
• Notification of response handled by BS WM thread so limited by setting of that WM.  If no WM specified then uses WLS default WM.
• Note this is a very short lived use of the thread
• After notification by callout BS WM thread that thread is released and execution continues on the original pipeline thread.
• Route/Callout to Proxy Service
• Request Pipeline of callee executes on requestor thread
• Response Pipeline of caller executes on response thread of requested proxy
• Throttling
• Request message may be queued if limit reached.
• Requesting thread is released (route node) or suspended (callout)

So what this means is that you may get deadlocks caused by thread starvation if you use the same thread pool for the business service in a route node and the business service in a callout from the response pipeline because the callout will need a notification thread from the same thread pool as the response pipeline.  This was the problem we were having.
You get a similar problem if you use the same work manager for the proxy request pipeline and a business service callout from that request pipeline.
It also means you may want to have different work managers for the proxy and business service in the route node.
Basically you need to think carefully about how threading impacts your proxy services.

OSB Nice Blog

http://allthingsmdw.blogspot.com/2012/09/osb-service-callouts-and-oql-part-3.html