BEA-110486 (Transaction BEA1-018F7B9BE9D9B3789CED) cannot complete commit processing because resource [cgDataSource] is unavailable

The error message is:

The solution was very easy, but taked much time to find.
Follow the steps:

1. Stop your server.
2. Go to [your domain]\servers\AdminServer\data\store\default.
3. Delete* _WLS_ADMINSERVER_[random number].DAT
4. Restart server

http://forums.oracle.com/forums/thread.jspa?threadID=692320

Option to force the JVM to send the User ID to the queue manager for authentication

Please add below string to setenv.ksh file

### Add option to force the JVM to send the User ID to the queue manager for authentication 

JAVA_OPTIONS=”${JAVA_OPTIONS} –Dcom.ibm.mq.jms.ForceUserID=true “

Five Cool Use Cases for the Spring Component of the SOA Suite 11g

1) http://www.slideshare.net/gschmutz/five-cool-use-cases-for-the-spring-component-in-oracle-soa-suite


2) https://blogs.oracle.com/rammenon/entry/spring_framework_samples

Performance Tuning :- Oracle SOA Suite 11g

When installed out-of-the-box, Oracle SOA Suite 11g performs adequately for a development environment. In this post, some key areas you need to focus on to improve your performance.

Light and heavy load tests were conducted on both synchronous and asynchronous Mediator services (not BPEL yet), and the recommendations are documented below. 

In summary:

·                     Upgrading to PS3 addresses memory instability issues

·                     Size your JVM appropriately

·                     Moving from Sun JDK to JRockit results in a 32% performance improvement

·                     Increasing the Mediator worker threads results in a 30% performance improvement for async services

·                     Changing the audit level from Development to Production results in a 46% performance improvement

·                     Changing the audit level from Production to Off results in a further 61% performance improvement

·                     Tuning the audit configuration causes Production and Off audit levels to perform equally the same

·                     Implementing parallel processing of routing rules may improve performance of Mediator anywhere from 4% to 509%

·                     It is recommended using parallel garbage collection

1. Use SOA Suite above 11.1.1.4

Upgrade to SOA Suite PS3 (11.1.1.4) or above. This resolves an enormous amount of memory issues.

2. Determine the size needed for your Java Heap Space

If you are running AIA Foundation Pack, then realistically you will need to set your Java Heap Space from 6GB to 10GB, depending on the amount of EBOs you are loading and the amount of composites you have. If you are running just SOA Suite, then you could manage with 2GB to 4GB.

1. Start up your SOA server, and wait until all composites are loaded2. Log in to the WebLogic Admin Console (at http://soaserver:7001/console)
3. Navigate to Home --> Servers --> soa_server1 --> Monitoring --> Performance
4. Click on "Garbage Collect"
5. Inspect the Heap Free Current
6. Though every environment is different, increase your heap space until you at least have 2 GB available

The Java Heap Space is configured in$MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:

PORT_MEM_ARGS="-Xms6144m -Xmx6144m ...

3. Set your PermSize (for Sun JDK only)

It is recommended a min and max of either 1GB or 1.5GB for your PermSize.

The PermSize is configured in $MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:

PORT_MEM_ARGS="... -XX:PermSize=1024m -XX:MaxPermSize=1024m ...

4. Set your Nursery Size (for JRockit only)
Generally speaking, the Nursery Size should be around 30% of your Java Heap Space.

The Nursery Size is configured in $MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.shas follows:

PORT_MEM_ARGS="... -Xns2048m ...

5. Use parallel garbage collection (for Sun JDK only)

Garbage collection (and some other recommended settings) is configured in$MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:

PORT_MEM_ARGS="... -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+ExplicitGCInvokesConcurrent -XX:-TraceClassUnloading ...

6. Use JRockit instead of Sun JDK

Performance testing has shown an average of 32% performance improvement when using JRockit versus Sun JDK.

7. Modify and tune the Audit Level

Adjusting the Audit Level and tuning it is necessary.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> Common Properties --> More SOA Infra Advanced Configuration Properties...
3. Click on "Audit Config"
4. Set the following values and click on "Apply" afterwards

AuditConfig/compositeInstanceStateEnabled = false    <-- for="" it="" must="" set="" span="" this="" to="" work="" you="">

AuditConfig/level = Production
AuditConfig/policies/Element_0/isActive = false
AuditConfig/policies/Element_0/name = Immediate
AuditConfig/policies/Element_1/isActive = true
AuditConfig/policies/Element_1/name = Deferred
AuditConfig/policies/Element_1/properties/Element_0/name = maxThreads
AuditConfig/policies/Element_1/properties/Element_0/value = 10
AuditConfig/policies/Element_1/properties/Element_1/name = flushDelay
AuditConfig/policies/Element_1/properties/Element_1/value = 5000
AuditConfig/policies/Element_1/properties/Element_2/name = batchSize
AuditConfig/policies/Element_1/properties/Element_3/value = 100

Performance tests have shown that changing from Development to Production results in an average of 46% performance improvement (without tuning the audit settings).

Performance tests have shown that changing from Production to Off results in an average of  61% performance improvement (without tuning the audit settings). 

Performance tests have shown that if you apply the audit tuning described above, both Production and Off perform the same, effectively translating to a 61% performance improvement of the Production audit level.

8. Modify Mediator Service Engine properties (for Mediator only)

Manipulating the threads will have a direct impact on the ability of the server to process asynchronous Mediator services.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> Mediator Properties
3. Edit the settings as follows:

Metrics Level:                                        Disabled
Parallel Worker Threads:                        12              <-- br="" experiment="" increasing="" this="" value="">Parallel Maximum Rows Retrieved:         600            <-- 50x="" above="" br="" set="" setting="" the="" to="">Parallel Locker Sleep Thread:                 1                <-- for="" o:p="" parallel="" reduces="" routing="" rules="" waits="">

Performance tests have shown that changing from 4 worker threads and  200 max rows to 12 worker threads and 600 rows results in an average of 30% performance improvement for asynchronous Mediator services and 0% performance improvement for synchronous Mediator services.

9. Reduce soa-infra Log Levels

Unless required, reduce the SOA Suite Log Levels to error.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> Logs --> Log Configuration
3. Set all Log Levels to ERROR:1 (SEVERE)
4. This must be repeated for all soa_server managed servers in your cluster 

Performance tests have shown that reducing this from NOTIFICATION:1 (INFO) to ERROR:1 (SEVERE) results in an average of  7% performance improvement.

10. Perform Database Tuning

These are the database parameters I used in a large scale implementation. However, remember that they are based on the database AWR reports, so you may not want to just go ahead and apply them blindly. They are here for your reference and have resulted in some improvement in our environment.

AUDIT_TRAIL            = NONE
DB_FILES               = 1024
DISK_ASYNCH_IO         = TRUE 
FILESYSTEMIO_OPTIONS   = SETALL
JOB_QUEUE_PROCESSES    = 10
LOG_BUFFER             = 209715200 
NLS_SORT               = BINARY
SESSION_CACHED_CURSORS = 500
PROCESSES              = 1500
SESSION_MAX_OPEN_FILES = 50
UNDO_MANAGEMENT        = AUTO
PLSQL_CODE_TYPE        = NATIVE
MEMORY_TARGET          = 0
SGA_TARGET             = 6g 
PGA_AGGREGATE_TARGET   = 2g 
TRACE_ENABLED          = FALSE

11. Modify BPEL Process Manager Properties (for BPEL only)

Manipulating the threads will have a direct impact on the ability of the server to process asynchronous BPEL processes.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> BPEL Properties
3. Edit the settings as follows:

Dispatcher System Threads:       10            <-- 10="" 2="" at="" br="" default="" from="" increase="" least="" of="" the="" to="" value="">Dispatcher Invoke Threads:         20            <-- br="" can="" depends="" handle="" if="" load="" targets="" the="" your="">Dispatcher Engine Threads:        30            <-- above="" br="" exceed="" not="" of="" should="" sum="" the="" two="" values="">Payload Validation:                    Disabled

Honestly, these numbers will vary based on the types of process designs you have.


Applicable Versions:

·                     Oracle SOA Suite 11g (11.1.1.4) or above

RestfulService using MyEclipse

http://www.myeclipseide.com/documentation/quickstarts/webservices_rest/

Important OSB Tips

http://anindyabhattacharjee.blogspot.com/2011/12/difference-in-retrieving-value-from-osb.html

http://svgonugu.wordpress.com/2011/06/15/common-mistakes-in-osb-message-flow/

Good article to handle empty elements while summation.


http://stackoverflow.com/questions/3563510/how-to-handel-empty-elements-during-doing-sum-of-elements-using-xslt

OSB Threading Model

Threading Model in OSB is explained with different scenarios:

Basic Thread Model

OSB goes to great lengths to avoid holding on to threads.  Lets start by looking at how how OSB deals with a simple request/response routing to a business service in a route node.

Most Business Services are implemented by OSB in two parts.  The first part uses the request thread to send the request to the target.  In the diagram this is represented by the thread T1.  After sending the request to the target (the Business Service in our diagram) the request thread is released back to whatever pool it came from.  A multiplexor (muxer) is used to wait for the response.  When the response is received the muxer hands off the response to a new thread that is used to execute the response pipeline, this is represented in the diagram by T2.

OSB allows you to assign different Work Managers and hence different thread pools to each Proxy Service and Business Service.  In our example we have the “Proxy Service Work Manager” assigned to the Proxy Service and the “Business Service Work Manager” assigned to the Business Service.  Note that the Business Service Work Manager is only used to assign the thread to process the response, it is never used to process the request.

This architecture means that while waiting for a response from a business service there are no threads in use, which makes for better scalability in terms of thread usage.

First Observation

Note that if the Proxy and the Business Service both use the same Work Manager then there is potential for starvation.  For example:

• Request Pipeline makes a blocking callout, say to perform a database read.
• Business Service response tries to allocate a thread from thread pool but all threads are blocked in the database read.
• New requests arrive and contend with responses arriving for the available threads.

Similar problems can occur if the response pipeline blocks for some reason, maybe a database update for example.

Solution

The solution to this is to make sure that the Proxy and Business Service use different Work Managers so that they do not contend with each other for threads.

Do Nothing Route Thread Model

So what happens if there is no route node?  In this case OSB just echoes the Request message as a Response message, but what happens to the threads?  OSB still uses a separate thread for the response, but in this case the Work Manager used is the Default Work Manager.

So this is really a special case of the Basic Thread Model discussed above, except that the response pipeline will always execute on the Default Work Manager.

Proxy Chaining Thread Model

So what happens when the route node is actually calling a Proxy Service rather than a Business Service, does the second Proxy Service use its own Thread or does it re-use the thread of the original Request Pipeline?

Well as you can see from the diagram when a route node calls another proxy service then the original Work Manager is used for both request pipelines.  Similarly the response pipeline uses the Work Manager associated with the ultimate Business Service invoked via a Route Node.  This actually fits in with the earlier description I gave about Business Services and by extension Route Nodes they “… uses the request thread to send the request to the target”.

Call Out Threading Model

So what happens when you make a Service Callout to a Business Service from within a pipeline.  The documentation says that “The pipeline processor will block the thread until the response arrives asynchronously” when using a Service Callout.  What this means is that the target Business Service is called using the pipeline thread but the response is also handled by the pipeline thread.  This implies that the pipeline thread blocks waiting for a response.  It is the handling of this response that behaves in an unexpected way.

When a Business Service is called via a Service Callout, the calling thread is suspended after sending the request, but unlike the Route Node case the thread is not released, it waits for the response.  The muxer uses the Business Service Work Manager to allocate a thread to process the response, but in this case processing the response means getting the response and notifying the blocked pipeline thread that the response is available.  The original pipeline thread can then continue to process the response.

Second Observation

This leads to an unfortunate wrinkle.  If the Business Service is using the same Work Manager as the Pipeline then it is possible for starvation or a deadlock to occur.  The scenario is as follows:

• Pipeline makes a Callout and the thread is suspended but still allocated
• Multiple Pipeline instances using the same Work Manager are in this state (common for a system under load)
• Response comes back but all Work Manager threads are allocated to blocked pipelines.
• Response cannot be processed and so pipeline threads never unblock – deadlock!

Solution

The solution to this is to make sure that any Business Services used by a Callout in a pipeline use a different Work Manager to the pipeline itself.

Summary

• Request Pipeline
• Executes on Proxy Work Manager (WM) Thread so limited by setting of that WM.  If no WM specified then uses WLS default WM.
• Route Node
• Request sent using Proxy WM Thread
• Proxy WM Thread is released before getting response
• Muxer is used to handle response
• Muxer hands off response to Business Service (BS) WM
• Response Pipeline
• Executes on Routed Business Service WM Thread so limited by setting of that WM.  If no WM specified then uses WLS default WM.
• No Route Node (Echo functionality)
• Proxy WM thread released
• New thread from the default WM used for response pipeline
• Service Callout
• Request sent using proxy pipeline thread
• Proxy thread is suspended (not released) until the response comes back
• Notification of response handled by BS WM thread so limited by setting of that WM.  If no WM specified then uses WLS default WM.
• Note this is a very short lived use of the thread
• After notification by callout BS WM thread that thread is released and execution continues on the original pipeline thread.
• Route/Callout to Proxy Service
• Request Pipeline of callee executes on requestor thread
• Response Pipeline of caller executes on response thread of requested proxy
• Throttling
• Request message may be queued if limit reached.
• Requesting thread is released (route node) or suspended (callout)

So what this means is that you may get deadlocks caused by thread starvation if you use the same thread pool for the business service in a route node and the business service in a callout from the response pipeline because the callout will need a notification thread from the same thread pool as the response pipeline.  This was the problem we were having.
You get a similar problem if you use the same work manager for the proxy request pipeline and a business service callout from that request pipeline.
It also means you may want to have different work managers for the proxy and business service in the route node.
Basically you need to think carefully about how threading impacts your proxy services.

OSB Nice Blog

http://allthingsmdw.blogspot.com/2012/09/osb-service-callouts-and-oql-part-3.html

Nice link to learn technology fast

http://www.tutorialspoint.com/index.htm

Deployment plan file to replace the JCA Adapters properties – Oracle SOA 11g

We can create and edit a configuration plan file in which we can replace the following attributes and properties:

•  Any composite, service component, reference, service, and binding properties in the SOA composite application file (composite.xml)
• Attribute values for bindings (for example, the location for binding.ws)
• schemaLocation attribute of an import in a WSDL file
• location attribute of an include in a WSDL file
• schemaLocation attribute of an include, import, and redefine in an XSD file
• Any properties in JCA adapter files

The configuration plan does not alter XSLT artifacts in the SOA composite application. If you want to modify any XSL, do so in the XSLT Mapper. Using a configuration plan is not useful. For example, you cannot change references in XSL using the configuration plan file. Instead, they must be changed manually in the XSLT Mapper in Oracle JDeveloper when moving to and from test, development, and production environments.

Generate the Plan file:

The plan file can be generated through JDeveloper or through ANT script.

Generating Plan file through JDeveloper:

Right Click on Composite.xml and click on generate config Plan.

Change the generated plan file contents accordingly to replace the properties and attributes of the composites.

Change the wsdlAndSchema section accordingly to replace the WSDL location and the schemaLocation of the WSDL’s and XSD’s

  

     

         http://localhost:7001/

         http://10.15.23.24:8004/

     

  

By default the wsdlAndSchema section will include the jca files to replace the JCA file properties.

The default configuration as shown below is not replacing the jca properties after the deployment of the composite.

     

D://Files

  D://Files//Sample     

  

The same should be changed as shown below to replace the JCA adapter properties 

File Adapter properties:

     

        

            D://Files

            D://Files//Sample

        

     

      

        

            false

            true

        

     

     

  

MQ Adapter properties:

                                  

       

            S_Order_Queue

            P_Order_Queue

       

       

  

The same searchReplace can be used to replace all the JCA adapters’ properties.

The plan file can be attached through ant script during the deployment of the composite to the server.

                 target="deploy">
           
           
           
           
           
           
           
           
            
       

DB Adapter Message Throttling in Oracle SOA 11g

DB throttling is the mechanism to control the number of database records processed by the SOA engine in a particular interval through DB Adapter. 

Throttling also can be used to control the number of records send to the end systems. If the throttling is not defined, the end systems may flood with number of messages that will affect the functioning of the end systems. Throttling parameters should be configured based on the end systems capacity to process the incoming messages.

Until Oracle SOA 11.1.1.6.0 the message throttling will not work as we expect, all the messages matching the where condition of the polling sql are processed in the same polling interval.

As of Oracle Adapters release 11.1.1.6.0 we can set the inbound DBAdapter property RowsPerPollingInterval to control the throttling. It acts as a limit on the number of records which can be processed in one polling interval. The default value is unlimited.

The Patch 12881289 should be applied to enable this for SOA 11.1.1.5.0 and earlier versions.

The maximum rows processed per second are:

Number of active nodes in SOA cluster x NumberOfThreads x RowsPerPollingInterval / PollingInterval

MaxTransactionSize can be thought of as RowsPerDatabaseTransaction or DatabaseFetchSize that is how many records will be fetched to DB Adapter engine from the database for each transaction. It does not affect how many rows can be processed in one polling interval period.

The one exception is the following configuration:

-distributed polling checked, usesSkipLocking="false"

In this one case RowsPerPollingInterval will default to MaxTransactionSize instead of unlimited in case of RowsPerPollingInterval property is not specified in the jca file.

If RowsPerPollingInterval is set to lower than MaxTransactionSize or MaxRaiseSize, they will be effectively lowered toRowsPerPollingInterval that means the value RowsPerPollingInterval will considered for MaxTransactionSize and MaxRaiseSize.

There is no UI support to add the RowsPerPollingInterval property. We have to add the property manually to the db.jca file for the inbound polling service. Add it to the same section as the properties MaxRaiseSize, MaxTransactionSize, and PollingInterval, in any order.

Transaction Management In OSB

Much awaited transaction management feature in Oracle Service Bus (OSB) message flows has been incorporated in the latest Oracle Service Bus release (OSB 11g). Now every OSB proxy service comes with two additional configuration parameters (check boxes) “Transaction Required” and “Same Transaction for Response” to deal with the transaction requirements.

Transaction Required, selecting this option ensures OSB execute the proxy service message flow in the context of a transaction. If a global transaction already exists, the transport provider propagates it in the request. If no global transaction exists, the message flow run time starts a new transaction. If the message flow run time starts a new transaction, the transaction-context begins before the service configuration is validated or run (for example, security checking or WS-I validation), and before message flow execution, ensuring that all processing and execution occurs in the transaction context.

If the message flow run time starts a transaction, quality of service (QoS) is exactly-once. However, if Service Callouts or Publish actions have the outbound QoS parameter set to best-effort, OSB executes those actions outside of the transaction context. To have OSB execute those actions in the same request transaction context, QoS on those actions need to be set to exactly-once. The service maintains its original messaging pattern (synchronous, asynchronous, one-way) regardless of the setting on this option.

This configuration has a catch; On selecting this option OSB invokes the system error handler for all failed transactions. Failed transactions cannot be catch in a user-configured error handler. For synchronous patterns, a transaction-exception is returned through the response. For asynchronous patterns, where the transaction is designed to be committed in the request, the exception is sent back on the request thread. Note that in asynchronous patterns, an error in the response that occurs after transaction committal in the request does not affect the transaction.

Another transactional configuration parameter is Same Transaction for Response, If the transaction of the request thread need to be propagated to the response thread this option also needs to be selected. On selecting this option, the message pattern becomes synchronous automatically, regardless of the initial message pattern setting.

Some do and don't while implementing OSB transaction options.

Always define the transaction timeouts for global transactions. Before setting the transaction timeout make sure that the timeout is sufficient to complete the message flow.

For those business requirements where request required transaction committal before sending the response, such as in a one-way, asynchronous pattern do not select Same Transaction for Response option. Selecting Same Transaction for Response option will override the pattern to synchronous.

 For making non-business interactions call (such as logging/message tracking services call) within a transactional flows set QoS to best effort. This will make the services to call outside the transaction on an event of their failure the main transaction will not be affected.

Article to learn technologies fast

http://www.tutorialspoint.com/index.htm

Weblogic Monitoring Dashboard to monitor the resources

Monitoring dashboard is the new feature of latest weblogic server to view diagnostic data without doing additional setup.

This is the extension of the WLDF framework. The Monitoring Dashboard provides views and tools for graphically presenting diagnostic data about servers and applications running on them. The underlying functionality for generating, retrieving, and persisting diagnostic data is provided by the WebLogic Diagnostics Framework. The Monitoring Dashboard provides additional tools for presenting that data in charts and graphs.

The Monitoring dashboard can be launched from the home page of the Web logic administration console or pointing to the direct URL.

Direct URL to access the Monitoring Dashboard - http://:/console/dashboard

The different resources like JMS, JVM, JDBC and Thread Pools can be monitored here.

The diagnostic data can be captured for a particular resource in a particular time by using the start and stop functionality.

The user also can create the custom views; Custom views are available only to the user who created them. Custom views are automatically persisted for the user and are in effect only for that user account and only in the current domain.

Refer the below URL for more details about dashboard

http://docs.oracle.com/cd/E14571_01/web.1111/e13714/dashboard.htm