Oracle BPEL Variable Sensors for logging

Logging in Oracle BPEL at times becomes necessary even though we got a powerful and user-friendly Enterprise Manager(EM) console which acts as a debugging tool where you can trace and track BPEL process instances. Such as, in BPEL process instance flow that got executed you may have many sequential activities with multiple loop statements within, it will be difficult to determine what has happened in the execution of such instances from the long list of activities showing up on EM console.

Logging in BPEL can be realised using Oracle BPEL Variable Sensors. Sensors in BPEL are of three different types; variable sensors are used for logging which can be triggered when value in the process variable gets changed. In order to get this implemented, you need to create a process variable, define sensor variable from the process variable, and then define and map the action that needs to be done when the sensor is triggered.  Note that the variable sensor is triggered each time when the value contained in the variable gets changed. 

1. Create a Process variable

Define a process variable from Create Variable screen of BPEL , define the variable of string data type.

2. Define Sensor variable from Process variable

In monitor view of the BPEL component file in JDeveloper, click on the Monitoring Objects option to select the Sensors option.

Select the Variable Sensor option from the Sensors window. Name the sensor and chose the target as process variable created in step 1.

3. Define Sensor Action

Click on the Sensor Action option from the drop down menu of Monitoring Objects in the Monitor View of BPEL Component in JDeveloper.

Create a Sensor Action with Publish Type as Database.

4. Map the Sensor Action to Sensor Variable

Edit the Sensor Variable created in step 2 to map it to the sensor action of step 3 from the Edit Variable Sensor window of Sensor Variable.

Now that you have configured BPEL variable sensors, any change in the value of BPEL sensor variable will be shown in the Sensor Values  tab of the BPEL flow instance of EM console.

Thus, you can use variable sensors to populate with meaningful data at various places in the BPEL program to capture the process instance flow at run time.

Minimum File Age Oracle SOA File/FTP Adapter Property

Oracle SOA File or the FTP adapter comes equipped with several properties during design time that can be used to meet the project requirements. Minimum File Age is one such property though may not be used all the time during the design of the File/FTP adapter components in the composite but when used, it does play a significant role in the way the files are picked from the File/FTP server.

Important points to be noted regarding the Minimum File Age Property:

• Minimum Age is applicable only in case of a File/FTP poller operation (Read or the GET)
• Minimum Age value is always specified in units of seconds in the File or the FTP JCA file of your composite though you may set the property using a different time unit in the File/FTP configuration wizard.
• The property more commonly used to process large files that takes longer time by the application to write to the File/FTP server location from where the File/FTP adapter polls it.
• When Minimum File Age property is set, the adapter takes into account the file modification time in order to pick the file.

Significance of Minimum File Age

Let me illustrate it with an example - Consider a use case, where you have multiple files with varying size written onto the FTP server location by some application. The large files may take more time to have them completely written on the FTP server path. During the time such large files are being written to the file location, the file modification time keeps changing.The FTP poller of the composite polling for a file on this file location with minimum file age unset(or set to zero) the poller may pick the file that's corrupt or invalid.   To avoid this, minimum age need to be set to an appropriate value in the FTP Adapter, that way only those files that are completely written are picked by the adapter for further processing.

The Minimum File Age value can be specified in the same screen of the File/FTP Adapter configuration wizard screen where you mention the polling frequency (as shown above)

Configuration plans for Oracle SOA Composites

Deployments of SOA composite(s) onto a SOA server environment is one of the challenging task for a SOA Developer/Admin, more so when certain composite or components parameters will need to be set differently from one server environment to another.  For instance, a composite with DB Adapter poller component deployed on a multi-node server(Clustered environment) will need to have a singleton property set to true to ensure that only one instance of adapter is triggered for a transaction. However, the singleton property may not hold good in a single node environment. The values of such properties and many such changes can be handled using the configuration plan during deployment of composites.

The configuration plan is applied to the composite during deployment. In this post, let's look at the way to generate a configuration plan, explain different section of a configuration plan and few of the ways the plan can be applied during composite deployments. Normally, the case is a developer's environment will not have any configuration plan and the default values specified in the project files are taken into account during composite deployment (It's true in the projects that I  had worked until now). The configuration plan is applied during composite deployment in server environment higher up to the development like test and production environments and values of the development environment can be over-ridden to environment specific values. Note that the configuration plan can also be defined at the SOA application level in which case it is applied to all projects within that application. In this post, to keep it simpler let's look at the plan being defined at the project level.


Generate a Configuration Plan

In order to create a configuration plan in JDeveloper, right click on the composite.xml file of the project and then select the Generate Config Plan option. 

Key in a meaningful value to your configuration plan file name and then click on Ok (I generally suffix the plan name with the environment name, that way one will know to which environment is the configuration plan will need to be applied as in my case I had a different config plan for stage and production environments).

Configuration Plan Explained

The configuration plan of the project so generated will contain changes to be done at two levels - composite.xml file, and the other to the WSDL and schema files defined in the project.

The composite section in turn contains different sub sections - import, component, service and reference sub sections. Key in the appropriate values to search and replace the URL/properties of sub sections to suit to your deployment environment.

The wsdlAndSchema section by default points to all the wsdl, schema and jca files in the project where a search and replace rules are mentioned. Change them with appropriate search and replace strings

Additionally, you can validate the configuration plan for valid tags and its syntax using the Validate Config Plan option of the menu popped up when you right click on the composite.xml file.

Applying Configuration plan during composite deployment

Configuration plan can be applied to the composite during deployment onto the server. Two such ways are mentioned below:

1. Using JDeveloper in the deployment wizard, while deploying to SAR file or to application server as shown below: 

2. In the Weblogic EM console, while deploying the Service Archive (SAR) file onto the server partition

Once deployed, you can validate if the properties specific to deployment environment mentioned in the configuration plan applied to the composites in the EM console.

Oracle FTP Adapter read file as Attachment

Oracle SOA Suite FTP Adapter is used to exchange file data - it can be a read or write a file to remote server file systems. The adapter built using the well known JCA architecture to transfer files in an heterogeneous Enterprise Information Systems (EIS).

In this post, lets look at the way to read large files opaquely using the FTP Adapter. By Opaque, I meant where the file data read as such doesn't matter and the objective is to just process the file contents read by the FTP Adapter as it is.

The ideal way to get this done is using the option Read File as Attachment which is available only when you check the inbound Get operation while configuring the FTP adapter in the composite.

Let's take this up with a use case - a composite designed to read large pdf file  from the FTP server and place the same file as-is on the local server where the SOA server is running.  I shall highlight only the important steps of design of this composite with JDeveloper IDE screen snapshot.

FTP Adapter used to Get File from the remote server

Step showing the FTP Adapter Configuration Wizard with Get File as operation type selected and Read File As Attachment option checked.

FTP Configuration Wizard with file pattern to be polled mentioned with file wildcard as *.pdf

File Adapter to write the same file to the local file system where the service is running

Step showing the file path where the file will be written and the file name pattern.

File Configuration wizard to select the attachment element reference, that way the data type is same as that of the FTP Get variable.

The composite file in design mode after creating the FTP and File Adapters is as below:

Add a BPEL service component to your composite, edit it to add a receive, assign and invoke activity and wire those activities/partnerlinks appropriately as shown below:

Edit the Assign activity to map the receive GET variable value with the invoke activity write variable:

Note that in the assign activity of the BPEL component the reference value is assigned and not the actual file content. The file content are stored in the ATTACHMENT table of the SOA Infra schema as BLOB data type in Attachment column against this reference value (column name is Key). The SCA engine will take care of fetching the file content from the ATTACHMENT table at run time before the invoke activity is executed to write the file content onto the local file system. 

The above method to read a file using FTP adapter is very efficient especially if there are large files to be read from the FTP server and the content of the file does not matter. You can even combine this with encryption or compression logic in your composite before the file is passed to the target location or for further processing.

Importance of serverTimeZone property of SOA FTP Adapter Connection JNDI

Oracle SOA Suite FTP Adapter can be used to transfer files to and from an FTP server. 

Before even you can use the FTP Adapter in your SOA Composite projects, an FTP Adapter Connection JNDI will need to be created. The set up can be done in the weblogic server against the FtpAdapter resource adapter application deployment by creating a JNDI entry in the Outbound Connection Pool. As part of the setup, mandatory  properties pertaining to the FTP server such as username, password, host and port will need to be mentioned. 

The serverTimeZone property is one such property which are often ignored when a request to create a new FTP connection JNDI is made. This property signifies the time zone of the server where the FTP application is running and can play a very pivotal role when polling a file after a minimum age delay. Say, for instance, you have a SOA composite that polls a file from a FTP server location after a minimum age delay of 2 minutes. If the server on which FTP application is running is on a different time zone when compared to that of the server where SOA composite is installed, the FTP poller of the SOA Composite will not pick the file as expected which may lead to lot of confusion during testing. In order to avoid such confusion, it is always preferred to set the serverTimeZone property when setting up the FTP Adapter Connection JNDI.

Now you may ask, I never bothered setting this property while configuring the FTP Adapter Connection JNDI, but still my services with FTP Adapter was working as expected. There may be some specific reasons for it:

• By default, serverTimeZone property is blank. When set to blank, the time zone of the SOA server is assumed. It could be that the SOA server time zone and FTP server time zone are one and the same.

• There was no minimum age delay specified in the FTP poller of your Composite project.

• The Adapter configured with non Get FTP operations - Put, List and Synchronized Get

It is hence recommended to set the serverTimeZone property of the FTP Connection JNDI to that of the FTP server time zone ID when you set it  up in the weblogic server especially required when your composite are having an FTP Poller in it with minimum file age set and with SOA and FTP server time zones are different; else expect the unexpected behaviour from the outcome of the service.

The time zone values that needs to go in to serverTimeZone property are those recognised by Java Classes.

No provision has been made for the SOA server to identify the Daylight Savings Time (DST) changes of the FTP server and adjust the FTP Adapter JNDI settings automatically; such changes will need to be done manually to the new time zone in the weblogic server.