Tuesday, December 20, 2011

Continuous integration for Oracle OSB projects using Ant and Hudson / Jenkins

Continuous integration (CI) is pervasive. This doesn't come from nothing, CI has many benefits, such as to avoid last minute integration hell and to improve software quality

The principles of Continuous integration
1. every commit to the SCM should be build so that you'll have an early feedback if the build breaks
2. automate build for consistency
3. automate deployment for consistency
4. automate test of the build artifact for consistency
5. archive the build artifact so that it'll be readily available (e.g. for further test)
6. keep the reports & metrics from build & test

You can also add additional steps for example when this commit breaks the test you can reject the commit by rolling back the deployment (using undeploy wlst task) & roll back the svn commit using merge backward:
svn merge -r currentver:previousver ; svn commit

This blog will show how to achieve these steps using Ant and Hudson/Jenkins. We use Ant since in many organizations Ant is already well adopted compared with Maven. In another blog we will discuss how to achieve the same goal using Maven and artifact repository (e.g. Artifactory or Nexus) which are handier than Ant particularly with respect to dependency management.

How it works:
1.Using Hudson/Jenkins to let the svn-commit triggers the Ant build
I chose Hudson/Jenkins since it's easy to use, has good features, scalable and recommended by many people (including folks working at Oracle). You can see Jenkins as a new version of Hudson, Jenkins was created to avoid legal problem with Oracle when Kohsuke Kawaguchi, the Hudson's creator, left Sun/Oracle.
Installing Hudson/Jenkins is easy, in Windows it can be run as a Window service. Hudson/Jenkins contains an embedeed Winstone servlet engine, so you can also run it using
java -jar hudson.war --httpPort=aportnumber
To install Hudson/Jenkins in Weblogic you need to add the deployment descriptor weblogic.xml to solve classpath conflicts of certain jars (depend on which version you install), also Hudson/Jenkins will not work in Weblogic servers with SOA/OSB extensions due to some conflicting settings.
You can add some plugins for example locale (to set languages), svn related plugins, trac (to recognize trac tag e.g. fixed#), cvs/svn browser viewer, html report, scp artifact repository, promote builds, among others.

You need to set some configurations such as JDK location, svn connection, Ant location, Maven location, Junit test report location, artifacts location, SMTP/email server for notifications.

For this example, I specify Hudson/Jenkins to pool the svn server every minute (set by the schedule "* * * * *" using cron format). When there is a new commit in the mysvnproject, the Ant ("main" target) will be invoked.

2. Checkout the new commited svn project using svntask:

<target name="checkout">
<delete failonerror="false" includeemptydirs="true" quiet="true"
dir="${servicename}" />
<svn username="${subversion.user}" password="${subversion.password}">
<checkout url="${subversion.path}" destPath="${servicename}" />

Build this code to an OSB project using ConfigExport

<target name="makeosbjar" depends="deletemetadata">

<!-- osb jar compile -->
<java dir="${osb.eclipse.home}"
<jvmarg line="-XX:MaxPermSize=256m" />
<arg line="-application com.bea.alsb.core.ConfigExport" />
<arg line="-data ${workdir}" />
<arg line="-configProject ${osb.config.project}" />
<arg line="-configJar ${jardir}/${packagename}" />
<arg line="-configSubProjects ${servicename}" />
<sysproperty key="weblogic.home" value="${osb.weblogic.home}" />
<sysproperty key="osb.home" value="${osb.home}" />

3. Deploy the jar to the OSB server using WLST import (a python script)

<target name="deployOSB">
<wlst fileName="${import.script}" debug="true" failOnError="false"
arguments="${wls.username} ${wls.password} ${wls.server} ${servicename} ${jardir}/${packagename} ${import.customFile}">
<script> <!-- run these before import.py -->
passphrase = "osb"

4. Run the SOAPUI web service test and generate the junit test report:
<target name="soapui-test">
<exec executable="cmd.exe" osfamily="windows" failonerror="false">
<arg line="/c ${testrunner.bat} -j -freports ${soapui.test}"/>

<junitreport todir="${testreportdir}">
<fileset dir="reports">
<include name="TEST-*.xml"/>
<report format="frames" todir="${testreportdir}/html"/>

Example of Hudson/Jenkins output:

Hudson also sent email notifications:

For the sake of clarity, there are some details omitted from this blog (e.g. the ant classpath for lib needed: svnant, svnjavahl, svnclientadapter, xmltask), these details will be found in the build.xml. Please download the build.xml, build.properties and the import.py wlst here.

The concepts in this blog work also for non OSB projects (e.g. Java/J2EE applications).

Source: Steve's blogs http://soa-java.blogspot.com/

Any comments are welcome :)

The file download is made possible by OpenDrive


Using the Oracle Service Bus Plug-ins for Workshop for WebLogic http://docs.oracle.com/cd/E13159_01/osb/docs10gr3/eclipsehelp/tasks.html
Using WLST http://docs.oracle.com/cd/E15051_01/wls/docs103/config_scripting/using_WLST.html
Biemond's blog http://biemond.blogspot.com/2010/07/osb-11g-ant-deployment-scripts.html
How-to-deploy-Hudson-to-weblogic http://jenkins.361315.n4.nabble.com/How-to-deploy-Hudson-to-weblogic-td3246817.html


Murph said...

I am working on an OSB project that has a similar approach. We use a different VCS but the same applies.
We are now moving into a phase where we will have multiple branches and be merging code. Given OSB code is primarily large ugly unreadable xml files I foresee major problems with conflicts and merges.

I looked on the Oracle forums and have not seen very much discussion.

Do you have any experience with this?


Steve Djajasaputra said...

Hi CM,
We have the same problem with the visual programming style of OSB. It's difficult to inspect the code line by line (e.g. in Trac/ SVN viewer) when we need to merge or understand the diffs between revisions. I don't dare to manipulate the OSB xml code since I am not sure how it works, so I often have to "redraw" in the OSB editor for refactoring :)


Mahender Akarapu said...
This comment has been removed by a blog administrator.
Pedro Tokarski said...

Hi Steve!

First of all, thanks for post!

I have this same scenario, and i'm trying to look at your example, but the link is broken! :(

Could you send me the source of this example?

Thank you!!

Steve Djajasaputra said...

Hi Mahender, Pedro,
It seems that the OpenDrive (where the file resides) has removed the file. I'll try to find it again and mail it to you.


PS. Mahender, I removed your comment since it contains your email address (I don't want you to become a spam victim).

Anonymous said...


Can you send me the project file as well? Im trying to figure out the variable values but didnt get them all yet...


Prasanna said...

Hi Steve,
I am trying to implement the same scenario for OSB and the link is broken. Can you please email me the file to prasanna.ganesan@gmail.com