WOA versus SOA

My discussion point in my last entry discussed how SOA is agnostic in which technology should be used to employ it. Joe McKendrick and Dion Hinchcliffe at ZDNet along with research I did for a client identified a growing interest in using REST to support SOA. ZDNet refers to it as Web Oriented Architecture, a subset of SOA.

Dion Hinchcliffe's discussion on WOA

Some points that are referred to in these articles identify these characteristics:

- HTTP(S)
- POX
- JSON

Effectively from the runup of Web 2.0 technologies, the ever increasing amount of public APIs, and the increasing amount of mashups that are being performed, Web Resources are more available then Web Services. WOA is being identified as the necessary amount of complexity to get the job done.

If we compare WOA to SOA, there are some distinct differences with one distinct one being the Service or Resource Contract. WSDL/XSD/WS-Policy are front in center for describing a Web Service. Within REST, the contract is implicit and is based on HTTP verbs and XML Schema. If developing a web service, the contract analysis, design and development effort is the main focus whereby the service foundation layer and data representation layer are identified. In WOA, the data representation layer is the focus instead.

Web Services != SOA

During a recent trip to a company technical summit a discussion came about related to how SOA is just Web Services. As a SOA Trainer, Architect and Implementer I live and breath it every day. Most peoples perspectives probably fall in this category and currently most of the SOA implementations that I have worked with have used Web Services. The key is that Web Services is just an implementation technology not a SOA.

Books from Thomas Erl, Intel Press and others have identified that SOA is a design paradigm that focuses on a series of principles that tie back into key business goals and benefits. Examples of these are presented at SOA Systems but include:

- Interoperability
- Federation
- Cooperative Development between Business Analysts and Technical Architects
- Vendor/Technology Choice
- ROI
- Reduced IT Effort
- Business Agility

Now a SOA could be constructed using CORBA, Java, SCA/SDO, Web Services, REST etc. The implementation mechanism that one chooses should support interoperability, federation etc. Currently XML Based technologies provide the best interoperability both in communication/exchange but in vendor support, standards, tooling but this may change over time and probably will.

So if a person asks me what SOA is, I simple identify it as a design paradigm for developing and evolutionary distributed architecture. It has its basis in OO, AOP, EAI, BPM and other styles. It is like the Bruce Lee of architectures having evolved from various fighting styles.

SOA Design Patterns

As part of the SOA Symposium I was excited to watch the release of the latest Prentice Hall title, SOA Design Patterns by Thomas Erl. This is a large effort and I was fortunate to be asked by Thomas to participate in a contributing author capacity. Over 80+ patterns are outlined and include many of the speakers at the SOA Symposium. It is good to see such cooperation and the development of more independent material for the international audience. Some of the patterns are available at www.soapatterns.org which will be updated to match the book as well as candidate design patterns.

Another title that I am very interested in reviewing is also another Prentice Hall title, Web Service Contract Design & Versioning for SOA.

SOA Symposium 1.0

Well it has been quite awhile since I have posted but now that I have a few minutes I figure I would discuss my experience at SOA Symposium 1.0 in Netherlands, Amsterdam. I am working for SOA Systems delivering SOA training this week and was fortunate to speak at three sessions and participate in a panel discussion with my peers. I was very impressed with the quality of sessions and speakers that were able to take time out and share their thoughts on the future of SOA. Check out www.soasymposium.com and look for publicly available content sometime in the future.

There was a lot of discussion on how SOA Governance is becoming more important along with more efficient platforms and alternative consumers. One common thread in my discussion with attendees was how important the people quotient is in creating a successful SOA.

Cape Clear ESB (Workday) Assemblies Context Options

One client I am assisting is the Cape Clear ESB and it Assembly functionality. This provides mediation support via a Spring Bean configuration file which is generated via graphical drag/drop tools. Because the Assembly Service is stateless and the project we were working on required some context sharing, we had to open up the covers and learn about org.springframework.context.ApplicationContext and org.springframework.beans.factory.InitializingBean. A single bean with a scope of singleton was developed to provide context for all instances of the assembly and when the assembly was loaded this information was initialized as part of the afterPropertiesSet() method.

Within each instance of the assembly, Cape Clear provides another context called com.capeclear.mediation.MediationContext which provides a setProperty() and getProperty() method. This provides simple storage of variables within an instance.

XACML: An option for authorization

I am currently taking the next step in security development learning about XACML from www.oasis-open.org. In the scenario that I am working on with the client, there Identity Management Solution will be acting as the Policy Authoring Point whereas the ESB that we have deployed will act as a Policy Enforcement Point and Policy Decision Point. Due to the fact that we have a clustered ESB a decision that we are mulling over is whether to have a central/shared directory for XACML Policies or use the support for XACML Request/Responses to do it real time. Our concern is the latency especially in a high volume environment and we are currently steering towards a shared directory. This will allow each ESB to take a portion of the load and keep the processing local.

As part of a SOA Solution we will be using Service Agents or Intermediaries to support the authentication (SAML) and authorization (XACML). The ESB provides out of the box interceptors for both of these and only requires that the Service Consumer be able to interact with the SAML Provider.

SAML

Well it's been awhile since my last post but work and life have been quite hectic. Currently I have a client that is attempting to integrate a Identity Management/Policy Decision application with their ESB. As part of this effort examination of the many options has presented itself and we have agreed to examine the Security Assertion Markup Language as a potential solution. The overall use case is to have internal/external clients use an SSO approach towards invoking available services.

To demonstrate this capability a SAML Assertion will be inserted into the SOAP Header of a request. The deployed Web Service in the ESB will include Message Processing Logic in the form of an interceptor. The interceptor will examine the SOAP Header and SAML Assertion. At this time, it will be absent of a Digital Signature which adds an additional layer of integrity for the Assertion. The other challenge is the use of SAML 1.0 versus the current supported version of 2.0. Fortunately, the PDP has support for SAML 1.0 eliminating the immediate need to custom code 2.0 support.

For a sample SAML Assertion, check out the Sun Web Site at:

Sun Sample

and OASIS which manages the specification.

OASIS Site

Service Scalability

Over the past few weeks I have spoken with several customers who are implementing SOA in various stages and have decided to employ Web Services as the foundation. One of the key concerns is scalability and the ability to handle large volume for public / private services. An approach that evolved from these discussions was the following:

- For new services, specify governance in the Service Analysis / Modeling phase. This should specify the SLA for the Service and help identify the potential reuse and usage pattern that may be experienced.

- Services typically begin with a single instance using protocols such as HTTP

- Service scalability should be tested via Performance tools to determine how it degrades with load, supportable performance etc.

- Services should be monitored using JMX or similar API to determine runtime usage and usage patterns.

- Make services stateless so that they can be easily clustered.

- If service usage is higher than anticipated, consider configuring a cluster of service instances and use Hardware or Software load balancing.

- If throttling is desired, have service instances read requests from a JMS Queue as this places an abstraction between the service consumer / service provider. Queues allow for asynchronous communication and persistence point for inbound requests. Services can then throttle how quickly requests are processed without losing messages.

Deployment / Test of simple BPEL Process in Apache ODE

In my previous post I spoke of the configuration / startup of Apache ODE. This morning I deployed one of the examples and tested it using the provided sendsoap.sh utility and SOAPUI. A couple of things were necessary to do this:

1) New processes are stored in a folder with the process name and include:
- BPEL File
- WSDL(s) File
- deploy.xml (deployment descriptor)
- other necessary files
2) The folder is stored under the directory.
3) On startup the server will load the processes and report on it within the server log file as shown below:



4) The Apache ODE User Guide then describes a way to test services using a script called sendsoap.sh under the Apache ODE Distribution bin directory. To get this to run I needed to open up permissions using chmod. One this was complete the syntax for executing this is:

/bin/sendsoap.sh (URL to the web service) (SOAP file sample)

or in the case of what is provided in the distro:

/bin/sendsoap.sh http://localhost:8080/ode/processes/helloWorld examples/HelloWorld2/testRequest.soap

When executed the following is returned in the client window:



The tomcat/catalina server window shows the following:



5) Testing in SOAPUI, the following was executed and the results are shown in the following image:
- Created new project in SOAPUI
- Selected WSDL
- Generated a message
- Invoked Service

Setting up Apache ODE within Tomcat / Axis2

So it has been a busy week and I have had little time to work on the Open Source BPEL investigation. I have decided for simplicty sake that I start with Apache Axis2 versus Apache ServiceMix. This will allow me to quickly fire in SOAP requests and not have to become familiar with the ServiceMix infrastructure. I will tackle this next and apply what is learned here to that.

So here are the steps that I followed to get Apache ODE installed within Axis2. I made the mistake of initially downloading the Axis2 binary distro versus Tomcat. The Apache ODE instructions work with Tomcat/Axis but not Axis2 standalone.

1) Download Apache Tomcat 6.0.16
2) Download Apache Axis2 1.3 WAR distribution
3) Download Apache ODE 1.1.1 WAR distribution
4) Copy axis2.war to tomcat/webapps
5) Copy ode.war to tomcat/webapps
6) Set JAVA_HOME
7) Run Tomcat using tomcat/bin/catalina.sh run

The console then shows the following on startup:



Success!

I then accessed the Axis2 Console and could see the Apache ODE Services by going to http://localhost:8080/ode.



Next will be deployment of a BPEL sample into Apache ODE and invoking using SOAPUI.

Cheers!

Eclipse BPEL Project

Just wanted to give a rundown of the Eclipse (Europa) BPEL Project and my impressions at the moment. The eclipse bpel project is located at:

Eclipse BPEL Project


Having experienced most of my BPEL development in either TextPad (BPEL 1.0) or in the Cape Clear Studio solution (BPEL 1.1), the Eclipse BPEL Project was not too difficult to slide into. Here are the highlights:

- BPEL Project is only focused on BPEL 2.0. No BPEL 1.1 at this time.
- Overall GUI is nicely organized and employs a very similar look and feel to the BPEL Editor in Cape Clear.




-Assignment Editor in the current release M3, does not provide a graphical drill down to the schema element level. You can select your messages but no further.




-Activities are easily selected and placed in the panel. The Properties window needs to be visible to then edit those value.
-No tab to edit the BPEL directly with context sensitive help. I opened the BPEL in an XML Editor and flipped between the windows.
-XPath is done manually rather than created via the selection tool.
-PartnerLink defintion process is not as easy as in Cape Clear but it does generate an artifact WSDL file that includes the partnerlink definition so you do not need to muck up your WSDLs with the elements.

Issues that I have experienced:

1) Currently the Web Service Tools that is bundled with the BPEL Project appears to have a bug with imported schema. This is obviously unrelated to the BPEL Project but the full-lifecycle of developing the Service Contract and using this for the BPEL is not working properly. Currently have to use inline schema instead of importing.

What I haven't tried:

1) Deploying generated BPEL 2.0 into a container. Will try Apache ODE in either Axis or ServiceMix to confirm execution.

SOAPUI Benefits

As part of the previously mentioned Continuous Integration effort I am undergoing, an effective/simple approach to Unit Testing needs to be identified. JUnit works well when performing assertions on Java. When testing Web Services, I decided to look for alternatives. One alternative is to use XMLUnit as I described in an earlier post. In order to test for the current client, a different approach was in order. Their current QA Staff are not familiar with Web Services so a Web Services centric Testing tool is an additional requirement. SOAPUI available at www.soapui.org provides a very simple and capable Test environment. GUI, command line and JUnit integration are provided in the freely available version.

So currently my Continuous Integration looks like:

- CruiseControl invoking Ant Script
- Ant Script invoking:
- Build / Validation Process
- Deploy Utility
- Junit TestCase
- SOAPUI Project

More to come....

OSS to the rescue for BPEL/ESB

Several of my customers are small independent start-ups that are heavily involved with OSS. Just recently a Web Service integration highlighted the amount of Java code necessary to broker interactions with external partners. The decision was made to examine the OSS marketplace for an ESB and BPEL development tools.

I will be examining the BPEL Project from Eclipse, Apache ODE and Apache ServiceMix. I was very suprised at the quality of Apache CXF and its inclusion in the ServiceMix container was one of the factors in proceeding with this evaluation.

The goal of the evaluation is to show the ability to compose a task service that will create users in two separate services (Apache CXF and Microsoft .Net). In addition, some of the mediation facilities will be examined for simplicity and flexibility.

Does anyone have thoughts on ServiceMix?

I may also attend a session at Sun related to their ESB/WS Tools next week. If I attend I will post my thoughts and how it rates with the niche Cape Clear solution and Apache solutons.

Continuous Builds with Web Services

In two of my current projects, I am working with clients that are moving to continuous builds from complete manual processes. What is a continuous build you may ask? Well anytime a change in the source code repository occurs, a build could be triggered and deployed to a development server. As part of the build process a suite of Unit Tests could be executed to confirm there is no regression in capabilities. Companies like Cape Clear employ this process to develop daily builds which are much improved in quality over the previous semi-automated processes.

The ingredients to this always falls into the same basic technologies and like most people OSS provides the answers:

- CruiseControl for the continuous build management
- Apache Ant for the build scripts
- JUnit for Unit Testing
- Some sort of Version Control Repository like SVN or SCM
- Web Service Container / ESB (Cape Clear or the like)

So is it difficult? Well I was suprised to see how quickly CruiseControl could be downloaded, started and configured through two configuration files (config.xml and build-.xml). In just a couple of hours the software was downloaded and a build was completed.

The necessary components to execute the build are the configurations described above but also the main build.xml that in the case of Cape Clear, invoked the Eclipse Buckminster Framework. This effectively creates a headless Eclipse Workspace and allows the ant script to run the validators, package the content, deploy it and test it. Since the Web Service/BPEL developer is not writing a compiled language, the validators are the key component in the build process.

Other things that I need to complete revolve around the rewriting of URLs or namespaces depending on which environment the build is being created for. Ant provides a soluton there via the replace feature which allows for the search/replace of tokens. This functionality is extremely helpful in complex BPEL where many partner services equates to many WSDL/endpoints. The other feature not complete is the execution of a JUnit test case using the Buckminster/Cape Clear launch feature.

Event Driven versus Orchestration

I had an interesting conversation the other day at a client who was very interested in an event driven approach towards integration versus the traditional centralized orchestration model. The discussion identified that within the event driven model, the state of the process would be maintained in the message versus in the singular centralized orchestration (BPEL). Both solutions have their strengths and drawbacks. One of the big drawbacks that was discussed was in monitoring/management. Operations likes to have a centralized, process oriented depiction of what is occurring. The event model we discussed would spread out the process across a wide range of intermediaries and the message would be updated along the way. Not an ideal situation if viewing from this perspective. Whereas, the BPEL Service provides a more centralized view/approach.

An option for Event Driven would be to utilize an auditing interceptor that would provide a centralized view/update of the process execution. Operations could then monitor this service to get the depiction they preferred.

A strength of the BPEL approach was in the composing of task services or business processes. An event approach everyone thought would not be as easy to compose, however the scalability/performance benefits is very intriguing. This discussion will continue and hopefully draw out distinct patterns on when one approach is better than another.

Cape Clear and the bigger picture

Well over a week ago Cape Clear was consumed by the SAAS marketplace. When I first heard the news I scratched my head but as SAAS gains steam, the need to share data via automation becomes more important. The increasing communication of data outside organizations has not decreased but only increased making simplified integration an imperative. Currently I am working with a several clients that are sharing data with business partners to provide a more complete solution. REST and SOAP interfaces are the standard and from experience are easier then the RosettaNet and EDI standards of the past. These standards spent a lot of time identifying a domain, event model, protocol etc that provided an out of the box roadmap for B2B.

These days the protocol and policy's are standardized however the event model and domain are custom to each. Most of the customers I am doing work for are implementing simple B2B models that share core data and simple event models. This is currently comprised of a WSDL on each end with customers invoking each others interface.

How does this tie into SAAS? Well if you are familiar with the first evolution such as Salesforce.com, there was a backlash from large organizations around storing their data offsite with poor APIs to extract and manipulate the data. After several iterations, Salesforce.com has improved their APIs and you see the second generation like Google, Workday and others providing compelling solutions for GIS, ERP etc. These services have taken off not only in small niche like applications but in the Fortune 1000 and Local/State/Federal Government.

What SAAS has exposed is the need for integration never decreases, it just sort of shifts from internal to external or if you look from a SOA perspective, the interoperability is baked in from the start.

The acquisition of Cape Clear is testament to this fact and identifies the need to use standards to simplify adoption.

BPEL Caching / Persistence

I recently attended a vendors presentation on some caching technology whereby Java Objects could be stored in an in-memory DB or caching layer. The architecture appeared to be quite sound in that it utilized a peer to peer network to manage copies of the information for reliability purposes. The demonstration sparked my interest in its application to stateful BPEL as persisting to a database is a costly exercise when executing large amount of transactions. Prior to / Following this I read subsequent articles at www.soamagazine.com which discussed BPEL Reliabilty using such a concept. A reliable in-memory grid caching store was discussed:

The original article:

Article 1


A current article:

Article 2

Application of technologies such as these will provide dramatic impacts to stateful BPEL Services that are more and more being utilized for Service Composition.

Apache CXF and SOAP Headers

Here is a link that helped me with the Apache CXF client capabilities. The requirement was to send a request to a web service which required a SOAP Header expressing authentication information. The WSDL for the Web Service did not express the request in the abstract portion but rather in the binding/concrete portion of the WSDL. Apache CXF generated a class to support the setting of the Authentication Header but no methods to add it to the request. Using Google and a decent portion of scanning/reading, I found that Apache CXF will not generate the appropriate methods to attach a SOAP Header. Java Annotations provided the answer as described at:

tapestryofthoughts.blogspot.com

By adding an annotation @WebParam along with the header=true flag and other appropriate values in the generated <####>SOAP.java as well as instantiating the Authentication Header via the generated class inside the <####>SOAP_Client.java, the client was sending a SOAP Header to the destination service.

Versioning Part 2

In my readings, I found the topic of versioning discussed in Thomas Erl's SOA Principals of Service Design. I wanted to provide a brief mention of Thomas Erl's vantage as it provides another view of the topic.

As I mentioned in the previous post, changes to the WSDL definition causes service consumers to break. Thomas' take on this is for core entity services and their associated contract to remain constant. Extension to those services can be performed which will not affect the service consumer. For Services that need to be dynamic and cannot be developed with longevity in mind, then SOA Governance is applied to manage versioning of services.

SOA Governance solutions that I have heard marketed in the past provided XML Routing solutions which would inspect requests and then use a lookup mechanism to route to the eventual destination. In this case, multiple versions of a service could exist and it would be up to the Governance solution to manage the Addressing aspects.

Another perspective of Service Versioning is James Pasley of Cape Clear and his examination of the current environment. His description of versioning best practices is available at:

http://developer.capeclear.com/versioning

and discusses the options for deploying various service versions concurrently.