Wednesday, December 19, 2007
One of the clients I work with has successfully employed a XML Schema definition for 3+ years now. The schemas have been allowed to evolve via the use of optional elements but those evolutions have been through a small group of Schema Owners and it is not modifyable via the developer community. Because of the inherent difficulty in changing the schema, it has prevented Schema creep and the potential interoperability damage quick changes could cause. The benefit of this has been interoperability across a wide range of technologies starting with four different app containers and two different JMS providers.
One other area of discussion is on versioning of Schemas/Services. One approach that I have seen is the use of the namespace and ESB technology to route based on unique namespaces. Namespace versioning can be something like:
This service would then have been developed in December 2007. People are familiar with this approach with some of the WS-* standards like WS-RM. The good part of this is that for me to invoke a service using this as the target namespace, requires that my SOAP Request call it out. The bad news is that changing this will break any clients that were developed against it. A quick solution is to support several services if the namespace changes by infrastructure like an ESB and XPath Routing. A simple XPath statement focusing on the SOAP Envelope could route this request to the correct destination. There are several other versioning solutions, check out developer.capeclear.com for a whitepaper on Service Versioning.
1) Create a single WSDL that had N number of operations matching the current operations that were currently spread across several WSDL.
2) Blow up the WSDLs and start from scratch creating a coarse grained interface that provided a single search operation. For the Web Service implementation, it would need to examine the query data to determine which type of search was being requested or a new element with an enumeration of search types could be added.
In both cases, revisiting the schema and collapsing the numerous elements that existed previously into a query input and search result was agreed upon.
To demonstrate the impact of solution 1 or 2, we then turned to BPEL and developed the service implementation as a BPEL Service. Why BPEL, well this service was meant to aggregate data from various sources depending on the input.
If we examine solution 1, the WSDL would have 8 operations. One option to support this was the use of a the Pick Activity and OnMessage Activity. One OnMessage per operation could be employed creating processing pipe creating an Event Driven design.
If we examine solution 2, the WSDL would have a single operation and thus would support a more traditional Receive/Reply Activity.
So in essence, the use of a Fine Grained WSDL impacted/reduced the options for BPEL development, whereas Coarse Grained WSDL did not.
Monday, December 10, 2007
To simplify the demonstration and complete the round-tripping exercise they were shooting for, we standardized the WSDL definitions to use the same Schema and to push the entire document across the wire between the services. In essence we standardized on a coarse definition. This then untethered the controller service they had defined in BPEL to just copy the top level element between services.
As changes occurred in the demo, the refactoring demonstrated that maintenance or updates were much simpler or non-existent.
From a business perspective using the business documents in the interfaces provides a SOA centric approach towards development. This approach doesn't always make sense as with very large documents, there may technology or complexity reasons to limit the use of such an approach. In those cases fine grained access make sense albeit with more work on the orchestration side.
Through his various books such as SOA Principles of Service Design and Service-Oriented Architecture a process of examining a current business process was defined. During the process, services are logically grouped into task, entity and utility. Each of these denotes a certain level of capability or interaction. The Modeling/Analysis phases help identify which bucket they belong in and confirm they follow his 8 SOA Principals such as Autonomy, Stateleness etc.
What did I get out of the session? Simply put a methodology that allowed the team I was working with to break a current process into its core services and model the message exchanges that would occur if implemented. This was done at a high level but demonstrated a simple approach that allowed both developers and a business analyst to communicate effectively.
Wednesday, November 7, 2007
Saturday, October 6, 2007
or at its home:
The purpose of this is to highlight the many options / directions the standards have gone as well as the large amount of vendor participation in these specifications. Obviously having vendors write standards is a double edged sword as they most often will have an offering following the standard release and will be pushing it on you and I. With the open source movement however, you will find a bunch of these standards available in Apache projects, Codehaus etc and the time/effort spent by other technologists can save time for us all.
One example of this was a project I worked on recently for a State Government contractor. There was a need to support basic authentication and integrate this with LDAP. Within a half an hour discussion, the decision was made to use WS-Security UsernameToken. The implementation took advantage of the WSSJ package from Apache and in a few hours this feature was ready to test. The alternative might have been a custom SOAP Header and the resulting WSDL / Interceptor work to build out the infrastructure. Not worth it when deadlines are tight!
So take a look at the BigSOAGrid and get up to speed on the latest and greatest. Maybe you will find a standards based option for your project needs!
Wednesday, September 26, 2007
Tuesday, September 11, 2007
So here are the diagrams we have covered:
- class diagram
- sequence diagram
- state machine diagram
- activity diagram
Still to go are:
- component diagram
- use case diagram
Based on the above it appears that BPEL could be described in a non-BPEL fashion using the activity diagram. An activity diagram could simply be described as a flow chart. It supports the definition of activities, looping, conditions etc. Sounds kinda like BPEL doesn't it.
Another diagram that appears to have some use is the component diagram. I am not 100% sure but the component diagram typically shows components in a system and how they interact with one another. Looking at Martin Fowler's publication UML Distilled (Third Edition), pg 141:
"Use component diagrams when you are dividing your system into components and want to show their inter-relationships through interfaces..."
Additional details in the publication and diagram examples appear to provide a way to visually describe a SOA where Web Services are the components.
I haven't researched this with other organizations but will trying to apply these prinicipals/approaches for some upcoming projects to help in the detailed design documentation phase.
Friday, September 7, 2007
Today I used the Apache CXF wsdl2java features to simply generate a client stub to a .Net service. In addition I needed to enable SSL communcation. From install to executing the client, it took 3-4 hours with a bit of debugging on the CXF client configuration file.
Current impressions are good documentation, clean generated code with no major issues.
Check it out at http://incubator.apache.org/cxf
Friday, August 31, 2007
I am now refactoring the current BPEL to use a new feature in Cape Clear BPEL, the XPath function getInitValueParam(). The getInitValueParam() function allows me to send in a property name as a string and get back its value. These properties are defined in the initParams section of the Cape Clear deployment descriptor (ws-application.xml).
I am also employing a while loop to examine the inbound XML for destination data and in combination with the above function, parameterize the destinations externally to the BPEL.
Here is the scenario, an inbound XML messages defines the names of the various intermediaries in the overall process. This is represented in an element called system as shown below:
The system element is unbounded but is an enumeration of valid destinations. Within the new BPEL, a counter is initialized and a while loop determines if the counter is less than or equal to the number of recipients. For each system I wanted to extract the system element value (the SOAP intermediary name) and then use the getInitValueParam() function to return the destinations URL . I then overwrite the PartnerLink value with this destination and I have performed dynamic endpoint assignment.
Sounds good right! Well I finally got it working through the help of a fellow BPEL blogger. The problem that I had was accessing the system element via array notation. My array value was defined as a variable called counter. I tried using a combination of XPath that would get me down to the system node and then combine this with bpws:getVariableData() of the counter value. No luck. I found this link which got me through the issue quickly:
Basically, it was necessary to use the concat function in a Copy activity to create a xpath query that builds the actual expression I was interested in, in this case something like:
concat('.../.../ns1:system[', bpws:getVariableData('counter'),']' )
then, this value was assigned to a variable called xpath. Then in the next Copy activity I refer to the new variable xpath:
bpws:getVariableData('...', 'parameters', bpws:getVariableData('xpath') )
A lot of work to do something so simple in Java. Oh well, the combination of these features is helping dramatically simplify my original BPEL and providing unlimited scalability for new systems via configuration instead of coding.
Saturday, August 25, 2007
The challenge I have is that my most important client does not have this style of service. Rather they are using One-Way/JMS style of communications. My thoughts here are centering around the use of a centrally available file system or database. Each of the SOAP Intermediaries would need to store the inbound and outbound requests to this storage point. Once the overall cycle is complete, the same XMLUnit features could be used to compare the entire request lifecycle.
Thursday, August 16, 2007
I don't know the answer on the current adoption of Rules Engines but for the right problem, it appears that it provides a compelling solution. I am currently investigating this sector to help identify if the technology is pluggable in a SOA and what type of business problems are best served. It appears that logic that is dynamic is the best candidate. The ability to update an external configuration without having to go through a typical release cycle if the logic was embedded in a bean is definitely attractive. The languages are pretty different between vendors, some using Natural Language approaches while others use a proprietary scripting language. The only constant is support for the JSR-094 specification.
Most of the vendors generate a java implementation or even web services so inclusion into a Web Services infrastructure is pretty straight forward.
So far I have looked at Jess and OpenRules and found the open source community to provide a very good starting point.
Monday, August 13, 2007
One of my current customers is also striving to implement a similar process. Currently development occurs in one major release a year and focuses on a monolothic EJB infrastructure. Unlike the previous company, the switch from a traditional waterfall process to a more agile/incremental environment has several challenges:
- History (the current organization was built around a single major release and has done so for many years)
- Platforms (the current infrastructure is comprised of very heavyweight EJB applications that are not flexible/agile)
The organization is tackling this on several fronts:
- Executive Support (most important is to get executive support on the use of a incremental/agile development approach which in turn will directly correlate to performance factors for the business)
- Organizational Changes (re-organizing the development team to focus on services)
- Simplification/Refactoring/Replacement (simplifying the exiting infrastructure for immediate gains and replacing the monolith with a series of simpler services for long term benefit)
- Putting out to pasture (placing the existing monolith into a legacy category thus limiting its scope/usage)
- Training (bringing in external consultants to guide the existing development staff on the best practices and fast track the simplification/replacement strategies).
Overall, I have been trying to do the same with the current ESB infrastructure. Through consistent interaction with operations and my desire to continue to refactor/simplify the production solution, improvements in reliability and customer support have been made. Incrementally I am removing / improving the production solution to make it extensible and more reliable for the large volumes of customer that interact with it. This process will never end and thus like the Toyota commercials, will hopefully result in the Corolla of ESBs (400,000 miles and still going!).
Wednesday, August 8, 2007
- Unique namespaces are critical/mandatory in XML Schema/WSDL as they are used to uniquely identify variables and their content.
- Schemas with imports should utilize xsd:ref otherwise bpel:assign activities become a nightmare. BPEL Variables that are generated from XML Schema/WSDL that import other schemas will each have their own namespaces. This makes copying root elements between two variables with different namespaces impossible using the bpel:copy activity. The option is to copy each element individually and see the BPEL script grow exponentially. BPEL 2.0 and its support of inline XSLT will eliminate this issue.
- Some Web Services are RPC Encoded versus Document Literal. bpws:getVariableData syntax is different when extracting values from a RPC Encoded BPEL Variable versus a Document Literal BPEL Variable. BPEL Editors eliminate this issue, however when editing by hand the later will include the variable, part and XPath to the element. RPC only includes the variable and part.
- One of the most simple but common mistakes is to not initialize the BPEL Variables with a literal assignment when using Document Literal services. This will cause run-time failures or other depending on the BPEL engine you are using.
Tuesday, July 31, 2007
- XML Schema - Pretty much the standard for data definition and more powerful than DTDs (nothing new here).
- XSLT/XPath - Standard for data transformation (haven't used XQuery yet).
- BPEL - Standard for Orchestration Web Services
- WS-RM - only have used it at one client but it provides reliability as advertised.
- WS-Security - not used in production but have several customers moving to it for up-coming releases.
- WS-Addressing - examining this and other specifications for a routing road map. Not currently used in production work that I have performed.
So as you can see there are still a lot of missing specifications that are floating in the WS-* ether as well as UDDI (an unpopular specification in project based SOA). The above is based on a sampling of 200+ customers that I have worked with from 2002.
Monday, July 30, 2007
- Topics are utilized when the information is time senstive (i.e. stock quote).
- Topics are utilized to broadcast of information to a wide audience (or subscribers).
- Queues are utilized when performing transactions.
There may be several other reasons to use one over the other, but this applied to this specific customer and helped address its usage.
Sunday, July 29, 2007
So on Friday I checked my gmail account and noticed that LinkedIn (http://www.linkedin.com/) had notified me of several contacts who were interested in touching base and joining my personal network. One was a fomer colleague and another was a former friend from college. Within a couple of hours I had caught up on 6 years with my colleague and 15 years with my college buddy. The part that is interesting is all I needed to do was post my LinkedIn profile (my personal WSDL). This included my destination (email address) and content. LinkedIn hosts some general operations and away you go.
So a single personal service in a short time frame brought me information relevant to me and simplified my networking life dramatically. Just imagine all the phone calls, mixers, conferences etc that you would have had to attend. Who's got the time!
Personal Integration is here and helping me and many others.
Saturday, July 28, 2007
- If securing two points (a client and a service for instance) and the assumption is that security is not required once it is received on either end, then transport level security is sufficient (HTTPS, SFTP etc.)
- If it is unknown how many systems/intermediaries will be touching the message then message level security should be utilized (WS-Security / XML-Encryption).
An interesting point by the security gurus is that transport level security has the potential of being at various strengths (i.e. 128 bit). As a message travels between more than one point, there is the potential for the privacy of the message to vary. This is not the case with message level security as the strength is specified within the SOAP Headers of the message and thus remains constant. This is an important consideration as a lot of state agencies begin integrating across departments/agencies which in turn may be across networks.
Friday, July 27, 2007
We simply resolved this issue by loading the SOAP request into a SOAP Client in Eclipse and doing an HTTP Post to the service.
There are many other reasons to using an ESB but just the flexibility in supporting multiple transports can help out in a pinch.
Thursday, July 26, 2007
Currently I am working with a variety of clients that are large and small using third party ESB, MOM or Open Source. In my travels I have run into the good and bad of SOA and when done right, the benefits it can provide.
As I navigate my customers problems and learn/implement the WS-* standards, I'll post what I find. Maybe this will help you or at a minimum draw some debate.