Chasing a Constantly Changing API

I was once waist deep into a development effort when a partnering development team informed us that they where changing their API again. The component we were developing depended on their components for data and capabilities. We had spent time interfacing with their component. We had been careful enough to isolate the interface to their API by creating a façade. This way, the two components were decoupled as much as possible and our component was somewhat isolated from future changes to the partners API.

Our partner was already changing their interface. Not a problem, right? The façade is the only subcomponent affected by this change. An engineer spends a few days reworking it and we move on. It’s that easy right? Well, no.

When this happens you need to make sure you find out why their interface is changing. Ask your partner why this change has been made and why this change was not thought of originally. If they do not have solid reasons for this, then be assured that their API will soon change again and you will find your team spending another couple of days updating the façade. I have seen this situation play out many times. Each time the partner swears that "this is the final change to the API" only to have it change again a week later.

Yes, the façade acts as a buffer to new changes but changing the façade still requires work. And no façade is completely perfect. Some changes might still bleed into other subcomponents causing many other needed changes. In fact, the more you need to update the façade, the greater the chance that the changes will effect all the subcomponents who utilize the façade.

In many situations you have no control over your partner's development practices. It is important to first identify when you may be dealing with an unstable API. Once you have identified the API as unstable, you have a few options other than using up all your resources chasing it.

1. Hold off on making the changes. Wait a few weeks before updating your façade to their latest API. You might also be able to wait till the end of the development effort to make the update. This way, you will skip a few interim API changes. I do have a word of caution on this. Make sure you’re not ignoring their changes; only skip the updates to your code. You want to make sure their changes do not drastically change or remove the needed data or functionalities you require from them. If you wait till the end of development, it might be too late to address the problem.



2. Send you partner your façade. Many partners will pushback on this. They might see this as more work for them, but it might actually save time for both of you. They have a better understanding of the new API and it will be a quicker change for them to make. They already know what changed in the API and why, thus cutting down on cross team meetings and emails.

Make sure you identify their unstable API as a risk to the project as early as possible. Assess your options; determine if your schedule, budget, or scope for the project is at risk. Chasing a changing API never seems like a big deal to the project until it is too late. If it is not identified and dealt with early then it will put the whole project at risk for failure.

Strategy in a SOA World – Three Strategic Principles

So your company has a niche product. You specialize in one aspect of a larger system. Your competitors are virtually sitting right there next you in the same system. But they have their niche and you have yours. You and your competitor are placed into a large Service-Oriented Architecture and both of your services provide utility to the end-user through your modular and accessible interfaces. To gain a competitive advantage, companies must follow three strategic principals: Do what is best for the entire system, Focus on the end-user, and Offer help to your competitors.

1. Do what is best for the entire system. Think of the system as your business ecosystem. To completely understand this analogy of a business ecosystem read The Keystone Advantage: What the New Dynamics of Business Ecosystems Mean for Strategy, Innovation, and Sustainability (Harvard Business School Press, 2004). Your competitors are in many ways your allies. Their niche adds value to the whole system just as yours does. Together your system competes against other systems and solutions. Before acting on any strategic moves, make sure the move is in best interest for your entire ecosystem.

2. Focus on the end-user. Make sure you are focused on the needs of the end-user. Your service may not interact directly with the end-user or it might be a very small piece of the system, but that does not mean you do not need to focus on the larger direct needs of the final customer. The more you are focused on the needs of the end-user, the better you will be at adding features to your services that end up being used by the customer. Many companies focus on extending their niche in any way possible. New features add to the utility of your component but if the end-user of the system has no need for those features your efforts have been wasted. You only can gain share of the system if the system’s end-user needs your added features.

3. Offer help to your competitors. Your competitors in the system are looking to add features to their own services. Position your services to add value to their future efforts. If your competitors can add value to their product they will utilize it. If utilizing a feature of your product is accessible (it is a SOA, so it should be) and it is cost effective, it will be very tempting for them to utilize your new feature. If your offer is tasty enough they will go for it. As long as you are able to constantly add value for their use, they will use your service over attempting to create that service themselves. At this point, your competitors are now more dependent on your services. At the same time you are expanding your share of the system.

SOAs are great for end-users because of their ability to trade in and out loosely coupled components. If your components are required by the end-user and/or required by competitors’ products, which are then required by the end-user, your components will have a strong foothold in the system. The stronger your foothold, the stronger your position will be among the players in the ecosystem. This long-term strategy will add market share and expand your competitive advantage.

A New Strategy for the Enterprise Software Industry (Post 2 of 4)

A SheeleyTech working paper

This is post two of a four post series.
1 A background of the enterprise software industry.

2. An examination of this industry using the analysis techniques in Clayton Christensen’s book Seeing What’s Next. I analyze the customers of this industry by categorizing them into undershot customers, overshot customers, and non-customers. This analysis determines which market segment offers the largest opportunity. I then look to see if the industry is signaling a change to target this market segment.

3. A competitive comparison determining which players are best to implement the strategy based on their resources, processes, and values.

4. My strategic road map for the company that is best fit to become the next leader in the enterprise software industry.


Customer Analysis
Undershot Customers are most likely very large corporations who will pay exorbitant amounts of money for slight improvements to their current enterprise systems. These customers can take advantage of economies of scale. The sizes of these businesses allow them to profit off of any improvement to the enterprise system.

Overshot Customers are small to medium size businesses that cannot adsorb big costs for every upgrade to the enterprise system. These companies tend not to pay for new improvements or the latest release. These companies have to anticipate big returns for any purchase or upgrade to the enterprise software.

Non-customers are small companies that tend to find cheaper alternatives that might not be the best solution but help get the job done. In these businesses you will find collections of excel applications that have been transformed into advanced accounting and management programs.

According to the US census bureau, in 2004, 99.7% of companies in the United States are small to medium size businesses[i]. If my overshot analysis is correct, then most businesses in the US are overshot customers. Anyone who works in an IT department that manages SAP or Peoplesoft enterprise systems, or knows someone who does, is aware that these systems are highly complex and upgrades are very costly. Integrating new features into these systems require IT departments to work weekends, cause downtime to the systems, and require an upfront purchase of a license and hardware for the new components.

According to Christensen’s model, if there are overshot customers in the industry then low-end disruptive innovations, displacing innovations, and downward migration of required skills will be occurring.

Looking at the software industry, there should be signals that these trends are emerging. We will look at the OnDemand business model as a low-end disruptive innovation, niche CRM and customer contact solutions as displacing innovations, and the emergence of Web Services and webapps as a downward migration of required skills.

The OnDemand Business Model is one that allows customers to use the software under a variety of payment plans. Companies with the business model offer solutions that do not require upfront costs for new hardware or software installations. The software is priced at different levels that allow their customers to pick the plan based on the level of usage required. These solutions give small and midsized businesses cheaper solutions than purchasing more expensive licenses and new hardware.

Niche CRM and Customer Contact Solutions are just a few displacing innovations that have emerged. Displacing innovations take aim at a specific feature of an existing product line. Solutions that target customer relationship management and customer contact campaigns are just a few of the areas where new players are sharpening their expertise. Their customers do not want to pay for an entire enterprise system or have already purchased one but are willing to pay for specialized features particularly needed for their industry.

Web Services and Webapps are examples of downward migration of required skills.
Web services are a means to define and expose application interfaces. Web services allow for easy upgrades and integrations of existing system. Web services are interfaces that define and verify integrated systems. Also, Web services allow enterprise systems to be loosely coupled, so that each component of the system is less dependent on the other components. This reduces the experience needed by IT departments to maintain and integrate new components into the systems and reduces the downtime required for most integration efforts. This also allows niche solutions to be created without complete knowledge of the other enterprise systems in their customers’ existing systems.

Webapps are applications that are accessed over the internet. These applications require now hardware, downloads, or installations. They are practically a webpage that looks and feels like an application. New enterprise solutions are emerging that are exclusively webapps. These solutions do not require an IT department or even a highly technical employee to do the setup.

These signals and trends are clear signs that the enterprise software industry is changing and is refocusing on the overshot customers who make up most of the business market. Next, I will analyze the competitive battles and determine which players are preparing to emerge as the next enterprise software leader.

[i] Small business is usually defined as under 100 employees and medium size businesses are usually defined as companies with 100 to 500 employees

A New Strategy for the Enterprise Software Industry (Post 1 of 4)

A SheeleyTech working paper

This is post one of a four post series.
1 A background of the enterprise software industry.

2. An examination of this industry using the analysis techniques in Clayton Christensen’s book Seeing What’s Next. I analyze the customers of this industry by categorizing them into undershot customers, overshot customers, and non-customers. This analysis determines which market segment offers the largest opportunity. I then look to see if the industry is signaling a change to target this market segment.

3. A competitive comparison determining which players are best to implement the strategy based on their resources, processes, and values.

4. My strategic road map for the company that is best fit to become the next leader in the enterprise software industry.


Background
In the 1970s, IBM and other software companies created the first widely used enterprise software applications. This disruptive technology allowed companies to digitally manage the business’ payroll and financial accounting. These systems were host-based. Host-based architectures perform all functions of the system on one machine. The storage of data (financials, and employee information), the processing of the data, and the presentation to the user are run on one mainframe.

Since this time, a series of sustaining innovations has occurred. Functions that address, human resource departments, customer relationship management, supply chain management, inventory management were added to enterprise software systems in order to meet the demands of the current market. The enterprise systems also changed into client-server architectures to increase the accessibility of the systems and to take advantage of processor power of multiple machines. These systems are a collection of dedicated database machines and dedicated processing machines. Users access these systems through client applications on their local PCs, many times through a web browser. These systems became widely popular and large enterprise companies such as SAP and Oracle capitalized on their demand.

In the past few years, new types of enterprise systems are emerging. Buzzwords such as OnDemand, Saas (Software as a Service), SOA (Service-Oriented Architectures), and webapp (web application) are used to describe these new types of systems. Companies like Salesforce.com, SoundBite Communications, WebEx, and RightNow Technologies Inc. are the new players in this field. These systems do not require the customer to host software on their networks. There is no IT department required to maintain the systems and many of these applications use an onDemand billing model. These models bill the customer each time the system is used or allow for multiple pricing plans rather than purchasing a license or other expensive one-time charges.

A TRUE benefit of using web services document style messaging over remote procedure call (RPC) style

Before reading this post, if you are not familiar with web services check out
http://www.ibm.com/developerworks/webservices/library/ws-docstyle.html
or
http://www.java-tips.org/java-ee-tips/enterprise-java-beans/document-handling-in-web-services-applica.html
or even
http://en.wikipedia.org/wiki/Web_service


The top three benefits of document over RPC are validation, asynchronous processing, and a less rigid contract between the client and the server. The first two benefits are more of a coding convenience than an added benefit and don’t really impress me. To me the third benefit is really the power of document style. But until my current project, I have yet to have a situation where I was able to utilize this powerful benefit. I want to explain my case for why a less rigid contract is the only powerful benefit and then I want to share with you a situation where documents style messaging is really the only option. (Note: there is this idea of using document style to save state, but my disagreement with this idea will soon become a completely separate post on SheeleyTech. For right now, just trust me. For most systems, don’t try to save state through web services.)

Validation in XML This benefit uses the power of XML. Document style uses the power of XML to validate the message where as with RPC the code behind the operation does the validating. So either the code does the validating of the XML. No big deal. (except that it allows you to have a less rigid contract, but that’s more than just validation)

Asynchronous Processing Ok another feature of document style, but I can name many other ways to get my RPC style call to be asynchronous for the client. Again, no big deal.

A Less Rigid Contract Now this benefit is cool. The idea here is when a client uses an RPC style web service; the client application relies on that specific method signature. Any change by the client of the parameters will break the client application. In other words, the interface cannot ever change unless it changes on the client as well. With documents style, this isn’t the case. Changes to the server interface that in RPC would change the interface, in document style only change the schema and therefore will not break the calling application.


So if you are like me, the last benefit sounds really cool, but where would you need it. I mean, I have used documents style in the past, but only to help future efforts that might need to make changes to the interface. …well, actually I used it only for the sake of using a documents style, RPC would have done the trick just as well. Besides, I’m really not quite sure how I could later change the schema without hurting any client applications. If they are relying on some data being in the Document message and I remove that data from the schema, won’t this cause them to have errors? This whole idea of using document for “great” benefits over RPC has really escaped me. ...until now.

My current project is enhancing a really cool application for the Air Force Research Lab. The need is for the system to be able to act as a standalone application (i.e. the client and the server are all on the same machine acting as a single application) and also act as a service and a client. (i.e. the client will be located on one machine and the server be on another machine handling service requests from whom ever is calling it).

The reason document style is needed for this design is because of the services that the server is exposing to the client. There is a ton of them (for this post we have N of them). Each service has its own set of parameters. In order to handle direct and web service mode, at first we thought we needed N times 2 handlers for each service (one to handle calls to the service directly and one to handle the call through web services). (See Diagram 1 below) Instead, if we use document style messaging for the web services, we can define the parameters for each service in the schema. We can have 1 handler for all calls thought web services and one handler for all calls to the services directly (See Diagram 2 below).

Diagram 1 A quickly made diagram of the system using RPC style messages

Diagram 2 A quickly made diagram of the system using document style messages

Before, the reason we needed N web service handlers was different parameters for each service. With document style, the parameters are defined in the schemas that define the structure of the XML document. The service proxies are needed to gather the required data for each service. Once the proxies have gathered the data, they can pass on the single parameter to the web service handler. The web service handler now only needs to make a single generic call that passes in the single parameter.

This design is simpler and contains fewer dependencies. It should also cut down on the time needed to build the system. I hope this now gives you a concrete real world use of web service document style messaging.