Over the last 28 years of running this business I have had hundreds of conversations with customers and partners about integration. In most cases when I have tried to obtain more details about exactly what they wanted to integrate to and how I have drawn a blank. It is as if the word ‘integration’ has some magical connotation that everyone is supposed to instantly understand. Being a very logical and technical person, I guess I fail the test because to me it is just a general term that covers a multitude of possibilities.
I have also designed and/or programmed many integrations in my life and I can honestly say that no two have ever been the same. So, assuming that you have a requirement for two or more of the application software products you use to ‘integrate’ how should you go about defining what is required so it is as clear and as unambiguous as possible to your partners, the people you will ask to do the work?
Integration is usually about sharing information and importantly, not duplicating effort (i.e., having to enter data twice) or duplicating information. Having to enter the same information twice into two different systems is just plain dumb and bad design. Maintaining duplicate copies of information is even dumber and dangerous because sooner or later they will get out of step and you will suffer from what we call a loss of data integrity. This is usually why we need integration, to share information and to avoid duplicate effort and duplicate information.
For our purpose let’s assume that we have two application systems, A and B, that we need to be ‘integrated’; both use a SQL database to store data. Applications A and B are produced by different vendors that haven’t worked together before. The first fact to face is that in the normal course of events each vendor is going to want the other vendor to do all the work and each vendor is going to want the other vendor to utilize its proprietary integration methodology (e.g., Application Program Interface (API) or Software Development Kit (SDK)). This is your first big challenge; you need to get the vendors to agree to work together because no matter how this turns out both are going to have to do work and contribute; you can’t complete an integration using just one vendor. That is, the most important thing you have to do is to manage the vendors involved. You can’t just leave it up to them; you need to manage the process from beginning to end.
The second most important thing you have to do is to actually define and document the integration processes as clearly as possible. Here is a checklist to guide you:
1. Will application A need to access (i.e., read) data held by application B?
2. Will application B need to access data held by application A?
3. How often and at what times is access to data required? All the time, once a day, once a week, only when something changes, only when there is new data, every time a new record is added to either application A or B, etc. What are the rules?
4. How is the data identified? That is, how does application A know what data it needs to access in the application B database? Is it by date or special code or some unique identifier? What are the rules that determine the data to be accessed?
5. Will application A need to transfer data to application B (i.e. write data to the B database)?
6. Will application B need to transfer data to application A?
7. How often and at what times is a transfer of data required? All the time, once a day, one a week, only when something changes, only when there is new data, every time a new record is added to either application A or B, etc. What are the rules?
8. How is the data identified? That is, how does application A know what data it needs to transfer to the application B database? Is it by date or special code or some unique identifier? What are the rules that determine the data to be transferred?
9. Does application A have an API or SDK?
10.What is the degree of difficulty (expertise required, time and cost) of programming to this API or SDK? Rate it on a scale of 1 to 10, 10 being the most difficult and most expensive and most time consuming.
11.Does application B have an API or SDK?
12.What is the degree of difficulty (expertise required, time and cost) of programming to this API or SDK? Rate it on a scale of 1 to 10, 10 being the most difficult and most expensive and most time consuming.
13.Is the vendor of application A happy to assign a suitable qualified technical person (not a sales person or pre-sales person) to be the interface?
14.Is the vendor of application B happy to assign a suitable qualified technical person (not a sales person or pre-sales person) to be the interface?
15.What is your timescale? When does it have to be completed? What is the ‘driver’ event?
16.What is your budget? Basically vendors work in a commercial environment and if they do work then they expect to get paid. As a rule, the size of the budget will depend directly upon your management skills and the quality of your integration specification.
Please keep it in mind that there are always multiple ways to integrate; there is never just a single solution. The best way is always the simplest way and this is usually also the lowest cost and quickest way as well as being the lowest cost to maintain. Think about the future, the most complex solution is always the most difficult and most expensive ongoing maintenance solution. Think KISS; minimize your pain and expense.
As a guideline, the vendor with the most work to do is usually the best one to be the ‘lead’ in the integration (remember, both have to be involved or it won’t work). So, if for example vendor A needs to read data from Vendor B’s database and then massage it and utilize it within application A then vendor A is the natural lead. All vendor B has to do is expose its API and provide the required technical assistance to vendor A so vendor A can successfully program to the API.
However, in the end it will usually be the vendor that is the most cooperative and most helpful that you will choose. If you choose the vendor you most trust and work best with to be the lead then you will maximize your chances of completing a successful integration on time and on budget.