The Challenge

An interesting challenge for Records Managers, Knowledge Managers and CIOs is the newer document management paradigm of being asked to manage all content according to the ‘Records-In-Place’ paradigm, without a single central repository. That is, to be responsible for all content across a myriad of locations controlled by a myriad of applications and a myriad of departments/organizations and people.

We all realize that Enterprise Content Management, or Content Services as it is now called by Gartner, is a moving target, constantly evolving with new challenges and new paradigms.

For example:  

• How do we filter out only relevant information from social media?
• How do we avoid capturing personal data and being culpable under privacy laws?
• How do we capture all emails containing sexism, racism and bullying without being guilty of an invasion of privacy of the individual?
• How do we meet all of our compliance obligations when our staff are spread across multiple states/counties/provinces and multiple countries with different legislation and compliance requirements?

All weighty challenges for the modern Records Manager, Knowledge Manager or CIO. Now we have a new challenge, how to manage multiple silos of information without a central repository.

Multi-Repository (multi-Silo) Systems

In multiple-repository systems we find multiple document stores or silos, local files, network file shares, local data bases, multiple file servers, multiple copies of SharePoint and multiple Cloud repositories like Dropbox, Box, iCloud, Google Cloud Storage and other hosted document storage. The CIO may proudly claim to manage multiple information silos but what he or she really has is a laissez faire document management ecosystem that may well be centrally monitored (hopefully) but is most certainly not centrally managed.

In the multiple silo model, the documents in our multiple locations are ‘managed’ by multiple people and multiple and independent applications (e.g., SharePoint, Google Docs, Office 365, etc.). We may have implemented another layer of software above all these diverse applications trying to keep up with what is happening but If I am just ‘watching’ then I don’t have an inviolate copy and I don’t have any control over what happens to the document. I am unable to enforce any standards. There is no ‘standard’ central control over versioning or retention and no control over the document life cycle or chain of evidence.

For example, you wouldn’t know if the document had since been moved to a different location that you are not monitoring. You wouldn’t know if it had been deleted. You wouldn’t know its relationship to other documents and processes in other silos. You wouldn’t know its context in your enterprise and therefore you wouldn’t know how relevant this document was. The important distinction is that under the multiple silo model you are ‘watching’ not managing; other multiple pieces of software are managing the life cycles and dispositions of the documents independently.

All you really know is that at a certain point in time a document existed and what its properties were at that time (e.g., historical ‘natural’ Metadata such as original filename, author, date created, etc.). However, you have no contextual Metadata, no transactional Metadata, no common indexing and no common Business Classification System. In this case, you don’t have a document management system, you have a laissez faire document management ecosystem, an assortment of independently ‘managed’ information silos. Most importantly, you are not able to link documents to business processes that transcend organizational structures and silos.

What are the issues?

Sure, SharePoint and Cloud silos make collaboration easier but at what cost? What can’t we do with this multi-silo ecosystem? Why doesn’t this solution meet the best-practice objectives of an enterprise Document Management System? What are the major areas where it falls short? How does the proliferation of multiple silos and content repositories affect us? What are our risks? Here is my assessment of the major shortfalls of this ‘Records-In-Place’ paradigm.

We are unable to: 

1. extract the critical insights that enterprise information should provide
2. define all the relationships that link documents to enterprise business processes
3. find the right information at the right time
4. provide a single access point for all content
5. Implement an effective, consistent enterprise-wide document security system
6. effectively protect against natural or man-made disasters
7. produce evidence-standard documents
8. minimize document handling costs
9. guarantee the integrity of a document
10. guarantee that a document is in fact the most recent version
11. guarantee that a document is not an older copy
12. minimize duplicate and redundant information
13. meet critical compliance targets like Sarbanes-Oxley Act (SOX) and the HIPAA
14. create secure, searchable archives for digital content
15. effectively secure all documents against loss
16. implement common enterprise version control
17. facilitate enterprise collaboration
18. Improve timeliness
19. manage enterprise document security and control
20. manage smaller and more reliable backups
21. achieve the lowest possible document management and archiving costs
22. deliver the best possible knowledge management access and search
23. guarantee consistent content
24. optimize management and executive time
25. standardize the types of documents and other content can be created within an organization.
26. define common use template to use for each type of document
27. standardize the Metadata required for each type of document
28. standardize where to store a document at each stage of its life cycle
29. control access to a document at each stage of its life cycle
30. move documents within the organization as team members contribute to the documents' creation, review, approval, publication, and disposition
31. implement a common set of policies that apply to documents so that document-related actions are audited, documents are retained or disposed of properly, and content that is important to the organization is protected
32. manage when and if a document has to be converted from one format to another as it moves through the stages of its life cycle
33. guarantee that all documents are treated as corporate records, that common retention policies are applied determining which documents must be retained according to legal requirements and corporate guidelines
34. guarantee enterprise-wide Regulatory compliance
35. produce an enterprise-wide audit trail
36. share information across departmental and/or silo boundaries
37. centrally manage the security access to documents/information across different areas of the organization
38. consistently classify documents as each repository may be used by a different department and be classified differently  
39. identify duplicates based on document name
40. easily find things based on metadata, as it wouldn’t be common across repositories
41. control access via AD single sign on
42. access all enterprise documents using a single license
43. centrally audit access and changes to metadata

What are your risks?  

Your risks are huge!

We have a large number of records and document management customers using our product RecFind 6 and with new customers the question always arises about how to best organize information in the RecFind 6 database. As the Metadata and business processes in RecFind 6 are 100% configurable, every customer ends up with a unique configuration.

Some records managers want the shared drives structure replicated in the database. Some want everything filed under a strict hierarchical classification system or Taxonomy. Some customers want the whole process simplified so end users clearly know where to file stuff and where to find stuff. Different managers in a single customer site will often disagree about how the information should be managed. Usually, the IT manager disagrees with the records manager and it is up to us to come up with an agreed and workable compromise; no easy task! There is no “one size fits all” paradigm here. We have grown to accept these discussions as part of every new installation.

Whereas I fully support the principles behind most EDRMS standards as espoused and recommended or even mandated by records management consultants I also find myself agreeing with most end users who just want the whole process simplified and expressed in natural language, not as an arcane, complex, inconsistent and difficult to navigate hierarchical classification system.

To wit, the way you classify information should not dictate how you store, manage and retrieve information.

I have written a paper of this exact subject and although it was in 2009 it is still 100% relevant. Please see this link Do You Really Need a Taxonomy? You don’t have to agree with me but please try to understand the message. End users want easy, fast access, not time-consuming complexity.

Maybe I should begin by telling you how we solve the problem at Knowledgeone Corporation and manage our emails, electronic documents and shared drives with a hybrid system. That is, a combination of RecFind 6 and shared drives. This is also a model we regularly recommend to our customers as an acceptable compromise; one that is simple to implement and one that always works.

I am obviously a big fan of making information as easy as possible to capture and as easy as possible to retrieve. This is especially important to the long-suffering end-user class who have no interest in becoming part-time records managers and who simply won’t use a system if it is too difficult to use and too time-consuming.

End users want direct access to information in the easiest and most timely fashion possible, they do not want to go through a third party or ‘information broker’. This means we need to have both a simple search system as well as a security system that ensures people only see what they are supposed to see.

And of course, the biggest problem with complex, hierarchical classification systems is that no two people file the same way and even a single user will often file things differently over time. This in itself makes the act of finding something by browsing through a classification hierarchy a hit and miss affair.

At Knowledgeone Corporation, we implemented a hybrid model that uses a simply structured shared drive resource plus automated tools to ensure everything that should be captured is captured. This approach is also all about separating the functionality of the Authoring packages (e.g., Word, Excel, Outlook, etc.) from the functionality of the EDRMS. They have different roles to play.

Let’s dispense with the notion that shared drives are evil just as we should dispense with the notion that paper is evil. Each has a part to play in a well management information management system

We use our product GEM to automatically capture all work related emails and we use our product RecCapture to automatically capture all work-related electronic documents from our shared drives. We all use a common shared drive structure to write and store our original electronic documents. Note that we do not use the feature in the RecFind 6 Button to force all ‘Saves’ into RecFind 6. We have this feature because the industry dictates it should be there but it is not popular and most customers never turn it on. Not everything you write should go into RecFind 6 and not everything you write is ready to go into RecFind 6 (though we do have a special ‘draft’ type for those customers that want drafts stored in RecFind 6).

We don’t use what you would call a formal taxonomy, we use what I call a ‘natural’ classification system. For us this means a classification system that perfectly reflects our business practices, processes and vocabulary. In our case, we are customer-centric so everything (apart from a little administrative and supplier stuff) is organized in customer or prospect folders and the lower levels are minimal, being things like Correspondence, Quotes and Orders.

Our RecFind 6 database is mostly based on customer and prospect files; it is our CRM. Customers and prospective customers are our core business just as members and cases are the core business of unions. Every industry has a core business and in my mind this should always be reflected in the classification system used so that it perfectly aligns with the work practices and processes and ‘language’ of most staff. Whenever I consult to a new organization I always try to first determine its core business and its natural language and then design the implementation around these.

We also use RecFind 6 to run our business so it is also our asset management system, our help desk and incident system, our project management system and our R&D development system. For these applications and others that we have implemented in RecFind 6, we have nothing outside of RecFind 6 to capture because all relevant information (e.g., customer support calls, details of meetings, phone calls, quotes, orders, annual leave request, etc.) are entered directly into RecFind 6 by our staff or captured automatically. RecFind 6 is our company repository and the source of all knowledge for my staff.

Because we are customer centric I need to be able to see everything about any customer or prospect in one place. For us this means centralizing on the Entity record (the Entity table is where we store the basic information on each customer or prospect). As RecFind 6 is a relational database we then store all related information in linked tables, all linked to and accessible from the Entity record with a single click.

In our RecFind 6 system, every piece of information I need to refer to is just one-click away once I view the entity record. I can also find any customer’s record instantly in RecFind 6 just by entering the customer number or a part of the organization name. Once I select the customer record, everything thing else I need to know is just one-click away and all links are viewable in a single screen. We are a customer-centric business and our RecFind 6 database is therefore organized as customer centric.

In practice, if someone at Knowledgeone Corporation wants to find something they always look first in RecFind 6 because it is a lot easier and faster than trying to search the shared drives or Outlook. Because we use automated tools (GEM and RecCapture) we are confident that everything that should be captured is captured. We don’t rely on our already too busy staff to remember to capture every important email or electronic document; it is done for them. All they have to do is search and create. Plus most of our information is stored behind customer/prospect/partner numbers in the Entity table so all information is both easy to browse and search (Text, Metadata, BOOLEAN, Saved Searches, etc.).

As a backup, every staff member has the Button installed (in Word, Excel, PowerPoint and Adobe Professional) but they rarely use it.

We have a security system configured around our management structure that works fine for us. As a Director for example, most of the stuff I save is with a basic security code (e.g., a letter to a customer) because everyone needs to be able to see it. However, as a Director I also have the right to save things with higher levels of security, e.g., Manager, Director, where appropriate with restricted access. Like all good security systems, it is simple but effective. I am not a fan of overcomplicating anything.

Our searching is also structured the same way. We have configured RecFind 6 to add the objects we need to search on as menu items in the search function just as we would do for any customer. We therefore have a Metadata search menu of Attachments (electronic documents, emails and images), Entities (Customers, Prospects, Partners and Suppliers), People, Incidents, Bugs, Quotes, Invoices, Timesheets, Support agreements, etc. We repeat this with Boolean searches. We make it as easy as possible and as logical as possible so our staff can find anything as fast as possible. After all, I am paying their salaries so I want them to be as productive as possible.

Most importantly, we provide multiple entry-points for searches. I can for example search directly for emails with a Metadata search, searching by a combination of Sender, Recipient, Date, Subject, etc. Alternatively, I can search for customer emails from within the Entity record just by clicking on a single link for all attachments for that customer. We give our staff multiple options just as we give our customers multiple options.

You can search on any field and different classes of users can have different Metadata to both view and search on. The security system determines what each class of user (security group) can both see and then do with the information they can see. That is, the security system determines what tables and fields (and electronic documents and emails) you can see and then what methods (Add, Modify, Clone, Delete, Search, Print, etc.) you can use. Each security group sees only what it needs to see and has only the functionality it needs to get the job done

Because the system is flexible, the records manager for example could choose to search for emails on the way they were classified (say a 3 level hierarchy) but end users could choose to search using a natural selection of Metadata fields such as Sender, Recipient, Subject, Content, Date or ranges of these fields combined in either a Metadata or BOOLEAN or (making it easy for end users) Saved search.

Its horses for courses!

Following the above hybrid approach also means that you can still implement and manage all the recordkeeping principles such as retention and disposal schedules, location tracking, auditing, etc.

My point is that it is possible to meet the needs of all classes of users without frustrating any group.  It just requires a hybrid approach and the configuration of the system to suit each class of user.

Making everyone happy is a lot better than making some people happy and some people unhappy. Why would you do this if you had a choice?

First a little history. We began in 1984 with our first document management application called DocFind marketed by the then Burroughs Corporation (now called Unisys). In June 1986 we sold the first version of RecFind, a fully-featured electronic records management system and a vast improvement on the DocFind product. Then we progressively added document imaging then electronic document management and workflow and then with RecFind 6 a brand new paradigm and an amalgam of all previous functionality; an Information management system able to run multiple applications concurrently with a complete set of enterprise content management functionality. RecFind 6 is the eighth completely new iteration of the iconic RecFind brand.

RecFind 6 was and is unique in our industry because it was designed to be what was previously called a Rapid Application Development system (RAD) but unlike previous examples, we provided the high level toolset so new applications could be inexpensively ‘configured’ (by using the DRM) not expensively programmed and new application tables and fields easily populated using Xchange. It immediately provided every customer with the ability to change almost anything they needed changed without needing to deal with the vendor (us).  Each customer had the same tools we used to configure multiple applications within a single copy of RecFind 6. RecFind 6 was the first ECM product to truly empower the customer and to release them from the expensive and time consuming process of having to negotiate with the vendor to “make changes and get things done.”

In essence, the future of the RecFind brand can be summarised as more of the same but as an even easier to use and more powerful product. Architecturally, we are moving away from the fat-client model (in our case based on the .NET smart-client paradigm) to the zero-footprint, thin-client model to reduce installation and maintenance costs and to support far more operating system platforms than just Microsoft Windows. The new version 2.6 web-client for instance happily runs on my iPad within the Safari browser and provides me with all the information I need on my customers when I travel or work from home (we use RecFind 6 as our Customer Relationship Management system or CRM). I no longer need a PC at home and nor do I need to carry a heavy laptop through airports.

One of my goals for the remainder of 2014 and 2015 following is to convince my customer base to move to the RecFind 6 web-client from the standard .NET smart-client. This is because the web-client provides tangible, measurable cost benefits and will be the basis for a host of new features as we gradually deprecate the .NET smart-client and expand the functionality of the web-client. We do not believe there is a future for the fat/smart-client paradigm; it has seen its day. Customers are rightfully demanding a zero footprint and the support of an extensive range of operating environments and devices including mobile devices such as smartphones and tablets. Our web-client provides the functionality, mobile device support and convenience they are demanding.

Of course the back-end of the product, the image and data repository, also comes in for major upgrades and improvements. We are sticking with MS SQL Server as our database but will incorporate a host of new features and improvements to better facilitate the handling of ‘big data’. We will continue to research and make improvements to the way we capture, store and retrieve data and because our customer’s databases are now so large (measured in hundreds of Gigabytes), we are making it easier and faster to both backup and audit the repository. The objectives as always are scalability, speed, security and robustness.

We are also adding new functionality to allow the customer to bypass our standard user interface (e.g., the .NET smart-client or web-client) and create their own user interface or presentation layer. The objective is to make it as easy as possible for the customer to create tailored interfaces for each operating unit within their organization. A simple way to think of this functionality is to imagine a single high level tool that lets you quickly and easily create your own screens and dashboards and program to our SDK.

On the add-in product front we will continue to invest in our add-in products such as the Button, the MINI API, the SDK, GEM, RecCapture, the High Speed Scanning Module and the SharePoint Integration Module. Even though the base product RecFind 6 has a full complement of enterprise content management functionality these add-on products provide options requested by our customers. They are generally a way to do things faster and more automatically.

We will continue to provide two approaches for document management; the end-user paradigm (RecFind 6 plus the Button) and the fully automatic capture and classification paradigm (RecFind 6 plus GEM and RecCapture). As has been the case, we also fully expect a lot of our customers to combine both paradigms in a hybrid solution.

The major architectural change is away from the .NET smart-client (fat-client) paradigm to the browser-based thin-client or web-client paradigm. We see this as the future for all application software, unconstrained by the strictures of proprietary operating systems like Microsoft Windows.

As always, our approach, our credo, is that we do all the hard work so you don’t have to. We provide the feature rich, scalable and robust image and data repository and we also provide all of the high level tools so you can configure your applications that access our repository. We also continue to invest in supporting and enhancing all of our products making sure that they have the feature set you require and run in the operating environments you require them to. We invest in the ongoing development of our products to protect your investment in our products. This is our responsibility and our contribution to our ongoing partnership.

As we all know, the world of IT and applications is changing rapidly and most of us application software vendors are trying to second-guess where the market is heading. The two key questions are:

1. How should we deliver applications? and
2. What should we be developing?

If we read and believe the IT press, especially the IT industry blogs, we should all be convinced by now that every application needs to be delivered on a mobile device. However, I am not fully convinced because I am a long-term and avid user of mobile devices, smartphone and iPad, and my experience tells me that mobile devices still don’t have the capabilities I need to be able to run all the applications I use. I also struggle to understand how to make my applications totally usable on mobile devices, especially smartphones.

For example, my smartphone is invaluable for checking and responding to emails when on the move. It is small, light, convenient (it sits in my shirt pocket) and has a long battery life. It also ‘connects’ to the Internet from most locations and 3G/4G and Wi-Fi services provide acceptable performance for email monitoring. But, it isn’t suitable for reading big documents and it isn’t suitable for lengthy responses. It is also painful when accessing web pages; the processor is too slow, the screen is just way too small and the QWERTY keyboard too small and too awkward for anything other than simple responses.

The iPad 2 is a lot better mainly because it has a bigger screen and more usable keyboard but it is still far from perfect.  Whenever I have to do real work (like writing this blog or writing program specifications), I end up working on my powerful laptop or desktop.

Strangely though, when I look at my laptop and desktop and all those messy cables and connections they look like museum pieces next to my iPad 2. In my opinion, the industry is somewhere between the old paradigm and the new paradigm but we haven’t got there just yet. Today’s mobile devices are a good first attempt but they don’t yet have what it takes to replace the desktop and laptop for serious business users.

The choice for us really comes down to developing and delivering software applications in either native mobile app mode (e.g., iPad apps developed in Xcode) or web-client mode (i.e., ‘thin-client’ applications that run in a browser and are developed using tools like HTML5, JavaScript and Ajax).

The web-client model is the best for us because it provides platform independence and the lowest cost delivery model. That is, it enables your application for all types of mobile devices as well as traditional notebooks and desktops and it is delivered just by the end user typing in a URL. It also only requires a single set of source code rather than the multiple sets of source code required to support native mobile apps for devices like the Android phone, iPad and Blackberry. It is therefore the lowest cost to develop and maintain and the lowest cost to roll out and support.

Ironically, the web-client model is also very old technology and I am surprised that after all these years we don’t have anything better to replace it.

As to what we should be developing, well that is literally the (multi) million dollar question. Our traditional fare is Enterprise Content Management software (ECM) or more simply, Information Management software. The ECM bag includes a host of horizontal market applications like document management software, records management software, contract management software, knowledge management software, etc. Our product RecFind 6 provides all of the above capabilities.

So, given that we already have a pretty clever and flexible ‘multi-application’ solution what should we replace it with or, what should we add to it? More importantly, what do customers need and want and even more importantly, what are they prepared to pay for?

Our customers happily tell us all the time about the new and extended functionality they would like to see in our products but usually their assumption is that we will fund the changes and provide the extended functionality free as part of a future upgrade. Usually they are right because we continually add new and improved features to successive upgrades provided under the customer’s maintenance agreement. However, for software vendors wanting to provide additional value and grow revenues, the real question is “is there a totally new product the majority of customers would need and want and be happy to pay for? “

I spend a lot of time thinking about this question. “What can I design and build that will provide significant value to a customer?” So much value in fact that the customer will be more than happy to outlay the funds to buy it. You might say it is the Holy Grail of software development, often called the ‘Killer App’. It may come as a surprise to those outside of our industry but many software developers spend enormous amounts of time and money building products no one buys.

A great idea doesn’t necessarily translate to success in the market. Similarly, because a customer says it wants something it does not necessarily follow that it will be willing to pay for it. As a software developer you have to ask the question, “If I build it, will you buy it?” This sounds a bit like Kevin Costner and his field of dreams movie, “Build it and they will come”, but in our case that isn’t necessarily true.

I have lots of ideas and have written lots of specifications and have built lots of applications but that killer app still eludes me. It must be time again to go out and ask our customers, “What would you like us to build? What application or feature or functionality would make a real difference to the running of your business? What application functionality do you need most of all? What do you believe will add the most value to your business? What would you like us to build next?”

Customers always have great ideas and they are often able to think outside the square. Software developers like us are more often than not too close to the problem. Now let’s see what they tell me, maybe that killer app is just around the corner, just like that next big lottery win.