How to manage the process via Natural Language

We have a large number of customers using our product RecFind 6 as a Records Management System and with new customers, the question always arises about how to best organize information in the RecFind 6 database to make it as easy as possible to manage and access. As the Metadata, Data Model and Business Processes in RecFind 6 are 100% configurable, every customer ends up with a unique configuration. As well as managing records, the end result also needs to be an Information Management System.

There is no one-solution that suits all. Some records managers want the shared drives structure replicated in the database. Some want everything filed under a strict hierarchical classification system or Taxonomy. Some IT Managers want the whole process simplified so end users clearly know where to file stuff and where to find stuff. Different managers in a single customer site will often disagree about how the information should be managed. It is usually up to us to come up with an agreed and workable compromise. There is no “one size fits all” paradigm here.

Whereas I fully support the principles behind most EDRMS standards (like ISO 15489) I also find myself agreeing with most end users who just want the whole process simplified and expressed in natural language, not as an arcane, complex, inconsistent and difficult to navigate hierarchical classification system.

The way you classify information should not dictate how you store, manage and retrieve information.

I have written a paper of this exact subject and although now somewhat old, it is still 100% relevant. You don’t need to agree with me, but please try to understand the message. End users want easy, fast access, not time-consuming complexity.

Let me tell you we solve the problem at Knowledgeone Corporation and manage our emails, electronic documents and shared drives with a hybrid system. We utilize a combination of RecFind 6 and shared drives. This is also a model we regularly recommend to our customers as an acceptable compromise; one that is simple to implement and one that always works.

I am a big fan of making information as easy as possible to capture and as easy as possible to retrieve (eDiscovery). This is especially important to the long-suffering end-user class who have no interest in becoming part-time records managers and who simply won’t use a system if it is too difficult to use and too time-consuming.

The biggest problem with complex, hierarchical classification systems is that no two people file the same way and even a single user will often file things differently over time. This in itself makes the act of finding something by browsing through a classification hierarchy a hit and miss affair.

At Knowledgeone Corporation, we implemented a hybrid model that uses a simply structured shared drive resource plus automated tools to ensure everything that should be captured is captured. This approach is also all about separating the functionality of the Authoring packages (e.g., Word, Excel, Outlook, etc.) from the functionality of the EDRMS. They have different roles to play.

We use our product GEM to automatically capture all work related emails and we use our product RecCapture to automatically capture all work-related electronic documents from our shared drives. We all use a common shared drive structure to write and store our original electronic documents.

We don’t use what you would call a formal taxonomy, we use what I call a ‘natural’ classification system. For us this means a classification system that perfectly reflects our business practices, processes and vocabulary. In our case, we are customer-centric so everything (apart from a little administrative and supplier stuff) is organized in customer or prospect folders and the lower levels are minimal, being things like Correspondence, Quotes and Orders.

The structure of our RecFind 6 database is mostly based on customer and prospect files; our Records Management System is also our CRM. Customers and prospective customers are our core business just as members and cases are the core business of unions. Every industry has a core business and, in my mind, this should always be reflected in the classification system used so that it perfectly aligns with the work practices, processes and ‘language’ of most staff. Whenever I consult with a new customer, I always try to first determine its core business and its natural language and then design the system around these.

Because we are customer centric, I need to be able to see everything about any customer or prospect in one place. For us this means focusing on the Entity record (the Entity table is where we store the basic information on each customer or prospect organization). As RecFind 6 is a relational database, we then store all related information in linked tables, all linked to and accessible from the Entity record with a single click.

Ease of Access for eDiscovery

In our RecFind 6 system, every piece of information I need to refer to is just one-click away once I view the customer’s entity record. Once I select the customer record, everything thing else I need to know is just one-click away and all links are viewable in a single screen.

We have configured our RecFind 6 security system around our management structure and that works fine for us. As a Director for example, most of the stuff I save (e.g., a letter, email or quote to a customer) is with a basic security code because everyone needs to be able to see it. However, as a Director I also have the right to save things with higher levels of security, e.g., Manager Director, with appropriate restricted access. Like all good security systems, it is simple but effective.

Our searching is also structured the same way. We have configured RecFind 6 to add the objects we need to search on as menu items in the search function just as we would do for any customer. We therefore have a Metadata search menu of Attachments (electronic documents, emails and images), Entities (Customers, Prospects, Partners and Suppliers), People, Incidents, Bugs, Quotes, Invoices, Timesheets, Support agreements, etc. We repeat this with Boolean searches. We make it as easy as possible and as logical as possible so our staff can find anything as fast as possible.

Most importantly, we provide multiple entry-points for searches. I can for example search directly for emails with a Metadata search, searching by a combination of Sender, Recipient, Date, Subject, Content, etc. Alternatively, I can search for customer emails from within the Entity record just by clicking on a single link for all emails and electronic documents for that customer.

You can search on any field and different classes of users can have different Metadata to both view and search on. The security system determines what each class of user (Security Group) can both see and then do with the information they can see. The members of each Security Group see only what they need to see and have only the functionality they need to get the job done

Following the above hybrid approach also means that you can still implement and manage all the recordkeeping principles such as retention and disposal schedules, location tracking, auditing, etc.

My point is that it is possible to meet the needs of all groups of users without frustrating any single group.  It just requires a hybrid approach and the configuration of the system to suit each class of user.

Making everyone happy is a lot better than making some people happy and some people unhappy. Why would you make users unhappy if you had a choice? Any Records Management System should also be an easy to use Information Management System.

I have written previously about ways to solve the shared drives problem (click here) and I have written numerous articles (and a book) about ways to manage emails and electronic/digital records. However, we still receive multiple requests from customers and prospective customers about the best, and simplest, way to effectively manage these problems.

The biggest stumbling block and impediment to progress in most cases is the issue of a suitable taxonomy or classification system. Time and time again I see people putting off the solution while they spend years and tens of thousands or hundreds of thousands of dollars grappling with the construction of a suitable taxonomy. I have written about this topic previously as well and if you want my recommendations please click on this link.

If you really want the simplest, easiest to understand, easiest to use and lowest cost way to solve all of the above problems then please forget about spending the next twelve to eighteen months grappling with the nuances of your classification system. It isn’t necessary.

What you need instead is a natural classification structure that reflects your business processes. Please give your long-suffering end users something they will instantly recognize and can easily work with because it is familiar from their day to day work. Give them something to work with that doesn’t require them to become amateur records managers battling to decipher a complex, hierarchical classification system that requires an intricate knowledge of classification theory to interpret correctly. Give them something that makes it as easy as possible to file everything in the right place first time with absolutely minimal effort. Give them something that makes it as easy as possible to find something.

What I am proposing isn’t a hundred-percent solution and it won’t suit every organization but I guarantee that it will turn chaos into order in any organization that implements it. You may well see it as an eighty-five-percent solution but that is a hell of a lot better than no solution. It is also easy and fast to implement and relatively low cost (you will need some form of RM software).

First up you need to make decisions about what kind of business you are.  Notice that I said “what kind of business you are” not “what kind of records you manage” or “how your business is structured”.  Most importantly, strongly resist the temptation to base your classification structure on your existing business structure or organization’s departments/agencies and instead base it on your most common business processes. Please refer to the following extract from:

Overview of Classification Tools for Records Management by the National Archives of Australia, ISBN 0 642 34499 X (an excellent reference document if you need to understand classification systems).

“Classifying records and business information by functions and activities moves away from traditional classification based on organisational structure or subject. Functions and activities provide a more stable framework for classification than organisational structures that are often subject to change through amalgamation, devolution and decentralisation. The structure of an organisation may change many times, but the functions an organisation carries out usually remain much the same over time.”

I would also strongly resist the temptation to build your classification structure on content; it is way too difficult. Instead, as I have said above, base it on your common business processes.

When I say classification structure I mean the way you name and organize folders in your shared drives. I can’t give you a generic solution because I am not that clever; I don’t know enough about your business. I can however, give you an example.

Please also remember that for the most part, we are dealing with unstructured source information; Word, Excel, PowerPoint, Emails, etc. Emails are a little easier to deal with because they have a limited but common structure, e.g., Date Received, Sender, Recipient, CC and Subject. With other electronic documents we are have far less information and are  usually limited to Author (not reliable), Date Created, Date Modified and Filename. Ergo, as I said earlier, trying to base a classification system on the content of unstructured documents is both difficult and inexact. It is certainly doable but you will have to spend a lot more money on consulting and sophisticated software to achieve your ends.

In my simple example of my simple system I am going to assume that your business is customer (or client) centric, i.e., as opposed to being case-centric or project-centric, etc. The top level of your classification structure therefore will be the client name and/or number. To make it as simple as possible I am going to propose only two levels. The second level represents your most common business processes, that is, what you do with each customer. So for example, I have:

Customer Name

     Correspondence

     Contracts

     Quotes & Proposals.

     Orders

     Incidents

I am also not going to differentiate between emails and other types of electronic documents, I am going to treat them all the same.

Now how does this simple system work?

1. Staff producing electronic documents don’t have their ‘own’ shared drive, all staff use the common classification structure. This is very important, let one or more people be exceptions and you no longer have a system you can rely on to meet your needs for reliable retrieval and any compliance legislation you are subject to.
2. Staff drag and drop or ‘save-as’ emails from their email client to the correct sub-folder.
3. Similarly, staff save (or drag and drop) electronic documents into the correct sub-folder. You can control access if required by applying security to electronic documents.
4. You purchase or build a document repository (based on any common database such as SQL Server, MySQL, etc.) and within this repository you replicate the folder structure of your shared drives with logical folders and subfolders.
5. You purchase or build a tool that constantly monitors the shared drives (e.g., using .NET Watcher technology) and that instantly captures a copy of any new or modified document (you do need to configure your repository to automatically version modified documents). You may also decide to automatically delete the original source document after it has been captured.
6. You build or purchase a records and document management software package that allows you to index, search and report on all the information in your repository.
7. You train your staff in how to save and search for information (shouldn’t take more than a half to one day) and then you go live.

I would also recommend applying a retention schedule based on sub folder (e.g., contracts) and date created and have the records management system automatically apply it to manage the lifecycle of captured documents. There is no sense in retaining information longer than you have to; it is also a dangerous practice.

Please note that the above is just an example and a very simple one at that. You need to determine the most appropriate folder structure for your organization.

WARNING

Do not let the folder structure become overly complex and unwieldy. If you do, it won’t work and you will end up with lots of stuff either not captured or captured to the wrong place. The basic rules are that if it takes more than few second to decide where to file something then it is too complex and that any structure more than 3 levels deep is too complex.

And finally, this isn’t just a theory, it is something we do in our organization and it is something many of our customers do. If you would like to read more on this approach there are some white papers and more explanations at this link. Alternatively, you can contact us and ask questions at this link.

Good luck.