What would you say is the biggest fallacy in the world of Sarbanes-Oxley (SOx) internal controls reviews? I ask this question when presenting a course on how to cut the cost of complying with this monstrous body of law and regulation, and it always draws a cynical comment from someone about something. Often these are good points, reflecting widely held and entirely reasonable views that the whole exercise of section 404 reviews and audits has, so far, been more costly than it was worth.
The answer I'm thinking of is perhaps less obvious, but liberating. The biggest fallacy is the idea that the Committee of Sponsoring Organizations of the Treadway Commission (COSO) internal controls framework (or any other framework used in this field) defines effective control. In fact, it gives considerations, but stops well short of saying how effective "effective" is. It's rather like writing a definition for the term "long piece of string" by just saying "consider the length of the string" but not saying how long "long" is.
Since "effective" is not defined (and neither is "reasonable assurance"), then questions about how much evidence to collect and how much work is needed cannot be resolved by rereading the guidance and rules. The right amount of work is a matter of negotiation, not definition.
The Opportunity to Rethink, Again
In 2007 the Securities and Exchange Commission (SEC) and Public Companies Accounting Oversight Board (PCAOB) have again issued documents urging companies and their external auditors to be flexible in applying the rules and guidance. Again, they are saying these reviews should be risk focused and top down, and again they are carefully contradicting myths and pseudo-rules invented by the big auditing firms and others.
They are also issuing slightly revised rules that remove some of the statements that helped prop up expensive and inefficient reviews and external audits in the past.
It is easy to feel overwhelmed by the complexity of the rules and the difficulty of interpreting the nuances of expression in the latest documents, but for most people, this is not necessary. The key point is that the regulators are inviting everyone to rethink what they have done in the past and arrive at new judgements about what is necessary. Even now there is no definition as to how much is enough, so it is all about using the new mood music in the renegotiations.
Key Controls: the Big Issue
At my course recently, the main issue people wanted to talk about was how to cut down the number of "key controls" they planned to test. In the jargon of Sarbanes-Oxley, a "key" control is not necessarily an important control. Your set of key controls is the set that, if tested and found to be operating effectively, gives sufficient assurance for a happy conclusion to the review.
For example, you might have 200 controls over a large scale activity but 5 of them are carried out toward the end of the process and confirm that the other 195 have done their work and that there are indeed no remaining reconciliation problems or other errors. Without the 195 earlier controls, there would be a huge number of errors coming through, and the 5 final checks would be little comfort. But if the 5 final checks usually find little or nothing requiring correction, then they alone might be enough for the key control set.
Most companies feel that they have in the past included far too many controls in their key controls set and therefore made work for themselves and their external auditors. Now they want to cut key controls out.
How To Cut Key Controls
The typical problem situation is that we have a set of thousands of key controls from an earlier year, collected with a very strong anticipation that these would become a standard set to be routinely tested according to an annual/quarterly cycle. We would like to cut that set down to size, but have to contend with the mindset of repetition and potential resistance from the external auditors. Here are some suggestions for how this might be done.
There are three periods during which key controls could be revised:
before we go out to review and retest,
while reviewing and retesting, and
next year when we take advantage of redesigned controls.
Even before setting out to review and retest, there is new information on which to base a revised selection, such as the new guidance from the regulators and views about possible mistakes made the previous year. Consequently, there are reasons that can be given for cutting out key controls:
On reflection the risk involved is low and we simply do not need to test or continue documenting the control.
Results from last year indicate that the risks in this area are such that we can reduce our requirement for testing this year, both in sample sizes and in the size of our key control set.
Results from last year show that the overall level of error is low, and high level reconciliations and other last-check controls that simply confirm all is well are all we need to test this year. The preceding, low level controls can come out of the set.
Once you start reviewing your documentation and retesting, there are additional opportunities to remove key controls on the grounds that you can gain alternative evidence. Here are some reasons you can give for removing controls from the set:
These controls can be dropped because earlier testing (e.g., inspection of documents) has confirmed that the inherent risk level is at or below the level we estimated in planning, so the need for controls evidence is lower.
These controls can be dropped because inspection of error figures from a later stage in processing has already provided evidence as to the level of error.
These controls can be dropped because we have uncovered more convincing, more detailed controls at entity level that we can rely on more than last time.
The following year, provided you made appropriate suggestions that were implemented, you can use those reasons again, but more so because the alternative controls are better value. For example:
These additional controls can be cut out because we are now able to rely more heavily on some highly effective entity level controls (e.g., analytical review of performance using the "explanations before variances" method).
Yet more controls can be cut out because we are relying more on newly automated controls that provide a final proof of correctness.
The "explanations before variances" method of analytical review of results is a stronger version of typical analytics. Typical analytical review involves looking at differences between the draft actual numbers and either last year's numbers or a budget. Having identified differences, you seek explanations for them. The weakness in this approach is that hindsight enables people to remember the explanations that work in the direction required to explain the differences, but other explanations that might be relevant are not remembered.
The stronger approach is to ask for the major things that have happened during the period, compute their likely impact on the numbers, and only then compare them with the draft actual numbers. Companies that base analytics on rolling forecasts may be doing this already.
Enabling These Changes
Don't forget that in addition to cutting down the key control set, you should also be shifting toward reliance on ongoing monitoring controls/assessment, powered by improved process health metrics, and more effective supervision conversations, all copied to the evidence database as well as fed up the line where it can help the business become more efficient and more reliable.
To do all this requires that the review/testing people are able to (1) change what they test as they go, and (2) make recommendations for control changes that will move toward a better design that is more efficient in every way, even though there is no deficiency in SOx terms. You will need to make necessary planning, training, and documentation changes to get people to think, and provide the flexibility to do so. You will also need to provide documentation that can capture the full range of evidence and resulting decisions, and prompt people to consider control improvements while keeping them separate from information about suspected deficiencies.
Finally, since this involves some innovation compared to previous years, the best project structure will be one with very rapid incremental delivery of completed sections of work. Forget the idea of having everyone throughout the company reaching the same point at the same time. You want to have small, pioneering teams completing narrow areas of work within a couple of weeks, right through to remediation, and learning rapidly from these complete experiences so that more and more increments can be kicked off and rapidly completed.
Most people agree that Section 404 compliance is still too expensive. Cutting out "key" controls is one way to reduce the cost and the regulators have provided some reasons for doing it. In addition, there are other reasons that have often been overlooked and underused. With no clear definition of how much work is enough, this is all helpful ammunition in the negotiations over what is truly key.
Opinions expressed in Expert Commentary articles are those of the author and are not necessarily held by the author's employer or IRMI. Expert Commentary articles and other IRMI Online content do not purport to provide legal, accounting, or other professional advice or opinion. If such advice is needed, consult with your attorney, accountant, or other qualified adviser.