How To Test Fewer Key Controls in a Sarbanes-Oxley Section 404 Project
July 2007
What would you say is the biggest fallacy
in the world of Sarbanes-Oxley (SOx) internal controls reviews? I ask this question
when presenting a course on how to cut the cost of complying with this monstrous
body of law and regulation, and it always draws a cynical comment from someone
about something. Often these are good points, reflecting widely held and entirely
reasonable views that the whole exercise of section 404 reviews and audits has,
so far, been more costly than it was worth.
by Matthew
Leitch
The answer I'm thinking of is perhaps less obvious, but liberating. The biggest
fallacy is the idea that the Committee of Sponsoring Organizations of the Treadway
Commission (COSO) internal controls framework (or any other framework used in
this field) defines effective control. In
fact, it gives considerations, but stops well short of saying how effective
"effective" is. It's rather like writing a definition for the term "long piece
of string" by just saying "consider the length of the string" but not saying
how long "long" is.
Since "effective" is not defined (and neither is "reasonable assurance"),
then questions about how much evidence to collect and how much work is needed
cannot be resolved by rereading the guidance and rules. The right amount of
work is a matter of negotiation, not definition.
The Opportunity To Rethink, Again
In 2007 the Securities and Exchange Commission (SEC) and Public Companies
Accounting Oversight Board (PCAOB) have again issued documents urging companies
and their external auditors to be flexible in applying the rules and guidance.
Again, they are saying these reviews should be risk focused and top down, and
again they are carefully contradicting myths and pseudo-rules invented by the
big auditing firms and others.
They are also issuing slightly revised rules that remove some of the statements
that helped prop up expensive and inefficient reviews and external audits in
the past.
It is easy to feel overwhelmed by the complexity of the rules and the difficulty
of interpreting the nuances of expression in the latest documents, but for most
people, this is not necessary. The key point is that the regulators are inviting
everyone to rethink what they have done in the past and arrive at new judgements
about what is necessary. Even now there is no definition as to how much is enough,
so it is all about using the new mood music in the renegotiations.
Key Controls: The Big Issue
At my course recently, the main issue people wanted to talk about was how
to cut down the number of "key controls" they planned to test. In the jargon
of Sarbanes-Oxley, a "key" control is not necessarily an important control.
Your set of key controls is the set that, if tested and found to be operating
effectively, gives sufficient assurance for a happy conclusion to the review.
For example, you might have 200 controls over a large scale activity but
5 of them are carried out toward the end of the process and confirm that the
other 195 have done their work and that there are indeed no remaining reconciliation
problems or other errors. Without the 195 earlier controls, there would be a
huge number of errors coming through, and the 5 final checks would be little
comfort. But if the 5 final checks usually find little or nothing requiring
correction, then they alone might be enough for the key control set.
Most companies feel that they have in the past included far too many controls
in their key controls set and therefore made work for themselves and their external
auditors. Now they want to cut key controls out.
How To Cut Key Controls
The typical problem situation is that we have a set of thousands of key controls
from an earlier year, collected with a very strong anticipation that these would
become a standard set to be routinely tested according to an annual/quarterly
cycle. We would like to cut that set down to size, but have to contend with
the mindset of repetition and potential resistance from the external auditors.
Here are some suggestions for how this might be done.
There are three periods during which key controls could be revised:
- before we go out to review and retest,
- while reviewing and retesting, and
- next year when we take advantage of redesigned controls.
Even before setting out to review and retest, there is new information on
which to base a revised selection, such as the new guidance from the regulators
and views about possible mistakes made the previous year. Consequently, there
are reasons that can be given for cutting out key controls:
-
On reflection the risk involved is low and we simply do not need to test
or continue documenting the control.
-
Results from last year indicate that the risks in this area are such
that we can reduce our requirement for testing this year, both in sample
sizes and in the size of our key control set.
-
Results from last year show that the overall level of error is low, and
high level reconciliations and other last-check controls that simply confirm
all is well are all we need to test this year. The preceding, low level
controls can come out of the set.
Once you start reviewing your documentation and retesting, there are additional
opportunities to remove key controls on the grounds that you can gain alternative
evidence. Here are some reasons you can give for removing controls from the
set:
-
These controls can be dropped because earlier testing (e.g., inspection
of documents) has confirmed that the inherent risk level is at or below
the level we estimated in planning, so the need for controls evidence is
lower.
-
These controls can be dropped because inspection of error figures from
a later stage in processing has already provided evidence as to the level
of error.
-
These controls can be dropped because we have uncovered more convincing,
more detailed controls at entity level that we can rely on more than last
time.
The following year, provided you made appropriate suggestions that were implemented,
you can use those reasons again, but more so because the alternative controls
are better value. For example:
-
These additional controls can be cut out because we are now able to rely
more heavily on some highly effective entity level controls (e.g., analytical
review of performance using the "explanations before variances" method).
-
Yet more controls can be cut out because we are relying more on newly
automated controls that provide a final proof of correctness.
The "explanations before variances" method of analytical review of results
is a stronger version of typical analytics. Typical analytical review involves
looking at differences between the draft actual numbers and either last year's
numbers or a budget. Having identified differences, you seek explanations for
them. The weakness in this approach is that hindsight enables people to remember
the explanations that work in the direction required to explain the differences,
but other explanations that might be relevant are not remembered.
The stronger approach is to ask for the major things that have happened during
the period, compute their likely impact on the numbers, and only then compare
them with the draft actual numbers. Companies that base analytics on rolling
forecasts may be doing this already.
Enabling These Changes
Don't forget that in addition to cutting down the key control set, you should
also be shifting toward reliance on ongoing monitoring controls/assessment,
powered by improved process health metrics, and more effective supervision conversations,
all copied to the evidence database as well as fed up the line where it can
help the business become more efficient and more reliable.
To do all this requires that the review/testing people are able to (1) change
what they test as they go, and (2) make recommendations for control changes
that will move toward a better design that is more efficient in every way, even
though there is no deficiency in SOx terms. You will need to make necessary
planning, training, and documentation changes to get people to think, and provide
the flexibility to do so. You will also need to provide documentation that can
capture the full range of evidence and resulting decisions, and prompt people
to consider control improvements while keeping them separate from information
about suspected deficiencies.
Finally, since this involves some innovation compared to previous years,
the best project structure will be one with very rapid incremental delivery
of completed sections of work. Forget the idea of having everyone throughout
the company reaching the same point at the same time. You want to have small,
pioneering teams completing narrow areas of work within a couple of weeks, right
through to remediation, and learning rapidly from these complete experiences
so that more and more increments can be kicked off and rapidly completed.
Summary
Most people agree that Section 404 compliance is still too expensive. Cutting
out "key" controls is one way to reduce the cost and the regulators have provided
some reasons for doing it. In addition, there are other reasons that have often
been overlooked and underused. With no clear definition of how much work is
enough, this is all helpful ammunition in the negotiations over what is truly
key.
Opinions expressed in Expert Commentary articles are those of the author and are
not necessarily held by the author's employer or IRMI. Expert Commentary articles
and other IRMI Online content do not purport to provide legal, accounting, or other
professional advice or opinion. If such advice is needed, consult with your attorney,
accountant, or other qualified adviser.