Virtually every auditor, whether internal or external, has to test the effectiveness of internal control procedures.
For an external auditor, internal controls effectiveness testing allows you to have comfort over an assertion without needing to test substantively. For an internal auditor, a significant part of their role is providing assurance to management and the audit committee over internal control effectiveness, especially if their organization is subject to Sarbanes-Oxley requirements.
Traditionally, sampling would be used to test internal control effectiveness, using sample size guidance, usually something like this:
The logic behind this guidance is that if the control has operated effectively for the sampled instances, you can be comfortable that it has operated effectively every time.
Unfortunately, in reality this is often not the case – as auditors we see it time and time again in well-known, large organizations, which appear to have strong controls environments. They get a case of the “This one’s” or “This time’s”:
When occurrences like these are identified using sampling with a few items, it is clear that there may be many, many others that are missed. Of even more concern, when using sampling, you potentially have a very high likelihood of missing these single control failures. Consider:
The only way to get any real comfort over the operating effectiveness of an internal control procedure is to test every instance of it running.
Now, if the control is conducted completely offline, outside of any systems, that is likely to be impractical. However most controls are now evidenced somewhere in an organization’s computer systems. By obtaining a download of transactions from these systems, you can very easily test the control against 100% of the transactions, using Computer Aided Audit Tools (CAATs; also known as “audit data analysis” or “data analytics”).
As an example, let’s look at what can happen to the “This one’s” and “This time’s” from our 5 sampling scenarios above when we utilize a data analytics tool:
With a data analytics tool, all of these analytics can be conducted in a matter of minutes. And, each of these tests can be made repeatable to easily run on a regular basis, or, if need be, by different auditors. These are just a few simple examples, but data analysis can be used in many different ways to test a wide range of internal controls.
