Abstract, reporting, data analysis
Tax & AccountingFinanceComplianceJuly 08, 2020

Are you the gatekeeper of dark data?

If you have ever met me, you will know that I am a voracious reader. I particularly like to keep up with trends in technology, business, and more importantly with Internal Audit. In the last six months, I've been particularly aware of the call to action to create more visual, trend-based, predictive reporting to senior management, the board, and other oversight bodies.

It has occurred to me though that if we are to trailblaze our way to the boardroom table, we need to have access to our data to identify the trends and predict the future. How will so many CAE's do this when their data, their work product, is captured and held hostage in workpapers?

I've been exploring visualization tools, machine learning, natural language processing, RPA tools, and advanced analytics. One thing has become clear to me, it's really hard to access data that is trapped in documents, spreadsheets, and presentations. While the collaboration features in your document might make it easier for you and your fellow auditor to work together, you are often saving this data to the Cloud, and it is not likely the same Cloud as your audit management system. This separation makes it far more challenging for new technologies to access all that valuable insight you've spent days and weeks creating - you have created dark data.

Dark data is data that is created or acquired through various means but is then not used in any manner to derive insights or for decision making. For most organizations, 65% of the data they create is dark, meaning they don't capture it in a format that makes it easy to read. They create more data than their analytics can consume; unstructured, document-based data that is not consistent; and data is missing or incomplete. Most internal audit departments have this challenge.

Want to know how many times in the last six months you have audited an area that is planning to roll out a new artificial intelligence algorithm? You can search on this criteria if you added it to the title of a workpaper or wrote up an issue, but chances are the real count is in your interview notes, testing spreadsheets, and document requests, all of which are workpapers and probably dark to your search engine, reporting, and technology tools.

I know that our habits have been to document our work in a format that we attach as evidence when we are ready to have others view it, but so many of the tools today provide database fields that can just as easily capture this same information – only now it is discoverable by your users just conducting a simple search using technologies that can analyze it for trends and insights.

It is time to take our data out of the dark and let it light the way to greater and more actionable insights.


Colleen Knuff - Senior Director, Product Management
Senior Director, Product Management
Colleen is a Certified Internal Auditor (CIA), Chartered Public Accountant (CPA), Certified Information Systems Auditor (CISA), and is certified in Risk Management Assurance (CRMA).