AuditNet®
Give Your Stat Sample the Boot!
by Mike Blakely
Have you ever been faced with an audit challenge that really stretches your stat sampling resources? Let's say that management has been informed (hot tip or otherwise) of alleged control exceptions that exist, but are likely infrequent. An example would be some employee expense reports that include unallowable expenses that are being reimbursed contrary to policy. What management wants to know is the actual dollar loss exposure. They have in their minds a threshold cutoff, say $50,000 /year and want you to inform them, with a very high level of confidence, whether this threshold has been exceeded. In other words, if your audit estimates that these unallowable expenses exceed $50,000 / year, with a high degree of confidence, then they will undertake some (costly) actions, otherwise, they will just send out reminders to employees that certain expenses are not reimbursable.
So let's say you go out, pull a random sample of 100 employee expense reports, and find four expense reports with unallowable amounts totaling say $100. You're excited and input the information into a statistical software calculator, but then very disappointed by the results. It comes up with a high degree of confidence that the total amount is at least $100 (what you found) and unlikely to exceed say $500,000. It also says that to get the confidence level you need, the required sample size will be about 4,000!
So what do you do? Go back to management and say that you pulled a sample but really can't answer their question. (Not good if you want to keep your job). Or do you go back and say you'll need to pull a sample of 4,000 and will get back to them in a few months (Also not a good option)? Is there an alternative or a better way to do it in this case?
Yes! Give your sample the BOOT! No, I don't mean discard it. There is actually a better way that was probably never covered in that undergrad Stat course you took years ago and also never mentioned in any of the Stat CPE courses you take every year. It's called bootstrapping.
Is this new? Definitely not! Does it have a solid statistical basis? Certainly! So how does it work, and what are the advantages and disadvantages?
Boot Strapping
Simplistically, boot strapping starts with just taking the sample results you’ve already obtained. Then use a computer program to sample the sample using replacement, thousands or tens of thousands of times and plot the results. This sounds complicated, but it's not. Here the sample was 100 items. So the computer program would select a random sample, with replacement, of size 100 from the sample of 100. The average of this sample would be recorded, and the process repeated thousands or tens of thousands of times. Then all these results would be plotted and the auditor would use the range of values where 80, 90, 95% of the sample means reside. This range would be used to estimate the population totals.
What are the disadvantages of bootstrapping? Well, it requires some computer resources, but those resources can probably be supplied using the machine you're using to view this article. It also requires some software (either buy or build).
What are the advantages? In a word, many! First, you can use the sample you already have and don't have to go out and select a huge sample. Second, the approach doesn't rely on any assumptions, such as what form or shape your population is. Both of these items can be huge advantages. Another advantage is that it will provide equal or more precise results than those obtained with traditional sampling.
So let's get back to your sample of 100. Unfortunately in this situation, the traditional statistical methods most commonly used by auditors will provide little in the way of information useful to management. But with the application of bootstrapping techniques applied to the sample already obtained, it is possible to provide management with information they can use. For example, the type of statement that could be made is "Based on a random sample taken, the auditors concluded, with a high degree of confidence, that the dollar amount of unallowable expense report items is at least $54,372 annually."
In next month's article we'll get into some examples and walk through the process of how all this is done.
Questions can be directed to the author at Mike.Blakley@ezrstats.com.
Article citation
Kvanli, A.H. and Schauer, R., A
The Bootstrap: What the Government Auditor Should Know,@ Journal of
Government Financial Management, Vol. 51, No. 3, 2002.
Web link to Article
The opinions, beliefs and viewpoints expressed by
the various authors and forum participants on this web site do not
necessarily reflect the opinions, beliefs and viewpoints of AuditNet®