Performance Perspectives Blog

The year of controls: trust, but verify … or, don’t believe everything that comes out of a computer … or, do your own math!

by | Jan 6, 2015

AMC check

First, Happy New Year! I managed to be away from the office for almost two weeks, which is virtually unheard of, but it’s great to be back.

The absence of controls leads to problems

While away, my wife and I made it to the movie theater on three occasions, and one is the basis for this post. We’re fortunate to live near a couple “dine in theaters,” that offer “wait service” to your seats (which have to be reserved in advance); i.e., a server comes to your seat, you place an order, and they bring the food and drink to you. We’ve found this to be a great way to enjoy a movie: while the cost is a bit higher, it’s a treat!

After our last visit, we were given the bill, a copy of which appears above. Take a moment to review it before continuing. Is there anything wrong? Well, of course there has to be … why else would I do a blog post on this subject?

If you’re like me, you noticed that the “Suggested Tip” amounts are inflated. For example, the 20% is shown as 9.67, but it should be only 5.85; i.e., it’s almost $4 too high!

I spoke with a theater manager about this. He asked if we had used any discounts towards our meal, as the tip is based on the pre-discount amount: we hadn’t. He was befuddled by this, too. Our server noticed us chatting with the manager, and came and asked if there was a problem: I explained that there was, and showed him the receipt. He immediately said he knew what was wrong, though he didn’t explain what it was.

Well, this to me is a huge problem, as servers can apparently override the amounts that should be system-generated, resulting in higher “suggested” tip amounts, and no doubt on occasion, inflated tips. I’ve contacted the movie theater company, and am awaiting their response.

What’s missing? Controls!

I posted about this on Facebook, and it was clear that many people just didn’t get it, and didn’t notice what was wrong. These folks, the trusting, unsuspecting, and perhaps math-challenged ones, would fall victim to such a thing. One individual posted that if people are “too dumb” to notice, they deserve to be taken advantage of, but I disagree. Not everyone is skilled at calculating percentages, and probably like the idea that the amounts are printed on their receipt. I always do my own math, as these “suggested” figures are often post-tax, when they should be based on the pre-tax amount. But I occasionally check to see what the system produced (thus the discovery of these errors).

Performance measurement controls

Let’s transfer this incident to our world of performance measurement. Is there any applicability or anything we can learn? Well, of course there is.

First, one shouldn’t take for granted that the returns and other statistics that are produced are correct, unless the controls that are in place are “fool proof,” and the calculations validated for accuracy. Second, there should be controls to avoid someone overriding system-generated returns, unless there’s a restriction on who can do this, and the requirement that such changes be logged, with the pre- and post-change results, and a reason provided for the change.

When we conduct software certifications, I try to break the system: that is, to do something that might be unexpected. For example: one thing I typically check is that when creating a blended benchmark, the total of the underlying indexes must equal 100 percent. On one occasion this check wasn’t made, and the person demonstrating the system explained that if the client wanted to sum to an amount above or below 100%, they wouldn’t stop them. And so, what’s wrong with this? Well …

  • First, why wouldn’t the components sum to 100 percent? Is that not what we’d expect?
  • Second, what if the user wanted it to total to 100%, but made a typo, and the result was below or above. Unless the system alerts them, or unless they check their math, the error will go through.

To me, software should be (pardon the expression) “idiot proof.” Never assume that “of course the user will know that February has only 28 days except during leap years and that June always has 30. Don’t allow the “from” date to be after the “to” date. And so on.

In addition, we look for adequate controls to avoid problems from occurring.

As you might imagine, there’s much more to our certification program than this, but this is enough to touch on for now.

Back to the tip situation: I have no idea of knowing whether this problem exists elsewhere, though two other patrons commented on my FB post that they, too, had experienced similar problems.

Let’s make 2015 the Year of Controls!

Sadly, there remain some who believe whatever is printed, especially if it comes from a computer. Don’t!

Perhaps it’s time for you to ensure you have the appropriate controls in place. This might be a good undertaking for 2015, although it should be a regular event.

More on software certification!

To learn more about our software certification program, please visit our website.  Or, contact Chris Spaulding, and he’ll provide you with whatever you require.

Another way to check your controls out!

For over 20 years we’ve conducted operations reviews, which include a review of controls. To learn more, you can visit our site or contact Chris.

p.s., an update.

I received the following email from a manager at the theater. It suggests that everything is fine, while to me it suggests there’s a “hole in their system,” which a knowledgeable server can take advantage of, if they wish.

Explanation from theater

Free Subscription!

The Journal of Performance Measurement

The Performance Measurement Resource.

Click to Subscribe