August 28, 2007

Perhaps They Should Have Tested More - Tennessee Lottery

For 23 days (July 28th through August 20th), the computer system that randomly generates the winning numbers for the Tennessee Lottery was incapable of producing a number with repeating digits (such as 2-5-2 or 3-7-7-7).

New Jersey-based Smartplay International, Inc. sold this computer system to Tennessee for $221, 200.
"Lottery officials have said the glitch occurred because of a computer coding error. A Smartplay employee typed in a "u" for "unique" instead of an "r" for "repeat" in the computer code, they said."
The Tennessee Lottery paid $5,000 to Gaming Laboratories, Inc, to test the system.  They didn't catch this error.
"Officials with that firm, Gaming Laboratories Inc., have returned the money to the lottery, Hargrove said."

How ironic [from Aug 8, 2007]:
"The Lottery has received some complaints from players about the new system, [Kym Gerlock, a spokeswoman for the Lottery] said.

"We hope players will accept and embrace the new technology," she said. "What people really want are winning numbers."

Computers could even be more effective in generating random numbers than lottery balls, said Frank Harrell, chairman of the biostatistics department at Vanderbilt University.

"The chance of a glitch is so nil," he said. "You could run the generator for centuries, and it would not repeat itself."

In response, the Tennessee Lottery is offering a "double refund" for those who had purchased lottery tickets that had no chance of winning (and who happened to keep their tickets).

There were indications that something was wrong less than a week after the new system was put in place.  But the Lottery may not have had the expertise to assess the situation properly
Internal lottery documents show that players began complaining about the strange drawing patterns less than a week after the old ball-drop system was replaced with computers that could generate random numbers for the twice-daily drawings.

Lottery officials found the patterns unusual but initially assigned them to chance, an analysis that numbers experts called flawed.

"This is one reason why rigorous probability theory and statistical methodologies were invented: Intuition can mislead us when thinking about chance and probabilities," said Chris Orum, a mathematician at Los Alamos National Laboratory who has worked with the Oregon Lottery.
And tests on the test system didn't reproduce the problem.
The method used to test the system is complicated and difficult to explain but resulted in the bad code affecting the live drawings, but not the feature that tests the system, Maida said. That's why lottery officials were not finding any flaws during repeated tests, he said.

This is not the first time a repeating-digit bug occurred in a computerized lottery system.
May, 2005:
California lottery officials traced the problem to November, when a contractor was replacing the 7-year-old computer hardware and software that draws the winning numbers each day.

The programmer inadvertently copied a line of software code that kept horses from appearing in more than one winning spot per race. The code guarantees that horse No. 8, for example, doesn't win both first and second place.

But when the code was mistakenly applied to the race time, it blocked the computer from picking any number that repeated.

And now the Tennessee Lottery is paying for an audit:
The Tennessee Lottery is paying KPMG between $80,000 and $95,000 to perform an audit, said Kym Gerlock, lottery spokeswoman.

[10/30/07 update]

Tennessee lottery officials have asked Smartplay and Gaming Laboratories to cover refunds to lottery players turning in losing tickets, increased prizes given out for several weeks and fees paid to verify that the system is now working properly.

This could cost the two companies $1.4 million.

Am I the only one who thinks that an experienced Developer and/or Tester should have done better?