False negative stretched routine software installation into four days of frustration

On Call As the weekend approaches, a question for our readers: was your week a success? Asking because in this week’s instalment of On Call – The Register's regular reader-contributed stories of techies being asked to assess the asinine – a failure to recognize success was the crux of the matter.

To start this tale, meet a reader we'll Regomize as "Dieter" who was once a Linux admin at one of the US's three big carmakers.

"The team I worked on took care of all sorts of Linux admin tasks, including working with software vendors to get their kit installed and tested in the corporate network," Dieter told On Call.

"Most of the time the software was already purchased and licensed by the time we sat down to install it. But there were those rare occasions where the IT leadership wanted to take the packages for a test drive before paying the vendor," he explained. When the bosses decided on that approach, they enabled verbose logging and debugging.

On one such project – involving a very well-known software vendor – preparing for this test required an installation process that took literally hours to run.

And then reported it had failed.

"The vendor team dived into the logs and worked for the rest of the day trying to determine what went wrong, but didn't uncover anything after reviewing the extensive log files," Dieter recalled.

Servers were therefore wiped ahead of a planned second installation attempt that was set to generate even more detailed logs.

That install also failed, which did not please Dieter's boss.

• Boss put project on progress bar timeline: three months … four … actually NOW!
• Cunningly camouflaged cable routed around WAN-sized hole in project budget
• That old box of tech junk you should probably throw out saves a warehouse
• Cheapest, oldest, slowest part fixed very modern Mac

On day three, the vendor returned, conducted a serious-looking meeting with serious-looking people wearing serious-looking suits, and tried again to install the software.And once again, went home defeated.

The next day, out came the big gun: a new installer that logged even more info.

It ran for most of the day, with Dieter and his colleagues watching line after line of text flash by.

After three failed installs, Dieter and his buddies knew exactly how long it would take before the installation would report failure. As they watched the clock tick towards the moment at which the previous installs had failed, their lack of faith was vindicated.

At the expected moment the dreaded message appeared: "Installation failed."

At which point the vendor team pounced on the keyboard and examined the last few lines of the output log. And discovered that the installation had in fact been a success.

Indeed, all four had worked.

But the conditional test the software used to check if it had been installed was hopeless at interpreting log files.

"In essence all of the many steps in the process passed and each one incremented the value of variable RESULT, but at the end they were checking the wrong variable," Dieter explained.

"Managers were less than impressed with the effort, and the response to the vendor from up the chain was something to the effect of 'Thanks for stopping by, no sale'."

Has your tech worked without reporting it's ready to roll? If so, click here to send On Call an email and we'll consider your yarn for a future Friday. ®