“They’ve got physical numbers on a piece of paper. It’s about control, manipulation, and coercion.”
Can you guess the speaker, or the context of that remark? Here are some hints: It involves shredded documents, a whistle-blower, and (potential) federal intervention. But before you reach for the obvious, know that it also involves radar guns, cameras, and masked men.
The speaker was Phil Janssen, who until last January oversaw the umpires of Major League Baseball (MLB), and who has spent a decade looking at ways to improve officiating. His comments to the National Labor Relations Board were merely the latest chapter in the sorry saga of Questec, an umpire-evaluation system that has become a hotly contested issue within MLB — one rife with lessons about the many ways that things can go wrong when technology, management, and those “physical numbers on a piece of paper” intersect.
Questec is billed (by the eponymous company that provides it) as an “umpire information system” comprising “measurement, replay, and analysis products.” That is, cameras, computers, and software are used in combination to assess how accurately umpires call balls and strikes. When MLB first began using the system, it was positioned as a teaching tool, a way to help umps understand which kinds of pitches tend to get called incorrectly. But Questec is capable of spitting out reports and rankings, and that proved hard to ignore, especially for a numbers-loving game like baseball. Soon a dispute broke out over whether the plumb home-plate umpiring assignment for the All-Star game should go to the umpire with the highest Questec rating. That led to allegations of document-shredding, hush money, and sundry other dealings that have provided fodder for sports columnists while reinforcing the sense that the true national pastime is litigation.
There isn’t a CFO in the land who’s oblivious to potential disruptions that can occur when technology, numerical analysis, performance measurement, and change management come into play simultaneously. The old saw “You can’t manage what you can’t measure” often gets turned on its head whenever technology provides data that didn’t exist before. When you have the capability to know what Websites your employees visit each day, or which employee processes the most invoices, or which business unit has shown the best year-on-year improvement, your job has just begun. The combination of the new and the incontrovertible holds an allure as potent as that of the corked bat, but hard numbers do not guarantee easy decisions.
That MLB leaders can now, thanks to technology, prove that an umpire calls X percent of the pitches he sees incorrectly doesn’t solve anything. In fact, it creates a situation in which MLB may make a colossal error thanks to an error-free system. No doubt the Questec brouhaha will make for an excellent business-school case study once the dust is swept off home plate. In the meantime, it provides a cautionary tale for any organization that confuses data with intelligence.