Are businesses making better judgments now that they have access to more data than ever before?

Finding the illusive large wisdom in big data was the headline of a piece I authored in December 2014. We’ve gone a long way since then in terms of storing, analysing, and managing vast amounts of data in a digital setting, but people still struggle to uncover meaningful nuggets that might enhance business results. I’m reviewing the concept to evaluate how far we’ve progressed and how much farther we still have to go.

Baseball is one of the clearest instances of how data can have a positive influence on a company. The book and film “Moneyball” depicted how Oakland Athletics general manager Billy Beane revolutionised baseball by adopting sophisticated statistical analysis instead of relying only on human scout reports. Baseball is now governed as much by analytics as it is by experienced players — but is there such a thing as too much data?

According to Alex Spier of the Boston Globe’s Sunday Baseball Notes column, Boston Red Sox Manager Alex Cora will have 11 coaches on staff this year for a total of 26 players. When you compare that to Terry Francona’s six for 25 in 2011, you can see that the figure has almost doubled.

This, according to Spier, is due in part to the increasing quantity of data that teams are gathering, which necessitates more people to watch, understand, and execute a strategy to exploit it. “The result?” wrote Spier. Several clubs now have three hitting coaches, and staffs continue to expand in an attempt to condense mountains of data into something that the 26 players on a roster can understand.”

“Startups that analyse and extract insights from rapid data have attracted a lot of attention from both public and private investors.” Deepak Jeevankumer is a character in the film Deepak Jeevankumer

Baseball serves as a testing ground for complex statistical analysis, and companies may learn a lot from how the sport handles growing statistics.

Companies dump data into data lakes, experiment with machine learning models, and utilise this data to make choices, but they still have to make basic decisions every day based on their ever-growing mountain of data. Sure, machines and software can assist — and they are improving — but anybody who has dealt with poorly targeted advertising or email marketing knows there is still a lot of work to be done.

The difference in play today versus 2014, according to Deepak Jeevankumer, managing director at Dell Technologies Capital, is that firms are increasingly focusing on delivering insights to the people who need them quicker. “Big Data is no longer as important as ‘quick data.'” “E-commerce customers, streaming media users, gamers, stock/crypto traders, and business marketers all want quick insights and expertise,” Jeevankumer said.

He thinks that data should be examined in real time while flowing and after a query is produced, and that entrepreneurs who build solutions that enable corporations (and baseball clubs) to do so would be more successful in the long run.