numbers-don't-liePersonally, I love David Brooks. His op-ed articles and book The Social Animal display a thoughtfulness and intellect that goes beyond most editorialists these days. His recent New York Times article What Data Can’t Do (February 18, 2013) seems like it could have been written by a Baynote data scientist. In particular, Brooks mentions the fact that “Data obscures values… data is never raw; it’s always structured according to somebody’s predispositions and values.

The end result looks disinterested, but, in reality, there are value choices all the way through, from construction to interpretation.” He would have enjoyed Scott Brave’s whitepaper on Personalization: Technology, Psychology and Science where he talked about exactly that. With all the hype on machine learning these days, one would think that data when produced by computers is infallible and inexplicably unencumbered. But it isn’t. Until computer learning can program itself, write its own algorithms and make its own assumptions, human fingerprints will continue to be all over the “machine” learning we all talk about so much. This is why we in the personalized ecommerce business get into “algorithm wars” all the time. The logic behind product or content recommendation engines relies heavily on the algorithms and the data capture required for ecommerce recommendations that display items such as recent popular, also bought and others.

This data science really does determine how relevant a recommendation can be in the context of a shopper’s visit. Add to this the fact that the retail merchandiser has to deploy the results in the right places at the right times, and you have about a 50/50 mix of human intervention and machine learning.