Since joining Baynote last year, I’ve learned a huge amount about ecommerce retailing and how the technical side of developing algorithms for personalized product recommendations interacts with marketing, merchandising and a retailer’s website.
I’ve observed some tension between the technical side, who advocate relying on the wisdom-of-the-crowds algorithms, and product merchandising, who want to hand-tune the output of the recommendation system. This plays out in the configuration of guides and rules. Product experts will try to decide which categories of products are alternatives, which are complementary, and how to tweak the guides to give what they think the customer wants. Those on the technical side want the algorithms to be left alone to learn from the data we collect – to allow the site’s millions of users to determine the merchandising.
On a recent flight to Europe, I finally had a chance to watch “Moneyball,” where Billy Bean (Brad Pitt) is tasked with rebuilding the Oakland A’s baseball team on a minor league budget. He hires Pete Brand (Jonah Hill), a quantitative economist, who runs the statistics on the players and identifies the championship team that they can afford. They can afford this team because everyone else undervalues them for reasons that have nothing to do with their playing ability, but rather for their perceived flaws. There’s a scene where each time one of the players chosen by the numbers is suggested to the talent scouts, the scouts, who “know baseball,” reject him for being too old, having a bad personality or not looking like a major league player. The scouts insist that their long experience of choosing players makes for more success than just looking at the numbers ever could.
Billy and Pete go by the numbers and the A’s finish first in the American League West with a season that includes a record-breaking winning streak of 20 consecutive games.
I see parallels with algorithmic, wisdom-of-the-crowds merchandising via recommendations, and hand-tuned guides and rules assembled by someone who “knows merchandising”. As algorithms become more sophisticated, are built from larger and larger databases of site users’ behavior, and use more features of the users’ behavior that indicate their intent, the need for merchandising rules will weaken and allow one- to-one personalization to optimally merchandise a site’s products for each web site visitor.