Blog Archives

Abusing Hadoop

By Robin Morris in A Word from the Engineers, Featured on July 11, 2013

blog image

Hadoop has, for good reasons, become the platform of choice for big data processing. It’s open source, it’s being developed rapidly, it runs on commodity hardware with no per-node licensing fees, and it has an active community and a steadily growing body of knowledge and experience built around it. All of which make adopting hadoop an easy sell for both start-ups and enterprise users who wish to migrate away from ... Read More »

Algorithms, Merchandising and Psychology – Part 2

By Robin Morris in A Word from the Engineers, Featured on May 9, 2013

blog image

In my last post, we discussed the recommendation algorithm, the merchandizing layer and the presentation layer as well as the fact that I am working on a KPI-optimizing algorithm that differs from a collaborative filtering algorithm. In running A/B tests on these differing approaches, we’ve found that the sites divide into two types. Type 1: ... Read More »

Algorithms, Merchandising and Psychology - Part 1

By Robin Morris in A Word from the Engineers, Featured on May 7, 2013

blog image

I was going to write this blog about feature hashing, a massively useful trick when building classifiers and predictive models.  It saves the time and complexity of building a dictionary and allows the hashed feature vector to be smaller than the number of possible features.  It is smaller because the number of actual features in a particular data set is often much ... Read More »

Machine Learning in Hive

By Robin Morris in A Word from the Engineers on March 20, 2013

blog image

As I’ve written before, we do a lot of our development here at Baynote in Hive, allowing us to leverage the power of our Hadoop cluster, whilst insulating us from writing low-level map-reduce jobs. One question that comes up on the hive mailing list from time-to-time is how to implement machine learning algorithms within Hive. Twitter told ... Read More »

A Word from the Engineers: Thanksgiving and Christmas are Different

By Robin Morris in A Word from the Engineers on January 31, 2013

blog image

Here in the engineering group at Baynote we had a big milestone at the end of 2012.  We successfully completed a major step of the platform updates we are putting together.  But more of that another time.  Here I want to look back at the last two months of last year, in terms the data we collect on e-commerce ... Read More »

12 Days of Shopping - Where Did That Package Come From?

By Robin Morris in Customer Experience, eCommerce, Featured on December 14, 2012

blog image

Yesterday, in my mail box, I found a package from Amazon.  In the package was a pair of gloves: Just a pair of gloves.  No invoice.  No packing slip.  No gift note.  Nothing, except a pair of gloves.  I hadn’t ordered a pair of gloves, and no one had told me they’d sent me a pair of gloves.  So, who did the mystery gloves come from? I went online, and ... Read More »

Writing a Hive Generic UDF

By Robin Morris in A Word from the Engineers, Featured on November 13, 2012

blog image

(How to convert an array<struct<target: bigint,quantity:int,price:float>> into an array<struct<target: bigint,quantity:int,price:float,externalid:string>> ) If you’ve been reading my blog posts over the last few months, you will have noticed that they’ve been focused more on the issues in the engineering side of Baynote, and less on the technical details. This blog post is an exception. It’s going to get very technical very quickly. Sorry. The back-end component of our ... Read More »

Why Amazon and Netflix have it Easy

By Robin Morris in A Word from the Engineers on October 8, 2012

blog image

Product recommendations on websites came into the public consciousness because of their prevalent adoption and continued use by two high-profile sites – Netflix and Amazon.  Amazon was first with a homegrown solution that offered recommendations based on what others browsed or bought that matched items that you bought.  Netflix next offered suggestions for films you might be interested in watching, based on ... Read More »

Find the Elephant

By Robin Morris in A Word from the Engineers on August 23, 2012

blog image

The last few weeks have been busy here in Baynote engineering. We’re about to go live with a new algorithm that runs on all of the new infrastructure that the platform team has been building over the last few months. The models are built on our hadoop cluster and incorporate a lot of the ideas I’ve written about in ... Read More »

Revenue, Margin, Seasonality and the Problem of Choosing the Best Recommendations

By Robin Morris in A Word from the Engineers on July 17, 2012

blog image

On the back of my business card it says, “Baynote: personalized shopping experiences.”  This can often seem very remote from what we in engineering work on day-to-day, where much time is spent in the technical guts of the products. So every now and again it’s good to think about the end goal that we’re working towards – providing ... Read More »