Decoding the complex online shopper’s mind and the key to unlocking it

Photo credit: creativedoxfoto / 123RF Stock Photo.

Maria had given birth to a beautiful baby boy three weeks earlier, and she was well on her way to losing those extra pounds. Spring was also just setting in and she was excited to finally browse through cotton knee-length non-maternity dresses.

But when she logged on to her favorite fashion site, she didn’t find even a single cotton knee-length dress. The site only recommended maternity dresses to her. She left instantly and the site lost a loyal customer.

What went wrong?

Was Maria’s case an algorithmic glitch? No. The personalization algorithms worked perfectly. She was shown maternity dress options based on what she historically browsed through and bought over the last nine months. Less importance was given to the shorter style of dresses she was recently looking at since there was simply not enough user data for the algorithms to pick up on.

Maria represents the proportion of disgruntled online shoppers who expect online stores to provide a retail experience that is tailored, dynamic, and specific to their shopping needs in real time. This case also represents the problem that AI-based personalization engine providers face: finding the balance between data-driven algorithms and real-time, contextual events that occur in a shopper’s immediate past (like Maria’s pregnancy) and long-term past that are not easily captured through conventional technology.

Fluctuations in browsing and spending patterns are caused by life events, and a page’s recommendations must be receptive to that. Most importantly, understanding the true intent of a shopper is a high priority for ecommerce companies.

Most brands use some form of personalization as a tool to build customer loyalty. However, shoppers are beginning to have unrealistic expectations of these engines. Highly optimized personalization engines require not only implicit data that is traceable by technology but also explicit user feedback data to successfully gauge the intent, context, and background of each online shopper.

Here comes the unavoidable glass ceiling.

The difficulties in cracking shopper intent

There are several challenges in deciphering customer intent.

First, a large proportion of customers accesses ecommerce sites through different channels. How does the intent and mindset of an Instagram shopper differ from that of a traditional brick-and-mortar store shopper? There is no established way to collect this type of data.

Second, there are customers who are unwilling to provide feedback regarding the products that are suggested, curtailing personalization. In the absence of this feedback, engines rely heavily on just clickstream data, which does not cover the whole story.

Third, while algorithms continually learn from the ranking systems and variables, the order of listing and the decision to recommend one product over the other is ultimately made by a human being. Several biases exist in the way the algorithms are trained for personalization. The recommendation system might favor one product over the other due to factors that may not be directly related to the shopper’s preferences.

Finally, algorithms require opportunities for rapid experimentation to iterate intelligently. While a subset of user behaviors can be modeled, it is unrealistic to expect all users to act the same. These atypical data points can skew the results and accuracy of these engines. Adding to this, ecommerce companies are mostly unwilling to permit experimentation on their user base.

This is not just a data science problem; this is a systemic one.

Decoding the psyche of online shoppers

Is convergence of big data and AI the next step?