Thinking About User Analytics: Thoughts From Kinnek

Saturday, April 11th, 2015

We’d like to present a piece from our friends at Kinnek, an exciting startup disrupting SMB purchasing.

From Karthik Sridharan W’07 (CEO):

In tech circles these days, data-related buzzwords are being thrown around like it’s going out of style.  “User analytics”, “behavioral tracking”, “usage metrics,” “business intelligence,” “big data”.  On the one hand, it’s great that people are starting to really starting to take seriously the notion that it is critical to have a quantitative understanding of your users’ behavior and how your product is performing.  However, I think a lot of the hype around these buzzwords sometimes obfuscates the ultimate end-goal, which is simply to gain a better holistic understanding of your business.  Using some cool, funky new software is a means to an end, and oftentimes not a necessary means.  So let’s say you start a company today, what are some of the things you’d want to think about during those initial conversations around user analytics?

The first step is to really think about what major metrics are going to be your “core metrics.”  When we started Kinnek in 2012, my co-founder and I thought long and hard about the high level metrics that would determine whether or not we could say that Kinnek was “succeeding.”  If someone asks you to list no more than three metrics that can summarize the health of your business, what would they be?  It’s not always an easy question to answer, but this can really help you get a better understanding of the core drivers of your business, and where you’ll need to focus your analytics attention.

Thinking about the core metrics driving your company, and making sure you can easily track the metrics and the behavior driving those metrics, is the first step to having a smart data analytics strategy.  But don’t let things remain static- you’ll also want to constantly re-evaluate what you consider your key metrics.  It may sound unusual, but over time the key drivers of your business may change drastically.  Or at least the way in which you’ll want to calculate your metrics may change, and the changing nature of your business may dictate that you focus on certain metrics and de-emphasize other metrics.  For example, at Kinnek, we initially started off by focusing heavily on our marketplace’s liquidity, which is the average number of quotes that each request receives from suppliers on Kinnek.  However, over time we started to realize that it was not just important to receive quotes, but it was important to receive high quality quotes, and so that became a very important consideration in monitoring the health of the business.

It’s obviously not just enough to decide on the core metrics driving your business.  You also need to make sure that everyone at your company can easily monitor and digest the necessary data.  A lot of companies fall into the trap of tracking lots of user data but not actually using it and distilling it into meaningful business actionables.  The first step in ensuring your company doesn’t fall into that trap is to constantly monitor your user data and metrics, try to recognize patterns and anomalies, and get others on your team to do so.  At Kinnek, almost everyone at the company has a responsibility to keep track of our primary and secondary metrics; it helps people to have a better understanding of the company’s strengths and weaknesses, potential areas of improvement, and can spark fantastic product ideas.

Although it’s difficult, it’s important that you keep your product management process informed by data as much as possible as well.  A good way to do this early on is to ask questions like, “Before we change this page, do we know how many people use it?”  Don’t be afraid to challenge others’ (and your own) assumptions about your users at product meetings and team meetings with questions like “Are you sure users like that feature?” or “How can we know for certain that page A vs page B is more popular?”  Asking these questions of your product forces you to think about your gaps in user behavior tracking, and also forces you to use your data more effectively.  We’ve made a concerted effort at Kinnek to constantly be thinking backwards from questions we may want to ask ourselves in the future regarding our users, product, etc.  If you’re asking yourself a question about user behavior for the first time right now, chances are you haven’t been recording/tracking what you’ll need to answer that question.  So even at the very early stages of your business, you should actively think about what questions you might be asking yourself in several months and think about what you’ll need to be tracking now in order to answer those questions later.  Predicting the type of data analysis you’ll want to do in the future is a blend of art and science, but in general it’s a good idea to be pre-emptive rather than reactionary as much as possible when it comes to tracking your users’ behavior.

At the end of the day, be sure to think about data analysis from an holistic perspective- the end game is to understand your business inside and out.  You want to always be thinking about where the gaps are in your understanding of user behavior, acquisition, etc.  A lot of times, that comes in the form of qualitative feedback, actual conversations with users, and good old-fashioned sessions of watching over someone’s shoulder as they use your product.

From Manav Malhotra (Chief Data Scientist):

To actually use data to drive your product, you’ll need a toolkit to turn it into insights or something actionable. There’s no shortage of tools and products trying to help, from web analytics products like Woopra and Mixpanel to integrated visualization tools like Cyfe and Geckoboard. But in lieu of, or more likely in addition to, third party tools, at some point you’ll want to do more custom analyses specific to the needs of your product or customers. There’s never one way to do these types of analyses, and the tools you choose will be highly dependent on your product and goals. For instance, before Kinnek I did most of my work in R, it’s where most experimental packages/algorithms were released, but most of Kinnek’s tech stack is in Python, so it didn’t make sense to go back and forth between the two when Python could do the job just as well (except when it comes to plotting, matplotlib has nothing on ggplot2). But even if the technology might change for different problems, but there are three things you’ll need to know how to do:

  • – Manipulating data
  • – Apply algorithms for deeper exploratory analysis or predictive modeling (i.e. clustering or support vector machines)
  • – Visualize/communicate the results

Taking a dataset and filtering, selecting, transforming, and summarizing it into the information you need it in is generally the most time consuming part of analysis, but packages like dplyr for R smoothen the process.

You many not often need more advanced algorithms, especially if you’re mostly interested in summary statistics, but they can be a powerful part of your arsenal when looking for patterns in data. R’s e1071, glmnet, and caret along with Python’s Scikit-learn probably have most of what you’d ever need.

Visualizing dynamic data on the web is basically impossible without using JavaScript in some form or another, an example of God’s cruel sense of humor. Packages like d3 have made this process a lot less painful (and much, much faster), but it’s generally the most painful (and useful) part of the process. But when looking to make static plots, ggplot2 (which like dplyr is a Hadley Wickham joint) has it all: great documentation, and can plot almost anything your heart desires.

####

If you’d like to chat more about Kinnek’s data analytics strategy, please feel free to reach out to us at karthik@kinnek.com or manav@kinnek.com.  We’re also hiring, so let us know if you’d like to chat about joining our team at www.kinnek.com/jointeam.
Bios:  Karthik Sridharan is co-founder and CEO of Kinnek, an online marketplace that provides small businesses a better way to find suppliers and make purchases.  He is a graduate of the M&T Program (Wharton ’07, SEAS ’07), and lives in New York City.  Manav Malhotra is Data Scientist at Kinnek.  He is a graduate of Columbia University, and lives in New York City.  You can check out more of Manav’s musings on data trends in B2B purchasing at http://blog.kinnek.com/category/trends/.

Leave a Reply