Nine Tips for Data Analysis

Posted on August 6, 2013

I have recently been thinking about the kind of analysis work I normally do and trying to put it into the most general and abstract framework that I can. This is a useful exercise because it strips everything back to first principles and makes it easier to see common ground between things that, at first, can look quite different.

Here is a quick list of nine tips that came out of this framework.

  1. Make sure that recorded outcomes match up with business goals in some way
  2. Sometimes directly measuring business outcomes is hard. In this case things speed things up by choosing proxy metrics. Check every now and then that the proxy metrics are still good proxies (i.e. you still have to measure the important stuff).
  3. Establish what it is possible to change first then focus on these dimensions. Whoever has commissioned the analysis will only have the scope to take certain actions so it is best to focus on these areas.
  4. Generally the analysis will involve seeing how the important stuff changes along the different dimensions.
  5. Always be aware that a lot of the relationships you find have confounding variable somewhere; people in neighbourhood X probably don’t have a higher AOV because they live in X; they live in X because they are rich and this also causes a higher AOV.
  6. But confounding variables can be interesting too - you just need to be aware of how they interact with the actionable dimensions.
  7. Some people like you to show your working when presenting conclusions, others only want the actionable points. The worst is people who say they want one but who then act like they want the other.
  8. Always keep that sense of playful curiosity that makes this type of work so much fun.
  9. “he most exciting phrase to hear in science, the one that heralds new discoveries, is not Eureka! (I found it!) but rather, ‘hmm… that’s funny…’” - Isaac Asimov