Nine Tips for Data Analysis
I have recently been thinking about the kind of analysis work I
normally do and trying to put it into the most general and abstract
framework that I can. This is a useful exercise because it strips
everything back to first principles and makes it easier to see common
ground between things that, at first, can look quite different.
Here is a quick list of nine tips that came out of this framework.
- Make sure that recorded outcomes match up with business goals in
- Sometimes directly measuring business outcomes is hard. In this
case things speed things up by choosing proxy metrics. Check every
now and then that the proxy metrics are still good proxies
(i.e. you still have to measure the important stuff).
- Establish what it is possible to change first then focus on these
dimensions. Whoever has commissioned the analysis will only have
the scope to take certain actions so it is best to focus on these
- Generally the analysis will involve seeing how the important stuff
changes along the different dimensions.
- Always be aware that a lot of the relationships you find have
confounding variable somewhere; people in neighbourhood X probably
don't have a higher AOV because they live in X; they live in X
because they are rich and this also causes a higher AOV.
- But confounding variables can be interesting too - you just need
to be aware of how they interact with the actionable
- Some people like you to show your working when presenting
conclusions, others only want the actionable points. The worst is
people who say they want one but who then act like they want the
- Always keep that sense of playful curiosity that makes this type
of work so much fun.
- "he most exciting phrase to hear in science, the one that heralds
new discoveries, is not Eureka! (I found it!) but rather,
'hmm… that's funny…'" - Isaac Asimov