Posts that are too self-centred, narcissitic, or short for the main archive.

July 17, 2020

I’ve been reading the Wikipedia page for Dunning-Kruger effect. There are some surprising things in it that I’ve detailed in this tweet thread. I think most people are mitaken about what Dunning and Kruger’s work actually tells us. But that isn’t what I’m writing about here. Wikipedia also has a section that suggests, maybe, the experimental findings are mathematically inevitable given how the experiment was setup. The intuition here is that if you are in, for example, the 96th percentile and you are asked to estimate your rank then you can only overestimate by 4 percent but you can underestimate by a lot more than that. The same idea is also true for those in the 4th percentile except there they have a lot more space to overestimate rather than underestimate. I’m going to make a simulation to investigate this. scores <- data.frame(actual=rnorm(1000)) ## Then simulate them estimating their competence ## The estimates are centred on their true competence with a bit of variation ## either way ## NB: There is no link between the precision of the estimate and competence scores$estimate

January 8, 2020

I’ve already written here about how I find September as much of a time for a fresh start as January. But this time around I’m feeling optimistic and I have several things I to look forward to over the rest of 2020. Does this mean my previous post was wrong? In terms of outcomes it must but I can offer the following mitigating circumstances: I was unusually quiet over the festive period. In fact I had two full weeks without having to do any client work at all which is the longest holiday I’ve had for some time. The weather today is lovely (albeit quite breezy) and I’ve been outside in the sun I’ve had two (!!) new business enquiries this week which is a lot for me and it makes me feel optimistic. Converting them into actual successful business opportunities is another story though… Because of all this I’ve been putting some time into thinking what I want to achieve in 2020. Sport My original rowing goal (set in September) for the season was to be in the top final for the single scull at one of the big multilane regattas. Because British Rowing have decided not to run

September 19, 2019

I have recently finished reading “The Book of Why” by Judea Pearl. It is not my first encounter with Pearl’s work; he is a leader in what he terms “the causal revolution” and this is the second book of his I have read on the subject. “The Book of Why” is a popular science overview and history of the field (particularly it’s division from statistics) but the first book of Pearl’s I read was his textbook “Causation” which was at the cutting edge of the field when he first published it. Why read “The Book of Why” when I’ve already read a much more advanced and detailed treatment - especially a popular science overview (since finishing my degree I have been too proud to read popular mathematics - this is my loss): I just didn’t get it when reading “Causation”. This is entirely normal when reading a maths textbook but, with hindsight, I also had the wrong expectations about what I was being told. I thought I would learn algorithms that infer causal models from raw data but this is not really what the book is about - it is more about confirming or denying the truth of a model

September 12, 2019

I always have a “back to school” feeling at the start of September even though it has been more than ten years since the end of my formal education. I thought that as my school and university days disappeared into the past that there would be a switch and I’d get a “new year, new me” feeling in January instead. This hasn’t happened and, after this length of time, I’m kind of doubtful it ever will. I don’t have kids, neither me nor my wife work in a school and it isn’t like we live in a university town either. So what is it about September that fills me with optimism for the coming twelve months - especially compared to January? First, let’s look at the reasons January sucks: It is very dark. I live around 55.5 degrees North and at the start of January there is only seven hours between sunrise and sunset. And, perhaps more importantly, there will have been less than eight hours between sunrise and sunset since the 22nd November. Although the days will be getting longer I’ll have been living in a dark fug for so long that it is difficult to see the light

August 29, 2019

This note is partly a test of my org-babel/hakyll setup and partly and exploration of a new (to me) form of the curse of dimensionality. What is the curse of dimensionality? Imagine a circle of radius 1 and the surrounding square. circleFun <- function(center = c(0,0),radius = 1, npoints = 100){ tt <- seq(0,2*pi,length.out = npoints) xx <- center[1] + radius * cos(tt) yy <- center[2] + radius * sin(tt) return(data.frame(x = xx, y = yy)) } circle <- circleFun() square <- data.frame(x=c(-1,1,1,-1,-1),y=c(-1,-1,1,1,-1)) ggplot(circle,aes(x=x,y=y)) + geom_path(color="green") + geom_path(data=square,color="blue") + theme_minimal() The circle takes up pi/4 ~= 79% of the