Archive

February 10, 2020

At the moment, my main hobby is rowing. A rowing boat with eight rowers has coxswain (cox) who steers the boat. He or she also has a microphone which is linked to speakers in the hull that they can use to communicate with and motivate the rowers. I have made a bot that imitates the styles and phrases often used by coxes during races. Generating the text takes around 3-5 minutes once you have pressed the button. Why not read more about how to make a bot like this below while you wait?
The coxwains in the training data often swear. Expect Robocox to do the same in the output Are you ready? How it all works An organisation called OpenAI trained a machine learning model on a very large amount of text from the Internet. We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training. The model they trained is called GTP-2 and some versions of it are open source and available for free. The

February 7, 2020

Here is a list of some of the links that I saved during January Discovering a formula for the cubic In the UK you learn the formula for the roots of a quadratic equation in high school. At my school we were also taught the “completing the square” method for solving a quadratic equation which can also be used to derive the quadratic formula. At university I learned that formulae also existed for the roots of cubic, quartic and quintic equations and I remember proving in a group theory course that no such formula can exist for polynomials of a higher degree than this. But I never learned what the formula for solving a cubic (or anything else) was. This post shows that you can easily derive the quadratic formula if you assume a certain form of the results (which is kind of obvious because you know the two roots are equidistant from the min/max point on the parabola). Then it shows how to go through a similar process for the cubic. It isn’t quite so easy because it is not as obvious what form the roots should take. How a cabal of romance writers cashed in on Amazon Kindle

January 29, 2020

Search engines use machine learning to develop the best possible ranking algorithm. Then, search engine optimisers (SEOs) pit themselves against this hidden algorithm, experimenting as they try to increase the rankings of a site. Only search engine employees really know which algorithms are used; in this post I will look at a known algorithm and then use techniques called “adversarial learning” to optimise against it. Working with a known algorithm is useful because it gives a controlled environment for running tests and optimisations. The algorithm we will be using is called Inception V3 and we will be using it to rank images of cats. The post will be in several sections: An introduction to Inception V3 and how we will use it to build the “cat engine” How to use something called “adversarial learning” to optimise images for the cat engine The methods in section two require knowing rather a lot about the Inception V3 algorithm. This section will show how to optimise an image whilst treating the algorithm as a “black box” where you don’t know how it works internally but only the input/output. Finally we make things even tougher by making

January 13, 2020

Here is a list of some of the links that I saved during December. My AI versus the Company AI AI has tremendous potential to free knowledge workers from drudgery. But is also has the potential to strip away identity and ownership of work as it has done for (e.g.) warehouse pickers. And that is before even talking about AI taking over people’s jobs entirely! The piece by Stripe Partners looks at the kind of AI tools that knowledge workers like working with and how AI can best be positioned as an enabler rather than a threat. It was produced as part of a piece of work with Google so I expect to see more of these ideas filtering through into Google products in the near future - the future is alreay here with things like Gmail’s auto replies and predictive text. What we discovered was that people wanted assistance to enhance their core work. Importantly, so that they retain agency, so that their skills becomes extended and augmented rather than replaced. For example, if your job is to design and run a workshop, you might want ideas – but you don’t want it to be designed for you. Why

January 6, 2020

More and more people are trying to train and apply machine learning models based on the pages a user on their site sees. This is often to do some kind of segmentation of the users based on the content they consume. In 2019 I’ve worked on a couple of models for this which have ended up with some things in common. Most users navigate the site with a sequence of pageviews. Some might branch out by opening new tabs and returning to old tabs but this is rare. It is tempting to use this sequence to train the model (e.g. with an LSTM layer). This takes much longer to train than a simpler model and the results aren’t that much better than what you get by using a “bag of words” type model where you only consider which pages were viewed and ignore the order. For sites (e.g. publishers) with frequently changing content the model will quickly go stale because the pages most users are viewing will fall outside the training set. If you train the model on the data from 2019 and users only look at pages published in 2020 then these will all be outside of what the