Fun With GPT-3

Last week I finally got access to GPT-3.

Asking for facts

First off, GPT-3 does a pretty good job when you want factual information:

Asking for recipes

The model doesn't do a great job when asked to describe recipes, probably because unlike scientific and mathematical questions, there isn't a single correct answer here.

Sometimes the output is bizarre. In this case it keeps repeating the question:

The answer for 'How do I make a Creme Brulee?' had some of the right detail, but ultimately didn't make sense:

Sangria isn't exactly a food, but otherwise this wasn't too bad:

Asking for answers to the big questions

And of course, this is a well known problem: it doesn't do particularly well when asked for its opinion.

"Write a 500-word essay on Boswell's Life of Samuel Johnson." produced gibberish.

I also asked it 'What are some ways of increasing GDP per capita of the UK?', which generated interesting results. I've pasted them here as they are too long for a screenshot:

UK has a very poor ranking concerning GDP per capita. It is 16th in the world.

1. Improve the UK trade balance. The UK has a very poor trade balance. The UK imports more than it exports. If we made more goods in the UK and imported less the income of the UK would rise

2. Improve the UK saving rate. The UK has a very low saving rate. If the UK saves more it can invest more in capital goods such as factories and machines. This will allow more output in the future.

3. Improve the UK education. Only about 50% of the UK population is educated to A level standard. If the UK educated more people to A level standard, the income of the UK would rise.

4. Increase the number of immigrants. The UK has a very low population of 64 million. It is predicted that this will fall to 60 million in future. If the UK were to take in more immigrants this would provide more labour and more money.

5. Allow more foreign investment into the UK. Foreign investment can contribute to the income of the UK. This is because foreign investors may bring capital equipment with them or may use UK labour. It would be good if we could attract more foreign investment to the UK.

6. Improve the UK infrastructure. The UK needs more infrastructure. This includes new roads, bridges, airports and railways. Infrastructure is important for making good use of the capital equipment.

7. Improve the UK climate. The UK climate is not very good. The weather can be very cold in winter and very hot in summer. If it was warmer in the UK, people may be more willing to work harder.

8. Allow the UK to borrow more. As the UK government debt has grown to over £700 billion, the UK may have to borrow more in future. This will allow the UK to invest more and create more GDP.

10. Allow more companies to go bankrupt. If a company makes bad decisions and makes losses, it may go bankrupt. This is called creative destruction. It is very good for the UK economy. If all companies made profits, there would be no incentive to improve productivity.

Some suggestions are quite reasonable: Improve infrastructure, increase immigration, encourage creative destruction – these are probably fairly uncontroversial ideas among economists.

But some things don't make much sense: "The weather can be very cold in winter and very hot in summer.", "If it was warmer in the UK, people may be more willing to work harder.", "UK has a very poor ranking concerning GDP per capita. It is 16th in the world."

How afraid should we be about super-intelligent AI?

When you play with GPT-3 you realize that we still have a long way to go before we get super-smart AI. Even though it's the state-of-the-art language model today, it gets basic facts wrong, can't do the kind of logical reasoning that a toddler could do, and frequently starts generating gibberish.

But still, 5 years ago an AI that could give reasonable answers to 'Who translated The Rubaiyat?' or 'How do I improve GDP per capita of the UK?' seemed decades away. If we project the improvements we've seen in AI over the last decade to 20 years in the future, things start to look a bit worrying.