9 Comments

Robin, by the time we reach your benchmark of Noticable Improvement, we are already dead.

Expand full comment

Re: research on thinking together. There is a (fairly old) line of work by OpenAI that goes in this direction:https://openai.com/blog/lea...Is that closer to what you had in mind?

Expand full comment

Cultural evolution got going among humans at a certain level of intelligence. Machines are already there - and they already have the social skills to talk to each other. Firewalls coming down between systems with shared memes will help - as will systems coming down in price. It might take a while yet, but it seems as though the winter is over and from here on in looks like an accelerating landslide.

Expand full comment

These models are all trained using variations of sgd, but sgd is very poorly understood in this context. Simple questions such as "why does sgd work for this highly non-convex function?" or "why do these models generalize well given that they have weights which fit the training data perfectly and give random results on test data?" do not have satisfactory answers as far as I know. Questions such as "Do some of these gradient updates contain the information that would be learned from self-talk?" do not seem obvious to me, regardless of how one chooses to make this question concrete.

Expand full comment

I think they are pretty open on their training method.

Expand full comment

I didn't say this is the only thing that would excite me.

Expand full comment

I'd be careful saying you won't be excited until some particular method is used. Surely it's better to say you won't be excited until some particular capability is achieved? Historically even experts in the field have a terrible track record predicting what methods will be needed to make progress.

Expand full comment

How do you know the training process for palm doesn't do something equivalent to an inner monologue?

Expand full comment

It would be easy to get PaLM to self-speak - but about what? Training by self-play requires an objective to optimize or a win condition. The current objective of language models is "have a realistic conversation" aka small talk. Not "solve a real-world problem". It is amazing what PaLM has modeled about the real-world but at no point does it have goals about real-world states (which is maybe good from an alignment perspective). It has no metric that tells it how far its speach acts take it from a real-world state because it is not embedded or "grounded" in a physical world. It is just dreaming. But embedding is possible - think Tesla FSD. I think there could be a big overhang of capability if you hook up PaLM to a game engine.

Expand full comment