Maybe. More likely, though, LLMs just aren't the answer at all.
Just in the last week both Demis Hassabis and Yann LeCun have seemingly moved towards the position popularised by Gary Marcus that what AI needs is world models. Maybe if some of the huge buckets of venture capital can be moved across we might see some progress there?
Surely Yann LeCun has been saying that for years? I will keep banging the drum for distinguishing between a world model ("this is a flat circular ceramic disc") and a conceptual model ("this is a plate") because I think they'll be creating all sorts of headaches down the road if they don't separate those right from the start. Humans constantly get the two mixed up and therein lie most of our problems.
He may have been saying it years AGO, but for the last decade has been riding the LLM gravy train as far as I can see, working at Meta. I suppose maybe he was privately still thinking it all along...
We've been seeing different interviews with him, then, Dom! For the year before he left Meta, pretty much every LinkedIn post or talk was about how predictive models wouldn't get us to AGI and we'd need something like JEPA. This video from last summer is typical: https://www.youtube.com/watch?v=4__gg83s_Do
He struggles for funding, unfortunately, because the truth of the current situation is not that it's an AI bubble, but rather that it's an LLM bubble. Or so I keep saying!
Maybe. More likely, though, LLMs just aren't the answer at all.
Just in the last week both Demis Hassabis and Yann LeCun have seemingly moved towards the position popularised by Gary Marcus that what AI needs is world models. Maybe if some of the huge buckets of venture capital can be moved across we might see some progress there?
Surely Yann LeCun has been saying that for years? I will keep banging the drum for distinguishing between a world model ("this is a flat circular ceramic disc") and a conceptual model ("this is a plate") because I think they'll be creating all sorts of headaches down the road if they don't separate those right from the start. Humans constantly get the two mixed up and therein lie most of our problems.
He may have been saying it years AGO, but for the last decade has been riding the LLM gravy train as far as I can see, working at Meta. I suppose maybe he was privately still thinking it all along...
We've been seeing different interviews with him, then, Dom! For the year before he left Meta, pretty much every LinkedIn post or talk was about how predictive models wouldn't get us to AGI and we'd need something like JEPA. This video from last summer is typical: https://www.youtube.com/watch?v=4__gg83s_Do
True, he's been behind JEPA for ages - much longer than a year, even.
He struggles for funding, unfortunately, because the truth of the current situation is not that it's an AI bubble, but rather that it's an LLM bubble. Or so I keep saying!