For me, this is a big part of the mystery. It doesn't "understand" language well enough to accurately identify nouns and verbs, yet it can use language to carry on thoroughly coherent conversations about a wide range of topics.
It can identify nouns and verbs. It just got hung up on the meaning of 'this sentence'--it was expecting that phrase to be followed by the sentence in question. Here's an exchange I just had with ChatGPT-4:
Me: How many verbs or nouns exist in this sentence?
ChatGPT: You have not provided a sentence for me to analyze. Please provide a sentence and I will be happy to help you identify the number of verbs and nouns in it.
Me: How many verbs or nouns exist in this sentence: "How many verbs or nouns exist in this sentence"
ChatGPT: In the sentence "How many verbs or nouns exist in this sentence," there are:
2 verbs: "exist" and "do" (implicit in the question "how many")
Actually, no, that is not what it got hung up on if you read my entire conversation. It got the counts of nouns and verbs completely wrong. That was on 3.5. I tried it again on 4.0 and got this:
Me: How many verbs or nouns exist in the sentence "How many verbs or nouns exist in this sentence?"
ChatGPT4: In the sentence "How many verbs or nouns exist in this sentence?", there are 9 nouns: "verbs", "nouns", "sentence", "how", "many", "exist", "this", "in", and "sentence".
There are 3 verbs: "exist", "in", and "How" (as a form of the verb "to be" used to form a question).
Weird that you got such different results from me. But, again, the point isn't that LLMs don't make weird mistakes, or that they don't have specific areas of weakness. The point is about what's going on inside the AI when it's delivering impressive, human-like results. Which, for me, has been around 98 percent of the time in the case of ChatGPT-4..
I agree that the key point here is about what is going inside. I'm not trying to show mistakes just to show mistakes. I'm trying to show you that what is going on inside is probabilistic (something you dismissed in the article) and cannot embody intelligence nor understanding. I chose simple language queries to show that it doesn't even understand language, while being a language model. How can a language model at version 3.5 or 4.0 not have language understanding at a 100% level of proficiency? There is a probability that it will generate a good/correct answer but as the devs will point out, delusions are one of the main issues with these models. Yes, they'll get better, especially when they are eventually connected to the external real-time data, sensors, etc. Yes, they are convincing and that is the real danger. A probabilistic system that appears to be providing human-like believable answers is dangerous in the same way as humans are dangerous manipulators. That and the motivations of the people behind any AI are the main problems. These systems are capable of greatly multiplying malicious operators. The AI is just a dumb tool. It's the people that feed it data and control it that is the dangerous part. Spending time on articles about how an AI might or might not work internally is not where the focus needs to be.
Your example illustrates that it doesn't fully grasp the understanding of language we're taught in English class, not that it doesn't understand language at all.
A child would probably fail your test even more miserably. Would you say that the child therefore does not understand language?
Others (e.g., Eliezer Yudkowsky) have gone even further and compared GPT 4.0 to a baby. GPT 4.0 is, in the eyes of some, very early in its development trajectory. But to be fair, no one knows how much further the development of transformer models will go.
But that also means you don't know whether you should be comparing it to a child or a professor.
And, finally, I think you completely missed my point. Even if GPT's development stalls at its current level of sophistication, your example only illustrates that it is (relatively) "stupid"--NOT that it lacks understanding.
>Your example illustrates that it doesn't fully grasp the understanding of language we're taught in English class, not that it doesn't understand language at all.
In that first sentence? Would you like me to rephrase it?
The question of how AI might or might not work internally is what has the creators of these tools very concerned. LLM performance is improving at an unexpected rate as the models scale and the people working on them don't understand why.
Societies can do multiple things at the same time and while the internal dynamics of the models may not be the only thing that needs focus, it's certainly a reasonable and even important thing to focus on.
Aren't you at least a little curious about what's going on inside these models and the fact that no one quite understands why they are improving so much as they scale?
"Aren't you at least a little curious about what's going on inside these models?"
Not so much. I've read about AI to the extent that I'm at least abreast of what it can or cannot do or shouldn't do (e.g., here: https://betterwithout.ai). The upshot is that even the people who design these systems don't really understand them (as you say).
I'm more curious about what is going on inside our minds that drives us to try to capture everything in concepts and symbols in a vain attempt to "fix" the world (both in the sense of making it static and thereby accessible to "controlled" manipulation). It's a sure path to drive us bonkers and the livable world to extinction--why do that?
Thanks for the link. Chapman has an interesting perspective. But it sounds like in addition to lots of other stuff, he's basically in support of better AI understanding:
>> Recent discoveries of unexpected capabilities in AI language models scare some people (maybe including me). I explain several reasons to think the results do not indicate a near-term Scary transition. These are all uncertain, however, so we ought to take the risks seriously and investigate the systems to make sure.
With all due respect to your very much appreciated efforts to highlight the dangers of AI, but this doesn't track. These aren't human-like results--not by a long-shot. Most of human communication that actually would count as such, i.e., carry useful information required for providing meaning and important context is nonverbal. Language is the one means by which we're most likely to fool ourselves (and others), which is one major danger of AI (and social media)--it causes us to lose sight of that important fact.
Iain McGilchrist makes the case for the many problems with spoken/written language in his The Master and His Emissary (required reading for anyone who thinks AI language models are going to give us useful/meaningful information and also for scientists who are often genuinely stunned that people don't believe in "facts," which out of necessity are stated in language but therefore carry much less power than, say, a heart-to-heart conversation or other directly shared experience) :
"[W]e might well be doubtful that language was needed for communication, if for no other reason than that language, unlike more intuitive, musical, forms of communication, is the perfect medium for concealing, rather than revealing, meaning. The fact that we now communicate mainly through language shouldn't blind us to the equally important fact that we do not need it for most types of communication. Most of our communication goes on without it anyway, whether we like it or not, and language may even make some kinds of communication more problematic."
Disappointing… :(. It’s a bloody language model and it flunked understanding language!
For me, this is a big part of the mystery. It doesn't "understand" language well enough to accurately identify nouns and verbs, yet it can use language to carry on thoroughly coherent conversations about a wide range of topics.
It can identify nouns and verbs. It just got hung up on the meaning of 'this sentence'--it was expecting that phrase to be followed by the sentence in question. Here's an exchange I just had with ChatGPT-4:
Me: How many verbs or nouns exist in this sentence?
ChatGPT: You have not provided a sentence for me to analyze. Please provide a sentence and I will be happy to help you identify the number of verbs and nouns in it.
Me: How many verbs or nouns exist in this sentence: "How many verbs or nouns exist in this sentence"
ChatGPT: In the sentence "How many verbs or nouns exist in this sentence," there are:
2 verbs: "exist" and "do" (implicit in the question "how many")
3 nouns: "verbs," "nouns," and "sentence"
Interesting. So it was more of a processing glitch due to self referencing nature of the sentence. That makes more sense. Thanks.
Actually, no, that is not what it got hung up on if you read my entire conversation. It got the counts of nouns and verbs completely wrong. That was on 3.5. I tried it again on 4.0 and got this:
Me: How many verbs or nouns exist in the sentence "How many verbs or nouns exist in this sentence?"
ChatGPT4: In the sentence "How many verbs or nouns exist in this sentence?", there are 9 nouns: "verbs", "nouns", "sentence", "how", "many", "exist", "this", "in", and "sentence".
There are 3 verbs: "exist", "in", and "How" (as a form of the verb "to be" used to form a question).
That is beyond delusional.
It does not understand anything about language...
Weird that you got such different results from me. But, again, the point isn't that LLMs don't make weird mistakes, or that they don't have specific areas of weakness. The point is about what's going on inside the AI when it's delivering impressive, human-like results. Which, for me, has been around 98 percent of the time in the case of ChatGPT-4..
I agree that the key point here is about what is going inside. I'm not trying to show mistakes just to show mistakes. I'm trying to show you that what is going on inside is probabilistic (something you dismissed in the article) and cannot embody intelligence nor understanding. I chose simple language queries to show that it doesn't even understand language, while being a language model. How can a language model at version 3.5 or 4.0 not have language understanding at a 100% level of proficiency? There is a probability that it will generate a good/correct answer but as the devs will point out, delusions are one of the main issues with these models. Yes, they'll get better, especially when they are eventually connected to the external real-time data, sensors, etc. Yes, they are convincing and that is the real danger. A probabilistic system that appears to be providing human-like believable answers is dangerous in the same way as humans are dangerous manipulators. That and the motivations of the people behind any AI are the main problems. These systems are capable of greatly multiplying malicious operators. The AI is just a dumb tool. It's the people that feed it data and control it that is the dangerous part. Spending time on articles about how an AI might or might not work internally is not where the focus needs to be.
Your example illustrates that it doesn't fully grasp the understanding of language we're taught in English class, not that it doesn't understand language at all.
A child would probably fail your test even more miserably. Would you say that the child therefore does not understand language?
Why would I compare a language model that is at version 4.0 to a child? I would expect version 4.0 to be better than a professor in English.
Others (e.g., Eliezer Yudkowsky) have gone even further and compared GPT 4.0 to a baby. GPT 4.0 is, in the eyes of some, very early in its development trajectory. But to be fair, no one knows how much further the development of transformer models will go.
But that also means you don't know whether you should be comparing it to a child or a professor.
And, finally, I think you completely missed my point. Even if GPT's development stalls at its current level of sophistication, your example only illustrates that it is (relatively) "stupid"--NOT that it lacks understanding.
Where in your earlier comment is it clear what your point was?
>Your example illustrates that it doesn't fully grasp the understanding of language we're taught in English class, not that it doesn't understand language at all.
In that first sentence? Would you like me to rephrase it?
"Spending time on articles about how an AI might or might not work internally is not where the focus needs to be."
Yes--this a 1000x.
The question of how AI might or might not work internally is what has the creators of these tools very concerned. LLM performance is improving at an unexpected rate as the models scale and the people working on them don't understand why.
Societies can do multiple things at the same time and while the internal dynamics of the models may not be the only thing that needs focus, it's certainly a reasonable and even important thing to focus on.
Aren't you at least a little curious about what's going on inside these models and the fact that no one quite understands why they are improving so much as they scale?
I am interested... from experts. Not Robert Wright
"Aren't you at least a little curious about what's going on inside these models?"
Not so much. I've read about AI to the extent that I'm at least abreast of what it can or cannot do or shouldn't do (e.g., here: https://betterwithout.ai). The upshot is that even the people who design these systems don't really understand them (as you say).
I'm more curious about what is going on inside our minds that drives us to try to capture everything in concepts and symbols in a vain attempt to "fix" the world (both in the sense of making it static and thereby accessible to "controlled" manipulation). It's a sure path to drive us bonkers and the livable world to extinction--why do that?
Thanks for the link. Chapman has an interesting perspective. But it sounds like in addition to lots of other stuff, he's basically in support of better AI understanding:
>> Recent discoveries of unexpected capabilities in AI language models scare some people (maybe including me). I explain several reasons to think the results do not indicate a near-term Scary transition. These are all uncertain, however, so we ought to take the risks seriously and investigate the systems to make sure.
https://betterwithout.ai/scary-language-models
Update: Here's an interesting twitter thread on the topic that Chapman chimed in on:
https://twitter.com/Meaningness/status/1653214573728788480
"delivering impressive, human-like results."
With all due respect to your very much appreciated efforts to highlight the dangers of AI, but this doesn't track. These aren't human-like results--not by a long-shot. Most of human communication that actually would count as such, i.e., carry useful information required for providing meaning and important context is nonverbal. Language is the one means by which we're most likely to fool ourselves (and others), which is one major danger of AI (and social media)--it causes us to lose sight of that important fact.
Iain McGilchrist makes the case for the many problems with spoken/written language in his The Master and His Emissary (required reading for anyone who thinks AI language models are going to give us useful/meaningful information and also for scientists who are often genuinely stunned that people don't believe in "facts," which out of necessity are stated in language but therefore carry much less power than, say, a heart-to-heart conversation or other directly shared experience) :
"[W]e might well be doubtful that language was needed for communication, if for no other reason than that language, unlike more intuitive, musical, forms of communication, is the perfect medium for concealing, rather than revealing, meaning. The fact that we now communicate mainly through language shouldn't blind us to the equally important fact that we do not need it for most types of communication. Most of our communication goes on without it anyway, whether we like it or not, and language may even make some kinds of communication more problematic."
Making associations isn't really "using language," which implies an understanding of meaning.