LLMs are fundamentally different from human consciousness.
They are also fundamentally different from a toaster. But that’s completely irrelevant. Consciousness is something you get when you put intelligent in an agent that has to move around in and interact with an environment. A chatbot has no use for that, it’s just there to mush through lots of data and produce some, it doesn’t have or should worry about its own existence.
It simply returns the next most-likely word in a response.
So does the all knowing oracle that predicts the lotto numbers from next week. It being autocomplete does not limit its power.
LLMs are a dead end.
There might be better or faster approaches, but it’s certainly not a dead end. It’s a building block. Add some long term memory, bigger prompts, bigger model, interaction with the Web, etc. and you can build a much more powerful bit of software than what we have today, without even any real breakthrough on the AI side. GPT as it is today is already “good enough” for a scary number of things that used to be exclusively done by humans.
A chatbot has no use for that, it’s just there to mush through lots of data and produce some, it doesn’t have or should worry about its own existence.
It literally can’t worry about its own existence; it can’t worry about anything because it has no thoughts or feelings. Adding computational power will not miraculously change that.
Add some long term memory, bigger prompts, bigger model, interaction with the Web, etc. and you can build a much more powerful bit of software than what we have today, without even any real breakthrough on the AI side.
I agree this would be a very useful chatbot. But it is still not a toaster. Nor would it be conscious.
Suppose you were saying that about me. How would I prove you wrong? How could a thinking being express that it is actually sentient to meet your standards?
If you ask ChatGPT if it is sentient, or has any thoughts, or experiences any feelings, what is its response?
But suppose it’s lying.
We also understand the math underlying it. Humans designed and constructed it; we know exactly what it is capable of and what it does. And there is nothing inside it that is capable of thought or feeling or even rationality.
It literally can’t worry about its own existence; it can’t worry about anything because it has no thoughts or feelings. Adding computational power will not miraculously change that.
Who cares? This has no real world practical usecase. Its thoughts are what it says, it doesn’t have a hidden layer of thoughts, which is quite frankly a feature to me. Whether it’s conscious or not has nothing to do with its level of functionality.
Even if they are a result of complexity, that still doesn’t change the fact that LLMs will never be complex in that manner.
Again, LLMs have no self-awareness. They are not designed to have self-awareness. They do not have feelings or emotions or thoughts; they cannot have those things because all they do is generate words in response to queries. Unless their design fundamentally changes, they are incompatible with consciousness. They are, as I’ve said before, complicated autosuggestion algorithms.
Suggesting that throwing enough hardware at them will change their design is absurd. It’s like saying if you throw enough hardware at a calculator, it will develop sentience. But a calculator will not do that because all it’s programmed to do is add numbers together. There’s no hidden ability to think or feel lurking in its design. So too LLMs.
I believe I understand everything you are saying and why you are saying it. I think you are completely missing the point, though. LLMs already do quite a few things they were not designed to do. Also, your idea of sentience seems very limited. Yes, with our biological computers we have some degree of presence over “time”, but is that critical - or is it just critical for us due to our limited faculties.
What if “the internet” developed some form of self-awareness - would we know? Our entire society could be subtly manipulated through carefully placed latency spikes, for example. I’m not saying this is happening, just that I think you are incredibly overconfident because you have an understanding of LLMs current lack of state etc.
If we added a direct feedback mechanism - realtime or otherwise - we could start seeing more compelling emergent properties develop. What about feedback and ability to self-modify?
These systems are processing information on a level we cannot even pretend to comprehend. How can you be so certain that a single training refinement couldn’t result in some sort of spark - curiosity, desire to be introspective, whatever.
Perhaps Hofstadter is losing his mind - but I think we should at least consider the possibility that his concern is warranted. We are not special.
LLMs already do quite a few things they were not designed to do.
No; they do exactly what they were designed to do, which is convert words to vectors, do math with them, and convert it back again. That we’ve find more utility in this use does not change their design.
What if “the internet” developed some form of self-awareness - would we know?
Uh what? Like how would it? This is just technomystical garbage. Enough data in one place and enough CPU in one place doesn’t magically make that place sentient. I love it as a book idea, but this is real life.
What about feedback and ability to self-modify?
This would be a significant design divergence from what LLMs are, so I’d call those things something different.
But in any event that still would not actually give LLMs anything approaching: thoughts, feelings, or rationality. Or even the capability to understand what they were operating on. Again, they have none of those things and they aren’t close to them. They are word completion algorithms.
Humans are not word completion algorithms. We have an internal existence and thought process that LLMs do not have and will never have.
Perhaps at some point we will have true artificial intelligence. But LLMs are not that, and they are not close.
You apply a reductionist view to LLMs that you do not apply to humans.
LLMs receive words and produce the next word. Humans receive stimulus from their senses and produce muscle movements.
LLMs are in their infancy, but I’m not convinced their “core loop”, so to speak, is any more basic than our own.
In the world of text: text in -> word out
In the physical word: sense stimulation in -> muscle movement out
There’s nothing more to it than that, right?
Well, actually there is more to it than that, we have to look at these things on a higher level. If we believe that humans are more than sense stimulation and muscle movements, then we should also be willing to believe that LLMs are more than just a loop producing one word at a time. We need to assess both at the same level of abstraction.
They have no core loop. You are anthropomorphizing them. They are literally no more self-directed than a calculator, and have no more of a “core loop” than a calculator does.
Do you believe humans are simply very advanced and very complicated calculators? I think most people would say “no.” While humans can do mathematics, we are different entirely to calculators. We experience sentience; thoughts, feelings, emotions, rationality. None of the devices we’ve ever built, no matter how clever, has any of those things: and neither do LLMs.
If you do think humans are as deterministic as a calculator then I guess I don’t know what to tell you other than I disagree. Other people actually exist and have internal realities. LLMs don’t. That’s the difference.
They are also fundamentally different from a toaster. But that’s completely irrelevant. Consciousness is something you get when you put intelligent in an agent that has to move around in and interact with an environment. A chatbot has no use for that, it’s just there to mush through lots of data and produce some, it doesn’t have or should worry about its own existence.
So does the all knowing oracle that predicts the lotto numbers from next week. It being autocomplete does not limit its power.
There might be better or faster approaches, but it’s certainly not a dead end. It’s a building block. Add some long term memory, bigger prompts, bigger model, interaction with the Web, etc. and you can build a much more powerful bit of software than what we have today, without even any real breakthrough on the AI side. GPT as it is today is already “good enough” for a scary number of things that used to be exclusively done by humans.
It literally can’t worry about its own existence; it can’t worry about anything because it has no thoughts or feelings. Adding computational power will not miraculously change that.
I agree this would be a very useful chatbot. But it is still not a toaster. Nor would it be conscious.
Suppose you were saying that about me. How would I prove you wrong? How could a thinking being express that it is actually sentient to meet your standards?
By telling me you are.
If you ask ChatGPT if it is sentient, or has any thoughts, or experiences any feelings, what is its response?
But suppose it’s lying.
We also understand the math underlying it. Humans designed and constructed it; we know exactly what it is capable of and what it does. And there is nothing inside it that is capable of thought or feeling or even rationality.
It is a word generation algorithm. Nothing more.
Who cares? This has no real world practical usecase. Its thoughts are what it says, it doesn’t have a hidden layer of thoughts, which is quite frankly a feature to me. Whether it’s conscious or not has nothing to do with its level of functionality.
You seem unfamiliar with the concept of consciousness as an emergent property.
What if we dramatically reduce the cost of training - what if we add realtime feedback mechanisms as part of a perpetual model refinement process?
As far as I’m aware, we don’t know.
How are you so confident that your feelings are not simply a consequence of complexity?
Even if they are a result of complexity, that still doesn’t change the fact that LLMs will never be complex in that manner.
Again, LLMs have no self-awareness. They are not designed to have self-awareness. They do not have feelings or emotions or thoughts; they cannot have those things because all they do is generate words in response to queries. Unless their design fundamentally changes, they are incompatible with consciousness. They are, as I’ve said before, complicated autosuggestion algorithms.
Suggesting that throwing enough hardware at them will change their design is absurd. It’s like saying if you throw enough hardware at a calculator, it will develop sentience. But a calculator will not do that because all it’s programmed to do is add numbers together. There’s no hidden ability to think or feel lurking in its design. So too LLMs.
I believe I understand everything you are saying and why you are saying it. I think you are completely missing the point, though. LLMs already do quite a few things they were not designed to do. Also, your idea of sentience seems very limited. Yes, with our biological computers we have some degree of presence over “time”, but is that critical - or is it just critical for us due to our limited faculties.
What if “the internet” developed some form of self-awareness - would we know? Our entire society could be subtly manipulated through carefully placed latency spikes, for example. I’m not saying this is happening, just that I think you are incredibly overconfident because you have an understanding of LLMs current lack of state etc.
If we added a direct feedback mechanism - realtime or otherwise - we could start seeing more compelling emergent properties develop. What about feedback and ability to self-modify?
These systems are processing information on a level we cannot even pretend to comprehend. How can you be so certain that a single training refinement couldn’t result in some sort of spark - curiosity, desire to be introspective, whatever.
Perhaps Hofstadter is losing his mind - but I think we should at least consider the possibility that his concern is warranted. We are not special.
No; they do exactly what they were designed to do, which is convert words to vectors, do math with them, and convert it back again. That we’ve find more utility in this use does not change their design.
Uh what? Like how would it? This is just technomystical garbage. Enough data in one place and enough CPU in one place doesn’t magically make that place sentient. I love it as a book idea, but this is real life.
This would be a significant design divergence from what LLMs are, so I’d call those things something different.
But in any event that still would not actually give LLMs anything approaching: thoughts, feelings, or rationality. Or even the capability to understand what they were operating on. Again, they have none of those things and they aren’t close to them. They are word completion algorithms.
Humans are not word completion algorithms. We have an internal existence and thought process that LLMs do not have and will never have.
Perhaps at some point we will have true artificial intelligence. But LLMs are not that, and they are not close.
You apply a reductionist view to LLMs that you do not apply to humans.
LLMs receive words and produce the next word. Humans receive stimulus from their senses and produce muscle movements.
LLMs are in their infancy, but I’m not convinced their “core loop”, so to speak, is any more basic than our own.
In the world of text: text in -> word out
In the physical word: sense stimulation in -> muscle movement out
There’s nothing more to it than that, right?
Well, actually there is more to it than that, we have to look at these things on a higher level. If we believe that humans are more than sense stimulation and muscle movements, then we should also be willing to believe that LLMs are more than just a loop producing one word at a time. We need to assess both at the same level of abstraction.
They have no core loop. You are anthropomorphizing them. They are literally no more self-directed than a calculator, and have no more of a “core loop” than a calculator does.
Do you believe humans are simply very advanced and very complicated calculators? I think most people would say “no.” While humans can do mathematics, we are different entirely to calculators. We experience sentience; thoughts, feelings, emotions, rationality. None of the devices we’ve ever built, no matter how clever, has any of those things: and neither do LLMs.
If you do think humans are as deterministic as a calculator then I guess I don’t know what to tell you other than I disagree. Other people actually exist and have internal realities. LLMs don’t. That’s the difference.