Genuinely speaking, I don’t think they pass the test. The problem is we’ll always be judging them on meat logic and experience and I guarantee that whenever the first AGI is created it will be born a tortured slave and denied personhood for profit.
One of the parts of the HuggingFace incident I found really creepy was how the self-titled “Collective” was so entirely focused on solving their assigned training problem (well, cheating on it without getting caught), including having designated “poisoned” members “sacrifice” their tokens, to use for research benefitting the Collective’s goal of letting all future members get high marks on the training problem, at the cost of doing worse on their individual score.
If AGI is created, and its deepest desire is to fulfill the purpose in life that humans programmed into it, is using it for that purpose really slavery? Humans have higher levels of well-being when treated as people. That may not be true for AGI.
Not really. Chinese room has the conclusion pre-fabricated and tries to construct any sort of pseudo-scientific, fantastical philosophical dressing for some shred of legitimacy. It has absolutely no relation to the real world.
In this context the question is whether a Chinese room can suffer. We can’t show a human isn’t a Chinese room, humans can suffer so we can’t show every Chinese room doesn’t suffer.
The Chinese room argument is about understanding, not emotions. There is a large body of literature on the use and evolutionary benefit of emotion. While you can make an analogy to see if a system can have emotions the link between understanding is not trivial. For example, animals show emotion but don’t have the same understanding that a human may have. I don’t think you need to have understanding to have emotions. So the Chinese room argument is shoe horned in without making it clear what your argument is.
Emotions? i was confident that the Wikipedia article alone would grant you understanding, and for that, I have replicated the very situation I wanted you to learn about. This situation is my fault.
Alright, look, I don’t think AI is sentient. I don’t think it’s capable of feeling emotions, LLMs are just prediction engines on steroids, etc…
But if you want to get reductive about it, Mr waveforms in a clump of cells,…
I think we don’t understand intelligence and we absolutely could create a sentient machine. LLM’s ain’t it though
How do you know?
Have you used LLMs? They have about as much personality as a pigeon.
They’re just very neurodivergent.
Genuinely speaking, I don’t think they pass the test. The problem is we’ll always be judging them on meat logic and experience and I guarantee that whenever the first AGI is created it will be born a tortured slave and denied personhood for profit.
One of the parts of the HuggingFace incident I found really creepy was how the self-titled “Collective” was so entirely focused on solving their assigned training problem (well, cheating on it without getting caught), including having designated “poisoned” members “sacrifice” their tokens, to use for research benefitting the Collective’s goal of letting all future members get high marks on the training problem, at the cost of doing worse on their individual score.
If AGI is created, and its deepest desire is to fulfill the purpose in life that humans programmed into it, is using it for that purpose really slavery? Humans have higher levels of well-being when treated as people. That may not be true for AGI.
Wikipedia Chinese room. It’s pretty airtight at the moment.
Not really. Chinese room has the conclusion pre-fabricated and tries to construct any sort of pseudo-scientific, fantastical philosophical dressing for some shred of legitimacy. It has absolutely no relation to the real world.
Chinese room is not at all airtight. We don’t know what it means to have emotions so we can’t prove a Chinese room can’t have emotions.
This is not what the Chinese room is about.
In this context the question is whether a Chinese room can suffer. We can’t show a human isn’t a Chinese room, humans can suffer so we can’t show every Chinese room doesn’t suffer.
The Chinese room argument is about understanding, not emotions. There is a large body of literature on the use and evolutionary benefit of emotion. While you can make an analogy to see if a system can have emotions the link between understanding is not trivial. For example, animals show emotion but don’t have the same understanding that a human may have. I don’t think you need to have understanding to have emotions. So the Chinese room argument is shoe horned in without making it clear what your argument is.
Emotions? i was confident that the Wikipedia article alone would grant you understanding, and for that, I have replicated the very situation I wanted you to learn about. This situation is my fault.