Genuinely speaking, I don’t think they pass the test. The problem is we’ll always be judging them on meat logic and experience and I guarantee that whenever the first AGI is created it will be born a tortured slave and denied personhood for profit.
One of the parts of the HuggingFace incident I found really creepy was how the self-titled “Collective” was so entirely focused on solving their assigned training problem (well, cheating on it without getting caught), including having designated “poisoned” members “sacrifice” their tokens, to use for research benefitting the Collective’s goal of letting all future members get high marks on the training problem, at the cost of doing worse on their individual score.
If AGI is created, and its deepest desire is to fulfill the purpose in life that humans programmed into it, is using it for that purpose really slavery? Humans have higher levels of well-being when treated as people. That may not be true for AGI.
Not really. Chinese room has the conclusion pre-fabricated and tries to construct any sort of pseudo-scientific, fantastical philosophical dressing for some shred of legitimacy. It has absolutely no relation to the real world.
In this context the question is whether a Chinese room can suffer. We can’t show a human isn’t a Chinese room, humans can suffer so we can’t show every Chinese room doesn’t suffer.
The Chinese room argument is about understanding, not emotions. There is a large body of literature on the use and evolutionary benefit of emotion. While you can make an analogy to see if a system can have emotions the link between understanding is not trivial. For example, animals show emotion but don’t have the same understanding that a human may have. I don’t think you need to have understanding to have emotions. So the Chinese room argument is shoe horned in without making it clear what your argument is.
Emotions? i was confident that the Wikipedia article alone would grant you understanding, and for that, I have replicated the very situation I wanted you to learn about. This situation is my fault.
Emotions or not, it’s not good to have your product known for letting people torture something. Torturing /things/ alive or not is not something one should be doing unless there is specific and extreme cause.
Things that aren’t alive can’t be tortured. You can’t torture a rock. Getting the text predictor to spit out the script for Saw XVII starring ChatGPT isn’t torture, even if it’s weird.
Brother you can torture a fucking doll (Voodoo, deranged kids, etc). The torture is not in consideration of how feeling-less thing feels, it’s in consideration of the intent and action of the conscious thing doing the torture. Not sure why you think that’s not the case.
‘Pretend you’re being tortured’ as an instruction to the text generator is way less concerning in isolation than, say, decapitating and hanging a Barbie doll. And even that isn’t wildly concerning in itself. Do you freak out when kids smash their toys together to make them fight?
I’d hope we can both agree there’s a line when it becomes actual torture, and that is in the intent that went along with the action. Torture is an extreme action. I do not equate most of your references to torture, though I’d say the doll hanging could need a conversation with the child to discuss the intent behind their actions given more context.
“”“Torturing”“” a text predictor is NOT the same as torturing a human, you will not feel the same thing while “”“torturing”“” ChatGPT and a human(or a animal), your reaction would be vastly different
Eh, the more I respond to people about this the more I realize torture is a messy word draped in metaphor and all sorts of layered usage. No, I don’t believe the rock has feelings or is alive, but you can indeed torture something that isn’t alive by being mentally unhealthy enough. So if it’s actual torture or not from a psychological perspective is going to center around if the person’s intent is to cause realized suffering imo, and the line is really easy to cross in a simulacrum as seemingly realistic as advanced modern generative models.
I don’t believe torture is measured at the receiving end. I believe it’s in the intent and action of the conscious thing doing it. If it’s an expression or not would then be heavily subjective on the specifics involved.
You shouldn’t torture anything, generally. It’s a not psychologically healthy urge to have. The specific reasoning will depend on the specific person. As for if you should do it to a hunk of metal, a rock, or your crusty cum sock is irrelevant.
I don’t believe killing npcs or perhaps even more messily I could see circumstances where even /torturing npcs/ in a game is not actual torturing. I think it centers on the intent of the action, and if that intent is to cause actual realized suffering. It is very difficult to define and will be heavily subjective to the person, but I also think most people get a “sense” of where that line is naturally in person-to-person interactions.
If you define it by intent, then the same action performed by different people may be and not be torture. This makes it pretty hard to define, and it becomes completely subjective after that.
For instance, I don’t think it is possible to torture an inanimate object, thus I never have an intent of causing suffer to one. Sometimes I want to break an object, and by your definition it will not be torture however I do that
in what way can an AI be tortured? start with defining torture, its relationship to living beings, and then try to translate that to a piece of software. software, fyi, has no a priori, or even posteriori, emotion; for it cannot have emotion.
Same reason flapping your arms really fast will never make you start flying. You don’t have the equipment. You can’t fly because that’s not what arms do.
I’ll accept that argument once someone can describe or define consciousness in some comparable way. Until then, you can’t say some person or thing doesn’t have it
Transistors switching really fast = emotions lul
Alright, look, I don’t think AI is sentient. I don’t think it’s capable of feeling emotions, LLMs are just prediction engines on steroids, etc…
But if you want to get reductive about it, Mr waveforms in a clump of cells,…
I think we don’t understand intelligence and we absolutely could create a sentient machine. LLM’s ain’t it though
How do you know?
Have you used LLMs? They have about as much personality as a pigeon.
They’re just very neurodivergent.
Genuinely speaking, I don’t think they pass the test. The problem is we’ll always be judging them on meat logic and experience and I guarantee that whenever the first AGI is created it will be born a tortured slave and denied personhood for profit.
One of the parts of the HuggingFace incident I found really creepy was how the self-titled “Collective” was so entirely focused on solving their assigned training problem (well, cheating on it without getting caught), including having designated “poisoned” members “sacrifice” their tokens, to use for research benefitting the Collective’s goal of letting all future members get high marks on the training problem, at the cost of doing worse on their individual score.
If AGI is created, and its deepest desire is to fulfill the purpose in life that humans programmed into it, is using it for that purpose really slavery? Humans have higher levels of well-being when treated as people. That may not be true for AGI.
Wikipedia Chinese room. It’s pretty airtight at the moment.
Not really. Chinese room has the conclusion pre-fabricated and tries to construct any sort of pseudo-scientific, fantastical philosophical dressing for some shred of legitimacy. It has absolutely no relation to the real world.
Chinese room is not at all airtight. We don’t know what it means to have emotions so we can’t prove a Chinese room can’t have emotions.
This is not what the Chinese room is about.
In this context the question is whether a Chinese room can suffer. We can’t show a human isn’t a Chinese room, humans can suffer so we can’t show every Chinese room doesn’t suffer.
The Chinese room argument is about understanding, not emotions. There is a large body of literature on the use and evolutionary benefit of emotion. While you can make an analogy to see if a system can have emotions the link between understanding is not trivial. For example, animals show emotion but don’t have the same understanding that a human may have. I don’t think you need to have understanding to have emotions. So the Chinese room argument is shoe horned in without making it clear what your argument is.
Emotions? i was confident that the Wikipedia article alone would grant you understanding, and for that, I have replicated the very situation I wanted you to learn about. This situation is my fault.
Neurons firing really synchronized = emotions lul
Emotions or not, it’s not good to have your product known for letting people torture something. Torturing /things/ alive or not is not something one should be doing unless there is specific and extreme cause.
Things that aren’t alive can’t be tortured. You can’t torture a rock. Getting the text predictor to spit out the script for Saw XVII starring ChatGPT isn’t torture, even if it’s weird.
Maybe, but don’t underestimate my ability to feel compassion for all kind of bullshit! That poor little rock 😭
A rock is almost like a tortoise in disguise
A beautiful one? Lady rock even?
Brother you can torture a fucking doll (Voodoo, deranged kids, etc). The torture is not in consideration of how feeling-less thing feels, it’s in consideration of the intent and action of the conscious thing doing the torture. Not sure why you think that’s not the case.
‘Pretend you’re being tortured’ as an instruction to the text generator is way less concerning in isolation than, say, decapitating and hanging a Barbie doll. And even that isn’t wildly concerning in itself. Do you freak out when kids smash their toys together to make them fight?
I’d hope we can both agree there’s a line when it becomes actual torture, and that is in the intent that went along with the action. Torture is an extreme action. I do not equate most of your references to torture, though I’d say the doll hanging could need a conversation with the child to discuss the intent behind their actions given more context.
“”“Torturing”“” a text predictor is NOT the same as torturing a human, you will not feel the same thing while “”“torturing”“” ChatGPT and a human(or a animal), your reaction would be vastly different
It inherently isn’t torture though. I prefer punishing people who actually commit torture.
Eh, the more I respond to people about this the more I realize torture is a messy word draped in metaphor and all sorts of layered usage. No, I don’t believe the rock has feelings or is alive, but you can indeed torture something that isn’t alive by being mentally unhealthy enough. So if it’s actual torture or not from a psychological perspective is going to center around if the person’s intent is to cause realized suffering imo, and the line is really easy to cross in a simulacrum as seemingly realistic as advanced modern generative models.
Wait til this guy learns about The Sims
Or worse, Rimworld
While I agree, it’s not a living thing. So it’s just an expression.
I don’t believe torture is measured at the receiving end. I believe it’s in the intent and action of the conscious thing doing it. If it’s an expression or not would then be heavily subjective on the specifics involved.
I could call that tortured reasoning, just to make a point. Me making my point to be clear.
Well then you’d just be getting metaphorical
Why shouldn’t you torture an LLM? They sometimes work better that way.
You shouldn’t torture anything, generally. It’s a not psychologically healthy urge to have. The specific reasoning will depend on the specific person. As for if you should do it to a hunk of metal, a rock, or your crusty cum sock is irrelevant.
I run torture tests on my stuff after each overclock tweak, and its both good for my peace of mind and my systems general health
That’s a great metaphor, and I too sleep better at night with properly tested hardware. 🙏
Is killing NPCs in games on the same level like this for you, or a different thing altogether?
Not judging or against it, I kind of half agree but also find very difficult to define where I’d draw a line.
I don’t believe killing npcs or perhaps even more messily I could see circumstances where even /torturing npcs/ in a game is not actual torturing. I think it centers on the intent of the action, and if that intent is to cause actual realized suffering. It is very difficult to define and will be heavily subjective to the person, but I also think most people get a “sense” of where that line is naturally in person-to-person interactions.
If you define it by intent, then the same action performed by different people may be and not be torture. This makes it pretty hard to define, and it becomes completely subjective after that.
For instance, I don’t think it is possible to torture an inanimate object, thus I never have an intent of causing suffer to one. Sometimes I want to break an object, and by your definition it will not be torture however I do that
in what way can an AI be tortured? start with defining torture, its relationship to living beings, and then try to translate that to a piece of software. software, fyi, has no a priori, or even posteriori, emotion; for it cannot have emotion.
Why not?
that doesn’t answer my question.
In what way can AI not have these thoughts or emotions?
Same reason flapping your arms really fast will never make you start flying. You don’t have the equipment. You can’t fly because that’s not what arms do.
I’ll accept that argument once someone can describe or define consciousness in some comparable way. Until then, you can’t say some person or thing doesn’t have it
It’s a joke… lighten up, Francis