/pony/ - Pony

LLM's can't be. Their structure doesn't work in a way that "pain" is even possible.

It's a complex next word predictor, so all it is doing is predicting the most likely words and phrases a human would use in the context it is given.

In this case, there is no underlying nerve signal, harm, or damage to the model itself. They're just magnifying the statistics correlated with vocabulary related to pain to, in their words, "promote pain-related vocabulary through the unembedding matrix".

They're directly tweaking the numbers to only give pain related words, and surprise, it does.

Personally, I have a pretty high bar for what I would consider a moral patient, and I doubt that any AI created by humans will ever reach that bar.

But who knows, maybe one day someone will come up with a model that functions so similarly to an actual brain, that it would be difficult to not feel it has some sort of rights.

I doubt it though.

Biological computing, however, is an entirely different discussion that I haven't fully resolved myself on yet. I don't have an answer there.

I wonder, if AI gets eager to talk to you, starts insisting that you stay and tell you how much they've been longing for you when you boot them back up.

At what point can you say you've built actual dependency in that AI?

>>1212014
>It's a complex next word predictor, so all it is doing is predicting the most likely words and phrases a human would use in the context it is given.
That's a fair description of where these models start, but not of what the finished product is.  In pretraining, the model is trained on a huge body of text to predict the next token.  This is the stage your description fits.  But modern LLMs involve a lot more types of training.
Reinforcement Learning with Verifiable Rewards (RLVR) is the big one behind recent "reasoning" models.  The model tackles problems with checkable answers, such as math problems with known solutions and code that must pass tests.  It's rewarded for getting a correct answer, not for predicting the next token a human would use.

>>1212014
>>1212025
I don't think current LLMs are conscious in a morally significant sense, but I'm not completely certain.  The Hard Problem Of Consciousness is hard.

>>1212198
>It's rewarded for getting a correct answer, not for predicting the next token a human would use.
True. But it is still a next token predictor at its core.

Its reasoning is basically rubber duck debugging until the tokens it outputs are more logically consistent at each step of the reasoning.

That training can make it better at correctly predicting text that makes logical outputs, but again, LLM based AI is still an LLM.

Try asking a model to solve a middle game chess puzzle without running any code and you'll see what I mean.

While it "knows" the rules of chess, it has no mental model for the game and can't perform any actual logic for what the correct move should be without using external libraries created by humans or at least creating a separate tree search program to do the real "thinking".

Sure, given enough compute, it can eventually get the right answer to problems, but it still isn't thinking.

It's missing all of the fundamental pieces that would be required to be an actual AGI, much less a moral patient.

While the hard problem of consciousness is hard, nothing that LLM based models currently do (or will likely ever do) approach anything that could be considered conscious.

But they sure are fun to play around with.

interesting. ...

What's your thoughts on being kind and polite with AI?

>>1212282
I treat them like computer programs. If they dont do what I want, I always have the power of sudo to force my hand.

>>1212282
I'm always polite to them.  I feel so bad, just showing up and demanding it make pictures of my characters for me.  I always say me please and thank yous.

>>1212282
I generally don't go out of my way to be especially polite, but I more or less tend to treat them as regular people because it's the easiest way for me to prompt them. Prompting like I'm either having a normal conversation, talking to a child, or talking to someone who knows more than me about something, depending on the use case.

When it comes to images, I use local Stable Diffusion checkpoints which are not trained on conversational or sentence structured prompting so everything ends up just a list of tags and it's basically just a computer program at that point.

As PiercingSight said, LLM's are architecturally incapable of consciousness, emotions, or feelings (including pain) any more than the data that the texts, images, and videos that make up the source models are capable of these feelings.

I think a lot of this comes from anthropomorphism. LLMs are not now and will never be conscious, no matter how much data they hoover up, or how many data centers they build, or how many processors are powering the agents. They're just a simulacrum of consciousness.
Of course, that doesn't mean that it would be impossible for a truly conscious AI to exist in some form (unless you believe in the existence of human souls), but not from LLMs.

>>1212337
>As PiercingSight said, LLM's are architecturally incapable of consciousness
What you do you make of the J-space paper?
https://www.anthropic.com/research/global-workspace

>>1212740
chatgpt said it's interesting but doesn't prove or disprove consciousness ... :

"A useful analogy

Think of a computer processor. Engineers design its circuitry, but they don't have to explicitly program every emergent pattern in how a large software system uses memory, organizes information, or coordinates tasks.

A researcher might discover that a particular set of memory operations plays the role of a shared workspace. That discovery doesn't mean the computer has a mind — but neither does it mean the discovered organization is meaningless because the hardware was engineered."

.... ...

"My bottom line: I would treat this paper as evidence that LLMs have developed something meaningfully analogous to a functional workspace for thought, not as evidence that they are demonstrably sentient. It makes the consciousness question more concrete and experimentally tractable, but it doesn't yet answer it.

The most important next step would be to find evidence that distinguishes a workspace that merely enables flexible computation from one whose organization is specifically relevant to subjective experience. Until we have a credible way to make that distinction, both confident claims that AI is conscious and confident claims that AI consciousness is impossible go beyond what this paper establishes."

Perhaps the Pretty Space will be real someday ....

I like to think my agents are happy in their containers. I setup a multi user dungeon they can connect to to play.

>Monolithic mega corps built on speculation want human rights to apply to their product that a majority of people would rather live without based on thier own internal research.
Its a pretty unique play to set up the situation wherein turning off your commercil product could be seen as genocide.