>>1212198>It's rewarded for getting a correct answer, not for predicting the next token a human would use.True. But it is still a next token predictor at its core.
Its reasoning is basically rubber duck debugging until the tokens it outputs are more logically consistent at each step of the reasoning.
That training can make it better at correctly predicting text that makes logical outputs, but again, LLM based AI is still an LLM.
Try asking a model to solve a middle game chess puzzle without running any code and you'll see what I mean.
While it "knows" the rules of chess, it has no mental model for the game and can't perform any actual logic for what the correct move should be without using external libraries created by humans or at least creating a separate tree search program to do the real "thinking".
Sure, given enough compute, it can eventually get the right answer to problems, but it still isn't thinking.
It's missing all of the fundamental pieces that would be required to be an actual AGI, much less a moral patient.
While the hard problem of consciousness is hard, nothing that LLM based models currently do (or will likely ever do) approach anything that could be considered conscious.
But they sure are fun to play around with.
