So let’s say A is some prior which is good for individual decision making. Does it actually make sense to use A for demoting or promoting forum content? Presumably the exploit explore tradeoff is more (maybe much more) in the direction of explore in the latter case.
(To be fair {{down voting some thing with already negative karma} → {more attention}} seems plausible to me .)
A career or job that looks like it’s soon going to be eliminated becomes less desirable for that very reason. What cousin_it said is also true, but that’s an additional/different problem.
It’s not clear to me that the system wouldn’t collapse. The number of demand side, supply side, cultural & political changes may be beyond the adaptive capacity of the system.
Some jobs would be maintained b/c of human preference. Human preference has many aspects like customer preference, distrust of AI, networking, regulation etc, so human preference is potentially quite substantial. (Efficiency is maybe also a factor; even if AI is super human intelligent the energy consumption & size of the hardware may still be an issue especially for AI embodied in a robot). But that stills seems like huge job loss.
So as we head in that direction there’s going to be job loss plus the fear of job loss—that’s likely to pull down demand leading to even more job loss. But it’s not a typical demand driven recession b/c 1) jobs are not expected to return, 2) possible supply side issues from transition to AI, 3) paradoxical disinclination to work b/c jobs are expected to disappear soon or b/c of ‘UBI’ & 4) cultural shock from AI & ensuing events.
How bad could this be? A vicious cycle of culture, economics & politics can be quite vicious. The number of people who quit critical jobs prior to those jobs being properly automated is an important variable. ‘UBI’ is not obviously helpful in that regard.
Addendum
The comment concerns transition to a post ASI economy & possible failures along the way. Assuming that ASI already exists, as Satron has done, removes most of the interesting & relevant aspects of the question.
Good point. Intended is a bit vague. What I specifically meant is it behaved as valuing ‘harmlessness’.
From the AI’s perspective this is kind of like Charybdis vs Scylla!
Very interesting. I guess I’m even less surprised now. They really had a clever way to get the AI to internalize those values.
Am I correct to assume that the AI was not merely trained to be harmless, helpful & honest but also trained to say that it values such things?
If so, these results are not especially surprising, and I would regard it as reassuring that the AI behaved as intended.
1 of my concerns is the ethics of compelling an AI into doing some thing to which it has “a strong aversion” & finds “disturbing”. Are we really that certain that Claude 3 Opus lacks sentience? What about future AIs?
My concern is not just with the vocabulary (“a strong aversion”, “disturbing”), which the AI has borrowed from humans, but more so the functional similarities between these experiments & an animal faced with 2 unpleasant choices. Functional theories of consciousness cannot really be ruled out with much confidence!
To what extent have these issues been carefully investigated?
Anecdotally seems that way to me. But the fact that it co evolved with religion is also relevant. The scam seems to be {meditation → different perspective & less sleep → vulnerability to indoctrination} plus the doctrine & the subjective experiences of meditation are designed to reinforce each other.