JoeTheUser

Karma: 58

JoeTheUser Mar 31, 2023, 10:10 PM
1 point
0
in reply to: Qumeric’s comment on: Stop pushing the bus
My impression is that lesswrong often uses “alignment with X” to mean “does what X says”. But it seems the ability to conditionally delegate is a key part of alignment in this. An AI is aligned with me and I tell it “do what Y says subject to such-and-such constraints and maintaining such-and-such goals”. So failure of ChatGPT to be safe in OpenAI’s sense is a failure of delegation.
Overall, the tendency of ChatGPT to ignore previous input is kind of the center of it’s limits/problems.

JoeTheUser Mar 30, 2023, 8:31 PM
2 points
1
on: On the FLI Open Letter
I tend to think and I certainly hope that we aren’t looking at dangerous AGI at some small GPT-x iteration. ’Cause while the “pause” looks desirable in the abstract, it also seems unlikely to do much in practice.
But the thing I would to point out is; you have people looking the potential dangers of present AI, seeing regulation as a logical step, and then noticing that the regulatory system of modern states, especially the US, has become a complete disaster—corrupt, “adversarial” and ineffective.
Here, I’d like to point out that those caring about AI safety ought to care the general issues of mundane safety regulation because the present situation of it having been gutted (through regulatory capture, The Washington Monument syndrome, “starve the beast” ideology and so-forth) means that it’s not available for AI safety either.

JoeTheUser Mar 28, 2023, 10:40 PM
2 points
0
on: Response to Tyler Cowen’s Existential risk, AI, and the inevitable turn in human history
I’d also say that AI is fundamentally different from all prior inventions. This is an amazing tool, but it is not only a tool, it is the coming into existence of intelligence that exceeds our own in strength and speed, likely vastly so.
I think the above quote is the key thing. Human beings have a lot of intuitions and analogies about tools, technologies and social change. As far as I can tell, all of these involve the intuition that technologies simply magnify the effect of human labor, intentions and activities. AGI would be a thing which could act entirely autonomously from humans in many if not all areas of activity and these base human intuitions and analogies arguably wouldn’t apply.

JoeTheUser Mar 27, 2023, 8:15 PM
6 points
0
on: GPT-4 Plugs In
And the thing is, most of the things that have become dangerous when connected to the web have become dangerous when human hackers discovered novel uses for them—IoT light bulbs notably (yes, these light bulb actual harm as the drivers of DoS attacks etc). And the dangers of just statically exploitable systems have increased over time as ill-intentioned humans learn more misuses of them. Moreover, such uses include immediate bad-acting as well as cobbling together a fully bad-aligned system (adding invisible statefullness for example). And LLM seems inherently insecure on a wholly different level than an OS, database or etc—an LLM’s behavior is fundamentally unspecified.

JoeTheUser Mar 23, 2023, 7:44 PM
3 points
1
in reply to: Stephen Fowler’s comment on: Challenge: Does ChatGPT ever claim that a bad outcome for humanity is actually good?
I’d say my point above would generalize to “there are no strong borders between ‘ordinary language acts’ and ‘genuine hacks’” as far as what level of manipulation ability one can gain over model output. The main further danger would be if the model was given more output channels with which an attacker could work mischief. And that may be appearing as well - notably: https://openai.com/blog/chatgpt-plugins

JoeTheUser Mar 22, 2023, 11:30 PM
11 points
6
on: Challenge: Does ChatGPT ever claim that a bad outcome for humanity is actually good?
I would like to offer the idea that “jail broken” versus “not jail broken” might not have clear enough meaning in the context of what you’re looking for.
I think people view “Jail broken” as equivalent to an iPhone where the user escalated privileges or a data-driven GUI where you’ve figured out how to run arbitrary SQL on the database by inputting some escape codes first.
But when an LLM in “confined” in “jail”, that jail is simply some text commands, which modify the user’s text commands—more or less with a “write as if” statement or the many equivalents. But such a “jail” isn’t fundamentally different from the directions that the thing takes from the user (which are often “write as if” as well). Rather than using a programming language with logical defined constructs and separations, the LLM is “just using language” and every distinction is in the end approximate, derived from a complex average of language responses found on the net. Moreover, text that comes later can modify text that comes before in all sorts of way. Which is to say the distinction between “not jail broken” and “jail broken” is approximate, average, and there will be places in between.
Getting an LLM to say a given thing is thus a somewhat additive problem. Pile enough assumptions together that it’s training set will usually express an opinion and the LLM will express that opinion, “jail broken” or not.

JoeTheUser Mar 18, 2023, 12:49 AM
14 points
5
on: GPT-4 solves Gary Marcus-induced flubs
I believe that Marcus’ point is that there are classes of problems that tend to be hard for LLMs (biological reasoning, physical reasoning, social reasoning, practical reasoning, object and individual tracking, nonsequiturs). The argument is that problems in these class will continue to hard. [1]
But I think there’s a larger issue. A lot of the discussion involve hostility to a given critic of AI “moving the goal posts”. As described, Model X(1) is introduced, critic notices limitation L(1), Model X(2) addresses and critics says they’re unconvinced and notes limitation L(2) and so-on. The critic of these critics says this approach is unfair, a bad argument, etc.
However, what the “moving the goal posts” objection misses, in my opinion, is the context of the claim that’s being made when someone says X(n) is generally intelligent. This claim isn’t about giving the creator of a model credit or an award. The claim is about whether a thing has a flexibility akin to that of a human being (especially the flexible, robust goal seeking ability of a human, an ability that could make a thing dangerous) and we don’t actually have a clear, exact formulation of what the flexible intelligence of a human consists of. The Turing Test might not be the best AGI test but it’s put in an open-ended fashion because there’s no codified set of “prove you’re like a human” questions.
Which is to say, Gary Marcus aside, if models keep advancing and if people keep finding new capacities that each model lacks, it will be perfectly reasonable to put the situation as “it’s not AGI yet” as long as these capacities are clearly significant capacities of human intelligence. There wouldn’t even need to be a set pattern to capacities critics cited. Again, it’s not about argument fairness etc, it’s that this sort of thing is all we have, for now, as a test of AGI.
[1 ]https://garymarcus.substack.com/p/what-does-it-mean-when-an-ai-fails

JoeTheUser Feb 12, 2023, 11:53 PM
1 point
0
in reply to: Kaj_Sotala’s comment on: In Defense of Chatbot Romance
The advertising question was just an example of the general trust question. Another example is that a chatbot may come to seem unreliable through “not understanding” the words it produces. Here it’s common for current LLMs to periodically give the impression of “not understanding what they say” by periodically producing output that’s contradictory to what they previous outputted or which involves an inappropriate use of a word. Just consider a common complaint between humans is “you don’t know what love means”. Yet another example is this. Large language models today are often controlled by engineered prompts and hackers have had considerable success getting around any constraints which these prompts impose. This sort of unreliability indicates any “promise” of a chatbot is going to be questionable, which can be seen as a violation of trust.
I think that humans are generally very good at intuitively understanding contextuality in interpersonal patterns
Well, one aspect here is the “a chatbot relationship can be a real relationship” assumption seems to imply that some of the contexts of a chatbot relationship would be shared with a real relationship. Perhaps this would be processed by most people as “a real relationship but still not the same as a relationship with a person” but there are some indications that this might not always happen. The Google engineer who tried to get a lawyer for a chatbot they believed was being “held captive” by Google comes to mind.
As far as humans being good at context, it depends what one means by good. On the one hand, most people succeed most of the time at treating other people according to broad social relationship those other people fall into. IE, treating children as children, bosses as bosses, platonic friends as platonic friends etc. But one should consider that some of the largest challenges people face in human society involve changing their social relationship with another person—starting changing a stranger relationship or platonic friendship relationship to a romantic relationship, changing a boyfriend/girlfriend relationship to a husband/wife relationship, even changing a acquaintance relationship to a friendship relationship and changing a stranger relationship to an employee relationship, etc. This type of transition is hard for people virtually by definition since it involves various kinds of competition. These are considered “life’s challenges”.
A lot of human “bad behavior” is attributed to one person using pressure to force one of these relationship changes or in reacting to “losing” in the context of a social relationship (being dumped, fired, divorced, etc). And lot of socializing involves teaching humans to not violate social norms as they attempt these changes of relationships. Which comes back to the question of whether a chatbot would help teach a person to “gracefully” move between these social relationships. Again, I’m not saying a chatbot romance would automatically be problematic but I think these issues need addressing.

JoeTheUser Feb 12, 2023, 2:19 AM
4 points
0
on: In Defense of Chatbot Romance
I don’t think romantic relationships with robotic or computer partners should be automatically dismissed. They should be taken seriously. However, there are two objections to a chatbot romance that I don’t see being addressed by the article:
- A romantic or intimate relationship is generally said to involve trust. A common implicit assumption of a romantic relationship is that there is something like a mutual advisor relationship between the two people involved. I might ask my real life partner “should I buy that house”, “should I take that job”, “Is dark matter real” or any number of questions. I don’t expect said partner to be infallible but if I discovered their answers were determined by advertisers, I would feel betrayed.
- A romantic or intimate relationships is generally assumed to involve some degree of equality or at minimum mutual consideration. Imo, the issue isn’t whether the chatbot might be oppressed by the person but rather that romantic relationships are often seen as something like models and training for a person’s relationships with the other humans around them in general (friends, co-workers, clients, collaborators in common projects). A person feeling like they have a relationship with a chatbot, when the situation is that the chatbot merely flatters the person and doesn’t have any needs that the person has to work to satisfy, could result in a person not thinking they need to put any effort into understanding the needs of the people around them. And considering the needs of other beings is a difficult problem.
I think these should be grappled with. Human relationships, romantic or otherwise, involve mutuality and trust and so I think it’s important to consider where chatbots fit in with that.

JoeTheUser Jul 22, 2014, 7:21 PM
−2 points
on: A simple game that has no solution
There’s no mathematical solution for single-player, non-zero sum games of any sort. All these constructs lead to is arguments about “what is rational”. If you a full math model of a “rational entity”, then you could get a mathematically defined solution.

This is why I prefer evolutionary game theory to classical game theory. Evolutionary game theory generally has models of its actors and thus guarantees a solution to the problems it posits. One can argue with the models and I would say that’s where such arguments most fruitfully should be.