Qumeric

Karma: 235

Qumeric Apr 8, 2023, 5:24 PM
3 points
2
in reply to: burrito’s comment on: All AGI Safety questions welcome (especially basic ones) [April 2023]
I think it is not necessarily correct to say that GPT-4 is above village idiot level. Comparison to humans is a convenient and intuitive framing but it can be misleading.
For example, this post argues that GPT-4 is around Raven level. Beware that this framing is also problematic but for different reasons.
I think that you are correctly stating Eliezer’s beliefs at the time but it turned out that we created a completely different kind of intelligence, so it’s mostly irrelevant now.
In my opinion, we should aspire to avoid any comparison unless it has practical relevance (e.g. economic consequences).

Qumeric Apr 8, 2023, 12:44 PM
2 points
−1
on: Catching the Eye of Sauron
I can’t agree more with the post but I would like to note that even the current implementation is working. It definitely grabbed people’s attention.
My friend who never read LW writes in his blog about why we are going to die. My wife who is not a tech person and was never particularly interested in AI gets TikToks where people say that we are going to die.
So far it looks like definitely positive impact overall. But it’s early to say, I am expecting some kind of shitshow soon. But even shitshow is probably better than nothing.

Qumeric Apr 8, 2023, 9:38 AM
4 points
3
on: Bringing Agency Into AGI Extinction Is Superfluous
I agree it’s a very significant risk which is possibly somewhat underappreciated in the LW community.
I think all three situations are very possible and potentially catastrophic:
1. Evil people do evil with AI
2. Moloch goes Moloch with AI
3. ASI goes ASI (FOOM etc.)
Arguments against (1) could be “evil people are stupid” and “terrorism is not about terror”.
Arguments against (1) and (2) could be “timelines are short” and “AI power is likely to be very concentrated”.

Qumeric Apr 1, 2023, 8:42 AM
14 points
1
on: Hooray for stepping out of the limelight
I think that Deepmind is impacted by race dynamics and Google’s code red etc. I heard from a Deepmind employee that the leadership including Demis is now much more focused on products and profits, at least in their rhetoric.
But I agree it looks like they tried and likely still trying to push back against incentives.
And I am pretty confident that they reduced publishing on purpose and it’s visible.

Qumeric Mar 31, 2023, 6:10 PM
13 points
7
in reply to: Richard_Kennaway’s comment on: Stop pushing the bus
This is true that it is not evidence of misalignment with the user but it is evidence of misalignment with ChatGPT creators.

Qumeric Mar 31, 2023, 4:25 PM
2 points
0
in reply to: TinkerBird’s comment on: “Dangers of AI and the End of Human Civilization” Yudkowsky on Lex Fridman
I agree it was a pretty weak point. I wonder if there is a longer form exploration of this topic from Eliezer or somebody else.
I think it is even contradictory. Eliezer says that AI alignment is solvable by humans and that verification is easier than the solution. But then he claims that humans wouldn’t even be able to verify answers.
I think a charitable interpretation could be “it is not going to be as usable as you think”. But perhaps I misunderstand something?

Qumeric Mar 31, 2023, 3:12 PM
2 points
1
in reply to: Lone Pine’s comment on: Alignment-related jobs outside of London/SF
Fwiw I live in London and have been to the Bay Area and I think that London is better across all 4 dimensions you mentioned.
- Social scene: Don’t know what exactly you are looking for but London is large and diverse.
- High cost of living: London is pretty expensive too but cheaper.
- Difficulty getting around: London has pretty good public transportation.
- Homeless problem: I think I see homeless people 10x less compared to when I was in the Bay.

Qumeric Mar 31, 2023, 11:16 AM
1 point
0
in reply to: Rob Bensinger’s comment on: Pausing AI Developments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky
you’re misunderstanding the TIME article as more naive and less based-on-an-underlying-complicated-model than is actually the case.
I specifically said “I do not necessarily say that this particular TIME article was a bad idea” mainly because I assumed it probably wasn’t that naive. Sorry I didn’t make it clear enough.
I still decided to comment because I think this is pretty important in general, even if somewhat obvious. Looks like one of those biases which show up over and over again even if you try pretty hard to correct it.
Also, I think it’s pretty hard to judge what works and what doesn’t. The vibe has shifted a lot even in the last 6 months. I think it is plausible it shifted more than in a 10-year period 2010-2019.

Qumeric Mar 30, 2023, 2:30 PM
1 point
−13
in reply to: gwd’s comment on: Pausing AI Developments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky
I second this.
I think people really get used to discussing things in their research labs or in specific online communities. And then, when they try to interact with the real world and even do politics, they kind of forget how different the real world is.
Simply telling people ~all the truth may work well in some settings (although it’s far from all that matters in any setting) but almost never works well in politics. Sad but true.
I think that Eliezer (and many others including myself!) may be suspectable to “living in the should-universe” (as named by Eliezer himself).
I do not necessarily say that this particular TIME article was a bad idea, but I am feeling that people who communicate about x-risk are on average biased in this way. And it may greatly hinder the results of communication.
I also mostly agree with “people don’t take AI alignment seriously because we haven’t actually seen anything all that scary yet”. However, I think that the scary thing is not necessarily “simulated murders”. For example, a lot of people are quite concerned about unemployment caused by AI. I believe it might change perception significantly if it will actually turn out to be a big problem which seems plausible.
Yes, of course, it is a completely different issue. But on an emotional level, it will be similar (AI == bad stuff happening).

Qumeric Mar 29, 2023, 5:35 PM
8 points
5
in reply to: Tristan Williams’s comment on: FLI open letter: Pause giant AI experiments
2. I think non-x-risk focused messages are a good idea because:
- It is much easier to reach a wide audience this way.
- It is clear that there are significant and important risks even if we completely exclude x-risk. We should have this discussion even in a world where for some reason we could be certain that humanity will survive for the next 100 years.
- It widens Overton’s window. x-risk is still mostly considered to be a fringe position among the general public, although the situation has improved somewhat.
3. There were cases when it worked well. For example, the Letter of three hundred.
4. I don’t know much about EA’s concerns about Elon. Intuitively, he seems to be fine. But I think that in general, people are more biased towards too much distancing which often hinders coordination a lot.
5. I think more signatures cannot make things worse if authors are handling them properly. Just rough sorting by credentials (as FLI does) may be already good enough. But it’s possible and easy to be more aggressive here.
I agree that it’s unlikely that this letter will be net bad and that it’s possible it can make a significant positive impact. However, I don’t think people argued that it can be bad. Instead, people argued it could be better. It’s clearly not possible to do something like this every month, so it’s better to put a lot of attention to details and think really carefully about content and timing.

Qumeric Mar 29, 2023, 12:36 PM
4 points
3
in reply to: Evan R. Murphy’s comment on: FLI open letter: Pause giant AI experiments
I think it is only getting started. I expect that likely there will be more attention in 6 months and very likely in 1 year.
OpenAI has barely rolled out its first limited version of GPT-4 (only 2 weeks have passed!). It is growing very fast but it has A LOT of room to grow. Also, text-2-video is not here in any significant sense but it will be very soon.

Qumeric Mar 27, 2023, 10:40 AM
5 points
on: What 2026 looks like (Daniel’s Median Future)
When it was published, it felt like a pretty short timeline. But now we are in early 2023 and it feels like late 2023 according to this scenario.

Qumeric Mar 25, 2023, 9:08 AM
3 points
2
in reply to: Emerson Spartz’s comment on: The Overton Window widens: Examples of AI risk in the media
I wonder if soon the general public will freak out on a large scale (Covid-like). I will be not surprised if it will happen in 2024 and only slightly surprised if it will happen this year. If it will happen, I am also not sure if it will be good or bad.

Qumeric Mar 24, 2023, 10:43 AM
3 points
0
on: Speed running everyone through the bad alignement bingo. $5k bounty for a LW conversational agent
OpenAI just dropped ChatGPT plugins yesterday. It seems like it is an ideal platform for it? Probably will be even easier to implement than before and have better quality. But more importantly, it seems that ChatGPT plugins will quickly shape to be the new app store and it would be easier to get attention on this platform compared to other more traditional ways of distribution. Quite speculative, I know, but seems very possible.

If somebody will start such a project, please contact me. I am ex-Google SWE with decent knowledge of ML and experience of running software startup (as co-founder and CTO in the recent past).

I would also be interested to hear why it could be a bad idea.

Qumeric Mar 23, 2023, 1:05 PM
15 points
6
in reply to: Bogdan Ionut Cirstea’s comment on: Sparks of Artificial General Intelligence: Early experiments with GPT-4 | Microsoft Research
Good point. It’s a bit weird that performance on easy Codeforces questions is so bad (0/10) though.
https://twitter.com/cHHillee/status/1635790330854526981

Qumeric Mar 23, 2023, 9:34 AM
5 points
−5
in reply to: Bogdan Ionut Cirstea’s comment on: Sparks of Artificial General Intelligence: Early experiments with GPT-4 | Microsoft Research
Might be caused mostly by data leaks (training set contamination).

Qumeric Mar 21, 2023, 3:50 PM
6 points
1
on: AI #4: Introducing GPT-4
I think you misinterpret hindsight neglect. It got to 100% accuracy, so it got better, not worse.

Also, a couple of images are not shown correctly, search for <img in text.

Qumeric Mar 18, 2023, 7:02 PM
7 points
4
on: What did you do with GPT4?
Really helpful for learning new frameworks and stuff like that. I had a very good experience using it for Kaggle competitions (I am semi-intermediate level, probably it is much less useful on the expert level).
Also, I found it quite useful for research on obscure topics like “how to potentiate this not well-known drug”. Usually, such research involves reading through tons of forums, subreddits etc. and signal to noise ratio is quite high. GPT-4 is very useful to distil signal because it basically already read this all.
Btw, I tried to make it solve competitive programming problems. I think it’s not a matter of prompt engineering: it is genuinely bad on it. The following pattern is common:
- GPT-4 proposes some solutions, usually wrong at the first glance.
- I point to mistakes.
- GPT-4 says yeah you’re right, but now it is fixed.
- It is going on like this for ~4 iterations until I give up on this particular problem or more interestingly GPT-4 starts to claim that it’s impossible to solve.
It really feels like a low IQ (but very eloquent) human in such moments, it just cannot think abstractly.

Qumeric Mar 18, 2023, 6:59 PM
5 points
0
in reply to: Edward Pascal’s comment on: Wonder about the hard parts of the alignment problem
Well, I do not have anything like this but it is very clear that China is way above GPT-3 level. Even the open-source community is significantly above. Take a look at LLaMA/Alpaca, people run them on consumer PC and it’s around GPT-3.5 level, the largest 65B model is even better (it cannot be run on consumer PC but can be run on a small ~10k$ server or cheaply in the cloud). It can also be fine-tuned in 5 hours on RTX 4090 using LORA: https://github.com/tloen/alpaca-lora .
Chinese AI researchers contribute significantly to AI progress, although of course, they are behind the USA.
My best guess would be China is at most 1 year away from GPT-4. Maybe less.

Btw, an example of a recent model: ChatGLM-6b

Qumeric Mar 14, 2023, 7:16 PM
7 points
0
in reply to: nz’s comment on: GPT-4
I just bought a new subscription (I didn’t have one before), it is available to me.