Wouldn’t it be better to accept contractual bindings and then at least have the opportunity to whistleblow (even if that means accepting the legal consequences)?
Or do you think that they have some kind of leverage by which the labs would agree to NOT contractually bind them? I’d expect the labs to just not allow them to evaluate the model at all were ARC to insist on or demand this.
I’m definitely not against reading your (and anyone else’s) blog posts, but it would be friendlier to at least outline or excerpt some of the post here too.
It looks like you didn’t (and maybe can’t) enter the ASCII art in the form Bing needs to “decode” it? For one, I’d expect line breaks, both after and before the code block tags and also between each ‘line’ of the art.
If you can, try entering new lines with <kbd>Shift</kbd>+<kbd>Enter</kbd>. That should allow new lines without being interpreted as ‘send message’.
I really like David’s writing generally but this ‘book’ is particularly strong (and pertinent to us here on this site).
The second section, What is the Scary kind of AI?, is a very interesting and (I think) useful alternative perspective on the risks that ‘AI safety’ do and (arguably) should focus on, e.g. “diverse forms of agency”.
The first chapter of the third (‘scenarios’) section, At war with the machines, provides a (more) compelling version of a somewhat common argument, i.e. ‘AI is (already) out to get us’.
The second detailed scenario, in the third chapter, triggered my ‘absurdity heuristic’ hard. The following chapter points out that absurdity was deliberate – bravo David!
The rest of the book is a surprisingly comprehensive synthesis of a lot of insights from LW, the greater ‘Modern Rationalist’ sphere, and David’s own works (much of which is very much related to and pertinent to the other two sets of insights). I am not ‘fully sold’ on ‘Mooglebook AI doom’, but I have definitely updated fairly strongly towards it.
This seems like the right trope:
Cartoonland Time—TV Tropes [WARNING: browsing TV Tropes can be a massive time sink]
That’s why I used a fatal scenario, because it very obviously cuts all future utility to zero
I don’t understand why you think a decision resulting in some person’s or agent’s death “cuts all future utility to zero”. Why do you think choosing one’s death is always a mistake?
I think I’d opt to quote the original title in a post here to indicate that it’s not a ‘claim’ being made (by me).
IIRC, RDIs (and I would guess EARs) vary quite significantly among the various organizations that calculate/estimate/publish them. That might be related to the point ChristianKI seemed to be trying to make. (Tho I don’t know whether ‘iron’ is one of the nutrients for which this is, or was, the case.)
I can’t tell what’s the output of ChatGPT or your prompts or commentary.
I don’t think ‘chronic fatigue syndrome’ is a great example of what the post discusses because ‘syndrome’ is a clear technical (e.g. medical) word already. Similarly, ‘myalgic encephalitis’ is (for most listeners or readers) not a phrase made up of common English words. Both examples seem much more clearly medical or technical terms. ‘chronic fatigue’ would be a better example (if it was widely used) as it would conflate the unexplained medical condition with anything else that might have the same effects (like ‘chronic overexertion’).
The only benefit of public schools anymore, from what I can tell, is that very wise and patient parents can use it to support their children in mastering Defense Against the Dark Arts.
Well, that and getting to play with other kids. Which is still pretty cool.
This may be, perhaps, an under-appreciated function of (public) school schooling!
I would think the title is itself a content warning.
I guess someone might think this post is or could be far more abstract and less detailed about the visceral realities than it is (or maybe even just using the topic as a metaphor at most).
What kind of specific content warning do you think would be appropriate? Maybe “Describes the dissection of human bodies in vivid concrete terms.”?
I was going to share it with you if you didn’t have it, but thanks!
Has anyone shared the link with you yet?
After a long day of work, you can kick back with projectlawful for a few hours, and then go to sleep. You can read projectlawful on the weekend. You can read projectlawful on vacation. It’s rest and rejuvenation and recharging …
I did NOT find this to be the case – I found it way TOO engaging and that it therefore, e.g. actively disrupted my ability to go to sleep. I also found the story to be extremely upsetting, i.e. NOT restful or rejuvenating. As-of now, it’s extremely bleak.
I very much DO like it and I am perfectly happy that it’s a glowfic. (There are some mildly confusing parts, probably because of it being a glowifc, but nothing too bad.)
This is also the first story for which I viscerally felt the utility of providing a ‘trigger warning’.
I don’t know what in particular makes you think the story is useful as “policy experience”. I’m skeptical that much ‘real world’ policy in any way resembles the story.
I thought it might be a reference to this:
I think ‘againstness’ is nearly perfect :)
I didn’t think anything was confusing!
‘Againstness’ felt like a nearly self-defining word to me.
Your course had a rough/sketched/outlined model based on other models at various levels and there’s a few example techniques based on it (in the course).
“againstness control” is totally sensible – just like, e.g. ‘againstness management’ and ‘againstness practice’, are too.
I think there’s an implied (and intriguing) element of using SNS arousal/dominance for, e.g. motivation. I think there are some times or circumstances in which ‘SNS flow’ is effective, e.g. competitive sports.
I don’t think your example techniques need a special term. I understood this post as ‘gesturing’ at something more than presenting anything comprehensive.
I think I’m missing a LOT of context you have about this. I very well could be – probably am – missing some point, but I also feel like you’re discouraging me from voicing anything that doesn’t assume whatever your point is. Is it just that “Stephen Wolfram is bad and everyone should ignore him.”? I honestly tried to investigate this, however poorly I might have done that, but this comment comes across as pretty hostile. Is it your intention to dissuade me from writing about this at all?
They bring it up because it is a shocking violation of norms, even commercial ones because zero money was at stake, for a rich entrepreneur like Wolfram to so punitively use the court system to go after an employee like that for something so utterly academic & a truly microscopic detail of CS theory where he knows full well that it will cost them a ton of money to fight it even for just a settlement and where the lawsuit could accomplish no good (in the tiny world of CAs, surely most people interested heard from the conference or were there). This was not “a literary convention”; this was cruel and vindictive.
I don’t what norms you’re referring to!
But, AFAIK, some commercial norms do in fact condone or allow for aggressive enforcement of NDAs, even where there are is (obviously) “zero money at stake”. I also guessed that the point of the lawsuit was maybe (partially) about ‘enforcing NDAs generally’ too, which is also one way it could ‘accomplish good’ even tho it didn’t prevent dissemination of the info in the paper itself.
If a member of any other research team, in defiance of their team leader (‘lead/principle investigator’), published a paper describing work done on a team project, is there no recourse or punishment expected?
Do researchers have a blanket right to publish at any time, on their own, about work they’ve done as an employee? That’s surprising if true! I wouldn’t think that’s true even in purely academic organizations.
How do you know that Wolfram’s ‘original plan’ wasn’t to allow Cook to publish the paper of his proof after the release of NKS? Is there some reason why making Cook wait was unconscionable? I’m sincerely curious! What’s the big deal? Why was Cook waiting so bad that of course he should have been able to publish his paper whenever he wanted? Or was making him wait a year fine, but three (or more) not?
Why didn’t Cook just retract his paper and settle with Wolfram immediately? I couldn’t find any relevant details about the suit and it appeared to me that the records were sealed, so I’d guess only the people involved could answer.
I don’t understand your theory of Cook. Why did he publish the paper when apparently Wolfram, his employer, didn’t want him to do that? Why didn’t he just retract or withdraw his paper? Why was he in the right in all of this? (Was he in the right about this?) Should Wolfram have just ignored Cook publishing the paper? Why shouldn’t he have asserted his legal right to control when the paper was published? What’s different about this specific paper than others, in both academia and commercial research? What am I missing?
I am very sympathetic to the lawsuit being “cruel and vindictive”! But I’m also sympathetic to being “cruel and vindictive”. If a (hypothetical) employee of mine openly defied my orders, e.g. released info about work I’d paid them to do and told them not to publish – I’d be pissed! If they had signed a legal agreement to NOT do that kind of thing too, I suspect I’d think of suing them myself. That the project was a non-fiction book doesn’t seem like it’d make this that much different. If nothing else, Cook deprived Wolfram of being able to market NKS as announcing that particular result.
As for the citation/credit generally, NKS is a large book and, AFAICT, it was NOT intended to be an ‘academic’ work. Had it been written in that ‘style’, I’d guess it would have been WAY too big to have been published at all.
Is there in fact a reasonable way that Wolfram could have cited other’s works, and described the contributions to the book itself, that would work not only for the work done by Matthew Cook, but everyone else involved?
I still don’t know very many details of all the contributions to just the Rule 110 proof paper!
(I don’t even know what academic paper authorship means. AFAIK, it is somewhat specific to the field
I am happy to admit that it’s certainly very likely that Wolfram failed to properly cite other’s work, or fairly credit their contributions. I am very sure that he does in fact have a reputation for that.
He nevertheless wrote NKS as it is: you can almost hear the teeth being pulled as he grudgingly mentions Cook at all instead of what was obviously the original plan (done with all the others) of just a brief mention of unspecified work, sort of like when a PI mentions a grad student in the acknowledgment section for ‘coding’ or ‘experiments’, using omission to take all the credit. NDA or no, that is not kosher citation.
I can’t “almost hear the teeth being pulled” but I’m even more confused – is Wolfram’s own citations ‘merely’ as bad as that done by many other PIs? I can’t tell what standard you’re judging him by, or how well I should expect him to fare by it, especially given his reputation for doing poorly in this area.
What does the distribution of how well others do in this regard look like? Are, e.g. 95% of researchers, or authors of ‘pop science’ books, basically ‘perfect’? Is Wolfram uniquely terrible?
I also can’t see what you mean by Wolfram “jamming himself in” in any of the quotes you provided.
I also don’t know that Cook didn’t do other “technical content” besides the Rule 110 proof.
I thought Wolfram was clear about NKS being a ‘team effort’ so I still think it’s (somewhat) reasonable that a much longer and more detailed description of more precise contributions were omitted.
I now think it is plausible that Wolfram sued “over literary conventions”:
I suspect that Wolfram just wanted to reveal the relevant proof himself, first, in his book NKS (A New Kind of Science), and that Matthew Cook probably was contractually obligated to allow Wolfram to do that.
Given that the two parties settled, and that Cook published his paper about his proof in Wolfram’s own journal (Complex Systems), two years after NKS was published, seems to mostly confirm my suspicions.
Do you think ARC should have traded publicizing the lab’s demands for non-disclosure instead of performing the exercise they did?
I think that would have been a bad trade.
I also don’t think there’s much value to them whistleblowing about any kind of non-disclosure that the lab’s might have demanded. I don’t get the sense there’s any additional bad (or awful) behavior – beyond what’s (implicitly) apparent from the detailed info ARC has already publicly released.
I think it’s very useful to maintain sufficient incentives for the lab’s to want to allow things like what ARC did.
Sure, tho I’d be much more terrified were they expropriated!