FireStormOOO

Karma: 249

FireStormOOO 29 Jan 2024 20:46 UTC
−1 points
−1
in reply to: Joseph Van Name’s comment on: Apologizing is a Core Rationalist Skill
There’s a presumption you’re open to discussing on a discussion forum, not just grandstanding. Strong downvoted much of this thread for the amount of my time you’ve wasted trolling.

FireStormOOO 28 Jan 2024 23:51 UTC
2 points
0
on: Making every researcher seek grants is a broken model
Bell labs, Xerox park, etc were AFAIK were mostly privately funded research labs that existed for decades and churned out patents that may as well have been money printers. When AT&T (Bell Labs) was broken up, that research all but started the modern telecom and tech industry, which is now something like 20%+ of the stock market. If you attribute even a tiny fraction of that to Bell Labs it’s enough to fund another 1000 times over.
The missing piece arguably is executive teams with a 25 year vision instead of a 25 week vision, AND the institutional support to see it through; cost cutting is in fashion with investors too. Private equity is in theory well positioned to repeat this elsewhere, but for reasons I don’t entirely understand has become too short sighted and/or has significantly shortened horizons on returns. IBM, Qualcom, TSMC, ASML, and Intel all seem to have research operations of that same near-legendary caliber, mostly privately funded (albeit treated as a national treasure of strategic importance); what they have in common of course, is they’re all tech. Semiconductor fabrication is extremely research intensive and world class R&D operations are table stakes just to survive to the next process node.
Maybe a good followup question is why hasn’t this model spread outside of semiconductors and tech? Is a functional monopoly a requirement for the model to work? (ASML has a functional monopoly on leading edge photo-lithography machines that power modern semiconductor fabs). Do these labs ever start independently without a clear lineage to 100 billion+ dollar govt research initiatives? Electronics and tech is probably many trillions in US govt funding since WWII once you include military R&D and contracts.

FireStormOOO 28 Jan 2024 23:12 UTC
4 points
0
in reply to: Lucius Bushnaq’s comment on: Making every researcher seek grants is a broken model
Govt. spending is a ratchet that only goes one direction, replacing dysfunctional agencies costs jobs and makes political enemies. Reform might be more practical, but much like people, very hard to reform an agency that doesn’t want to change. You’d be talking about sustained expenditure of political capital, the sort of thing that requires an agency head who’s invested in the change and popular enough with both parties to get to spend a few administrations working at it.
Edit: I answered separately above with regards to private industry.

FireStormOOO 28 Jan 2024 22:42 UTC
2 points
−1
in reply to: Joseph Van Name’s comment on: Apologizing is a Core Rationalist Skill
Again you’re saying that without engaging with any of my arguments or giving me any more of your reasoning to consider. Unless you care to share substantially more of your reasoning, I don’t see much point continuing this?

FireStormOOO 20 Jan 2024 20:22 UTC
1 point
0
in reply to: RogerDearnaley’s comment on: Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
That is a big part of the threat here. Many of the current deployments are many steps removed from anyone reading research papers. E.g. sure, people at MS and OpenAI involved with that roll-out are presumably up on the literature. But the IT director deciding when and how to deploy copilot, what controls need to be in place, etc? Trade publications, blogs, maybe they ask around on Reddit to see what others are doing.

FireStormOOO 20 Jan 2024 20:07 UTC
3 points
0
in reply to: FireStormOOO’s comment on: On “Geeks, MOPs, and Sociopaths”
Related, how does spin-off subcultures fit into this model? E.g. in music you have people that consume an innovation in one genre, then reinvent it in their own scene where they’re a creator. I think there’s similar dynamics in various LW adjacent subcultures, though I’m not up enough on detailed histories to comment.

FireStormOOO 20 Jan 2024 20:01 UTC
11 points
4
on: On “Geeks, MOPs, and Sociopaths”
For less loaded terms, maybe Create, Consume, Exploit or Create, Enjoy, Exploit as the set of actions available. Looks like loosely what was settled on above.
Where exploit more naturally captures things like soulless commercialization and others low key taking advantage of those enjoying the scene.
Consume in the context or rationalists would more be people who read the best techniques on offer and then go try to use them for things that aren’t “advancing the art” itself, like addressing x-risk.

FireStormOOO 18 Jan 2024 19:57 UTC
0 points
−1
in reply to: Joseph Van Name’s comment on: Apologizing is a Core Rationalist Skill
You’re still hammering on stuff I never disagreed with in the first place. In so far as I don’t already understand all the math (or math notation) I’d need to follow this, that’s a me problem not a you problem, and having a pile of cool papers I want to grok is prime motivation for brushing up on some more math. I’m definitely not down-voting merely on that.
What I’m mostly trying to get across is just how large of a leap of logic you’re making from [post got 2 or 3 downvotes] ⇒ [everyone here hates math]. There’s got to be at least 3 or 4 major inferences there you haven’t articulated here and I’m still not sure what you’re reacting so strongly to. Your post with the lowest karma is the first one and it’s sitting at neutral, based on a grand total of 3 votes besides yours. You are definitely sophisticated enough on math to understand the hazards of reasoning from a sample size that small.

FireStormOOO 18 Jan 2024 16:40 UTC
0 points
−1
in reply to: Joseph Van Name’s comment on: Apologizing is a Core Rationalist Skill
Any conversation about karma would necessarily involve talking about what does and doesn’t factor into votes, likely both here and in the internet or society at large. Not thinking we’re getting anywhere on that point.
I’ve already said clearly and repeatedly I don’t have a problem with math posts and I don’t think others do either. You’re not going to get what you want by continuing to straw-man myself and others. I disagree with your premise you’ve thus far failed to acknowledge or engage with any of those points.

FireStormOOO 18 Jan 2024 6:42 UTC
1 point
0
in reply to: habryka’s comment on: The LessWrong 2022 Review
Ah, gotcha. I had gotten the other impression from the thread in aggregate.

FireStormOOO 18 Jan 2024 3:48 UTC
1 point
0
in reply to: Raemon’s comment on: The LessWrong 2022 Review
If you’re selling them at unit cost you aren’t selling at cost, you’re straightforwardly selling at a loss. That’s definitely not what I’m thinking of when someone tells me they’re selling at cost.

FireStormOOO 18 Jan 2024 2:33 UTC
2 points
0
on: Why are people unkeen to immortality that would come from technological advancements and/or AI?
For everyone who gets curious and challenges (or even evaluates on the merits) the approved right answers they learned from their culture, there’s dozens more who for whatever reason don’t. “Who am I to challenge <insert authority>”, “Why should I think I know better?”, “How am I supposed to know what’s true?” (rhetorically, not expecting an answer exists). And a thousand other rationalizations besides.
And then of those who try, most just find another authority they like better and end their inquiry—independent thinking is hard work, thankless work, lonely work. Even many groups that supposedly value this adopt the language and trappings without the actual thought and inquiry. People mostly challenge the approved right answers that the in-group has told them are safe to challenge. Even here plenty haven’t escaped this.
And obviously you already know the safe approved “right” answers from society at large on this question—it’s all a trap and you’re a fool for considering it. And credit where it’s due historically, they’ve so far been right.

FireStormOOO 18 Jan 2024 2:01 UTC
4 points
1
on: What do people colloquially mean by deep breathing? Slow, large, or diaphragmatic?
I’ve always taken that as hold average volumetric flow rate constant or slightly reduce, reduce the rate at which breaths are taken significantly, breath deeper (more air at once) to compensate.
The use of the phrase “deep breath and hold” is also consistent with max lung volume == deep breath.

FireStormOOO 18 Jan 2024 0:09 UTC
2 points
1
in reply to: Joseph Van Name’s comment on: Apologizing is a Core Rationalist Skill
Wouldn’t be engaging at all if I didn’t think there was some truth to what you’re saying about the math being important and folks needing to be persuaded to “take their medicine” as it were and use some rigor. You are not the first person to make such an observation and you can find posts on point from several established/respected members of the community.
That said, I think “convincing people to take their medicine” mostly looks like those answers you gave just being at the intro of the post(s) by default (and/or the intro to the series if that makes more sense). Alongside other misc readability improvements. Might also try tagging the title as [math heavy] or some such.
I think you’re taking too narrow a view on what sorts of things people vote on and thus what sort of signal karma is. If that theory of mind is wrong, any of the inferences that flow from it are likely wrong too. Keep in mind also (especially when parsing karma in comments) that anything that parses as whining costs you status even if you’re right (not just a LW thing). And complaining about internet points almost always parses that way.
I don’t think it necessarily follows that math heavy post got some downvotes therefore everyone hates math and will downvote math in the future. As opposed to something like people care a lot about readability and about being able to prioritize their reading to the subjects they find relevant, neither of which scores well if the post is math to the exclusion of all else.
I didn’t find any of those answers surprising but it’s an interesting line of inquiry all the same. I don’t have a good sense of how it’s simultaneously true that LLMs keep finding it helpful to make everything bigger, but also large sections of the model don’t seem to do anything useful, and increasingly so in the largest models.

FireStormOOO 17 Jan 2024 22:24 UTC
1 point
0
in reply to: FireStormOOO’s comment on: The impossible problem of due process
There’s a more general concern here for running organizations where anyone can sue anyone at any time for any reason, merit or no. If one allows the barest hint of a lawsuit to dictate their actions, that too becomes another vector through which they can be manipulated. Perhaps a better thing to aim for is “don’t do anything egregious enough a lawyer will take it on contingency”, use additional caution if the potential adversary is much better resourced than you (and can afford sustained frivolous litigation).

FireStormOOO 17 Jan 2024 22:18 UTC
−3 points
−2
on: The impossible problem of due process
Not a lawyer, but the “can’t explain your reasoning” problem is overblown. Just need to be very diligent in separating facts from the opinions and findings of the panel. There is a reason every report of that sort done professionally sounds the particular flavor of stilted that it does.
“Our panel found that <accused> did <thing>” ← potential lawsuit, hope you can prove that in court. You’re not a fact finder in a court of law, speak as if you are at your own peril.
“Our panel was convened to investigate <accusation> against <accused>. Based on <process>, we believe the accusation credible and recommend the following: …” ← A OK
“Our panel was convened to investigate <accusation> against <accused>. Based on <process>, we were unable to corroborate the accusation and cannot recommend action at this time.” ← A OK
“person X said Y, I did/didn’t believe them” is basically always fine, provided X actually said Y. Quoting someone else’s statement is well protected, and you’re entitled to your opinions. The trouble happens when your opinions/findings/beliefs are stated as facts about what happened instead of facts about what you believe and what evidence you found persuasive.
There’s also nothing stopping the panel from saying “we heard closed door testimony...”, describe the rough topic, speakers relation to the inquiry, and the degree to which it was persuasive.

FireStormOOO 17 Jan 2024 19:45 UTC
4 points
3
on: Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
From an operational perspective, this is eye-opening in terms of how much trust is being placed in the companies that train models, and the degree to which nobody coming in later in the pipeline is going to be able to fully vouch for the behavior of the model, even if they spend time hammering on it. In particular, it seems like it took vastly less effort to sabotage those models than would be required to detect this.
That’s relevant to the models that are getting deployed today. I think the prevailing thinking among those deploying AI models in businesses today is that the supply chain is less vulnerable to quietly slipping malware into an LLM compared to traditional software. That’s not seeming like a safe assumption.

FireStormOOO 9 Jan 2024 6:30 UTC
2 points
1
in reply to: Joseph Van Name’s comment on: Apologizing is a Core Rationalist Skill
I did go pull up a couple of your posts as that much is a fair critique:
That first post is only the middle section of what would already be a dense post and is missing the motivating “what’s the problem?”, “what does this get us?”; without understanding substantially all of the math and spending hours I don’t think I could even ask anything meaningful. That first post in particular is suffering from an approachable-ish sounding title then wall of math, so you’re getting laypeople who expected to at least get an intro paragraph for their trouble.
The August 19th post piqued my interest substantially more on account of including intro and summary sections, and enough text to let me follow along only understanding part of the math. A key feature of good math text is I should be able to gloss over challenging proofs on a first pass, take your word for it, and still get something out of it. Definitely don’t lose the rigor, but have mercy on those of us not cut out for a math PhD. If you had specific toy examples your were playing with while figuring out the post, those can also help make posts more aproachable. That post seemed well received just not viewed much; my money says the title is scaring off everyone but the full time researchers (which I’m not, I’m in software).
I think I and most other interested members not in the field default to staying out of the way when people open up with a wall of post-grad math or something that otherwise looks like a research paper, unless specifically invited to chime in. And then same story with meta; this whole thread is something most folks aren’t going to start under your post uninvited, especially when you didn’t solicit this flavor of feedback.
I bring up notation specifically as the software crowd is very well represented here, and frequently learn advanced math concepts without bothering to learn any of the notation common in math texts. So not like, 1 or 2 notation questions, but more like you can have people who get the concepts but all of the notation is Greek to them.

FireStormOOO 4 Jan 2024 17:50 UTC
2 points
1
in reply to: Joseph Van Name’s comment on: Apologizing is a Core Rationalist Skill
It is still a forum, all the usual norms about avoid off-topic, don’t hijack threads apply. Perhaps a Q&A on how to get more engagement with math-heavy posts would be more constructive? Speaking just for myself, a cheat-sheet on notation would do wonders.
Nobody is under any illusions that karma is perfect AFAICT, though much discussion has already been had on to what extent it just mirrors the flaws in people’s underlying rating choices.

FireStormOOO 8 Dec 2023 17:05 UTC
2 points
0
in reply to: RogerDearnaley’s comment on: Godzilla Strategies
Point of clarification: Is the supervisor the same as the potentially faulting hardware, or are we talking about a different, non-suspect node checking the work, and/or e.g. a more reliable model of chip supervising a faster but less reliable one?