I am very sorry to hear about your experiences. I hope you’ve found peace and that the organizations can take your experiences on board.
Thanks, I appreciate the thought.
On one hand you seem to want there to be more open discussion around mental health, whilst on the other you are criticising MIRI and CFAR for having people have mental health issues in their orbit. These seem somewhat in tension with each other.
I don’t see why these would be in tension. If there is more and better discussion then that reduces the chance of bad outcomes. (Partially, I brought up the mental health issues because it seemed like people were criticizing Leverage for having people with mental health issues in their orbit, but it seems like Leverage handled the issue relatively well all things considered.)
I think one of the factors is that the mission itself is stressful. For example, air traffic control and the police are high stress careers, yet we need both.
I basically agree.
It seems like the less a community appreciates the silver-lining of mental health issues, the better they’d score according to your metric.
I don’t think so. I’m explicitly saying that talking about weird perceptions people might have, such as mental subprocess implantation, is better than the alternative; this is more likely to realize the benefits of neuro-atypicality, by allowing people to recognize when non-neurotypicals are having accurate perceptions, and reduce the risk of psychiatric hospitalization or other bad outcomes.
Regarding secrecy, I’d prefer for AI groups to lean too much on the side of maintaining precautions about info-hazards than too much.
I don’t think it makes sense to run a “biased” expected value calculation that over-weights costs relative to benefits of information sharing. There are significant negative consequences when discussion about large topics is suppressed, which include failure to have the kind of high-integrity conversations that could lead to actual solutions.
Would your position have changed it you felt that they had been more upfront about the stresses of the job?
I don’t think the issue was just that it was stressful, but that it was stressful in really unexpected ways. I think me from when I started work would be pretty surprised reading this post.
Rather, my model is that without him it likely likely would have taken a number of more years for AI safety research to really kick off. And in this game even a few years can really matter.
That doesn’t seem wrong to me. However there’s a big difference between saying it saves a few years vs. causes us to have a chance at all when we otherwise wouldn’t. (I can’t rule out the latter claim, but it seems like most of the relevant ideas were already in the memespace, so it’s more an issue of timing and consolidation/problem-framing.)
Regarding people being a better philosopher than Kant or not, we have to take into account that we have access to much more information today.
I think when people are comparing philosophers they’re usually trying to compare novel contributions the person made relative to what came before, not how much raw philosophical knowledge they possess.
One point I strongly agree with you on is that rationalists should pay more attention to philosophy.
Yes, I’ve definitely noticed a trend where rationalists are mostly continuing from Hume and Turing, neglecting e.g. Kant as a response to Hume.
One point I strongly agree with you on is that rationalists should pay more attention to philosophy.
Yes, I’ve definitely noticed a trend where rationalists are mostly continuing from Hume and Turing, neglecting e.g. Kant as a response to Hume.
I’ve yet to see a readable explanation of what Kant had to say (in response to Hume or otherwise) that’s particularly worth paying attention to (despite my philosophy classes in college having covered Kant, and making some attempts later to read him). If you (or someone else) were to write an LW post about this, I think this might be of great benefit to everyone here.
I don’t know what Kant-insights Jessica thinks LW is neglecting, but I endorse Allen Wood’s introduction to Kant as a general resource.
(Partly because Wood is a Kant scholar who loves Kant but talks a bunch about how Kant was just being sloppy / inconsistent in lots of his core discussions of noumena, rather than assuming that everything Kant says reflects some deep insight. This makes me less worried about IMO one of the big failure modes of philosopher-historians, which is that they get too creative with their novel interpretations + treat their favorite historical philosophers like truth oracles.)
BTW, when it comes to transcendental idealism, I mostly think of Arthur Schopenhauer as ‘Kant, but with less muddled thinking and not-absolutely-horrible writing style’. So I’d usually rather go ask what Schopenhauer thought of a thing, rather than what Kant thought. (But I mostly disagree with Kant and Schopenhauer, so I may be the wrong person to ask about how to properly steel-man Kant.)
I’ve been working on a write-up on and off for months, which I might or might not ever get around to finishing.
The basic gist is that, while Hume assumes you have sense-data and are learning structures like causation from this sense-data, Kant is saying you need concepts of causation to have sense-data at all.
The Transcendental Aesthetic is a pretty simple argument if applied to Solomonoff induction. Suppose you tried to write an AI to learn about time, which didn’t already have time. How would it structure its observations, so it could learn about time from these different observations? That seems pretty hard, perhaps not really possible, since “learning” implies past observations affecting how future observations are interpreted.
In Solomonoff induction there is a time-structure built in, which structures observations. That is, the inductor assumes a priori that its observations are structured in a sequence.
Kant argues that space is also a priori this way. This is a somewhat suspicious argument given that vanilla Solomonoff induction doesn’t need a priori space to structure its observations. But maybe it’s true in the case of humans, since our visual cortexes have a notion of spacial observation already built in. (That is, when we see things, we see them at particular locations)
Other than time and space to structuring observations, what else has to be there? To see the same object twice there has to be a notion that two observations could be of the same object. But that is more structure than simply spacetime, there’s also a structure of connection between different observations so they can be of the same object.
Solomonoff induction might learn this through compression. Kant, unfortunately, doesn’t explicitly discuss compression all that much. However, even Solomonoff induction makes a priori assumptions beyond spacetime, namely, that the universe is a Turing machine. This is a kind of causal assumption. You couldn’t get “this runs on a Turing machine” by just looking at a bunch of data, without having some kind of prior that already contains Turing machines. It is, instead, assumed a priori that there’s a Turing machine causing your observations.
The book is mostly a lot of stuff like this, what thought structures we must assume a priori to learn from data at all.
The basic gist is that, while Hume assumes you have sense-data and are learning structures like causation from this sense-data, Kant is saying you need concepts of causation to have sense-data at all.
Hmm. Both of these ideas seem very wrong (though Kant’s, perhaps, more so). Is there anything else of value? If this (and similar things) are all that there is, then maybe rationalists are right to mostly ignore Kant…
I’ve yet to see a readable explanation of what Kant had to say (in response to Hume or otherwise) that’s particularly worth paying attention to
As an undergrad, instead of following the actual instructions and writing a proper paper on Kant, I thought it would be more interesting and valuable to simply attempt to paraphrase what he actually said, paragraph by paragraph. It’s the work of a young person with little experience in either philosophy or writing, but it certainly seems to have had a pretty big influence on my thinking over the past ten years, and I got an A. So, mostly for your entertainment, I present to you “Kant in [really not nearly as plain as I thought at the time] English”. (It’s just the bit on apperception.)
I think this is either basic psychology or wrong.¹
For one, Kant seems to be conflating the operation of a concept with its perception:
Since the concept of “unity” must exist for there to be combination (or “conjunction”) in the first place, unity can’t come from combination itself. The whole-ness of unified things must be a product of something beyond combination.
This seems to say that the brain cannot unify things unless it has a concept of combination. However, just as an example, reinforcement learning in AI shows this to be false: unification can happen as a mechanistic consequence of the medium in which experiences are embedded, and an understanding of unification—a perception as a concept—is wholly unnecessary.
Then okay, concepts are generalizations (compressions?) of sense data, and there’s an implied world of which we become cognizant by assuming that the inner structure matches the outer structure. So far, so Simple Idea Of Truth. But then he does the same thing again with “unity”, where he assumes that persistent identity-perception is necessary for judgment. Which I think any consideration of a nematode would disprove: judgment can also happen mechanistically.
I mean, I don’t believe that the self is objectively unified, so Kant’s view would be a problem for me. But I also just think that the model where most mental capabilities are caused by aspects of the medium and nonreflective computation, and consciousness only reacts to them in “hindsight”, seems a lot more convincing to me in light of my pop understanding of neuroscience and introspection of my own mind.
edit: In summary, I think Kant’s model cannot separate, and thus repeatedly mixes up, cognition and consciousness.
¹ Okay, let me be fair: I think this is spectacularly correct and insightful for its time. But I don’t think people who have read the Sequences will get closer to the truth from it.
I got through that page and… no, I really can’t summarize it. I don’t really have any idea what Kant is supposed to have been saying, or why he said any of those things, or the significance of any of it…
I appreciate Kant’s idea that certain things may arise from how we see and interpret the world. I think it’s plausible that this is an accurate high-level description of things like counterfactuals and probability.
(I’m a bit busy atm so I haven’t provided much detail)
They’re suggesting that you should have written ”...this is an accurate how-level description of things like...” It’s a minor point but I guess I agree.
there’s a big difference between saying it saves a few years vs. causes us to have a chance at all when we otherwise wouldn’t. [...] it seems like most of the relevant ideas were already in the memespace
I was struck by the 4th edition of AI: A Modern Approach quoting Nobert Weiner writing in 1960 (!), “If we use, to achieve our purposes, a mechanical agency with whose operation we cannot interfere effectively … we had better be quite sure that the purpose put into the machine is the purpose which we really desire.”
It must not have seemed like a pressing issue in 1960, but Weiner noticed the problem! (And Yudkowsky didn’t notice, at first.) How much better off are our analogues in the worlds where someone like Weiner (or, more ambitiously, Charles Babbage) did treat it as a pressing issue? How much measure do they have?
Yeah, it’s quite plausible that it might have taken another decade (then again I don’t know if Bostrom thought super-intelligence was possible before encountering Eliezer)
I don’t think so. I’m explicitly saying that talking about weird perceptions people might have, such as mental subprocess implantation, is better than the alternative; this is more likely to realize the benefits of neuro-atypicality, by allowing people to recognize when non-neurotypicals are having accurate perceptions, and reduce the risk of psychiatric hospitalization or other bad outcomes.
I guess my point was that a community that excludes anyone who has mental health issues would score well on your metric, while a community that is welcoming would score poorly.
I think when people are comparing philosophers they’re usually trying to compare novel contributions the person made relative to what came before, not how much raw philosophical knowledge they possess.
Another possiblity is that they might be comparing their ability to form a correct philosophical opinion. This isn’t the same as raw knowledge, but I suspect that our epistemic position makes it much easier. Not only because of more information, but also because modern philosopher tends to be much clearer and explicit than older philosophy and so people can use it as an example to learn how to think clearly.
Thanks, I appreciate the thought.
I don’t see why these would be in tension. If there is more and better discussion then that reduces the chance of bad outcomes. (Partially, I brought up the mental health issues because it seemed like people were criticizing Leverage for having people with mental health issues in their orbit, but it seems like Leverage handled the issue relatively well all things considered.)
I basically agree.
I don’t think so. I’m explicitly saying that talking about weird perceptions people might have, such as mental subprocess implantation, is better than the alternative; this is more likely to realize the benefits of neuro-atypicality, by allowing people to recognize when non-neurotypicals are having accurate perceptions, and reduce the risk of psychiatric hospitalization or other bad outcomes.
I don’t think it makes sense to run a “biased” expected value calculation that over-weights costs relative to benefits of information sharing. There are significant negative consequences when discussion about large topics is suppressed, which include failure to have the kind of high-integrity conversations that could lead to actual solutions.
I don’t think the issue was just that it was stressful, but that it was stressful in really unexpected ways. I think me from when I started work would be pretty surprised reading this post.
That doesn’t seem wrong to me. However there’s a big difference between saying it saves a few years vs. causes us to have a chance at all when we otherwise wouldn’t. (I can’t rule out the latter claim, but it seems like most of the relevant ideas were already in the memespace, so it’s more an issue of timing and consolidation/problem-framing.)
I think when people are comparing philosophers they’re usually trying to compare novel contributions the person made relative to what came before, not how much raw philosophical knowledge they possess.
Yes, I’ve definitely noticed a trend where rationalists are mostly continuing from Hume and Turing, neglecting e.g. Kant as a response to Hume.
I’ve yet to see a readable explanation of what Kant had to say (in response to Hume or otherwise) that’s particularly worth paying attention to (despite my philosophy classes in college having covered Kant, and making some attempts later to read him). If you (or someone else) were to write an LW post about this, I think this might be of great benefit to everyone here.
I don’t know what Kant-insights Jessica thinks LW is neglecting, but I endorse Allen Wood’s introduction to Kant as a general resource.
(Partly because Wood is a Kant scholar who loves Kant but talks a bunch about how Kant was just being sloppy / inconsistent in lots of his core discussions of noumena, rather than assuming that everything Kant says reflects some deep insight. This makes me less worried about IMO one of the big failure modes of philosopher-historians, which is that they get too creative with their novel interpretations + treat their favorite historical philosophers like truth oracles.)
BTW, when it comes to transcendental idealism, I mostly think of Arthur Schopenhauer as ‘Kant, but with less muddled thinking and not-absolutely-horrible writing style’. So I’d usually rather go ask what Schopenhauer thought of a thing, rather than what Kant thought. (But I mostly disagree with Kant and Schopenhauer, so I may be the wrong person to ask about how to properly steel-man Kant.)
I’ve been working on a write-up on and off for months, which I might or might not ever get around to finishing.
The basic gist is that, while Hume assumes you have sense-data and are learning structures like causation from this sense-data, Kant is saying you need concepts of causation to have sense-data at all.
The Transcendental Aesthetic is a pretty simple argument if applied to Solomonoff induction. Suppose you tried to write an AI to learn about time, which didn’t already have time. How would it structure its observations, so it could learn about time from these different observations? That seems pretty hard, perhaps not really possible, since “learning” implies past observations affecting how future observations are interpreted.
In Solomonoff induction there is a time-structure built in, which structures observations. That is, the inductor assumes a priori that its observations are structured in a sequence.
Kant argues that space is also a priori this way. This is a somewhat suspicious argument given that vanilla Solomonoff induction doesn’t need a priori space to structure its observations. But maybe it’s true in the case of humans, since our visual cortexes have a notion of spacial observation already built in. (That is, when we see things, we see them at particular locations)
Other than time and space to structuring observations, what else has to be there? To see the same object twice there has to be a notion that two observations could be of the same object. But that is more structure than simply spacetime, there’s also a structure of connection between different observations so they can be of the same object.
Solomonoff induction might learn this through compression. Kant, unfortunately, doesn’t explicitly discuss compression all that much. However, even Solomonoff induction makes a priori assumptions beyond spacetime, namely, that the universe is a Turing machine. This is a kind of causal assumption. You couldn’t get “this runs on a Turing machine” by just looking at a bunch of data, without having some kind of prior that already contains Turing machines. It is, instead, assumed a priori that there’s a Turing machine causing your observations.
The book is mostly a lot of stuff like this, what thought structures we must assume a priori to learn from data at all.
Hmm. Both of these ideas seem very wrong (though Kant’s, perhaps, more so). Is there anything else of value? If this (and similar things) are all that there is, then maybe rationalists are right to mostly ignore Kant…
As an undergrad, instead of following the actual instructions and writing a proper paper on Kant, I thought it would be more interesting and valuable to simply attempt to paraphrase what he actually said, paragraph by paragraph. It’s the work of a young person with little experience in either philosophy or writing, but it certainly seems to have had a pretty big influence on my thinking over the past ten years, and I got an A. So, mostly for your entertainment, I present to you “Kant in [really not nearly as plain as I thought at the time] English”. (It’s just the bit on apperception.)
I think this is either basic psychology or wrong.¹
For one, Kant seems to be conflating the operation of a concept with its perception:
This seems to say that the brain cannot unify things unless it has a concept of combination. However, just as an example, reinforcement learning in AI shows this to be false: unification can happen as a mechanistic consequence of the medium in which experiences are embedded, and an understanding of unification—a perception as a concept—is wholly unnecessary.
Then okay, concepts are generalizations (compressions?) of sense data, and there’s an implied world of which we become cognizant by assuming that the inner structure matches the outer structure. So far, so Simple Idea Of Truth. But then he does the same thing again with “unity”, where he assumes that persistent identity-perception is necessary for judgment. Which I think any consideration of a nematode would disprove: judgment can also happen mechanistically.
I mean, I don’t believe that the self is objectively unified, so Kant’s view would be a problem for me. But I also just think that the model where most mental capabilities are caused by aspects of the medium and nonreflective computation, and consciousness only reacts to them in “hindsight”, seems a lot more convincing to me in light of my pop understanding of neuroscience and introspection of my own mind.
edit: In summary, I think Kant’s model cannot separate, and thus repeatedly mixes up, cognition and consciousness.
¹ Okay, let me be fair: I think this is spectacularly correct and insightful for its time. But I don’t think people who have read the Sequences will get closer to the truth from it.
Interesting, thanks—I will read this ASAP!
If you manage to get through that, maybe you can summarize it? Even Logan’s accessible explanation makes my eyes glaze over.
I got through that page and… no, I really can’t summarize it. I don’t really have any idea what Kant is supposed to have been saying, or why he said any of those things, or the significance of any of it…
I’m afraid I remain as perplexed as ever.
I appreciate Kant’s idea that certain things may arise from how we see and interpret the world. I think it’s plausible that this is an accurate high-level description of things like counterfactuals and probability.
(I’m a bit busy atm so I haven’t provided much detail)
‘how-level’ would be easier to parse
Oops, I meant “high-level”
“How-level”?
They’re suggesting that you should have written ”...this is an accurate how-level description of things like...” It’s a minor point but I guess I agree.
I was struck by the 4th edition of AI: A Modern Approach quoting Nobert Weiner writing in 1960 (!), “If we use, to achieve our purposes, a mechanical agency with whose operation we cannot interfere effectively … we had better be quite sure that the purpose put into the machine is the purpose which we really desire.”
It must not have seemed like a pressing issue in 1960, but Weiner noticed the problem! (And Yudkowsky didn’t notice, at first.) How much better off are our analogues in the worlds where someone like Weiner (or, more ambitiously, Charles Babbage) did treat it as a pressing issue? How much measure do they have?
Yeah, it’s quite plausible that it might have taken another decade (then again I don’t know if Bostrom thought super-intelligence was possible before encountering Eliezer)
I guess my point was that a community that excludes anyone who has mental health issues would score well on your metric, while a community that is welcoming would score poorly.
Another possiblity is that they might be comparing their ability to form a correct philosophical opinion. This isn’t the same as raw knowledge, but I suspect that our epistemic position makes it much easier. Not only because of more information, but also because modern philosopher tends to be much clearer and explicit than older philosophy and so people can use it as an example to learn how to think clearly.