Am I the only one who finds parts of the early story rather dystopian? He sounds like a puppet being pulled around by the AI, gradually losing his ability to have his own thoughts and conversations. (That part’s not written, but it’s the inevitable result of asking the AI every time he encounters struggle.)
Definitely not the only one. I think the only way I would be halfway comfortable with the early levels of intrusion that are described is if I were able to ensure the software is offline and entirely in my control, without reporting back to whoever created it, and even then, probably not.
Part of me envys the tech-optimists for their outlook, but it feels like sheer folly.
I am pretty worried about the bad versions of everything listed here, and think the bad versions are what we get by default. But, also, I think figuring out how to get the good versions is just… kinda a necessary step along the path towards good futures.
I think there are going to be early adopters who a) take on more risk from getting fucked , but b) validate the general product/model. There will also be versions that are more “privacy first” with worse UI (same as there are privacy-minded FB clones nobody uses).
Some people will choose to stay grounded… and maybe (in good futures) get to have happy lives, but, in some sense they’ll be left behind.
In a good future, they get left behind by people who use some sort of… robustly philophically and practically safe version of these sorts of tools. In bad worlds, they get left behind by hollowed out nonconscious shells of people (or, more likely, just paperclipped)
I’m currently working on a privacy-minded set of tools for recording my thoughts (keystrokes, audio transcripts, keystrokes), that I use for LLM augmented thought. (Alongside metacognition training that, among other things, is aimed at preserving my mind as I start relying on those tools more and more).
I have some vague hope that if we make it to a good enough intermediate future that it seems worth prioritizing, I can also prioritize getting the UI right so the privacy-minded versions don’t suck compared to the Giant Corporate Versions.
Oh yes. It’s extremely dystopian. And extremely lonely, too. Rather than having a person, actual people around him to help, his only help comes from tech. It’s horrifyingly lonely and isolated. There is no community, only tech.
Also, when they died together, it was horrible. They literally offloaded more and more of themselves into their tech until they were powerless to do anything but die. I don’t buy the whole ‘the thoughts were basically them’ thing at all. It was at best, some copy of them.
There can be made an argument for it qualitatively being them, but quantitatively, obviously not.
Am I the only one who finds parts of the early story rather dystopian? He sounds like a puppet being pulled around by the AI, gradually losing his ability to have his own thoughts and conversations. (That part’s not written, but it’s the inevitable result of asking the AI every time he encounters struggle.)
I am reminded of Scott’s “whispering earring” story (https://www.reddit.com/r/rational/comments/e71a6s/the_whispering_earring_by_scott_alexander_there/). But I’m not sure whether that’s actually bad in general rather than specifically because the earring is maybe misaligned.
Definitely not the only one. I think the only way I would be halfway comfortable with the early levels of intrusion that are described is if I were able to ensure the software is offline and entirely in my control, without reporting back to whoever created it, and even then, probably not.
Part of me envys the tech-optimists for their outlook, but it feels like sheer folly.
I am pretty worried about the bad versions of everything listed here, and think the bad versions are what we get by default. But, also, I think figuring out how to get the good versions is just… kinda a necessary step along the path towards good futures.
I think there are going to be early adopters who a) take on more risk from getting fucked , but b) validate the general product/model. There will also be versions that are more “privacy first” with worse UI (same as there are privacy-minded FB clones nobody uses).
Some people will choose to stay grounded… and maybe (in good futures) get to have happy lives, but, in some sense they’ll be left behind.
In a good future, they get left behind by people who use some sort of… robustly philophically and practically safe version of these sorts of tools. In bad worlds, they get left behind by hollowed out nonconscious shells of people (or, more likely, just paperclipped)
I’m currently working on a privacy-minded set of tools for recording my thoughts (keystrokes, audio transcripts, keystrokes), that I use for LLM augmented thought. (Alongside metacognition training that, among other things, is aimed at preserving my mind as I start relying on those tools more and more).
I have some vague hope that if we make it to a good enough intermediate future that it seems worth prioritizing, I can also prioritize getting the UI right so the privacy-minded versions don’t suck compared to the Giant Corporate Versions.
Oh yes. It’s extremely dystopian. And extremely lonely, too. Rather than having a person, actual people around him to help, his only help comes from tech. It’s horrifyingly lonely and isolated. There is no community, only tech.
Also, when they died together, it was horrible. They literally offloaded more and more of themselves into their tech until they were powerless to do anything but die. I don’t buy the whole ‘the thoughts were basically them’ thing at all. It was at best, some copy of them.
There can be made an argument for it qualitatively being them, but quantitatively, obviously not.