I’m relatively new to this site and have been trying to read the backlog this past week so maybe I’ve missed some things, but from my vantage point it seems like your are trying to do, Eliezer, is come up with a formalized theory of friendly agi that will later be implemented in code using, I assume, current software development tools on current computer architectures. Also, your approach to this AGI is some sort of bayesian optimization process that is ‘aligned’ properly as to ‘level-up’ in such a way as to become and stay ‘friendly’ or benevolent towards humanity and presumably all sentient life and the environment that supports them. Oh ya, and this bayesian optimization process is apparently recursively self-improving so that you would only need to code some seedling of it (like a generative process such as a mandelbrot set) and know that it will blossom along the right course. That, my friends, is a really tall order and I do not envy anyone who tries to take on such a formidable task. I’m tempted to say that it is not even humanly possible (without a manhattan project and even then maybe not) but I’ll be bayesian and say the probability is extremely low.
I think you are a very bright and thoughtful young guy and from what I’ve read seem like more of a philosopher than an engineer or scientist, which isn’t a bad thing, but to transition from philosophizing to engineering is not trivial especially when philosophizing upon such complex issues.
I can’t even imagine trying to create some trivial new software without prototyping and playing around with drafts before I had some idea of what it would look like. This isn’t Maxwell’s equations, this is messy self-reflective autonomous general intelligence, there is no simple, elegant theory for such a system. So get your hands dirty and take on a more agile work process. Couldn’t you at least create a particular component of the AI, such as a machine vision module, that would show your general approach is feasible? Or do you fear that it would spontaneously turn into skynet? Does your architecture even have modules, or are you planning some super elegant bayesian quine? Or do you even have an architecture in mind?
Anyway, good luck and I’ll continue reading, if for nothing else then entertainment.
I’m relatively new to this site and have been trying to read the backlog this past week so maybe I’ve missed some things, but from my vantage point it seems like your are trying to do, Eliezer, is come up with a formalized theory of friendly agi that will later be implemented in code using, I assume, current software development tools on current computer architectures. Also, your approach to this AGI is some sort of bayesian optimization process that is ‘aligned’ properly as to ‘level-up’ in such a way as to become and stay ‘friendly’ or benevolent towards humanity and presumably all sentient life and the environment that supports them. Oh ya, and this bayesian optimization process is apparently recursively self-improving so that you would only need to code some seedling of it (like a generative process such as a mandelbrot set) and know that it will blossom along the right course. That, my friends, is a really tall order and I do not envy anyone who tries to take on such a formidable task. I’m tempted to say that it is not even humanly possible (without a manhattan project and even then maybe not) but I’ll be bayesian and say the probability is extremely low.
I think you are a very bright and thoughtful young guy and from what I’ve read seem like more of a philosopher than an engineer or scientist, which isn’t a bad thing, but to transition from philosophizing to engineering is not trivial especially when philosophizing upon such complex issues.
I can’t even imagine trying to create some trivial new software without prototyping and playing around with drafts before I had some idea of what it would look like. This isn’t Maxwell’s equations, this is messy self-reflective autonomous general intelligence, there is no simple, elegant theory for such a system. So get your hands dirty and take on a more agile work process. Couldn’t you at least create a particular component of the AI, such as a machine vision module, that would show your general approach is feasible? Or do you fear that it would spontaneously turn into skynet? Does your architecture even have modules, or are you planning some super elegant bayesian quine? Or do you even have an architecture in mind?
Anyway, good luck and I’ll continue reading, if for nothing else then entertainment.