I can (poorly) simulate a Linux terminal in my head because of my experience of working in them. I suppose it shouldn’t be too surprising.
I assume it’s mostly trained on help articles and tutorials. Which begs the question, what would it be like of it were actually trained on a terminal?
Can we train an ML system to simulate a function? Multiple times. Millions. Now simulate a million functions. Now train another ML system using the parameters of the first system as the input, with the expected output being the original function.
Would we then have a system that can predict how a given ML system configuration will function?
Now with those functions, randomly knock out part of the code, and fill by inference. Compile/interpret the code (retry when it doesn’t compile, or maybe leave it with a heavy penalty). Feed those functions into our first system.
Would we then have a system two that knows how to make changes to a system one causing a specific change in behavior?
I don’t know how far a model trained explicitly on only terminal output could go, but it makes sense that it might be a lot farther than a model trained on all the text on the internet (some small fraction of which happens to be terminal output). Although I also would have thought GPT’s architecture, with a fixed context window and a fixed number of layers and tokenization that isn’t at all optimized for the task, would pay large efficiency penalties at terminal emulation and would be far less impressive at it than it is at other tasks.
Assuming it does work, could we get a self-operating terminal by training another GPT to roleplay the entering commands part? Probably. I’m not sure we should though...
I can (poorly) simulate a Linux terminal in my head because of my experience of working in them. I suppose it shouldn’t be too surprising.
I assume it’s mostly trained on help articles and tutorials. Which begs the question, what would it be like of it were actually trained on a terminal?
Can we train an ML system to simulate a function? Multiple times. Millions. Now simulate a million functions. Now train another ML system using the parameters of the first system as the input, with the expected output being the original function.
Would we then have a system that can predict how a given ML system configuration will function?
Now with those functions, randomly knock out part of the code, and fill by inference. Compile/interpret the code (retry when it doesn’t compile, or maybe leave it with a heavy penalty). Feed those functions into our first system.
Would we then have a system two that knows how to make changes to a system one causing a specific change in behavior?
Is this all just nonsense?
I don’t know how far a model trained explicitly on only terminal output could go, but it makes sense that it might be a lot farther than a model trained on all the text on the internet (some small fraction of which happens to be terminal output). Although I also would have thought GPT’s architecture, with a fixed context window and a fixed number of layers and tokenization that isn’t at all optimized for the task, would pay large efficiency penalties at terminal emulation and would be far less impressive at it than it is at other tasks.
Assuming it does work, could we get a self-operating terminal by training another GPT to roleplay the entering commands part? Probably. I’m not sure we should though...