I’m curious how bash does it. Suppose you do cat <(cat), this gives you
One cat process with stdout writing to a pipe
One cat /proc/self/fd/11 process with file descriptors 3 and 11 both reading from this pipe.
So if I had to guess: bash calls pipe() to get two ends of a pipe. Let’s say these have file descriptors 10 and 11. Then it forks twice, which copies the list of file descriptors. We now have three processes with file descriptors 10 (open for writing) and 11 (open for reading). Parent process closes both ends, one child closes the read end, one child closes the write end.
The writing child process uses dup2() or something to set its stdout to be the pipe, then calls exec("cat", []) or whatever.
The reading child process has open file descriptor 11, and can access it as a file at /proc/self/fd/11. So it calls exec("cat", ["/proc/self/fd/11"]). The process which is now cat still has that open file descriptor, but it doesn’t know that, it just knows it’s been given a path; it opens that path and gets file descriptor 3, also reading from that pipe.
So if you want to duplicate this in python:
You’ll need to figure out how to access the pipe() function.
Do the various subprocess libraries close file descriptors? If not, you can maybe just pass them /proc/self/fd/(fileno) as the path to read/write. If they do, you might need to do the fork and exec manually?
Maybe you also need to figure out “use this specific file descriptor for the stdin/stdout of this process”? Not sure. At minimum a fork/exec would work if you do need to do this, but maybe there’s something better.
I’m curious how bash does it. Suppose you do
cat <(cat), this gives youOne
catprocess with stdout writing to a pipeOne
cat /proc/self/fd/11process with file descriptors 3 and 11 both reading from this pipe.So if I had to guess: bash calls
pipe()to get two ends of a pipe. Let’s say these have file descriptors 10 and 11. Then it forks twice, which copies the list of file descriptors. We now have three processes with file descriptors 10 (open for writing) and 11 (open for reading). Parent process closes both ends, one child closes the read end, one child closes the write end.The writing child process uses
dup2()or something to set its stdout to be the pipe, then callsexec("cat", [])or whatever.The reading child process has open file descriptor 11, and can access it as a file at /proc/self/fd/11. So it calls
exec("cat", ["/proc/self/fd/11"]). The process which is nowcatstill has that open file descriptor, but it doesn’t know that, it just knows it’s been given a path; it opens that path and gets file descriptor 3, also reading from that pipe.So if you want to duplicate this in python:
You’ll need to figure out how to access the
pipe()function.Do the various subprocess libraries close file descriptors? If not, you can maybe just pass them
/proc/self/fd/(fileno)as the path to read/write. If they do, you might need to do the fork and exec manually?Maybe you also need to figure out “use this specific file descriptor for the stdin/stdout of this process”? Not sure. At minimum a fork/exec would work if you do need to do this, but maybe there’s something better.
None of this is very high confidence.