ProPuke
ProPuke2mo ago

Piping stdout from a ChildProcess while it's running

Is it possible in Deno to read from a ChildProcess's stdout while it is running (before it has completed)? I'm using deno.Command and .spawn() to create a ChildProcess.
If I then try reading from process.stdout.getReader() it does not return anything until the process has terminated and an output status is returned. If I try reading direct from process.stdout I get the printed warning "Can't collect output because stdout is locked". (my bad, this was called from later calling output() instead of awaiting status) Is it possible to read from a ChildProcess stdout while it is interactively running?
5 Replies
bartlomieju
bartlomieju2mo ago
Yes, this is the exact scenario that is supported. Do you have a cod Ethan doesn't work as expected that you can share?
ProPuke
ProPuke2mo ago
So the following program prints "1\n", waits 3 seconds, then prints "2\n":
#include <stdio.h>
#include <unistd.h>

int main() {
printf("1\n");
sleep(3);
printf("2\n");
return 0;
}
#include <stdio.h>
#include <unistd.h>

int main() {
printf("1\n");
sleep(3);
printf("2\n");
return 0;
}
But if I run the following in deno I get no output for 3 seconds, then all 4 bytes at once when the program terminates:
const process = new Deno.Command('./test', { stdout: 'piped' }).spawn();

(async() => {
const reader = process.stdout.getReader();
while(true){
const read = await reader.read();
if(read.value){
console.log(`read ${read.value.length} bytes`);
}
if(read.done){
console.log('read complete');
break;
}
}
})();

const status = await process.status;
console.log(`process finished with code ${status.code}`);
const process = new Deno.Command('./test', { stdout: 'piped' }).spawn();

(async() => {
const reader = process.stdout.getReader();
while(true){
const read = await reader.read();
if(read.value){
console.log(`read ${read.value.length} bytes`);
}
if(read.done){
console.log('read complete');
break;
}
}
})();

const status = await process.status;
console.log(`process finished with code ${status.code}`);
read 4 bytes
read complete
process finished with code 0
read 4 bytes
read complete
process finished with code 0
A simpler example would also be:
const process = new Deno.Command('./test', { stdout: 'piped' }).spawn();

for await (const data of process.stdout.values()){
console.log(`read ${data.length} bytes: ${data}`);
}
const process = new Deno.Command('./test', { stdout: 'piped' }).spawn();

for await (const data of process.stdout.values()){
console.log(`read ${data.length} bytes: ${data}`);
}
This also waits 3 seconds, and then reads all 4 bytes at once:
read 4 bytes: 49,10,50,10
read 4 bytes: 49,10,50,10
In all cases nothing from stdout is readable until the program has terminated.
bartlomieju
bartlomieju2mo ago
Your C program is buffering the output - it's only flushed when your program finishes If you add setbuf(stdout, NULL); in the first line of main() your Deno program behaves as you expect
ProPuke
ProPuke2mo ago
Right you are. That does fix it. However, this is not necessary when running tasks in the terminal, which does display each line separately. Is it possible to duplicate this behaviour in Deno? hmm.. I guess making deno emulate/present as a TTY so that programs buffer less is a different question. Thanks, you've addressed everything here!
bartlomieju
bartlomieju2mo ago
Sounds good! Happy to help