ProPuke•3mo ago

Piping stdout from a ChildProcess while it's running

Is it possible in Deno to read from a ChildProcess's stdout while it is running (before it has completed)? I'm using deno.Command and .spawn() to create a ChildProcess.
If I then try reading from process.stdout.getReader() it does not return anything until the process has terminated and an output status is returned. ~~If I try reading direct from process.stdout I get the printed warning "Can't collect output because stdout is locked".~~ (my bad, this was called from later calling output() instead of awaiting status) Is it possible to read from a ChildProcess stdout while it is interactively running?

5 Replies

bartlomieju•3mo ago

Yes, this is the exact scenario that is supported. Do you have a cod Ethan doesn't work as expected that you can share?

ProPuke•3mo ago

So the following program prints "1\n", waits 3 seconds, then prints "2\n":

#include <stdio.h>
#include <unistd.h>

int main() {
    printf("1\n");
    sleep(3);
    printf("2\n");
    return 0;
}

#include <stdio.h>
#include <unistd.h>

int main() {
    printf("1\n");
    sleep(3);
    printf("2\n");
    return 0;
}

But if I run the following in deno I get no output for 3 seconds, then all 4 bytes at once when the program terminates:

const process = new Deno.Command('./test', { stdout: 'piped' }).spawn();

(async() => {
    const reader = process.stdout.getReader();
    while(true){
        const read = await reader.read();
        if(read.value){
            console.log(`read ${read.value.length} bytes`);
        }
        if(read.done){
            console.log('read complete');
            break;
        }
    }
})();

const status = await process.status;
console.log(`process finished with code ${status.code}`);

const process = new Deno.Command('./test', { stdout: 'piped' }).spawn();

(async() => {
    const reader = process.stdout.getReader();
    while(true){
        const read = await reader.read();
        if(read.value){
            console.log(`read ${read.value.length} bytes`);
        }
        if(read.done){
            console.log('read complete');
            break;
        }
    }
})();

const status = await process.status;
console.log(`process finished with code ${status.code}`);

read 4 bytes
read complete
process finished with code 0

read 4 bytes
read complete
process finished with code 0

A simpler example would also be:

const process = new Deno.Command('./test', { stdout: 'piped' }).spawn();

for await (const data of process.stdout.values()){
    console.log(`read ${data.length} bytes: ${data}`);
}

const process = new Deno.Command('./test', { stdout: 'piped' }).spawn();

for await (const data of process.stdout.values()){
    console.log(`read ${data.length} bytes: ${data}`);
}

This also waits 3 seconds, and then reads all 4 bytes at once:

read 4 bytes: 49,10,50,10

read 4 bytes: 49,10,50,10

In all cases nothing from stdout is readable until the program has terminated.

bartlomieju•3mo ago

Your C program is buffering the output - it's only flushed when your program finishes If you add setbuf(stdout, NULL); in the first line of main() your Deno program behaves as you expect

ProPuke•3mo ago

Right you are. That does fix it. However, this is not necessary when running tasks in the terminal, which does display each line separately. Is it possible to duplicate this behaviour in Deno? hmm.. I guess making deno emulate/present as a TTY so that programs buffer less is a different question. Thanks, you've addressed everything here!

bartlomieju•3mo ago

Sounds good! Happy to help