piping for async sub process

In Deno.process, is there a way to set its STDOUT to something I can read line by line while that primary process is still running?
12 Replies
AapoAlas
AapoAlas2y ago
Deno CLI APIs – Deno.run | Deno Doc
Spawns new subprocess. RunOptions must contain at a minimum the opt.cmd, an array of program arguments, the first of which is the binary. const p = Deno.run({ cmd: ["curl", "https://example.com"], }); const status = await p.status(); Subprocess uses same working directory as parent process unless opt.cwd is specified. Environmental variable...
Deno CLI APIs – Deno.Process | Deno Doc
Represents an instance of a sub process that is returned from {@linkcode Deno.run} which can be used to manage the sub-process.
AapoAlas
AapoAlas2y ago
Or stdin if you meant reading line by line inside the spawned process.
dan.the.discloser
Thank you! This works, but is VERY ugly * Is there a better way to terminate the reading loop? * Is there a better way to deal with the reads not filling the buffer?
const cmd = ["sh", "long-running-script"]

const proc = Deno.run({stdout: "piped" , cmd: cmd});

async function handleAll(){
const decoder = new TextDecoder()
const buf =new Uint8Array(10000);
while (true){
const n = await proc.stdout.read(buf)
if (n === null){ break;}
const outout = decoder.decode(buf)
console.log(outout.slice(0,n));
}
}

const promises = [handleAll() , proc.status()]

await Promise.allSettled(promises)
const cmd = ["sh", "long-running-script"]

const proc = Deno.run({stdout: "piped" , cmd: cmd});

async function handleAll(){
const decoder = new TextDecoder()
const buf =new Uint8Array(10000);
while (true){
const n = await proc.stdout.read(buf)
if (n === null){ break;}
const outout = decoder.decode(buf)
console.log(outout.slice(0,n));
}
}

const promises = [handleAll() , proc.status()]

await Promise.allSettled(promises)
AapoAlas
AapoAlas2y ago
https://deno.land/std@0.161.0/streams/mod.ts will probably have the helpers you want.
dan.the.discloser
I read through the doc's and the sources (to the extent I can) and I just don't get it. Clearly what I am doig is not right. error: TS2339 [ERROR]: Property 'pipeThrough' does not exist on type 'Reader & Closer & { readable: ReadableStream<Uint8Array>; }'.
async function handleAll(){
const output = await proc.stdout.pipeThrough((new TextDecoderStream()))
console.dir(output)
}
async function handleAll(){
const output = await proc.stdout.pipeThrough((new TextDecoderStream()))
console.dir(output)
}
Can you point me at a simple example?
AapoAlas
AapoAlas2y ago
stdout.readable.pipeThrough
dan.the.discloser
Thank you -- i will try it Thank you! This is much less ugly! Is there also a way to make the control of the while loop less ugly?
async function handleAll() {
const outputStream = await proc.stdout.readable.pipeThrough(
new TextDecoderStream(),
);
const r = outputStream.getReader();
let firstOutput = { done: false };
while (!firstOutput.done) {
firstOutput = await r.read();
console.log(firstOutput);
}
return r;
}
async function handleAll() {
const outputStream = await proc.stdout.readable.pipeThrough(
new TextDecoderStream(),
);
const r = outputStream.getReader();
let firstOutput = { done: false };
while (!firstOutput.done) {
firstOutput = await r.read();
console.log(firstOutput);
}
return r;
}
AapoAlas
AapoAlas2y ago
I presume your end goal is not to just log the output? You might consider implementing your eventual use case as a writable stream. But if you just need to get the output then yeah, a while loop doing awaits is pretty much the way. You can improve it a bit by doing:
let firstOut;
do {
firstOut = await r.read();
} while (!firstOut.done);
let firstOut;
do {
firstOut = await r.read();
} while (!firstOut.done);
dan.the.discloser
thank you a, a "do while" was exactly what I was hoping for indeed there is more work to do with the results, but I am trying to understands. the basics. when I tried
async function handleAll() {
const outputStream = await proc.stdout.readable.pipeThrough(
new TextDecoderStream(),
);
for await (const firstOutput of outputStream) {
console.log(firstOutput);
}
}
async function handleAll() {
const outputStream = await proc.stdout.readable.pipeThrough(
new TextDecoderStream(),
);
for await (const firstOutput of outputStream) {
console.log(firstOutput);
}
}
it waited for the process to finish and then gives me all the reads rapidly rather tthan giving to me as they become available
AapoAlas
AapoAlas2y ago
Humm...
dan.the.discloser
The problem was with my environment, the iterator works fine. Sorry to be stupid.