venego•6mo ago

Can't spawn a shell properly using Deno!

const bash = (() => {
  const bashProc = spawn("bash");

  bashProc.stdout.on("data", (data: string) => {
    console.log(`stdout: ${data}`);
  });

  bashProc.stderr.on("data", (err: string) => {
    console.error(`stderr: ${err}`);
    throw Error(err);
  });

  bashProc.on("close", (code: string) => {
    console.log(`child process exited with code ${code}`);
  });

  return bashProc.stdin.write;
})();
bash('echo lll');

const bash = (() => {
  const bashProc = spawn("bash");

  bashProc.stdout.on("data", (data: string) => {
    console.log(`stdout: ${data}`);
  });

  bashProc.stderr.on("data", (err: string) => {
    console.error(`stderr: ${err}`);
    throw Error(err);
  });

  bashProc.on("close", (code: string) => {
    console.log(`child process exited with code ${code}`);
  });

  return bashProc.stdin.write;
})();
bash('echo lll');

I know there is Deno.run, but I need a way to be able to access the std outputs and errors. I'm really new to Deno, sorry if this is a dumb question.

21 Replies

Deno AI Helper•6mo ago

No worries, there are no dumb questions when you're learning something new! 🦕 In Deno, you can use Deno.Command to spawn a subprocess and interact with its standard input, output, and error streams. The code you've provided looks like it's using a Node.js-style API, which won't work directly in Deno because Deno has a different API for subprocesses. Here's how you can achieve similar functionality in Deno:

const command = new Deno.Command("bash", {
  stdin: "piped",
  stdout: "piped",
  stderr: "piped",
});

const process = command.spawn();

// Write to the stdin of the bash process
const encoder = new TextEncoder();
await process.stdin.write(encoder.encode("echo lll\n"));
process.stdin.close();

// Read from the stdout of the bash process
const decoder = new TextDecoder();
const stdout = await process.stdout.readable.getReader().read();
console.log("stdout:", decoder.decode(stdout.value));

// Read from the stderr of the bash process
const stderr = await process.stderr.readable.getReader().read();
console.error("stderr:", decoder.decode(stderr.value));

// Wait for the process to exit and log the exit code
const status = await process.status;
console.log(`child process exited with code ${status.code}`);

// Don't forget to close the resources
process.stdout.close();
process.stderr.close();

const command = new Deno.Command("bash", {
  stdin: "piped",
  stdout: "piped",
  stderr: "piped",
});

const process = command.spawn();

// Write to the stdin of the bash process
const encoder = new TextEncoder();
await process.stdin.write(encoder.encode("echo lll\n"));
process.stdin.close();

// Read from the stdout of the bash process
const decoder = new TextDecoder();
const stdout = await process.stdout.readable.getReader().read();
console.log("stdout:", decoder.decode(stdout.value));

// Read from the stderr of the bash process
const stderr = await process.stderr.readable.getReader().read();
console.error("stderr:", decoder.decode(stderr.value));

// Wait for the process to exit and log the exit code
const status = await process.status;
console.log(`child process exited with code ${status.code}`);

// Don't forget to close the resources
process.stdout.close();
process.stderr.close();

Please note that you need to grant the --allow-run permission to your Deno process to spawn subprocesses. Also, the subprocesses are not run in a Deno sandbox, so they have the same permissions as if you were to run the command from the command line yourself. The above example demonstrates how to write to the subprocess's standard input and read from its standard output and error streams. It's important to handle the resources properly by closing them after you're

Deno AI Helper•6mo ago

done to avoid any resource leaks. If you have any more questions or need further clarification, feel free to ask!

Helpful Links: Creating a Subprocess | Deno Docs Subprocesses: Collecting Output - Deno by Example Deno 1.29: Custom npm registry support Deno 1.21 Release Notes

venego•6mo ago

The stdin appears to not have a function write!

error: Uncaught (in promise) TypeError: process.stdin.write is not a function
await process.stdin.write(encoder.encode("echo lll\n"));
                    ^
    at file:///home/venego/home/dev/lab/test.js:42:21

error: Uncaught (in promise) TypeError: process.stdin.write is not a function
await process.stdin.write(encoder.encode("echo lll\n"));
                    ^
    at file:///home/venego/home/dev/lab/test.js:42:21

raunioroo•6mo ago

Not a dumb question, the web standard stream api can feel a bit weird / surprising. stdin.getWriter() should give you a thing that has the write()method Somewhat like so, untested :)

const writer = myprocess.stdin.getWriter(); 
writer.write((new TextEncoder()).encode("Foo"));
writer.write((new TextEncoder()).encode("Bar"));
writer.write(binarystuff_as_is);

const writer = myprocess.stdin.getWriter(); 
writer.write((new TextEncoder()).encode("Foo"));
writer.write((new TextEncoder()).encode("Bar"));
writer.write(binarystuff_as_is);

pyrote•6mo ago

If you only care about stdout and stderr and you can wait for the process to complete you could use something like this,

const bash = (cmd: string) => {

const output = new Deno.Command("bash", { args: ["-c", cmd] }).outputSync();

const decoder = new TextDecoder();

console.log("stdout:",decoder.decode(output.stdout));
console.error("stderr:",decoder.decode(output.stderr));

};

bash('echo lll');

const bash = (cmd: string) => {

const output = new Deno.Command("bash", { args: ["-c", cmd] }).outputSync();

const decoder = new TextDecoder();

console.log("stdout:",decoder.decode(output.stdout));
console.error("stderr:",decoder.decode(output.stderr));

};

bash('echo lll');

Calling spawn() creates a subprocess that has to be accessed with readable streams. Thus the ^^^ getWriter()/getReader() needs to be used. If you need to interact with the command while its running you would use spawn().

venego•6mo ago

This a really clean code that works, but I really need one shell so I can set variables and stuff, (creating a post-installation script for installing Debian). Got it, the code the AI wrote barely works, I got other erros that makes me want to write this from scratch, would you please suggest a link to the docs about this topic?

venego•6mo ago

I hope this is a good start: https://examples.deno.land/subprocesses-spawn

Subprocesses: Spawning - Deno by Example

For more complex usecases, we don't simply want the output of some command. In this case, we can spawn a subprocess and interact with it. -- Deno by example is a collection of annotated examples for how to use Deno, and the various features it provides.

venego•6mo ago

~~Here is what I tried:~~ It works now

const command = new Deno.Command("bash", {
  args: [],
  stdin: "piped",
  stdout: "piped",
  stderr: "piped",
});
const process = command.spawn();
const textEncoder = new TextEncoder();
const textDecoder = new TextDecoder();

const bash = async (cmdString)=>{

  const writer = process.stdin.getWriter();
  writer.write(textEncoder.encode(cmdString));
  writer.releaseLock();
  await process.stdin.close();

  const output = await process.output();

  const stdout = output.stdout;
  console.log(textDecoder.decode(stdout));

  const stderr = output.stderr;
  console.log(textDecoder.decode(stderr));
}

bash('echo "finally works');

const command = new Deno.Command("bash", {
  args: [],
  stdin: "piped",
  stdout: "piped",
  stderr: "piped",
});
const process = command.spawn();
const textEncoder = new TextEncoder();
const textDecoder = new TextDecoder();

const bash = async (cmdString)=>{

  const writer = process.stdin.getWriter();
  writer.write(textEncoder.encode(cmdString));
  writer.releaseLock();
  await process.stdin.close();

  const output = await process.output();

  const stdout = output.stdout;
  console.log(textDecoder.decode(stdout));

  const stderr = output.stderr;
  console.log(textDecoder.decode(stderr));
}

bash('echo "finally works');

~~No output is shown in the console.~~ How do I release the lock for the following commands?

error: Uncaught (in promise) TypeError: Can't collect output because stdout is locked
  const output = await process.output();

error: Uncaught (in promise) TypeError: Can't collect output because stdout is locked
  const output = await process.output();

output.releaseLock() didn't work!

raunioroo•6mo ago

Hmm. The command api reference https://deno.land/api@v1.39.0?s=Deno.Command comes to mind, and then for the stream stuff, MDN docs: https://developer.mozilla.org/en-US/docs/Web/API/Streams_API

venego•6mo ago

The MDN link seems to be talking about network streams, does deno use network to talk to sub-processes? or is it just the same protocol?

raunioroo•6mo ago

It's just the same standard API. Deno used to have it's own streams API for file access, but its deprecated if not already removed, and now the web streams api is the only one that should be used. web streams api as in the name of the api. in deno it does access files directly, no network stuff involved :) I take the intent is to be able to call bash()multiple times but for a single process? And now it works but only once, subsequent calls to bash()throw errors, right?

venego•6mo ago

Thanks I'll give the mdn page a good read. That's good to know. Oh, let me share an update...

const command = new Deno.Command("bash", {
  args: [],
  stdin: "piped",
  stdout: "piped",
  stderr: "piped",
});
const process = command.spawn();
const textEncoder = new TextEncoder();
const textDecoder = new TextDecoder();

const bash = async (cmdString)=>{
  const writer = process.stdin.getWriter();
  writer.write(textEncoder.encode(cmdString));
  writer.releaseLock();
  await process.stdin.close();
  
  // const reader = await process.stdout.getReader();
  // const ll = await reader.read();
  // console.log(ll)
  const output = await process.output();

  const stdout = output.stdout;
  console.log(textDecoder.decode(stdout));

  const stderr = output.stderr;
  console.log(textDecoder.decode(stderr));
}

await bash('echo "finally works"');
await bash('echo "finally works"');

const command = new Deno.Command("bash", {
  args: [],
  stdin: "piped",
  stdout: "piped",
  stderr: "piped",
});
const process = command.spawn();
const textEncoder = new TextEncoder();
const textDecoder = new TextDecoder();

const bash = async (cmdString)=>{
  const writer = process.stdin.getWriter();
  writer.write(textEncoder.encode(cmdString));
  writer.releaseLock();
  await process.stdin.close();
  
  // const reader = await process.stdout.getReader();
  // const ll = await reader.read();
  // console.log(ll)
  const output = await process.output();

  const stdout = output.stdout;
  console.log(textDecoder.decode(stdout));

  const stderr = output.stderr;
  console.log(textDecoder.decode(stderr));
}

await bash('echo "finally works"');
await bash('echo "finally works"');

The input has to be open and closed multiple times is what I assume. I've tried await process.stdin.open(); but there is no such method. here is the error i get on the second call to bash:

error: Uncaught (in promise) TypeError: The stream is closing or is closed.
  writer.write(textEncoder.encode(cmdString));

error: Uncaught (in promise) TypeError: The stream is closing or is closed.
  writer.write(textEncoder.encode(cmdString));

If I don't close the writer/stdin, it just hangs there with no output. If the input could be written to (like const output = await process.output(); but for inputs), that would be really nice; no streams are required for what I'm trying to input. I imagine it would be like: await process.setInput('inputtt').

raunioroo•6mo ago

wait a bit, need to test a bit to remember how it works :)

venego•6mo ago

No rush, I'm bruteforcing some random methods myself 🙂

raunioroo•6mo ago

import { TextLineStream } from "https://deno.land/std@0.211.0/streams/text_line_stream.ts";

const command = new Deno.Command("bash", {
  args: [],
  stdin: "piped",
  stdout: "piped",
  stderr: "piped",
});
const process = command.spawn();

const textEncoder = new TextEncoder();

const writer = process.stdin.getWriter();

const listen = async (stream, cb) => {
  const reader = stream.pipeThrough(new TextDecoderStream()).pipeThrough(
    new TextLineStream(),
  ).getReader();
  while (true) {
    const { done, value } = await reader.read();
    if (done) {
      break;
    }
    cb(value);
  }
};

listen(process.stdout, (data) => {
  console.log("bash stdout says: " + data);
});
listen(process.stderr, (data) => {
  console.log("bash stderr says: " + data);
});

const bash = async (cmdString) => {
  await writer.write(textEncoder.encode(cmdString));
};

await bash('echo "finally works 1"\n');
await bash('echo "finally works 2"\n');
const wait_for_exit = await process.status;

import { TextLineStream } from "https://deno.land/std@0.211.0/streams/text_line_stream.ts";

const command = new Deno.Command("bash", {
  args: [],
  stdin: "piped",
  stdout: "piped",
  stderr: "piped",
});
const process = command.spawn();

const textEncoder = new TextEncoder();

const writer = process.stdin.getWriter();

const listen = async (stream, cb) => {
  const reader = stream.pipeThrough(new TextDecoderStream()).pipeThrough(
    new TextLineStream(),
  ).getReader();
  while (true) {
    const { done, value } = await reader.read();
    if (done) {
      break;
    }
    cb(value);
  }
};

listen(process.stdout, (data) => {
  console.log("bash stdout says: " + data);
});
listen(process.stderr, (data) => {
  console.log("bash stderr says: " + data);
});

const bash = async (cmdString) => {
  await writer.write(textEncoder.encode(cmdString));
};

await bash('echo "finally works 1"\n');
await bash('echo "finally works 2"\n');
const wait_for_exit = await process.status;

^ listening for input is a bit involved as deno can't really know when bash is done with a command you send (unless you create a new process for each command). you can only collect output as it comes, and perhaps like in here, run a callback when hit with a newline. so, only one getWriter() call, reuse the returned instance. No locking or closing stuffs. If you dynamically want to send and receive stuff to the process like here, you can't use await process.output() - as you already handle reading and writing yourself. await process.status works to just wait until the program ends without attempting to read output again. and writer.write() does not add newlines like console.log, so you need to add that for bash to recognize the input

venego•6mo ago

Thanks, I think this is exactly what I needed. I'm just confused by what you just said: if the output's end can't be detected, how do I make sure I don't the second command while the 1st still running? The last line tho, seems to be unnecessary as deno does that without it (or something weird is blocking the process from exiting).

raunioroo•6mo ago

haha good question.. it is indeed a bit tricky. for oneliner outputs you could perhaps wait for the callback to be called at least once before issuing more commands yeah might be unnecessary. or alternatively, you may need to call process.kill() to allow deno to exit

venego•6mo ago

I'm thinking of a complex way that involves bash variables after each command, might be the worst way to do it tho.

raunioroo•6mo ago

yeah any solution will probably be at least a bit ugly or complex :) really it could be so much simpler if you could run one bash instance per one command, but if not, just a bit of legwork.

venego•6mo ago

How does this code smell?

const vars = {
  comandRunning: false,
}
const sleep = (t: number) => new Promise((resolve) => setTimeout(resolve, t));

listen(process.stdout, (data) => {
  if(data == 'endofcommand[uniquestufuniquestufuniquestuf]'){
    vars.comandRunning = false;
  }
  console.log("bash stdout says: " + data);
});
listen(process.stderr, (data) => {
  console.log("bash stderr says: " + data);
});
const bash = async (cmdString) => {
  while(1){
    if(!vars.commandRunnsing){
      vars.comandRunning = true;
      await writer.write(textEncoder.encode(`${cmdString}; echo "endofcommand[uniquestufuniquestufuniquestuf]"\n`));
    }

    await sleep(500);
  }
};

const vars = {
  comandRunning: false,
}
const sleep = (t: number) => new Promise((resolve) => setTimeout(resolve, t));

listen(process.stdout, (data) => {
  if(data == 'endofcommand[uniquestufuniquestufuniquestuf]'){
    vars.comandRunning = false;
  }
  console.log("bash stdout says: " + data);
});
listen(process.stderr, (data) => {
  console.log("bash stderr says: " + data);
});
const bash = async (cmdString) => {
  while(1){
    if(!vars.commandRunnsing){
      vars.comandRunning = true;
      await writer.write(textEncoder.encode(`${cmdString}; echo "endofcommand[uniquestufuniquestufuniquestuf]"\n`));
    }

    await sleep(500);
  }
};

Not too bad right? I guess I'll try to mix between oneliner and this method, and maybe gradually find new ways to split commands into smaller independent chunks. Either way, thanks a lot for helping @pyrote and @raunioroo. You saved me a lot of reading ❤️

raunioroo•6mo ago

<:hooray_deno:1035517542200004688>