jeff.hykin
jeff.hykin17mo ago

Deno.run Strange Issue

I use Deno.run + stdout/signals/etc a lot, but this behavior is beyond my current debugging abilities I have a command nix eval ... which works fine in a python subshell, and on the commandline. However, Deno.run with piped stdout makes it hang indefinitely. To be explicit, in this block: process.status().then(callback).catch(callback) the callback doesn't run until a .kill() is used (SIGHUP/SIGINT/SIGKILL). Tested on Deno 1.29.4 and Deno 1.22.0 for good measure. What is even more strange is trying to bypass the problem with hackery also fails. All of these round-about execution methods (translated to Deno.run) also hang indefinitely: - bash -c '[command]' - bash ./file_with_command.sh - bash -i ./file_with_command.sh In particular the bash -i method not only hangs, but also exclusively makes it so that ctrl+C, ctrl+D, and even ctrl+Z are ignored; I have to kill the deno process from another termial. Env: bash -c env prints out the EXACT same env vars for Deno/Python/CLI Cwd: deno/python/cli all have the same working directory Interactivity: running it with deno repl has been the same as deno run Piping: Manually closing the stdout pipe doesn't work/help. Removing stdout: 'piped', makes command not-hang (but defeats the point). Adding stderr: 'piped' still causes a hang. I don't even know what else could be possibly causing a difference (Deno & Python code attached as comments)
7 Replies
jeff.hykin
jeff.hykin17mo ago
Here's the deno code (does need nix: https://nixos.org/download.html ):
const process = Deno.run({
"cmd": [
"nix",
"eval",
"-I", "nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz",
"--impure",
"--expr", "(builtins.attrNames (import <nixpkgs> {}))",
],
"stdout": "piped"
})
console.debug(`process is:`,process)
console.debug(`await process.status() is:`,await process.status())
const process = Deno.run({
"cmd": [
"nix",
"eval",
"-I", "nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz",
"--impure",
"--expr", "(builtins.attrNames (import <nixpkgs> {}))",
],
"stdout": "piped"
})
console.debug(`process is:`,process)
console.debug(`await process.status() is:`,await process.status())
And here is the python for comparison
def run(*cmd, timeout_sec=None):
from subprocess import Popen, PIPE
proc = Popen(cmd, stdout=PIPE, stderr=PIPE)
stdout, stderr = proc.communicate()
return stdout.decode('utf-8')[0:-1], stderr.decode('utf-8')[0:-1]

print(
run(
"nix",
"eval",
"-I", f'nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz',
'--impure',
'--expr', '(builtins.attrNames (import <nixpkgs> {}))'
)
)
def run(*cmd, timeout_sec=None):
from subprocess import Popen, PIPE
proc = Popen(cmd, stdout=PIPE, stderr=PIPE)
stdout, stderr = proc.communicate()
return stdout.decode('utf-8')[0:-1], stderr.decode('utf-8')[0:-1]

print(
run(
"nix",
"eval",
"-I", f'nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz',
'--impure',
'--expr', '(builtins.attrNames (import <nixpkgs> {}))'
)
)
And for good measure here is also Ruby code, which also doesn't hang
puts `nix eval \
-I nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz \
--impure \
--expr '(builtins.attrNames (import <nixpkgs> {}))'`
puts `nix eval \
-I nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz \
--impure \
--expr '(builtins.attrNames (import <nixpkgs> {}))'`
ioB
ioB17mo ago
I haven't had a chance to look into why this might be hanging, but I will point out that Deno.run is being deprecated in favor of Deno.Command which has an objectively clearer api. I will look into this when I get the chance
AapoAlas
AapoAlas17mo ago
Deno.run does have some hanging issue as well. If memory serves then one way around it is to await Promise.all() all the three outputs of the run, not only status but stdout and stderr as well.
jeff.hykin
jeff.hykin17mo ago
I was looking into that but are there docs for Deno.Command? I expanded every section on https://deno.land/api@v1.30.3 and ctrl+f still returns nothing It also doesn't seem to exist on 1.29.4
jeff.hykin
jeff.hykin17mo ago
wow... this actually works/doesnt hang. Thanks @aapoalas
var process = Deno.run({
"cmd": [
"nix",
"eval",
"-I", "nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz",
"--impure",
"--expr", "(builtins.attrNames (import <nixpkgs> {}))",
],
"stdout": "piped"
})

Promise.all([
process.output(),
]).then(_=>console.log("outputs finished"))
await process.status()
var process = Deno.run({
"cmd": [
"nix",
"eval",
"-I", "nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz",
"--impure",
"--expr", "(builtins.attrNames (import <nixpkgs> {}))",
],
"stdout": "piped"
})

Promise.all([
process.output(),
]).then(_=>console.log("outputs finished"))
await process.status()
AapoAlas
AapoAlas17mo ago
It's probably still under the "unstable" checkbox. IIRC the PR stabilising it has only just been merged or is about to be.
ioB
ioB17mo ago
It is going to be part of v1.31.0 iirc