jeff.hykin•2y ago

Deno.run Strange Issue

I use Deno.run + stdout/signals/etc a lot, but this behavior is beyond my current debugging abilities I have a command nix eval ... which works fine in a python subshell, and on the commandline. However, Deno.run with piped stdout makes it hang indefinitely. To be explicit, in this block: process.status().then(callback).catch(callback) the callback doesn't run until a .kill() is used (SIGHUP/SIGINT/SIGKILL). Tested on Deno 1.29.4 and Deno 1.22.0 for good measure. What is even more strange is trying to bypass the problem with hackery also fails. All of these round-about execution methods (translated to Deno.run) also hang indefinitely: - bash -c '[command]' - bash ./file_with_command.sh - bash -i ./file_with_command.sh In particular the bash -i method not only hangs, but also exclusively makes it so that ctrl+C, ctrl+D, and even ctrl+Z are ignored; I have to kill the deno process from another termial. Env: bash -c env prints out the EXACT same env vars for Deno/Python/CLI Cwd: deno/python/cli all have the same working directory Interactivity: running it with deno repl has been the same as deno run Piping: Manually closing the stdout pipe doesn't work/help. Removing stdout: 'piped', makes command not-hang (but defeats the point). Adding stderr: 'piped' still causes a hang. I don't even know what else could be possibly causing a difference (Deno & Python code attached as comments)

7 Replies

jeff.hykin•2y ago

Here's the deno code (does need nix: https://nixos.org/download.html ):

const process = Deno.run({
    "cmd": [
        "nix",
        "eval",
        "-I", "nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz",
        "--impure",
        "--expr", "(builtins.attrNames (import <nixpkgs> {}))",
    ],
    "stdout": "piped"
})
console.debug(`process is:`,process)
console.debug(`await process.status() is:`,await process.status())

const process = Deno.run({
    "cmd": [
        "nix",
        "eval",
        "-I", "nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz",
        "--impure",
        "--expr", "(builtins.attrNames (import <nixpkgs> {}))",
    ],
    "stdout": "piped"
})
console.debug(`process is:`,process)
console.debug(`await process.status() is:`,await process.status())

And here is the python for comparison

def run(*cmd, timeout_sec=None):
    from subprocess import Popen, PIPE
    proc = Popen(cmd, stdout=PIPE, stderr=PIPE)
    stdout, stderr = proc.communicate()
    return stdout.decode('utf-8')[0:-1], stderr.decode('utf-8')[0:-1]

print(
    run(
        "nix",
        "eval",
        "-I", f'nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz',
        '--impure',
        '--expr', '(builtins.attrNames (import <nixpkgs> {}))'
    )
)

def run(*cmd, timeout_sec=None):
    from subprocess import Popen, PIPE
    proc = Popen(cmd, stdout=PIPE, stderr=PIPE)
    stdout, stderr = proc.communicate()
    return stdout.decode('utf-8')[0:-1], stderr.decode('utf-8')[0:-1]

print(
    run(
        "nix",
        "eval",
        "-I", f'nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz',
        '--impure',
        '--expr', '(builtins.attrNames (import <nixpkgs> {}))'
    )
)

And for good measure here is also Ruby code, which also doesn't hang

puts `nix eval \
    -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz \
    --impure \
    --expr '(builtins.attrNames (import <nixpkgs> {}))'`

puts `nix eval \
    -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz \
    --impure \
    --expr '(builtins.attrNames (import <nixpkgs> {}))'`

ioB•2y ago

I haven't had a chance to look into why this might be hanging, but I will point out that Deno.run is being deprecated in favor of Deno.Command which has an objectively clearer api. I will look into this when I get the chance

AapoAlas•2y ago

Deno.run does have some hanging issue as well. If memory serves then one way around it is to await Promise.all() all the three outputs of the run, not only status but stdout and stderr as well.

jeff.hykin•2y ago

I was looking into that but are there docs for Deno.Command? I expanded every section on https://deno.land/api@v1.30.3 and ctrl+f still returns nothing It also doesn't seem to exist on 1.29.4

jeff.hykin•2y ago

wow... this actually works/doesnt hang. Thanks @aapoalas

var process = Deno.run({
    "cmd": [
        "nix",
        "eval",
        "-I", "nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz",
        "--impure",
        "--expr", "(builtins.attrNames (import <nixpkgs> {}))",
    ],
    "stdout": "piped"
})

Promise.all([
    process.output(),
]).then(_=>console.log("outputs finished"))
await process.status()

var process = Deno.run({
    "cmd": [
        "nix",
        "eval",
        "-I", "nixpkgs=https://github.com/NixOS/nixpkgs/archive/aa0e8072a57e879073cee969a780e586dbe57997.tar.gz",
        "--impure",
        "--expr", "(builtins.attrNames (import <nixpkgs> {}))",
    ],
    "stdout": "piped"
})

Promise.all([
    process.output(),
]).then(_=>console.log("outputs finished"))
await process.status()

AapoAlas•2y ago

It's probably still under the "unstable" checkbox. IIRC the PR stabilising it has only just been merged or is about to be.

ioB•2y ago

It is going to be part of v1.31.0 iirc