raunioroo•10mo ago

await writer.write(buffer) never resolves on Deno.Command

Hiho. I'm creating a Deno.Command that writes a buffer to vipsthumbnail via stdin (and reads the result via stdout). It works most of the time, but sometimes gets stuck on await writer.write(source);. Source is the result of await Deno.readFile(path). The big problem is that there is no error, the writer.write() promise is just never resolved. So I have no idea how to debug, catch errors or even cancel this after timeout or something. I can process a couple of files (all with a new Deno.Command) but then after a few files it gets stuck and the promise never resolves. Any ideas how to begin solving what the problem is.

    const cmd = new Deno.Command(this.vipspath + this.vipsthumbnail, {
      args: args,
      stdin: "piped",
      stderr: "piped",
      stdout: "piped",
    }).spawn();

    const writer = cmd.stdin.getWriter();
    console.log("START WRITING", typeof source, source.byteLength);
    await writer.write(source); // <--- STUCK HERE MAY NEVER RESOLVE
    console.log("YAY! WE GOT HERE");
    await writer.ready;
    await writer.close();

    const cmd = new Deno.Command(this.vipspath + this.vipsthumbnail, {
      args: args,
      stdin: "piped",
      stderr: "piped",
      stdout: "piped",
    }).spawn();

    const writer = cmd.stdin.getWriter();
    console.log("START WRITING", typeof source, source.byteLength);
    await writer.write(source); // <--- STUCK HERE MAY NEVER RESOLVE
    console.log("YAY! WE GOT HERE");
    await writer.ready;
    await writer.close();

18 Replies

Joe Hildebrand•10mo ago

Are you sure that vipsthumbnail is actually reading stdin if stdin hasn't closed? If the output buffer fills up, writer.write() will block. you could try await Promise.all([writer.write(), writer.close()]) to find out

raunioroo•10mo ago

hm. no I'm not sure :) it appears to read, I can process a couple of files just fine. is it possible vipsthumbnail does read stdin, but then stops reading it when it encounters some kind of error parsing a certain file?

Joe Hildebrand•10mo ago

anything's possible, of course. All programs have bugs.

raunioroo•10mo ago

this doesnt seem to help with getting it unstuck

Joe Hildebrand•10mo ago

however, when vipsthumbnail exits without reading pending stdin, I would have expected await writer.write to throw.

raunioroo•10mo ago

the weird thing is that vipsthumbnail prints stuff to stderr, like _file "" does not exist, but then it still gives a proper result. maybe it first tries to open a file named "stdin", prints some errors, and only then reverts to the code path that actually reads stdin. (you are supposed to call it with "stdin" in place of the input file name argument)

jeff.hykin•10mo ago

might be unrelated but this sounds similar to this old problem https://discord.com/channels/684898665143206084/1074484890562736188/1074682730073313340

raunioroo•10mo ago

how would I go about writing a timeout to abort the writer.write() if it takes x amount of time

Joe Hildebrand•10mo ago

What if you throw cmd.status() into the Promise.all? Do you get a zero exit code?

raunioroo•10mo ago

await Promise.all([writer.write(source), writer.close(), cmd.status]); just makes it fail on the first file (same result, never resolves)

jeff.hykin•10mo ago

This uses Deno.run but it should be the same general logic https://github.com/jeff-hykin/quickr/blob/139ebd639bb474be1b0b77a3c3f273ca36c93d6d/main/run.js#L592C23-L592C23

GitHub

quickr/main/run.js at 139ebd639bb474be1b0b77a3c3f273ca36c93d6d · je...

💾 📦 Tools for Deno. Contribute to jeff-hykin/quickr development by creating an account on GitHub.

raunioroo•10mo ago

Copilot to the rescue, it wrote this nicely working timeout function, so at least I can move on when it fails.

    const timeout = new Promise((_, reject) =>
      setTimeout(() => reject(new Error("Operation timed out")), 1000)
    );
    try {
      await Promise.race([writer.write(source), timeout]);
    } catch (error) {
      console.log("writing to vipsthumbnail timed out");
      return null;
    }

    const timeout = new Promise((_, reject) =>
      setTimeout(() => reject(new Error("Operation timed out")), 1000)
    );
    try {
      await Promise.race([writer.write(source), timeout]);
    } catch (error) {
      console.log("writing to vipsthumbnail timed out");
      return null;
    }

jeff.hykin•10mo ago

did you not actually want to cancel the process on timeout?

raunioroo•10mo ago

oh yeah I should do that, close everything before returning

jeff.hykin•10mo ago

okay. if you try to kill a process that already stopped it'll get an error. The code I linked handles that case, although you'll have to integrate it with the write-timeout code