EthanThatOneKid•14mo ago

Deno Deploy Queue trouble in production

I'm encountering an issue with Deno Queue in my Deno Deploy project, and I'm hoping to get some guidance on it. Here's a brief summary of the problem: Problem description: In my Deno Deploy project, I'm trying to use Deno Queue, but it's not behaving as expected in production. During development, I received a log message stating that Deno Queue features are enabled in production, but I'm unable to get it to work as intended in the production environment. Code Sample: Here's the code I've written for this functionality:

import type { ShorterOptions } from "shorter/lib/shorter/mod.ts";
import { shorter } from "shorter/lib/shorter/mod.ts";

/**
 * TTLMessage is a message received from the TTL channel.
 */
export interface TTLMessage {
  channel: "ttl";
  data: {
    alias: string;
    actor: ShorterOptions["actor"];
  };
}

/**
 * isTTLMessage checks if the message is a TTL message.
 */
export function isTTLMessage(m: unknown): m is TTLMessage {
  return (m as TTLMessage).channel === "ttl";
}

/**
 * makeTTLMessageListener makes a TTL message listener.
 */
export function makeTTLMessageListener(
  githubPAT: string,
) {
  /**
   * listenToTTLChannel listens to the TTL channel.
   */
  return async function listenToTTLChannel(m: unknown) {
    if (!isTTLMessage(m)) {
      return;
    }

    // Remove the shortlink from persisted storage.
    await shorter({
      githubPAT,
      actor: m.data.actor,
      // Omitting the destination will remove the alias.
      data: { alias: m.data.alias },
    })
      .catch((error) => {
        if (error instanceof Error) {
          console.error(error);
        }
      });
  };
}

/**
 * addTTLMessage adds a TTL message to the TTL channel.
 *
 * See:
 * https://docs.deno.com/kv/manual/queue_overview#queues-on-deno-deploy
 */
export async function addTTLMessage(
  kv: Deno.Kv,
  data: TTLMessage["data"],
  delay: number,
) {
  return await kv.enqueue(
    { channel: "ttl", data },
    { delay },
  );
}

import type { ShorterOptions } from "shorter/lib/shorter/mod.ts";
import { shorter } from "shorter/lib/shorter/mod.ts";

/**
 * TTLMessage is a message received from the TTL channel.
 */
export interface TTLMessage {
  channel: "ttl";
  data: {
    alias: string;
    actor: ShorterOptions["actor"];
  };
}

/**
 * isTTLMessage checks if the message is a TTL message.
 */
export function isTTLMessage(m: unknown): m is TTLMessage {
  return (m as TTLMessage).channel === "ttl";
}

/**
 * makeTTLMessageListener makes a TTL message listener.
 */
export function makeTTLMessageListener(
  githubPAT: string,
) {
  /**
   * listenToTTLChannel listens to the TTL channel.
   */
  return async function listenToTTLChannel(m: unknown) {
    if (!isTTLMessage(m)) {
      return;
    }

    // Remove the shortlink from persisted storage.
    await shorter({
      githubPAT,
      actor: m.data.actor,
      // Omitting the destination will remove the alias.
      data: { alias: m.data.alias },
    })
      .catch((error) => {
        if (error instanceof Error) {
          console.error(error);
        }
      });
  };
}

/**
 * addTTLMessage adds a TTL message to the TTL channel.
 *
 * See:
 * https://docs.deno.com/kv/manual/queue_overview#queues-on-deno-deploy
 */
export async function addTTLMessage(
  kv: Deno.Kv,
  data: TTLMessage["data"],
  delay: number,
) {
  return await kv.enqueue(
    { channel: "ttl", data },
    { delay },
  );
}

In this code, I've implemented functions for handling TTL (Time To Live) messages using Deno Queue. However, when I try to use this code in production, it doesn't work as expected. I'm wondering if anyone has experienced a similar issue or if there's something specific I should be aware of when using Deno Queue in a production environment. Thanks in advance!

10 Replies

EthanThatOneKidOP•14mo ago

I have been referencing the recent blog post https://deno.com/blog/queues and all the links within it.

Deno Blog

Announcing Deno Queues

Introducing Deno Queues - zero config, scalable messaging with a guaranteed at-least-once delivery. This new primitive builds on the foundation set by Deno KV, and is available today in the Deno JavaScript runtime and Deno Deploy.

cknight•14mo ago

Can you elaborate on what isn't working as expected?

EthanThatOneKidOP•14mo ago

It never executes. I tested with various delays.

cknight•14mo ago

Does it deliver without a delay? And does it work locally?

EthanThatOneKidOP•14mo ago

if (import.meta.main) {
  await main();
}

/**
 * main is the entrypoint for the Shorter application command.
 */
export async function main() {
  // Set up queue listener.
  const kv = await Deno.openKv();
  kv.listenQueue(makeTTLMessageListener(GITHUB_TOKEN));

  // Start the server.
  Deno.serve(
    { port: PORT, onListen },
    makeHandler(kv),
  );
}

if (import.meta.main) {
  await main();
}

/**
 * main is the entrypoint for the Shorter application command.
 */
export async function main() {
  // Set up queue listener.
  const kv = await Deno.openKv();
  kv.listenQueue(makeTTLMessageListener(GITHUB_TOKEN));

  // Start the server.
  Deno.serve(
    { port: PORT, onListen },
    makeHandler(kv),
  );
}

No delivery. I’ll double check if it works locally.

cknight•14mo ago

I think you can also hoist all the main() code to top level, no need for await main, etc. Not sure that's your problem, but just a pointer. Anyway, I'd add console.log messages after kv.listenQueue to validate it was setup properly and also after kv.enqueue to validate the message was sent. Check your Deploy logs tab as well for any errors.

EthanThatOneKidOP•14mo ago

Thanks for the pointers. I’ll start on a little repro repo to help clear up any hurdles for myself and anyone else who it may be of use to.

EthanThatOneKidOP•14mo ago

After some experimenting, it’s possible to establish a Deno Queue in the top-level scope, in an abstracted main function, and with an in-memory kv instance, all with and without delays. Here are the manually-testable examples: https://github.com/EthanThatOneKid/deno_queues/tree/main/examples

GitHub

deno_queues/examples at main · EthanThatOneKid/deno_queues

🦕 Example usage of https://deno.com/blog/queues! Contribute to EthanThatOneKid/deno_queues development by creating an account on GitHub.

igorzi•14mo ago

@EthanThatOneKid (He/Him) - are you not able to get queue messages delivered on Deploy? Can you please DM me your Deploy project?

EthanThatOneKidOP•14mo ago

Wow thank you for following up with me in DMs! At the time of opening this forum post something was fishy, but now all seems to be working as expected. For me, this forum post is resolved. cc @igorzi