zing•2mo ago

Sequential queue processing with Deno KV listenQueue?

I'm working on an API endpoint that needs to process lots of calls every second. I want to make this call asynchronous so that the API returns fast and I can offload the data to be processed in the background, so I thought of using Deno.kv queues(https://docs.deno.com/deploy/kv/manual/queue_overview/). The only issue is that there is a lot of async code inside the processing jobs, and when I queue up a lot of them they seem to overlap in processing, which causes issues when accessing shared resources(db connections specifically). Is there a way to make kv.listenQueue process each entry in sequence? This is effectively my queue code:

const kv = await Deno.openKv();

kv.listenQueue(async (msg: EvaluateRequest) => { await service.evaluate(msg) })

const kv = await Deno.openKv();

kv.listenQueue(async (msg: EvaluateRequest) => { await service.evaluate(msg) })

1 Reply

eli•2mo ago

I have the same question. You may also want to note that the docs explicitly warn about the lack of guaranteed in-order processing, so you could potentially process the requests themselves out of sequence at high throughput, if that matters to you.

Sequential queue processing with Deno KV listenQueue?

Did you find this page helpful?