zing•2mo ago

Deno Kv: "Database is locked" on enqueue

I'm running 2 Deno services. Service 1 gathers info and posts it to service 2 via HTTP post, where 2 immediately offloads the request to a Deno.kv queue and returns. There are a lot of posts being made, and once a couple thousand events are queued I'm noticing that inconsistently service 2 will throw the following error:

[uncaught application error]: TypeError - database is locked

request: {
  url: "http://localhost:8080/api/events",
  method: "POST",
  hasBody: true
}
response: { status: 500, type: "text", hasBody: true, writable: true }

    at async doAtomicWriteInPlace (ext:deno_kv/01_db.ts:629:10)
    at async Kv.enqueue (ext:deno_kv/01_db.ts:163:26)
    at async file:///<my_filepath>/src/api/events.controller.ts:13:5
    at async dispatch (https://jsr.io/@oak/oak/17.0.0/middleware.ts:90:7)
    at async dispatch (https://jsr.io/@oak/oak/17.0.0/middleware.ts:90:7)
    at async dispatch (https://jsr.io/@oak/oak/17.0.0/middleware.ts:90:7)
    at async default (file:///<my_filepath>/src/middleware/errorhandling.middleware.ts:9:9)
    at async dispatch (https://jsr.io/@oak/oak/17.0.0/middleware.ts:90:7)
    at async Application.#handleRequest (https://jsr.io/@oak/oak/17.0.0/application.ts:605:9)

[uncaught application error]: TypeError - database is locked

request: {
  url: "http://localhost:8080/api/events",
  method: "POST",
  hasBody: true
}
response: { status: 500, type: "text", hasBody: true, writable: true }

    at async doAtomicWriteInPlace (ext:deno_kv/01_db.ts:629:10)
    at async Kv.enqueue (ext:deno_kv/01_db.ts:163:26)
    at async file:///<my_filepath>/src/api/events.controller.ts:13:5
    at async dispatch (https://jsr.io/@oak/oak/17.0.0/middleware.ts:90:7)
    at async dispatch (https://jsr.io/@oak/oak/17.0.0/middleware.ts:90:7)
    at async dispatch (https://jsr.io/@oak/oak/17.0.0/middleware.ts:90:7)
    at async default (file:///<my_filepath>/src/middleware/errorhandling.middleware.ts:9:9)
    at async dispatch (https://jsr.io/@oak/oak/17.0.0/middleware.ts:90:7)
    at async Application.#handleRequest (https://jsr.io/@oak/oak/17.0.0/application.ts:605:9)

the controller handling these calls looks like the following:

eventsRouter.post("/", async (ctx) => {
    const data: Dto = await ctx.request.body.json();

    await kv.enqueue({ type: "handle_event", details: data }, cfg.KV_OPTIONS);

    ctx.response.status = 200;
});

eventsRouter.post("/", async (ctx) => {
    const data: Dto = await ctx.request.body.json();

    await kv.enqueue({ type: "handle_event", details: data }, cfg.KV_OPTIONS);

    ctx.response.status = 200;
});

It doesn't seem to be breaking consistently after 'x amount of queued items' or anything like that, from what I can see. I'm not quite sure what is causing it, perhaps anyone else has some experience with this? Thanks for any help

2 Replies

zingOP•2mo ago

Apparently this happens when multiple processes try access the kv db file at the same time. I'm not sure how to prevent it as of yet. For now I've gotten around it by implementing a retry mechanism for the enqueue call, it just attempts to enqueue in a try catch within a for loop a couple times. I haven't run into a case where it fails to enqueue more than once so this seems to be fine for what I need for now.

CodyC•2mo ago

I'd recommend opening a bug for this. Sounds like a bug in their local Kv implementation.

Deno Kv: "Database is locked" on enqueue

Did you find this page helpful?