Deno•3y ago

Deno KV: Should I expect 'value too large' errors inserting a JSON file with kv.set?

I was trying out kv and quickly ran into something I found surprising;

let kv = await Deno.openKv('omg.db')

let hieroglyphsJSON = await Deno.readTextFile('./hieroglyphs.json')
let hieroglyphs = JSON.parse(hieroglyphsJSON)

kv.set(["hieroglyphs"], hieroglyphs)

let kv = await Deno.openKv('omg.db')

let hieroglyphsJSON = await Deno.readTextFile('./hieroglyphs.json')
let hieroglyphs = JSON.parse(hieroglyphsJSON)

kv.set(["hieroglyphs"], hieroglyphs)

hieroglyphs.json

hieroglyphs.json

is not huge (360K), and yet I get this error:

$ deno run --unstable --allow-write --allow-read kv.js
error: Uncaught (in promise) TypeError: value too large (max 65536 bytes)
kv.set(["hieroglyphs"], hieroglyphs)
   ^
    at Kv.set (ext:deno_kv/01_db.ts:87:41)
    at file:///Users/me/kv.js:8:4
    at eventLoopTick (ext:core/01_core.js:183:11)

$ deno run --unstable --allow-write --allow-read kv.js
error: Uncaught (in promise) TypeError: value too large (max 65536 bytes)
kv.set(["hieroglyphs"], hieroglyphs)
   ^
    at Kv.set (ext:deno_kv/01_db.ts:87:41)
    at file:///Users/me/kv.js:8:4
    at eventLoopTick (ext:core/01_core.js:183:11)

I feel like I’m missing something obvious; am I trying to do something kv wasn’t designed for?

TIA

girl pisser•8/29/23, 5:10 PM

Error is obvious, you can put maximum 65536 bytes of data and apparently your data is bigger than that.

patwasalinguistOP•8/29/23, 6:00 PM

forgive my ignorance.

that's the maximum size that can be inserted as a value? that seems very small to me.

girl pisser•8/29/23, 6:05 PM

KV is still an unstable feature, so it's subject to change, maybe you can create a discussion about it on GitHub or at #general

NDH•8/29/23, 7:23 PM

There are plans for Blob storage. Also you could break that up into 64k chunks and use a key sequence to fetch (get) the chunks.

patwasalinguistOP•8/29/23, 9:28 PM

I see, thank you both