D
Deno

help

Deno KV: Should I expect 'value too large' errors inserting a JSON file with kv.set?

Ppatwasalinguist8/29/2023
I was trying out kv and quickly ran into something I found surprising;
let kv = await Deno.openKv('omg.db')

let hieroglyphsJSON = await Deno.readTextFile('./hieroglyphs.json')
let hieroglyphs = JSON.parse(hieroglyphsJSON)

kv.set(["hieroglyphs"], hieroglyphs)
let kv = await Deno.openKv('omg.db')

let hieroglyphsJSON = await Deno.readTextFile('./hieroglyphs.json')
let hieroglyphs = JSON.parse(hieroglyphsJSON)

kv.set(["hieroglyphs"], hieroglyphs)
hieroglyphs.json is not huge (360K), and yet I get this error:
$ deno run --unstable --allow-write --allow-read kv.js
error: Uncaught (in promise) TypeError: value too large (max 65536 bytes)
kv.set(["hieroglyphs"], hieroglyphs)
^
at Kv.set (ext:deno_kv/01_db.ts:87:41)
at file:///Users/me/kv.js:8:4
at eventLoopTick (ext:core/01_core.js:183:11)
$ deno run --unstable --allow-write --allow-read kv.js
error: Uncaught (in promise) TypeError: value too large (max 65536 bytes)
kv.set(["hieroglyphs"], hieroglyphs)
^
at Kv.set (ext:deno_kv/01_db.ts:87:41)
at file:///Users/me/kv.js:8:4
at eventLoopTick (ext:core/01_core.js:183:11)
I feel like I’m missing something obvious; am I trying to do something kv wasn’t designed for? TIA
Mmeow :38/29/2023
Error is obvious, you can put maximum 65536 bytes of data and apparently your data is bigger than that.
Ppatwasalinguist8/29/2023
forgive my ignorance. that's the maximum size that can be inserted as a value? that seems very small to me.
Mmeow :38/29/2023
KV is still an unstable feature, so it's subject to change, maybe you can create a discussion about it on GitHub or at #general
ABAltair 680b8/29/2023
There are plans for Blob storage. Also you could break that up into 64k chunks and use a key sequence to fetch (get) the chunks.
// foreach chunk - index++
kv.set(["hieroglyphs", "chunks", index], chunk)
// then
const iter = kv.list<string>({ prefix: ["hieroglyphs"] });
// foreach chunk - index++
kv.set(["hieroglyphs", "chunks", index], chunk)
// then
const iter = kv.list<string>({ prefix: ["hieroglyphs"] });
Ppatwasalinguist8/29/2023
I see, thank you both

Looking for more? Join the community!

Recommended Posts
Deno language server stops working in VS CodeAfter some random time the language server stops working and I suddenly get code errors like this: `Getting free disk space without using child_process.Hello everyone! I would like to create a small CLI tool using Typescript and Deno. I would like to mtest --filter a stepCan I only run a certain step in the test?fastwebsockets how to end client calls and avoid Error in websocket connection Unexpected EOFWhat is the issue in this client code that causes the error? ```rust pub async fn connect() -> Resuldeno compile --include argumentIn `deno compile`, how can i include a whole directory instead of just a file? Also, is there a way `deno run` as a library`deno run` does a lot of useful things behind the scenes, many of which are not exposed as a libraryPermissions with npm/node importsI'm trying to run a command and it's like I need to add `--allow-read` permissions because of the `nBroadcastChannelWhen using BroadcastChannel api, in the background does deno send the broadcast to every instance orIs it possible to create a TCP server on Deno Deploy?Is it possible to create a TCP server on Deno Deploy?oak server can't accept requests from another server?hi, I'm trying out deno with oak to make a simple server. I set up cors(x/cors), it accepts requestsaddEventListener for keydown in terminal via deno?Is there something like `addEventlistener("keydown")` in deno, so i can capture terminal inputs?Deno KV Type ErrorWhen I run await (await kv.get<typeThing>(["example"])).id (the value is an object), it returns a ty