NDH
NDH11mo ago

Kv encodes integer as double

Anyone know why Deno.kv encodes an integer in a mutli-part key as a double?
const key = ["int", 3]
encodes to: [2,105,110,116,0,33,192,8,0,0,0,0,0,0]
const key = ["int", 3]
encodes to: [2,105,110,116,0,33,192,8,0,0,0,0,0,0]
the - 33, 192, 8 above represents a Double (code 33 = Double) with value 3 It should encode to:
[2,105,110,116,0,21,3]
[2,105,110,116,0,21,3]
21, 3 represents (code 21 = integer) 3
6 Replies
ioB
ioB11mo ago
doubles are a safe way to represent JS numbers I assume that's about as deep as the reasoning goes
NDH
NDH11mo ago
It's very inefficient. The lib I've ported handles integers without issue. Also the rust codec seems to handle integers, but my testing shows that it is not working.
pub fn encode_key(key: &Key) -> std::io::Result<Vec<u8>> {
let mut output: Vec<u8> = vec![];
for part in &key.0 {
match part {
KeyPart::String(key) => {
output.push(STRING);
escape_raw_bytes_into(&mut output, key.as_bytes());
output.push(0);
}
KeyPart::Int(key) => {
bigint::encode_into(&mut output, key)?;
}
KeyPart::Float(key) => {
double::encode_into(&mut output, *key);
}
KeyPart::Bytes(key) => {
output.push(BYTES);
escape_raw_bytes_into(&mut output, key);
output.push(0);
}
KeyPart::False => {
output.push(FALSE);
}
KeyPart::True => {
output.push(TRUE);
}
}
}
Ok(output)
}
pub fn encode_key(key: &Key) -> std::io::Result<Vec<u8>> {
let mut output: Vec<u8> = vec![];
for part in &key.0 {
match part {
KeyPart::String(key) => {
output.push(STRING);
escape_raw_bytes_into(&mut output, key.as_bytes());
output.push(0);
}
KeyPart::Int(key) => {
bigint::encode_into(&mut output, key)?;
}
KeyPart::Float(key) => {
double::encode_into(&mut output, *key);
}
KeyPart::Bytes(key) => {
output.push(BYTES);
escape_raw_bytes_into(&mut output, key);
output.push(0);
}
KeyPart::False => {
output.push(FALSE);
}
KeyPart::True => {
output.push(TRUE);
}
}
}
Ok(output)
}
ioB
ioB11mo ago
could you share the rest of the code in the match? I don't think this is a bug
NDH
NDH11mo ago
Thanks for looking! I'll have to modify my FDB-codec to match this odd behaviour. I'm building a KV utility that will display the raw k: blob and v: blob from the SQLite kv table. I re-read the docs. Looks like the only 'int' supported is Bigint, 1n. I'll modify my lib to match. Thanks again. 😊 Yes changing the key to 3n does allow encoding to integer. This returns [21, 3] where (code 21 = integer, and 3 is the value) For a large data sets, this saves 7 bytes per key.
ioB
ioB11mo ago
good to know
NDH
NDH11mo ago
// key = ["int", "3"] <string, string>
[2,105,110,116,0,2,51,0]

// key = ["int", 3n] <string, Bigint>
[2,105,110,116,0,21,3]

// key = ["int", 3] <string, number>
[2,105,110,116,0,33,192,8,0,0,0,0,0,0]
// key = ["int", "3"] <string, string>
[2,105,110,116,0,2,51,0]

// key = ["int", 3n] <string, Bigint>
[2,105,110,116,0,21,3]

// key = ["int", 3] <string, number>
[2,105,110,116,0,33,192,8,0,0,0,0,0,0]