NDH•11mo ago

Kv encodes integer as double

Anyone know why Deno.kv encodes an integer in a mutli-part key as a double?

const key = ["int", 3]
encodes to: [2,105,110,116,0,33,192,8,0,0,0,0,0,0]

const key = ["int", 3]
encodes to: [2,105,110,116,0,33,192,8,0,0,0,0,0,0]

the - 33, 192, 8 above represents a Double (code 33 = Double) with value 3 It should encode to:

[2,105,110,116,0,21,3]

[2,105,110,116,0,21,3]

21, 3 represents (code 21 = integer) 3

6 Replies

ioB•11mo ago

doubles are a safe way to represent JS numbers I assume that's about as deep as the reasoning goes

NDH•11mo ago

It's very inefficient. The lib I've ported handles integers without issue. Also the rust codec seems to handle integers, but my testing shows that it is not working.

pub fn encode_key(key: &Key) -> std::io::Result<Vec<u8>> {
  let mut output: Vec<u8> = vec![];
  for part in &key.0 {
    match part {
      KeyPart::String(key) => {
        output.push(STRING);
        escape_raw_bytes_into(&mut output, key.as_bytes());
        output.push(0);
      }
      KeyPart::Int(key) => {
        bigint::encode_into(&mut output, key)?;
      }
      KeyPart::Float(key) => {
        double::encode_into(&mut output, *key);
      }
      KeyPart::Bytes(key) => {
        output.push(BYTES);
        escape_raw_bytes_into(&mut output, key);
        output.push(0);
      }
      KeyPart::False => {
        output.push(FALSE);
      }
      KeyPart::True => {
        output.push(TRUE);
      }
    }
  }
  Ok(output)
}

pub fn encode_key(key: &Key) -> std::io::Result<Vec<u8>> {
  let mut output: Vec<u8> = vec![];
  for part in &key.0 {
    match part {
      KeyPart::String(key) => {
        output.push(STRING);
        escape_raw_bytes_into(&mut output, key.as_bytes());
        output.push(0);
      }
      KeyPart::Int(key) => {
        bigint::encode_into(&mut output, key)?;
      }
      KeyPart::Float(key) => {
        double::encode_into(&mut output, *key);
      }
      KeyPart::Bytes(key) => {
        output.push(BYTES);
        escape_raw_bytes_into(&mut output, key);
        output.push(0);
      }
      KeyPart::False => {
        output.push(FALSE);
      }
      KeyPart::True => {
        output.push(TRUE);
      }
    }
  }
  Ok(output)
}

ioB•11mo ago

could you share the rest of the code in the match? I don't think this is a bug

NDH•11mo ago

Thanks for looking! I'll have to modify my FDB-codec to match this odd behaviour. I'm building a KV utility that will display the raw k: blob and v: blob from the SQLite kv table. I re-read the docs. Looks like the only 'int' supported is Bigint, 1n. I'll modify my lib to match. Thanks again. 😊 Yes changing the key to 3n does allow encoding to integer. This returns [21, 3] where (code 21 = integer, and 3 is the value) For a large data sets, this saves 7 bytes per key.

ioB•11mo ago

good to know

NDH•11mo ago

// key = ["int", "3"] <string, string>
[2,105,110,116,0,2,51,0]

// key = ["int", 3n]  <string, Bigint>
[2,105,110,116,0,21,3]

// key = ["int", 3]  <string, number>
[2,105,110,116,0,33,192,8,0,0,0,0,0,0]

// key = ["int", "3"] <string, string>
[2,105,110,116,0,2,51,0]

// key = ["int", 3n]  <string, Bigint>
[2,105,110,116,0,21,3]

// key = ["int", 3]  <string, number>
[2,105,110,116,0,33,192,8,0,0,0,0,0,0]