ReallyBored•11mo ago

Problem with parsing `base64url` from a `Buffer (node)`

help , it works in bun but not in Deno, I don't know if is a bug from them or it's not implemented here,it also works in node:

import BufferProto from "node:buffer"

 BufferProto.Buffer.from(
    "IntcImhlbGxvXCI6XCJoZGQvZStpXCJ9Ig"
  ,
    "base64url"
 ).toString() 

//deno: ""
//bun: "{\"hello\":\"hdd/e+i\"}"
//node:"{\"hello\":\"hdd/e+i\"}"

import BufferProto from "node:buffer"

 BufferProto.Buffer.from(
    "IntcImhlbGxvXCI6XCJoZGQvZStpXCJ9Ig"
  ,
    "base64url"
 ).toString() 

//deno: ""
//bun: "{\"hello\":\"hdd/e+i\"}"
//node:"{\"hello\":\"hdd/e+i\"}"

8 Replies

Deno AI Helper•11mo ago

Based on the documentation, Deno does not have built-in support for the base64url encoding in the Buffer.from() method. The Buffer.from() method in Node.js does not support base64url encoding either. The discrepancy you're seeing might be due to differences in how the Buffer.from() method is implemented in bun, Node.js, and Deno. In Deno, you can use the built-in TextDecoder and atob functions to decode base64 strings. However, these functions do not support base64url directly. You would need to convert the base64url string to a regular base64 string first. Here's how you can do it:

const base64url = "IntcImhlbGxvXCI6XCJoZGQvZStpXCJ9Ig";
const base64 = base64url.replace(/-/g, '+').replace(/_/g, '/');
const decoded = atob(base64);
console.log(decoded);

const base64url = "IntcImhlbGxvXCI6XCJoZGQvZStpXCJ9Ig";
const base64 = base64url.replace(/-/g, '+').replace(/_/g, '/');
const decoded = atob(base64);
console.log(decoded);

Please note that this information is not explicitly stated in the documentation, but it's a common way to handle base64url in JavaScript.

Helpful Links: node: specifiers | Manual | Deno Deno 1.7 Release Notes File System API | Deploy Docs A list of every web API in Deno

ReallyBored•11mo ago

That's actually incorrect, the official types includes 'base64url'

Deno AI Helper•11mo ago

I apologize for the confusion. The official Node.js documentation does not list 'base64url' as a valid encoding for the Buffer.from() method. However, if 'base64url' is working in your Node.js environment, it might be due to a third-party library or a custom implementation that extends the native Buffer functionality. In Deno, you might need to manually convert the 'base64url' string to a regular base64 string before decoding it. Here's how you can do it:

const base64url = "IntcImhlbGxvXCI6XCJoZGQvZStpXCJ9Ig";
const base64 = base64url.replace(/-/g, '+').replace(/_/g, '/');
const decoded = atob(base64);
console.log(decoded);

const base64url = "IntcImhlbGxvXCI6XCJoZGQvZStpXCJ9Ig";
const base64 = base64url.replace(/-/g, '+').replace(/_/g, '/');
const decoded = atob(base64);
console.log(decoded);

This code replaces '-' with '+' and '_' with '/' in the 'base64url' string, converting it to a regular base64 string. Then it uses the atob() function to decode the base64 string.

Helpful Links: Hex and Base64 Encoding - Deno by Example A list of every web API in Deno node: specifiers | Manual | Deno Hashing - Deno by Example

ReallyBored•11mo ago

Well, I will need to make a deno's only implementation that will be slower than bun's

pyrote•11mo ago

I'm seeing the same bug with Buffer.from and base64url. Its returning a buffer that is mostly zeros. The AI doesn't know that Deno has a module for Base64URL.

import { encode as encodeBase64url, decode as decodeBase64url } from "https://deno.land/std/encoding/base64url.ts";

const text_decode = (d) => new TextDecoder().decode(new Uint8Array(d));

const url_str = 'IntcImhlbGxvXCI6XCJoZGQvZStpXCJ9Ig';
const buffer = decodeBase64url(url_str);
const buffer_str = text_decode(buffer);
console.log(`Text ${buffer_str}`);

const text = '"{\\"hello\\":\\"hdd/e+i\\"}"';
const base64_text = encodeBase64url(text);
console.log(`Base64URL ${base64_text}`);

import { encode as encodeBase64url, decode as decodeBase64url } from "https://deno.land/std/encoding/base64url.ts";

const text_decode = (d) => new TextDecoder().decode(new Uint8Array(d));

const url_str = 'IntcImhlbGxvXCI6XCJoZGQvZStpXCJ9Ig';
const buffer = decodeBase64url(url_str);
const buffer_str = text_decode(buffer);
console.log(`Text ${buffer_str}`);

const text = '"{\\"hello\\":\\"hdd/e+i\\"}"';
const base64_text = encodeBase64url(text);
console.log(`Base64URL ${base64_text}`);

ReallyBored•11mo ago

That would do, I hope is fast because the other implementation works but it's super slow

lcasdev•11mo ago

Hey folks, do you mind opening a bug about this? We'll fix it!

ReallyBored•11mo ago

I will indeed! done!