AapoAlas Comments - Deno

AapoAlas

•Created by Meepso on 10/19/2024 in #help

Async Unsafe Callback?

The only case when this would be possible without cooperation from the caller is when the caller performs this callback from a foreign thread: In that case we can block that foreign thread while we perform the asynchronous work on the main thread. But if the exact same call is performed on the main thread, there is no way to (unilaterally) block the main thread for the caller while still using the main thread to perform the asynchronous JavaScript work.

4 replies

•Created by Meepso on 10/19/2024 in #help

Async Unsafe Callback?

No, this is not possible as it would (in the general case) require cooperation from the caller.

4 replies

•Created by Nanashi. on 8/12/2024 in #help

Rust's `-> !` equivalent in TS

https://github.com/microsoft/TypeScript/issues/12825

4 replies

•Created by Nanashi. on 8/12/2024 in #help

Rust's `-> !` equivalent in TS

never is the closest to -> ! but https://github.com/microsoft/TypeScript/issues/15991

4 replies

•Created by BowTiedGnome on 8/11/2024 in #help

FFI on windows and handles

I believe you've made a mistake with the handling of tokenHandle parameter: In OpenProcessHandle it is an out pointer of type PHANDLE so a pointer to a HANDLE. In GetTokenInformation it is just a HANDLE but you're still passing in the same Uint8Array. You should be passing in the contents of it, so that which you've logged as token intptr

4 replies

•Created by Kuwazy on 5/1/2024 in #help

Do you know how I can call a C variadic function in FFI Deno?

Also this: https://discord.com/channels/684898665143206084/956626010248478720/1233095033935237213

4 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

Looks good to me! 🙂

21 replies

•Created by René. on 4/11/2024 in #help

Help with usage of unsafe apis in webview_deno

Hmm, @divy or @eliassjogreen (denosaurs) maybe?

6 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

Related: If I did not plug these yet, then take a look at: * Denonomicon: https://denonomicon.deno.dev/introduction * WTF is FFI? videos: https://youtu.be/9f9Ujeods5o?si=qV9YeRyI_TiHxYsK * I also spend most of this seemingly Rust talk talking about Deno FFI: https://youtu.be/QDd4Iu_-MTw?si=VdZm_TF8KnNxl7xW

21 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

Re: The WeakMap. It should actually be created outside of the function scope, so that it itself cannot be GC'd. You can then start code-golfing to something like (pseudo-code incoming because I'm on mobile):

const BUF_TO_STRING_BUFS_MAP = new WeakMap<Uint8Array, Uint8Array[]>();
function toCStringArray(strings: string[]) {
  const buf = new BigUint64Array(strings.length);
  const chunks = strings.map((str, I) => {
    const chunk = encode(str + "\0");
    buf[i] = Deno.UnsafePointer.valueOf(chunk);
    return chunk;
  });
  BUF_TO_STRINGS_BUF_MAP.set(buf, chunks);
  return new Uint8Array(buf.buffer);
}

const BUF_TO_STRING_BUFS_MAP = new WeakMap<Uint8Array, Uint8Array[]>();
function toCStringArray(strings: string[]) {
  const buf = new BigUint64Array(strings.length);
  const chunks = strings.map((str, I) => {
    const chunk = encode(str + "\0");
    buf[i] = Deno.UnsafePointer.valueOf(chunk);
    return chunk;
  });
  BUF_TO_STRINGS_BUF_MAP.set(buf, chunks);
  return new Uint8Array(buf.buffer);
}

21 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

And there is indeed an advantage to using Uint8Array as your go-to-choice whenever you pass data into FFI as "buffer": Due to reasons, Deno has to choose one buffer type to optimise for, and Uint8Array is the most versatile one we can choose so that one it is. As a result, with a trivial benchmark you should see perhaps even a 100x difference in performance between the exact same FFI call but just changing from a BigUint64Array into a Uint8Array.

21 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

Return is reusing the underlying buffer.

21 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

Your WeakMap should be a map from your buf into chunks. Right now you're saying "until ptr exists, chunk must exist." Then you immediately let the ptr object go out of scope and potentially get garbage collected. The chunk is then free to be GC'd as well.

21 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

Deno internally does fun stuff with a WeakMap in Deno.UnsafePointer.of(arrayBuffer). This returns an opaque pointer object and we put it in as a key into a WeakMap with arrayBuffer as the value. The result is that as long as the pointer object lives, the buffer memory also lives.

21 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

Alternatively, you can just encode the strings one by one and create pointers to those resulting buffers. Then, allocate an array of pointers and write the pointers into that. Finally, use a WeakMap to bind the string-buffers to stay alive as long as the pointers-buffer is alive. This gets rid of the extra copy but retains the N+1 allocations.

21 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

This does suffer from doing two memory copies for the strings instead of one. Also you're doing N+1 memory allocations where N is the number of strings. You could calculate the encoded length from the string lengths times some constant. Any constant below 3 (IIRC) does mean that it's theoretically possible for the estimated encoded length to be too small, but that is fairly unlikely. This would mean no extra memory copy and only one allocation. If the buffer ends up being too small, you can detect that based on the encoder's return value (bytes read, bytes written) and create a new, larger buffer to replace the first one. This would be an extra memory copy but it's very, very unlikely to happen.

21 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

And here's a usage example: https://github.com/aapoalas/libclang_deno/blob/5d6d8ed33627c6f12949efc15c3f7ccf07f26b3a/lib/mod.ts#L364-L380

21 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

The basic idea is this: Create a buffer that is large enough to hold params.length pointers plus all of the param strings encoded. Then encode the strings into the buffer and write in pointers that point to the start of each string (so essentially self-references; the buffer contains pointers that point within the buffer).

21 replies

•Created by carragom on 4/9/2024 in #help

How to go from a JS string[] to a C array of pointers

Here's an example of how I pass string args: https://github.com/aapoalas/libclang_deno/blob/main/lib/utils.ts#L9-L46

21 replies

•Created by kitty on 4/8/2024 in #help

Deno.UnsafeCallback Error (bypass string from c++ to deno)

No problem. Let me know if you find any more questions about FFI 🙂 Also if you didn't know yet, check out Denonomicon, and the WTF is FFI videos on Youtube

9 replies