NeTT•3y ago

Are pointers supported in Fast API return types?

If no, how do I return pointers?

31 Replies

Unknown User•3y ago

Message Not Public

NeTTOP•3y ago

Nevermind I realized that I can just cast the pointer to an integer and make it work From what I've seen, it was easier to pass the typed array as a parameter and let the native code do the allocation directly into the typed array now I have a new question. How do I know my FFI call is utilising Fast API?

Unknown User•3y ago

Message Not Public

NeTTOP•3y ago

#[no_mangle]
pub unsafe extern "C" fn no_op() -> isize {
  -97
}

#[no_mangle]
pub unsafe extern "C" fn no_op() -> isize {
  -97
}

With

no_op: {
  parameters: [],
  result: "i64",
},

no_op: {
  parameters: [],
  result: "i64",
},

Takes 26.3ns on average. While

no_op: {
  parameters: [],
  result: "pointer",
},

no_op: {
  parameters: [],
  result: "pointer",
},

Takes 16-17ns on average.

Unknown User•3y ago

Message Not Public

NeTTOP•3y ago

no_op_i64    26.49 ns/iter   (24.97 ns … 70.14 ns)  25.66 ns  43.22 ns  48.23 ns

no_op_ptr    17.58 ns/iter   (16.54 ns … 45.23 ns)  17.03 ns  27.49 ns  28.51 ns

no_op_i64    26.49 ns/iter   (24.97 ns … 70.14 ns)  25.66 ns  43.22 ns  48.23 ns

no_op_ptr    17.58 ns/iter   (16.54 ns … 45.23 ns)  17.03 ns  27.49 ns  28.51 ns

copypasted from deno bench returning 97 instead of -97 is twice as fast btw

Unknown User•3y ago

Message Not Public

NeTTOP•3y ago

no idea an ACTUAL no-op takes like 4ns

NeTTOP•3y ago

with

#[no_mangle]
pub unsafe extern "C" fn no_op() {
}

#[no_mangle]
pub unsafe extern "C" fn no_op() {
}

Unknown User•3y ago

Message Not Public

NeTTOP•3y ago

and returntype void when defining symbols

Unknown User•3y ago

Message Not Public

NeTTOP•3y ago

Ig this is actually going through fast api considering my older no-op calls (from a few weeks ago) were taking around 40ns

AapoAlas•3y ago

Pointers are supported as return types, as are 64 bit integers. Under the hood it works so that an extra buffer argument is passed into the call, our trampoline writes the pointer / integer into the buffer, and then on JS side we check (for positive integers) if the number has no upper bits in which case we return the lower 32 bits as a number. Otherwise, we take a BigInt out of the buffer and check if it can be safely represented as a number. If it can, we return a number. Else, the BigInt.

NeTTOP•3y ago

so uhh can I make an FFI call explicitly use the slow API? for comparison

AapoAlas•3y ago

Yes, mark it with callback: true. Fast API will throw if you try to call into JS so we need a way to mark functions that may call back and use slow calls for those.

NeTTOP•3y ago

yea thanks that works can see a 10x difference between the fast call and the slow call

AapoAlas•3y ago

You can also force a deopt to slow by using eg. null or Uint32Array for a buffer type, or null for pointer etc.

NeTTOP•3y ago

NeTTOP•3y ago

(that 3.61ns was a fluke) or wait

NeTTOP•3y ago

NeTTOP•3y ago

becomes slower when called after a dummy bench?

AapoAlas•3y ago

Bench has some issues with the very first bench call.

NeTTOP•3y ago

btw, could something here cause the call to use the ops layer instead of the fast api?

call_fn: {
      parameters: ["buffer", "buffer", "u32", "u32"],
      result: "pointer",
},

call_fn: {
      parameters: ["buffer", "buffer", "u32", "u32"],
      result: "pointer",
},

AapoAlas•3y ago

If either of the first two parameters is not a Uint8Array then it'll deopt to the ops layer. (Well, not quite ops layer but similar.)

NeTTOP•3y ago

theyre both technically Uint8Arrays

const x_for_ffi = new Uint8Array(x.buffer);
const y_for_ffi = new Uint8Array(y.buffer);

const x_for_ffi = new Uint8Array(x.buffer);
const y_for_ffi = new Uint8Array(y.buffer);

these two variables are the first two params

AapoAlas•3y ago

Okay, that should be fine. Though do note that it's been seen elsewhere that doing new Uint8Array(buffer) can be surprisingly slow. If you can move it up out of the loop / bench it will probably help a lot.

NeTTOP•3y ago

actually now that I see it, same params but I removed the function body from the rust function

NeTTOP•3y ago

without removing function body from rust function

NeTTOP•3y ago

okay seems some loops were taking too much time I was iterating through a slice of 10k numbers 4 times so it took 40 microseconds for each of them. Like this:

while i < n {
    mean_x += x[i];
    mean_y += y[i];
    i += 1;
}

while i < n {
    mean_x += x[i];
    mean_y += y[i];
    i += 1;
}

so in the end, the call was actually going through fast API but the actual program ended up being slow Works fine now my dumbass didn't set opt-level when compiling with rustc