Mike Wilkerson
Mike Wilkerson
DDeno
Created by David Alsh on 1/29/2024 in #help
How to call a JS function from Rust?
Something that's still bugging me about this is memory management. call_with_args requires that the arguments are &[Global<Value>]. And in the example code, getting a global seems to involve: 1. creating a local 2. promoting it to global But does that then mean it's not eligible for garbage collection? In my case, this function calling would happen many times in a long-running process, where the range of possible args is unlimited. So if every invocation of the function involves creating globals that won't be garbage collected, doesn't that mean memory leak?
15 replies
DDeno
Created by David Alsh on 1/29/2024 in #help
How to call a JS function from Rust?
Ok, it looks to me like what I did with lexical scoping is the same solution as what's in those gists. Only those are cleaner overall. The final gist has this:
let request = {
let mut scope = &mut runtime.handle_scope();
let request = serde_v8::to_v8(scope, "asdsd").unwrap();
Global::new(&mut scope, request)
};
let call = runtime.call_with_args(&function, &[request]);
let request = {
let mut scope = &mut runtime.handle_scope();
let request = serde_v8::to_v8(scope, "asdsd").unwrap();
Global::new(&mut scope, request)
};
let call = runtime.call_with_args(&function, &[request]);
The implementation of call_with_args is where the second mutable borrow happens: it also invokes handle_scope. I'd pulled that apart because I thought (not yet understanding scopes), that it should be the same scope used in both places. So I could clean mine up to look more like that, but in principle, both have these two features: 1. using lexical scopes to fix the double mutable borrow 2. invoking handle_scope() twice I think I want to consider that confirmation ...?
15 replies
DDeno
Created by David Alsh on 1/29/2024 in #help
How to call a JS function from Rust?
For now, I got it to build and run by enclosing the first mutable borrow within a lexical scope and having it return the Future from the call_with_args. Then in a second lexical scope, do the run_event_loop. Like this:
let fut = {
let mut scope = worker.js_runtime.handle_scope();
// ...
JsRuntime::scoped_call_with_args(&mut scope, &fn_global, &args)
};

let string_value = {
worker.run_event_loop(false).await?;
let result = fut.await?;
let mut scope = worker.js_runtime.handle_scope();
let local = deno_core::v8::Local::new(&mut scope, result);
// The function I'm calling returns a `String`.
deno_core::serde_v8::from_v8::<String>(&mut scope, local)?
};
let fut = {
let mut scope = worker.js_runtime.handle_scope();
// ...
JsRuntime::scoped_call_with_args(&mut scope, &fn_global, &args)
};

let string_value = {
worker.run_event_loop(false).await?;
let result = fut.await?;
let mut scope = worker.js_runtime.handle_scope();
let local = deno_core::v8::Local::new(&mut scope, result);
// The function I'm calling returns a `String`.
deno_core::serde_v8::from_v8::<String>(&mut scope, local)?
};
This does get the result I want. But I don't yet understand what's happening with scopes well enough to reason about the fact that a separate scope is used in the second block. It seems like that might be a problem, but I'm not sure what that would be. Still learning...
15 replies
DDeno
Created by David Alsh on 1/29/2024 in #help
How to call a JS function from Rust?
Oh, and related, the docs for call_with_args say:
The event loop must be polled seperately for this future to resolve. If the event loop is not polled, the future will never make progress.
I verified that, because when I successfully invoke the function, it hangs forever, not making progress. But calling run_event_loop also involves a mutable borrow of js_runtime. So it results in the same complaint from the borrow checker.
15 replies
DDeno
Created by David Alsh on 1/29/2024 in #help
How to call a JS function from Rust?
I'm trying to do the same thing and have got this working up to the point of invoking call_with_args. Now the problem is with the borrow checker. That call_with_args requires a mutable borrow of the js_runtime. But it's already been mutably borrowed up here:
let mut scope = js_runtime.handle_scope();
let mut scope = js_runtime.handle_scope();
Any suggestions for how to resolve that?
15 replies
DDeno
Created by Mike Wilkerson on 5/3/2023 in #help
Embedding, snapshotting, and 7-bit ascii requirement
@.bartlomieju sure, thanks. For what it's worth, I can control all of this code. In fact, the module I'm importing is already something that was produced by a custom Rollup configuration. I'll attempt to just replace utf-8 characters with escape codes on that bundle.
15 replies
DDeno
Created by Mike Wilkerson on 5/3/2023 in #help
Embedding, snapshotting, and 7-bit ascii requirement
My conclusion from this, then, is that if I want to use any valid ES module, I should not load it via extension!. Furthermore, I should not expect to be able to snapshot any valid ES module either. ...which circles back to your initial suggestion: replace any UTF-8 characters with escape codes, if I want to load such a module in an extension, in order to create a snapshot.
15 replies
DDeno
Created by Mike Wilkerson on 5/3/2023 in #help
Embedding, snapshotting, and 7-bit ascii requirement
Ok, thanks.
15 replies
DDeno
Created by Mike Wilkerson on 5/3/2023 in #help
Embedding, snapshotting, and 7-bit ascii requirement
Ok, that makes sense about why you'd do that optimization, and the possible escape code workaround. I still wonder why I only hit this ASCII requirement when trying to load it as an extension. If I recall correctly, I was having this same problem with the ASCII requirement when I was just loading the module in an extension, even without creating a snapshot. Is the higher-level approach (via load_side_module()) replacing the UTF-8 characters automatically for me? Or is it some how loading the module in a way that does not have an ASCII requirement?
15 replies
DDeno
Created by Mike Wilkerson on 5/3/2023 in #help
Embedding, snapshotting, and 7-bit ascii requirement
It seems like the recent video live stream addresses exactly my concern, except that it doesn't address the roadblock I'm hitting: https://www.youtube.com/watch?v=zlJrMGm-XeA When I attempt to use the snapshotting APIs described in the video, I run into other problems. The method shown there uses an extension, and loads the code using a deno_core::ExtensionBuilder, .esm(), and include_js_files!(). When I load the exact same JavaScript module in an extension like that, I get an error like "Extension code must be 7-bit ASCII...". So there is evidently some difference in how this JavaScript module is being loaded and processed into these two loading scenarios. The higher-level approach, loading as a "side module" in the JsRuntime that is part of the MainWorker provided by the deno_runtime crate works as expected, but is less efficient than it might be if I were leveraging snapshotting. The lower-level approach, using an extension with deno_core does not work because this 7-bit ascii requirement is violated. What am I missing?
15 replies
DDeno
Created by Mike Wilkerson on 5/3/2023 in #help
Embedding, snapshotting, and 7-bit ascii requirement
My scenario works when I use the deno_runtime crate, the MainWorker API, and I use include_str!() to include the string contents of that module into the binary, create a ModuleCode from it, and pass that to load_side_module(). Something like this:
let main_module = Url::parse("http://localhost/main").unwrap();
let side_module = Url::parse("http://localhost/side").unwrap();
let permissions = PermissionsContainer::allow_all();

let mut worker = MainWorker::bootstrap_from_options(main_module.clone(), permissions, options);
let code = include_str!("js/my-bundle.js");
let side_module_code: ModuleCode = FastString::Static(code);
let side_module_id_fut = worker
.js_runtime
.load_side_module(&side_module, Some(side_module_code));
let side_module_id = futures::executor::block_on(side_module_id_fut).unwrap();
let _ = worker.js_runtime.mod_evaluate(side_module_id);
futures::executor::block_on(worker.js_runtime.run_event_loop(false)).unwrap();
let main_module = Url::parse("http://localhost/main").unwrap();
let side_module = Url::parse("http://localhost/side").unwrap();
let permissions = PermissionsContainer::allow_all();

let mut worker = MainWorker::bootstrap_from_options(main_module.clone(), permissions, options);
let code = include_str!("js/my-bundle.js");
let side_module_code: ModuleCode = FastString::Static(code);
let side_module_id_fut = worker
.js_runtime
.load_side_module(&side_module, Some(side_module_code));
let side_module_id = futures::executor::block_on(side_module_id_fut).unwrap();
let _ = worker.js_runtime.mod_evaluate(side_module_id);
futures::executor::block_on(worker.js_runtime.run_event_loop(false)).unwrap();
After that completes, the runtime is in a state where I can run scripts that use that side module. However, what I think is happening here is that every time this embed code runs, it's having to create a new module, re-parse the JavaScript, and re-execute the module in order to put the runtime into a state where the module is available for use by some "main module", or some ad-hoc code that I may later run with execute_script(). (Not to mention that the binary will have to include the JavaScript source code a string, making for a larger binary.) So what it seems like I should be doing is leveraging the snapshotting feature to do the equivalent of this side module loading work in a build.rs, take a snapshot, and then load that snapshot in the main.rs (or lib.rs) of my crate.
15 replies