orvit
orvit•3y ago

deno_core Extension with Global Access

Is there a way to make a Deno Extension be accessed by Extension.op() instead of Deno.core.ops.op()?
38 Replies
Doctor 🤖
Doctor 🤖•3y ago
Does this work?
const Extension = Deno.core.ops
Extension.op()
const Extension = Deno.core.ops
Extension.op()
orvit
orvitOP•3y ago
Of course that does work, but that'd require asking everyone to add that, I'm looking for a way to make it so whenever deno_core::JsRuntime::execute_script is run, it'll just work.
bartlomieju
bartlomieju•3y ago
currently it's not possible OOTB, you need to handle it yourself
orvit
orvitOP•3y ago
So how do extensions like console and fetch have their stuff globally? I'm willing to add a JS file to the extension.
bartlomieju
bartlomieju•3y ago
the global scope is assembled in a file called 99_main.js that is executed after all extensions have been loaded and added their relevant stuff to different window.__bootstrap namespaces
orvit
orvitOP•3y ago
So is there no way to implement a similar functionality as that into my own project without forking?
bartlomieju
bartlomieju•3y ago
you certainly can, which crates do you use? do you use deno_runtime crate?
orvit
orvitOP•3y ago
Yes, runtime.
bartlomieju
bartlomieju•3y ago
if you use deno_runtime it will be harder right now - @crowlkats and I are working on a PR that will make it much easier (https://github.com/denoland/deno/pull/16597) that will allows to extend global scope as you see fit
bartlomieju
bartlomieju•3y ago
all in all you should still be able to assign whatever you want to globalThis after all JS code from crates executes
orvit
orvitOP•3y ago
What's the goal for that pr? (If any)
bartlomieju
bartlomieju•3y ago
to allow creating V8 snapshots from existing V8 snapshots so you can extend what's already in deno_runtime and change it as you see fit mainly useful for a windowing project Leo is working on
orvit
orvitOP•3y ago
Around how long should I expect to wait for that
bartlomieju
bartlomieju•3y ago
a week probably
orvit
orvitOP•3y ago
So next patch? Alright. I appreciate all of this work. Thanks for the knowledge.
bartlomieju
bartlomieju•3y ago
yes, next patch release
orvit
orvitOP•3y ago
@.bartlomieju i see it was released, how would i properly add to the globalThis?
bartlomieju
bartlomieju•3y ago
@crowlkats can you give a quick rundown?
crowlKats
crowlKats•3y ago
a live example of this is the cli in the deno repo. first is the build script: first off you need to define a path for the snapshot and path for where you have your js files https://github.com/denoland/deno/blob/main/cli/build.rs#L460-L463 and then the actual snapshot creation https://github.com/denoland/deno/blob/main/cli/build.rs#L271-L317 then you need to get the snapshot during execution https://github.com/denoland/deno/blob/main/cli/js.rs and pass it to your WorkerOptions's startup_snapshot. then, in a js file in the folder you defined as js folder you do whatever you need to do, inside the usual iief; at the end of the iief you add to the global scope whatever you want with window.__bootstrap.globalScope.windowOrWorkerGlobalScope.myThing = myThing
orvit
orvitOP•3y ago
can this be done with extensions instead of everything in the build.rs? I dont exactly follow correctly
crowlKats
crowlKats•3y ago
I am not sure I understand
orvit
orvitOP•3y ago
How would I do this with an extension? Do I have to put everything in the js folder, or can I integrate extensions into this as well?
crowlKats
crowlKats•3y ago
ah, at the time it wasnt possible, but now it is i can write you up how in a bit
orvit
orvitOP•3y ago
Could I have this please?
crowlKats
crowlKats•3y ago
my bad, forgot about it. will do it now
orvit
orvitOP•3y ago
Thank you 😄
crowlKats
crowlKats•3y ago
sadly there is no live example. first instead of doing https://github.com/denoland/deno/blob/main/cli/build.rs#L474-L475, just use a vec with deno_runtime::js::get_99_main() in it. Then, in deno_core::snapshot_util::CreateSnapshotOptions there is a new field called extensions_with_js; to that you pass a vec of extensions you want to use to add additional js files. that should be all
orvit
orvitOP•2y ago
If I want all the default extensions in the cli, would I have to add those myself?
error: failed to run custom build command for `runtime v0.1.0 (/workspaces/runtime)`

Caused by:
process didn't exit successfully: `/workspaces/runtime/target/debug/build/runtime-7279b5b14e8d927f/build-script-build` (signal: 5, SIGTRAP: trace/breakpoint trap)
--- stderr
Unknown external reference 0x21.
<unresolved>
error: failed to run custom build command for `runtime v0.1.0 (/workspaces/runtime)`

Caused by:
process didn't exit successfully: `/workspaces/runtime/target/debug/build/runtime-7279b5b14e8d927f/build-script-build` (signal: 5, SIGTRAP: trace/breakpoint trap)
--- stderr
Unknown external reference 0x21.
<unresolved>
hmm
use deno_core::snapshot_util;
use std::path::PathBuf;

fn main() {
let snapshot_path = PathBuf::from(std::env::var_os("OUT_DIR").unwrap()).join("EVO_SNAPSHOT.bin");
create_snapshot(snapshot_path.clone());

println!("cargo:rustc-env=EVO_SNAPSHOT_PATH={}", snapshot_path.to_str().unwrap());
}

fn create_snapshot(snapshot_path: PathBuf) {
snapshot_util::create_snapshot(snapshot_util::CreateSnapshotOptions {
cargo_manifest_dir: env!("CARGO_MANIFEST_DIR"),
snapshot_path,
startup_snapshot: Some(deno_runtime::js::deno_isolate_init()),
extensions: vec![],
extensions_with_js: vec![],
additional_files: vec![deno_runtime::js::get_99_main()],
compression_cb: None,
});
}
use deno_core::snapshot_util;
use std::path::PathBuf;

fn main() {
let snapshot_path = PathBuf::from(std::env::var_os("OUT_DIR").unwrap()).join("EVO_SNAPSHOT.bin");
create_snapshot(snapshot_path.clone());

println!("cargo:rustc-env=EVO_SNAPSHOT_PATH={}", snapshot_path.to_str().unwrap());
}

fn create_snapshot(snapshot_path: PathBuf) {
snapshot_util::create_snapshot(snapshot_util::CreateSnapshotOptions {
cargo_manifest_dir: env!("CARGO_MANIFEST_DIR"),
snapshot_path,
startup_snapshot: Some(deno_runtime::js::deno_isolate_init()),
extensions: vec![],
extensions_with_js: vec![],
additional_files: vec![deno_runtime::js::get_99_main()],
compression_cb: None,
});
}
@crowlkats it errors out if you leave extensions as a vec. Is this intended? It seems you have to include every single extension the runtime uses for a successful build. Seems kinda frustrating, and would make maintaining the list frustrating.
crowlKats
crowlKats•2y ago
unsure how we could improve that really @.bartlomieju thoughts?
orvit
orvitOP•2y ago
I mean it wouldn't be challenging to add all the extensions to my project, it's just maintaining that list would be frustrating. The only solution I could think of for improving maintainability is making a function to get all default extensions built into deno_runtime, but that seems like an unnecessary solution to something that isn't much of a problem. Yet again, the cli also includes all those extensions, so a function would allow one list for the runtime and cli to use. runtime build script cant access runtime nvm
bartlomieju
bartlomieju•2y ago
well we had this one idea, that each Extension would specify its name in the builder and then we could add .depends_on(&["name1", "name2"]) and panic if one of the extensions hasn't been yet registered that would make debugging problems much easier in such case
crowlKats
crowlKats•2y ago
ah yea, though that still seems not perfect maybe make a function that returns a vec of extensions in deno_runtime?
bartlomieju
bartlomieju•2y ago
that's also an option
orvit
orvitOP•2y ago
That was the idea I had, but unless you want to do that in deno_core or made a new package for that, you couldn't access it in the runtime build script
crowlKats
crowlKats•2y ago
yea thats the one problem with it... though would still be better than nothing
orvit
orvitOP•2y ago
The only that would work with everything would be to create an ext module that re-exports everything but I don't see that being a good solution to a small problem Also, any specific reason why runtime defines all the default runtime extensions as extensions_with_js instead of just extensions like in the cli?
bartlomieju
bartlomieju•2y ago
Yes, if you put them in extensions_with_js the JS code defined in extension will be executed, it only need to happen once so that why they are passed in extensions in the cli
orvit
orvitOP•2y ago
ah
use deno_core::snapshot_util;
use deno_runtime::{permissions::Permissions, *};

use std::{env, path::PathBuf};

fn main() {
if env::var("HOST").unwrap() != env::var("TARGET").unwrap() {
panic!("Cross-compiling is not supported");
}

let evo_snapshot_path = PathBuf::from(env::var("OUT_DIR").unwrap()).join("EVO_SNAPSHOT.bin");
create_evo_snapshot(evo_snapshot_path.clone());

println!(
"cargo:rustc-env=EVO_SNAPSHOT_PATH={}",
evo_snapshot_path.to_str().unwrap()
);
}

fn create_evo_snapshot(snapshot_path: PathBuf) {
// from github.com/denoland/deno/blob/master/cli/build.rs (formatted + comments removed)
let extensions = vec![
deno_webidl::init(),
deno_console::init(),
deno_url::init(),
deno_tls::init(),
deno_web::init::<Permissions>(deno_web::BlobStore::default(), Default::default()),
deno_fetch::init::<Permissions>(Default::default()),
deno_cache::init::<deno_cache::SqliteBackedCache>(None),
deno_websocket::init::<Permissions>("".to_owned(), None, None),
deno_webstorage::init(None),
deno_crypto::init(None),
deno_webgpu::init(false),
deno_broadcast_channel::init(
deno_broadcast_channel::InMemoryBroadcastChannel::default(),
false,
),
deno_node::init::<Permissions>(None),
deno_ffi::init::<Permissions>(false),
deno_net::init::<Permissions>(None, false, None),
deno_napi::init::<Permissions>(false),
deno_http::init(),
deno_flash::init::<Permissions>(false),
];

snapshot_util::create_snapshot(snapshot_util::CreateSnapshotOptions {
cargo_manifest_dir: env!("CARGO_MANIFEST_DIR"),
snapshot_path,
startup_snapshot: Some(js::deno_isolate_init()),
extensions,
extensions_with_js: vec![],
additional_files: vec![js::get_99_main()],
compression_cb: None,
});
}
use deno_core::snapshot_util;
use deno_runtime::{permissions::Permissions, *};

use std::{env, path::PathBuf};

fn main() {
if env::var("HOST").unwrap() != env::var("TARGET").unwrap() {
panic!("Cross-compiling is not supported");
}

let evo_snapshot_path = PathBuf::from(env::var("OUT_DIR").unwrap()).join("EVO_SNAPSHOT.bin");
create_evo_snapshot(evo_snapshot_path.clone());

println!(
"cargo:rustc-env=EVO_SNAPSHOT_PATH={}",
evo_snapshot_path.to_str().unwrap()
);
}

fn create_evo_snapshot(snapshot_path: PathBuf) {
// from github.com/denoland/deno/blob/master/cli/build.rs (formatted + comments removed)
let extensions = vec![
deno_webidl::init(),
deno_console::init(),
deno_url::init(),
deno_tls::init(),
deno_web::init::<Permissions>(deno_web::BlobStore::default(), Default::default()),
deno_fetch::init::<Permissions>(Default::default()),
deno_cache::init::<deno_cache::SqliteBackedCache>(None),
deno_websocket::init::<Permissions>("".to_owned(), None, None),
deno_webstorage::init(None),
deno_crypto::init(None),
deno_webgpu::init(false),
deno_broadcast_channel::init(
deno_broadcast_channel::InMemoryBroadcastChannel::default(),
false,
),
deno_node::init::<Permissions>(None),
deno_ffi::init::<Permissions>(false),
deno_net::init::<Permissions>(None, false, None),
deno_napi::init::<Permissions>(false),
deno_http::init(),
deno_flash::init::<Permissions>(false),
];

snapshot_util::create_snapshot(snapshot_util::CreateSnapshotOptions {
cargo_manifest_dir: env!("CARGO_MANIFEST_DIR"),
snapshot_path,
startup_snapshot: Some(js::deno_isolate_init()),
extensions,
extensions_with_js: vec![],
additional_files: vec![js::get_99_main()],
compression_cb: None,
});
}
it works but it looks like it'd be annoying to maintain