orvit•2y ago

deno_core Extension with Global Access

Is there a way to make a Deno Extension be accessed by Extension.op() instead of Deno.core.ops.op()?

38 Replies

Does this work?

const Extension = Deno.core.ops
Extension.op()

const Extension = Deno.core.ops
Extension.op()

orvit•2y ago

Of course that does work, but that'd require asking everyone to add that, I'm looking for a way to make it so whenever deno_core::JsRuntime::execute_script is run, it'll just work.

bartlomieju•2y ago

currently it's not possible OOTB, you need to handle it yourself

orvit•2y ago

So how do extensions like console and fetch have their stuff globally? I'm willing to add a JS file to the extension.

bartlomieju•2y ago

the global scope is assembled in a file called 99_main.js that is executed after all extensions have been loaded and added their relevant stuff to different window.__bootstrap namespaces

orvit•2y ago

So is there no way to implement a similar functionality as that into my own project without forking?

bartlomieju•2y ago

you certainly can, which crates do you use? do you use deno_runtime crate?

orvit•2y ago

Yes, runtime.

bartlomieju•2y ago

if you use deno_runtime it will be harder right now - @crowlkats and I are working on a PR that will make it much easier (https://github.com/denoland/deno/pull/16597) that will allows to extend global scope as you see fit

GitHub

[WIP] Move global scopes to separate files by bartlomieju · Pull Re...

DON'T MERGE YET

bartlomieju•2y ago

all in all you should still be able to assign whatever you want to globalThis after all JS code from crates executes

orvit•2y ago

What's the goal for that pr? (If any)

bartlomieju•2y ago

to allow creating V8 snapshots from existing V8 snapshots so you can extend what's already in deno_runtime and change it as you see fit mainly useful for a windowing project Leo is working on

orvit•2y ago

Around how long should I expect to wait for that

bartlomieju•2y ago

a week probably

orvit•2y ago

So next patch? Alright. I appreciate all of this work. Thanks for the knowledge.

bartlomieju•2y ago

yes, next patch release

orvit•2y ago

@.bartlomieju i see it was released, how would i properly add to the globalThis?

bartlomieju•2y ago

@crowlkats can you give a quick rundown?

crowlKats•2y ago

a live example of this is the cli in the deno repo. first is the build script: first off you need to define a path for the snapshot and path for where you have your js files https://github.com/denoland/deno/blob/main/cli/build.rs#L460-L463 and then the actual snapshot creation https://github.com/denoland/deno/blob/main/cli/build.rs#L271-L317 then you need to get the snapshot during execution https://github.com/denoland/deno/blob/main/cli/js.rs and pass it to your WorkerOptions's startup_snapshot. then, in a js file in the folder you defined as js folder you do whatever you need to do, inside the usual iief; at the end of the iief you add to the global scope whatever you want with window.__bootstrap.globalScope.windowOrWorkerGlobalScope.myThing = myThing

orvit•2y ago

can this be done with extensions instead of everything in the build.rs? I dont exactly follow correctly

crowlKats•2y ago

I am not sure I understand

orvit•2y ago

How would I do this with an extension? Do I have to put everything in the js folder, or can I integrate extensions into this as well?

crowlKats•2y ago

ah, at the time it wasnt possible, but now it is i can write you up how in a bit

orvit•2y ago

Could I have this please?

crowlKats•2y ago

my bad, forgot about it. will do it now

orvit•2y ago

Thank you 😄

crowlKats•2y ago

sadly there is no live example. first instead of doing https://github.com/denoland/deno/blob/main/cli/build.rs#L474-L475, just use a vec with deno_runtime::js::get_99_main() in it. Then, in deno_core::snapshot_util::CreateSnapshotOptions there is a new field called extensions_with_js; to that you pass a vec of extensions you want to use to add additional js files. that should be all

orvit•2y ago

If I want all the default extensions in the cli, would I have to add those myself?

error: failed to run custom build command for `runtime v0.1.0 (/workspaces/runtime)`

Caused by:
  process didn't exit successfully: `/workspaces/runtime/target/debug/build/runtime-7279b5b14e8d927f/build-script-build` (signal: 5, SIGTRAP: trace/breakpoint trap)
  --- stderr
  Unknown external reference 0x21.
  <unresolved>

error: failed to run custom build command for `runtime v0.1.0 (/workspaces/runtime)`

Caused by:
  process didn't exit successfully: `/workspaces/runtime/target/debug/build/runtime-7279b5b14e8d927f/build-script-build` (signal: 5, SIGTRAP: trace/breakpoint trap)
  --- stderr
  Unknown external reference 0x21.
  <unresolved>

hmm

use deno_core::snapshot_util;
use std::path::PathBuf;

fn main() {
    let snapshot_path = PathBuf::from(std::env::var_os("OUT_DIR").unwrap()).join("EVO_SNAPSHOT.bin");
    create_snapshot(snapshot_path.clone());

    println!("cargo:rustc-env=EVO_SNAPSHOT_PATH={}", snapshot_path.to_str().unwrap());
}

fn create_snapshot(snapshot_path: PathBuf) {
    snapshot_util::create_snapshot(snapshot_util::CreateSnapshotOptions {
        cargo_manifest_dir: env!("CARGO_MANIFEST_DIR"),
        snapshot_path,
        startup_snapshot: Some(deno_runtime::js::deno_isolate_init()),
        extensions: vec![],
        extensions_with_js: vec![],
        additional_files: vec![deno_runtime::js::get_99_main()],
        compression_cb: None,
    });
}

use deno_core::snapshot_util;
use std::path::PathBuf;

fn main() {
    let snapshot_path = PathBuf::from(std::env::var_os("OUT_DIR").unwrap()).join("EVO_SNAPSHOT.bin");
    create_snapshot(snapshot_path.clone());

    println!("cargo:rustc-env=EVO_SNAPSHOT_PATH={}", snapshot_path.to_str().unwrap());
}

fn create_snapshot(snapshot_path: PathBuf) {
    snapshot_util::create_snapshot(snapshot_util::CreateSnapshotOptions {
        cargo_manifest_dir: env!("CARGO_MANIFEST_DIR"),
        snapshot_path,
        startup_snapshot: Some(deno_runtime::js::deno_isolate_init()),
        extensions: vec![],
        extensions_with_js: vec![],
        additional_files: vec![deno_runtime::js::get_99_main()],
        compression_cb: None,
    });
}

@crowlkats it errors out if you leave extensions as a vec. Is this intended? It seems you have to include every single extension the runtime uses for a successful build. Seems kinda frustrating, and would make maintaining the list frustrating.

crowlKats•2y ago

unsure how we could improve that really @.bartlomieju thoughts?

orvit•2y ago

I mean it wouldn't be challenging to add all the extensions to my project, it's just maintaining that list would be frustrating. The only solution I could think of for improving maintainability is making a function to get all default extensions built into deno_runtime, but that seems like an unnecessary solution to something that isn't much of a problem. ~~Yet again, the cli also includes all those extensions, so a function would allow one list for the runtime and cli to use.~~ runtime build script cant access runtime nvm

bartlomieju•2y ago

well we had this one idea, that each Extension would specify its name in the builder and then we could add .depends_on(&["name1", "name2"]) and panic if one of the extensions hasn't been yet registered that would make debugging problems much easier in such case

crowlKats•2y ago

ah yea, though that still seems not perfect maybe make a function that returns a vec of extensions in deno_runtime?

bartlomieju•2y ago

that's also an option

orvit•2y ago

That was the idea I had, but unless you want to do that in deno_core or made a new package for that, you couldn't access it in the runtime build script

crowlKats•2y ago

yea thats the one problem with it... though would still be better than nothing

orvit•2y ago

The only that would work with everything would be to create an ext module that re-exports everything but I don't see that being a good solution to a small problem Also, any specific reason why runtime defines all the default runtime extensions as extensions_with_js instead of just extensions like in the cli?

bartlomieju•2y ago

Yes, if you put them in extensions_with_js the JS code defined in extension will be executed, it only need to happen once so that why they are passed in extensions in the cli

orvit•2y ago

use deno_core::snapshot_util;
use deno_runtime::{permissions::Permissions, *};

use std::{env, path::PathBuf};

fn main() {
    if env::var("HOST").unwrap() != env::var("TARGET").unwrap() {
        panic!("Cross-compiling is not supported");
    }

    let evo_snapshot_path = PathBuf::from(env::var("OUT_DIR").unwrap()).join("EVO_SNAPSHOT.bin");
    create_evo_snapshot(evo_snapshot_path.clone());

    println!(
        "cargo:rustc-env=EVO_SNAPSHOT_PATH={}",
        evo_snapshot_path.to_str().unwrap()
    );
}

fn create_evo_snapshot(snapshot_path: PathBuf) {
    // from github.com/denoland/deno/blob/master/cli/build.rs (formatted + comments removed)
    let extensions = vec![
        deno_webidl::init(),
        deno_console::init(),
        deno_url::init(),
        deno_tls::init(),
        deno_web::init::<Permissions>(deno_web::BlobStore::default(), Default::default()),
        deno_fetch::init::<Permissions>(Default::default()),
        deno_cache::init::<deno_cache::SqliteBackedCache>(None),
        deno_websocket::init::<Permissions>("".to_owned(), None, None),
        deno_webstorage::init(None),
        deno_crypto::init(None),
        deno_webgpu::init(false),
        deno_broadcast_channel::init(
            deno_broadcast_channel::InMemoryBroadcastChannel::default(),
            false,
        ),
        deno_node::init::<Permissions>(None),
        deno_ffi::init::<Permissions>(false),
        deno_net::init::<Permissions>(None, false, None),
        deno_napi::init::<Permissions>(false),
        deno_http::init(),
        deno_flash::init::<Permissions>(false),
    ];

    snapshot_util::create_snapshot(snapshot_util::CreateSnapshotOptions {
        cargo_manifest_dir: env!("CARGO_MANIFEST_DIR"),
        snapshot_path,
        startup_snapshot: Some(js::deno_isolate_init()),
        extensions,
        extensions_with_js: vec![],
        additional_files: vec![js::get_99_main()],
        compression_cb: None,
    });
}

use deno_core::snapshot_util;
use deno_runtime::{permissions::Permissions, *};

use std::{env, path::PathBuf};

fn main() {
    if env::var("HOST").unwrap() != env::var("TARGET").unwrap() {
        panic!("Cross-compiling is not supported");
    }

    let evo_snapshot_path = PathBuf::from(env::var("OUT_DIR").unwrap()).join("EVO_SNAPSHOT.bin");
    create_evo_snapshot(evo_snapshot_path.clone());

    println!(
        "cargo:rustc-env=EVO_SNAPSHOT_PATH={}",
        evo_snapshot_path.to_str().unwrap()
    );
}

fn create_evo_snapshot(snapshot_path: PathBuf) {
    // from github.com/denoland/deno/blob/master/cli/build.rs (formatted + comments removed)
    let extensions = vec![
        deno_webidl::init(),
        deno_console::init(),
        deno_url::init(),
        deno_tls::init(),
        deno_web::init::<Permissions>(deno_web::BlobStore::default(), Default::default()),
        deno_fetch::init::<Permissions>(Default::default()),
        deno_cache::init::<deno_cache::SqliteBackedCache>(None),
        deno_websocket::init::<Permissions>("".to_owned(), None, None),
        deno_webstorage::init(None),
        deno_crypto::init(None),
        deno_webgpu::init(false),
        deno_broadcast_channel::init(
            deno_broadcast_channel::InMemoryBroadcastChannel::default(),
            false,
        ),
        deno_node::init::<Permissions>(None),
        deno_ffi::init::<Permissions>(false),
        deno_net::init::<Permissions>(None, false, None),
        deno_napi::init::<Permissions>(false),
        deno_http::init(),
        deno_flash::init::<Permissions>(false),
    ];

    snapshot_util::create_snapshot(snapshot_util::CreateSnapshotOptions {
        cargo_manifest_dir: env!("CARGO_MANIFEST_DIR"),
        snapshot_path,
        startup_snapshot: Some(js::deno_isolate_init()),
        extensions,
        extensions_with_js: vec![],
        additional_files: vec![js::get_99_main()],
        compression_cb: None,
    });
}

it works but it looks like it'd be annoying to maintain