Damian Reeves•2mo ago

Using Deno core in an app that already had its own tokio runtime

I am using the deno_core and deno_ast crates to "Roll my own JavaScript runtime", as seen here (https://github.com/denoland/roll-your-own-javascript-runtime/blob/main/src/main.rs#L153-L159 ), but I don't know how to launch my deno runtime properly within my existing tokio runtime. I've tried using a Handle and spawning a thread and using block_on, like below:

pub(crate) fn run_js(args: &RunArgs) {
    let file_path = args.file.to_owned().into_string().unwrap();
    let handle = tokio::runtime::Handle::current();
    std::thread::spawn(move || {
        // Using Handle::block_on to run async code in the new thread.
        if let Err(error) = handle.block_on(run_js(file_path.as_str())) {
            eprintln!("error: {error}");
        }
    })
    .join()
    .unwrap();
}

pub(crate) fn run_js(args: &RunArgs) {
    let file_path = args.file.to_owned().into_string().unwrap();
    let handle = tokio::runtime::Handle::current();
    std::thread::spawn(move || {
        // Using Handle::block_on to run async code in the new thread.
        if let Err(error) = handle.block_on(run_js(file_path.as_str())) {
            eprintln!("error: {error}");
        }
    })
    .join()
    .unwrap();
}

But the moment I execute the example script, it fails at setTimeout.

console.log("Waiting 2 seconds for timeout...");
setTimeout(() => {
  console.log("Hello World from timeout!");
}, 2000);

console.log("Waiting 2 seconds for timeout...");
setTimeout(() => {
  console.log("Hello World from timeout!");
}, 2000);

I end up with output like below:

[out]: "Waiting 2 seconds for timeout..."
Error:   × Main thread panicked.
  ├─▶ at /Users/my_user_name/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_unsync-0.3.4/src/task.rs:56:3
  ╰─▶ assertion failed: Handle::current().runtime_flavor() == RuntimeFlavor::CurrentThread
  help: set the `RUST_BACKTRACE=1` environment variable to display a backtrace.

fatal runtime error: failed to initiate panic, error 5

[out]: "Waiting 2 seconds for timeout..."
Error:   × Main thread panicked.
  ├─▶ at /Users/my_user_name/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_unsync-0.3.4/src/task.rs:56:3
  ╰─▶ assertion failed: Handle::current().runtime_flavor() == RuntimeFlavor::CurrentThread
  help: set the `RUST_BACKTRACE=1` environment variable to display a backtrace.

fatal runtime error: failed to initiate panic, error 5

GitHub

roll-your-own-javascript-runtime/src/main.rs at main · denoland/rol...

Contribute to denoland/roll-your-own-javascript-runtime development by creating an account on GitHub.

8 Replies

Damian Reeves•2mo ago

Can any guidance be guided on how to run deno in its own pool for an app that already has an existing tokio runtime?

Deno AI Helper•2mo ago

The error you're encountering seems to be related to the interaction between the Deno runtime and the Tokio runtime, specifically with the setTimeout function. The assertion failure indicates that the current Tokio handle is expected to be of the RuntimeFlavor::CurrentThread, but it seems that it's not the case in your setup. The documentation provides an example of how to create a single-threaded Tokio runtime and execute a function that runs JavaScript code using deno_core. Here's a snippet from the documentation that shows how to set up the Tokio runtime:

fn main() {
    let runtime = tokio::runtime::Builder::new_current_thread()
        .enable_all()
        .build()
        .unwrap();
    if let Err(error) = runtime.block_on(run_js("./example.js")) {
        eprintln!("error: {}", error);
    }
}

fn main() {
    let runtime = tokio::runtime::Builder::new_current_thread()
        .enable_all()
        .build()
        .unwrap();
    if let Err(error) = runtime.block_on(run_js("./example.js")) {
        eprintln!("error: {}", error);
    }
}

In this example, the new_current_thread builder method is used to create a single-threaded Tokio runtime, which is compatible with the Deno runtime's expectations for setTimeout and other asynchronous operations. If you already have an existing Tokio runtime that is not single-threaded, you might need to adjust your setup to ensure that the Deno runtime is running within a single-threaded context. Here's a general approach you could take: 1. Ensure that your existing Tokio runtime is single-threaded. If it's not, you may need to create a separate single-threaded runtime for the Deno-related tasks. 2. Instead of spawning a new thread and using Handle::block_on, consider using the spawn_blocking function provided by Tokio, which offloads blocking operations to a thread where blocking is acceptable. Here's a conceptual example of how you might use spawn_blocking: ```rust pub(crate) fn run_js(args: &RunArgs) { let file_path =

Deno AI Helper•2mo ago

args.file.to_owned().into_string().unwrap(); tokio::task::spawn_blocking(move || { let local = tokio::runtime::Builder::new_current_thread() .enable_all() .build() .unwrap(); local.block_on(async { if let Err(error) = run_js(file_path.as_str()).await { eprintln!("error: {error}"); } }) }) .await .unwrap(); }

``

Please note that this is a general advice and may not directly solve your issue without further adjustments. The actual implementation details may vary based on the specifics of your existing Tokio runtime and how you've integrated

deno_core

.

If you continue to face issues, it might be helpful to look into how

deno_core` expects the Tokio runtime to be configured, especially regarding the thread model and the event loop. 🦕

Helpful Links: Roll your own JavaScript runtime Roll your own JavaScript runtime, pt. 3 Workers | Deno Docs

Damian Reeves•2mo ago

So after hacking around a bit, this seems to work.

pub(crate) fn run_js(args: &RunArgs) {
    let file = args.file.to_owned();
    std::thread::spawn(move || {
        let file_path = file.to_str();
        file_path.map(|path| {
            println!("Running JavaScript file: {}", path);

            let runtime = tokio::runtime::Builder::new_current_thread()
                .enable_all()
                .build()
                .unwrap();

            if let Err(error) = runtime.block_on(run_js(path)) {
                eprintln!("error: {error}");
            }
        });
    })
    .join()
    .unwrap();
}

pub(crate) fn run_js(args: &RunArgs) {
    let file = args.file.to_owned();
    std::thread::spawn(move || {
        let file_path = file.to_str();
        file_path.map(|path| {
            println!("Running JavaScript file: {}", path);

            let runtime = tokio::runtime::Builder::new_current_thread()
                .enable_all()
                .build()
                .unwrap();

            if let Err(error) = runtime.block_on(run_js(path)) {
                eprintln!("error: {error}");
            }
        });
    })
    .join()
    .unwrap();
}

I changed args.file to an Arc<OsString> instead of an OsString.

bartlomieju•2mo ago

@Damian Reeves you don't really need to use a separate Tokio runtime unless your use case prohibits it You can safely run it in the context of your existing Tokio runtime

Damian Reeves•2mo ago

I found that when I tried to use a Handle to call sync code such as the setTimeout from the repo that my code exited with a panic. I’d love to not use another Tokyo runtime and instead have a dedicated pool for my workloads that use deno_core but I’m not familiar enough with either tokio or eeno to accomplish this

bartlomieju•2mo ago

Oh I see the panic now Then you're out of luck - Deno can only work with "current thread" Tokio runtimes, not multithreaded That's because a JsRuntime must be pinned to a certain thread and can't be moved to another thread

Damian Reeves•2mo ago

Interesting, let me see if I can change my app to use current_thread, yet still make a dedicated thread for the deno work from my main app