Deno•2y ago

Using Deno core in an app that already had its own tokio runtime

I am using the deno_core and deno_ast crates to "Roll my own JavaScript runtime", as seen here (https://github.com/denoland/roll-your-own-javascript-runtime/blob/main/src/main.rs#L153-L159 ), but I don't know how to launch my deno runtime properly within my existing tokio runtime.

I've tried using a

Handle

Handle

Handle

Handle and spawning a thread and using block_on, like below:

pub(crate) fn run_js(args: &RunArgs) {
    let file_path = args.file.to_owned().into_string().unwrap();
    let handle = tokio::runtime::Handle::current();
    std::thread::spawn(move || {
        // Using Handle::block_on to run async code in the new thread.
        if let Err(error) = handle.block_on(run_js(file_path.as_str())) {
            eprintln!("error: {error}");
        }
    })
    .join()
    .unwrap();
}

pub(crate) fn run_js(args: &RunArgs) {
    let file_path = args.file.to_owned().into_string().unwrap();
    let handle = tokio::runtime::Handle::current();
    std::thread::spawn(move || {
        // Using Handle::block_on to run async code in the new thread.
        if let Err(error) = handle.block_on(run_js(file_path.as_str())) {
            eprintln!("error: {error}");
        }
    })
    .join()
    .unwrap();
}

pub(crate) fn run_js(args: &RunArgs) {
    let file_path = args.file.to_owned().into_string().unwrap();
    let handle = tokio::runtime::Handle::current();
    std::thread::spawn(move || {
        // Using Handle::block_on to run async code in the new thread.
        if let Err(error) = handle.block_on(run_js(file_path.as_str())) {
            eprintln!("error: {error}");
        }
    })
    .join()
    .unwrap();
}

pub(crate) fn run_js(args: &RunArgs) {
    let file_path = args.file.to_owned().into_string().unwrap();
    let handle = tokio::runtime::Handle::current();
    std::thread::spawn(move || {
        // Using Handle::block_on to run async code in the new thread.
        if let Err(error) = handle.block_on(run_js(file_path.as_str())) {
            eprintln!("error: {error}");
        }
    })
    .join()
    .unwrap();
}

But the moment I execute the example script, it fails at

setTimeout

setTimeout

setTimeout

setTimeout.

console.log("Waiting 2 seconds for timeout...");
setTimeout(() => {
  console.log("Hello World from timeout!");
}, 2000);

console.log("Waiting 2 seconds for timeout...");
setTimeout(() => {
  console.log("Hello World from timeout!");
}, 2000);

console.log("Waiting 2 seconds for timeout...");
setTimeout(() => {
  console.log("Hello World from timeout!");
}, 2000);

console.log("Waiting 2 seconds for timeout...");
setTimeout(() => {
  console.log("Hello World from timeout!");
}, 2000);

I end up with output like below:

[out]: "Waiting 2 seconds for timeout..."
Error:   × Main thread panicked.
  ├─▶ at /Users/my_user_name/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_unsync-0.3.4/src/task.rs:56:3
  ╰─▶ assertion failed: Handle::current().runtime_flavor() == RuntimeFlavor::CurrentThread
  help: set the `RUST_BACKTRACE=1` environment variable to display a backtrace.

fatal runtime error: failed to initiate panic, error 5

[out]: "Waiting 2 seconds for timeout..."
Error:   × Main thread panicked.
  ├─▶ at /Users/my_user_name/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_unsync-0.3.4/src/task.rs:56:3
  ╰─▶ assertion failed: Handle::current().runtime_flavor() == RuntimeFlavor::CurrentThread
  help: set the `RUST_BACKTRACE=1` environment variable to display a backtrace.

fatal runtime error: failed to initiate panic, error 5

[out]: "Waiting 2 seconds for timeout..."
Error:   × Main thread panicked.
  ├─▶ at /Users/my_user_name/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_unsync-0.3.4/src/task.rs:56:3
  ╰─▶ assertion failed: Handle::current().runtime_flavor() == RuntimeFlavor::CurrentThread
  help: set the `RUST_BACKTRACE=1` environment variable to display a backtrace.

fatal runtime error: failed to initiate panic, error 5

[out]: "Waiting 2 seconds for timeout..."
Error:   × Main thread panicked.
  ├─▶ at /Users/my_user_name/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_unsync-0.3.4/src/task.rs:56:3
  ╰─▶ assertion failed: Handle::current().runtime_flavor() == RuntimeFlavor::CurrentThread
  help: set the `RUST_BACKTRACE=1` environment variable to display a backtrace.

fatal runtime error: failed to initiate panic, error 5

GitHub

roll-your-own-javascript-runtime/src/main.rs at main · denoland/rol...

Contribute to denoland/roll-your-own-javascript-runtime development by creating an account on GitHub.

Damian ReevesOP•6/4/24, 2:14 PM

Can any guidance be guided on how to run deno in its own pool for an app that already has an existing tokio runtime?

Deno AI Helper•6/4/24, 3:14 PM

The error you're encountering seems to be related to the interaction between the Deno runtime and the Tokio runtime, specifically with the

setTimeout

setTimeout

function. The assertion failure indicates that the current Tokio handle is expected to be of the RuntimeFlavor::CurrentThreadRuntimeFlavor::CurrentThread, but it seems that it's not the case in your setup.

The documentation provides an example of how to create a single-threaded Tokio runtime and execute a function that runs JavaScript code using deno_coredeno_core. Here's a snippet from the documentation that shows how to set up the Tokio runtime:

fn main() {
    let runtime = tokio::runtime::Builder::new_current_thread()
        .enable_all()
        .build()
        .unwrap();
    if let Err(error) = runtime.block_on(run_js("./example.js")) {
        eprintln!("error: {}", error);
    }
}

fn main() {
    let runtime = tokio::runtime::Builder::new_current_thread()
        .enable_all()
        .build()
        .unwrap();
    if let Err(error) = runtime.block_on(run_js("./example.js")) {
        eprintln!("error: {}", error);
    }
}

In this example, the new_current_threadnew_current_thread builder method is used to create a single-threaded Tokio runtime, which is compatible with the Deno runtime's expectations for

setTimeout

setTimeout

and other asynchronous operations.

If you already have an existing Tokio runtime that is not single-threaded, you might need to adjust your setup to ensure that the Deno runtime is running within a single-threaded context. Here's a general approach you could take:

Ensure that your existing Tokio runtime is single-threaded. If it's not, you may need to create a separate single-threaded runtime for the Deno-related tasks.
Instead of spawning a new thread and using Handle::block_onHandle::block_on, consider using the spawn_blockingspawn_blocking function provided by Tokio, which offloads blocking operations to a thread where blocking is acceptable.

Here's a conceptual example of how you might use spawn_blockingspawn_blocking:

```rust
pub(crate) fn run_js(args: &RunArgs) {
let file_path =

Deno AI Helper•6/4/24, 3:14 PM

args.file.to_owned().into_string().unwrap();
tokio::task::spawn_blocking(move || {
let local = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
local.block_on(async {
if let Err(error) = run_js(file_path.as_str()).await {
eprintln!("error: {error}");
}
})
})
.await
.unwrap();
}

``

Please note that this is a general advice and may not directly solve your issue without further adjustments. The actual implementation details may vary based on the specifics of your existing Tokio runtime and how you've integrated

``

Please note that this is a general advice and may not directly solve your issue without further adjustments. The actual implementation details may vary based on the specifics of your existing Tokio runtime and how you've integrated

deno_core

.

If you continue to face issues, it might be helpful to look into how

.

If you continue to face issues, it might be helpful to look into how

deno_core` expects the Tokio runtime to be configured, especially regarding the thread model and the event loop.

Helpful Links:
Roll your own JavaScript runtime
Roll your own JavaScript runtime, pt. 3
Workers | Deno Docs

Damian ReevesOP•6/4/24, 3:57 PM

So after hacking around a bit, this seems to work.

pub(crate) fn run_js(args: &RunArgs) {
    let file = args.file.to_owned();
    std::thread::spawn(move || {
        let file_path = file.to_str();
        file_path.map(|path| {
            println!("Running JavaScript file: {}", path);

            let runtime = tokio::runtime::Builder::new_current_thread()
                .enable_all()
                .build()
                .unwrap();

            if let Err(error) = runtime.block_on(run_js(path)) {
                eprintln!("error: {error}");
            }
        });
    })
    .join()
    .unwrap();
}

pub(crate) fn run_js(args: &RunArgs) {
    let file = args.file.to_owned();
    std::thread::spawn(move || {
        let file_path = file.to_str();
        file_path.map(|path| {
            println!("Running JavaScript file: {}", path);

            let runtime = tokio::runtime::Builder::new_current_thread()
                .enable_all()
                .build()
                .unwrap();

            if let Err(error) = runtime.block_on(run_js(path)) {
                eprintln!("error: {error}");
            }
        });
    })
    .join()
    .unwrap();
}

I changed args.fileargs.file to an Arc<OsString>Arc<OsString> instead of an OsStringOsString.

bartlomieju•6/4/24, 10:20 PM

@Damian Reeves you don't really need to use a separate Tokio runtime unless your use case prohibits it

bartlomieju•6/4/24, 10:20 PM

You can safely run it in the context of your existing Tokio runtime

Damian ReevesOP•6/4/24, 10:27 PM

I found that when I tried to use a Handle to call sync code such as the setTimeout from the repo that my code exited with a panic.
I’d love to not use another Tokyo runtime and instead have a dedicated pool for my workloads that use deno_core but I’m not familiar enough with either tokio or eeno to accomplish this

bartlomieju•6/5/24, 1:35 PM

Oh I see the panic now

bartlomieju•6/5/24, 1:35 PM

Then you're out of luck - Deno can only work with "current thread" Tokio runtimes, not multithreaded

bartlomieju•6/5/24, 1:35 PM

That's because a JsRuntimeJsRuntime must be pinned to a certain thread and can't be moved to another thread

Damian ReevesOP•6/5/24, 3:28 PM

Interesting, let me see if I can change my app to use current_thread, yet still make a dedicated thread for the deno work from my main app

Using Deno core in an app that already had its own tokio runtime

Similar Threads

Similar Threads

Similar Threads