Nick•8mo ago

Return lazy struct fields from Op

My goal is to build a plugin system which allows invoking user defined code. One of the features I am looking to support is the ability to invoke user provided functions with function parameters, an example:

// Ideally, I can model this in an Op:
type Input = {
  lazyValueA: () => Promise<number>;
  lazyValueB: () => Promise<number>;
}

// Example usage
const userCallback: (input: Input) => Promise<number> = async ({lazyValueA, lazyValueB}) => {
  return (await lazyValueA()) + (await lazyValueB());
}

// Ideally, I can model this in an Op:
type Input = {
  lazyValueA: () => Promise<number>;
  lazyValueB: () => Promise<number>;
}

// Example usage
const userCallback: (input: Input) => Promise<number> = async ({lazyValueA, lazyValueB}) => {
  return (await lazyValueA()) + (await lazyValueB());
}

I have explored these paths but neither have been successful: - model the Op return as an Input struct with v8::Function fields (while returning v8::Function values is supported) - model the Op return as an Input as Future values, but serializing Future is obviously problematic Reading through the deno_ops and v8_serde packages, I don't see an obvious way to accomplish this (very open to thoughts about how to restructure this, feels like im working against the tooling here)

3 Replies

Deno AI Helper•8mo ago

I wasn't able to find the answer to your question in the official Deno documentation. However, based on the code snippet and the paths you've explored that you provided in your message, I can offer some general advice. From what you've described, it seems like you're trying to pass functions (or lazy values) through Deno's plugin system. In JavaScript and TypeScript, functions are not serializable out of the box, which means you cannot directly send them across boundaries such as Web Workers or plugins without some form of serialization. The issue with trying to serialize functions or Future values is that they contain more than just data; they contain behavior or a state that is pending, respectively. This is why you're encountering difficulties. One potential solution, which is not based on the Deno documentation and should be double-checked for accuracy, is to consider an approach where you serialize the instructions for creating the lazy values rather than the values themselves. This could involve sending a serializable representation of the operations needed to be performed and then reconstructing the functions on the other side of the boundary. For example, instead of sending functions, you could send an object with the information necessary to create the function: ```typescript type InputSpec = { lazyValueASpec: SerializableDataForA; lazyValueBSpec: SerializableDataForB; } // Then on the receiving end, you would have a way to interpret these specs and create the functions function createLazyValueA(spec: SerializableDataForA): () => Promise<number> { // Interpret the spec and return a function } function createLazyValueB(spec: SerializableDataForB): () => Promise<number> { // Interpret the spec and return a function } // Usage const input: Input

Deno AI Helper•8mo ago

= { lazyValueA: createLazyValueA(receivedSpec.lazyValueASpec), lazyValueB: createLazyValueB(receivedSpec.lazyValueBSpec), }; ``` This approach requires that you define a way to serialize and deserialize the specifications for your lazy values. It's a more indirect method but could potentially work around the limitations of serializing functions or futures. Please note that this advice is speculative and not based on the Deno documentation. It's important to verify the feasibility of this approach in the context of Deno's plugin system and the specific requirements of your application.

Helpful Links: Deno 1.0 Deno 1.9 Release Notes Data Modeling in TypeScript | Deno Docs Roll your own JavaScript runtime, pt. 2

Nick•8mo ago

ah this seems more correct, I’ll explore this direction