Stéphane
Stéphane•15mo ago

API architecture for generic callback

Hi there, I have a general question about code architecture for an API I'm doing. Basically, it listens to MIDI messages from a native library and I use an FFI binding to forward the messages to the user. A message is represented by an array of bytes, and from that you can describe its data and how to interpret it. I made a layer that converts the raw data to a typed object, but what I want to do is offer a choice to the user so he can retrieve either the raw data or the converted typed object. What I have so far is like that : 
interface InputCallbackParams<T extends Message<MessageData> | number[]> {
  message: T;
  deltaTime?: number;
}

export interface InputCallbackOptions {
  callback?: (params: InputCallbackParams<Message<MessageData>>) => void;
  rawCallback?: (params: InputCallbackParams<number[]>) => void;
}
interface InputCallbackParams<T extends Message<MessageData> | number[]> {
  message: T;
  deltaTime?: number;
}

export interface InputCallbackOptions {
  callback?: (params: InputCallbackParams<Message<MessageData>>) => void;
  rawCallback?: (params: InputCallbackParams<number[]>) => void;
}
And the method that the user can use to listen to messages : 
onMessage(
    options: InputCallbackOptions,
  ): void {
    this.callback = Deno.UnsafeCallback.threadSafe(
      RtMidiCCallbackCallbackDefinition,
      (
        deltaTime: number,
        message: Deno.PointerValue<unknown>,
        messageSize: number | bigint,
      ) => {
        const msg_data = new Uint8Array(
          new Deno.UnsafePointerView(message!).getArrayBuffer(
            messageSize as number,
          ),
        );

        if (options.callback !== undefined) {
          options.callback({
            message: decodeMessage(msg_data),
            deltaTime,
          });
        }

        if (options.rawCallback !== undefined) {
          options.rawCallback({
            message: Array.from(msg_data),
            deltaTime,
          });
        }
      },
    );
    rtmidi.rtmidi_in_set_callback(this.device, this.callback!.pointer, null);
  }
onMessage(
    options: InputCallbackOptions,
  ): void {
    this.callback = Deno.UnsafeCallback.threadSafe(
      RtMidiCCallbackCallbackDefinition,
      (
        deltaTime: number,
        message: Deno.PointerValue<unknown>,
        messageSize: number | bigint,
      ) => {
        const msg_data = new Uint8Array(
          new Deno.UnsafePointerView(message!).getArrayBuffer(
            messageSize as number,
          ),
        );

        if (options.callback !== undefined) {
          options.callback({
            message: decodeMessage(msg_data),
            deltaTime,
          });
        }

        if (options.rawCallback !== undefined) {
          options.rawCallback({
            message: Array.from(msg_data),
            deltaTime,
          });
        }
      },
    );
    rtmidi.rtmidi_in_set_callback(this.device, this.callback!.pointer, null);
  }
And I wonder if there's a better way to do that and if it looks understandable as an API user. Any thoughts welcome 🙂
21 Replies
Stéphane
StéphaneOP•15mo ago
Usage would look like this, with the user having the choice between the callbacks 
midi_in.onMessage({
  rawCallback: ({ message }) => {
    console.log("Raw callback");
    console.log(message);
  },
  callback: ({ message }) => {
    console.log("Typed callback");
    console.log(midi.MessageType[message.type]);
    console.log(message.data);
  },
});
midi_in.onMessage({
  rawCallback: ({ message }) => {
    console.log("Raw callback");
    console.log(message);
  },
  callback: ({ message }) => {
    console.log("Typed callback");
    console.log(midi.MessageType[message.type]);
    console.log(message.data);
  },
});
Deno AI Helper
Deno AI Helper•15mo ago
Based on the documentation, it seems like you're on the right track with your current approach. You're providing a flexible interface for the user to handle either raw data or a converted typed object, which is a good practice for API design. However, the documentation doesn't provide a specific answer to your question about whether there's a better way to structure your API. It's important to note that the design of an API can vary greatly depending on the specific use case and requirements. One suggestion that isn't explicitly mentioned in the documentation but could improve the usability of your API is to consider using an event-driven approach. Instead of passing callbacks to the onMessage method, you could emit events when a message is received and allow users to listen for these events. This could make your API more intuitive and flexible, as users could listen for specific types of messages or handle messages in different parts of their code. Remember, this suggestion isn't explicitly supported by the documentation, so it's just a general advice based on common practices in API design. 🦕
Helpful Links: Deno.Reader | Runtime APIs | Deno Deno.ReaderSync | Runtime APIs | Deno MessagePort | Runtime APIs | Deno Worker | Runtime APIs | Deno
Stéphane
StéphaneOP•15mo ago
Cool suggestion. I'm doing that and now encounter a weird issue. I created this interface (+ the alternative for raw messages) 
export interface MidiMessageEvent extends CustomEvent {
  detail: {
    message: Message<MessageData>;
    deltaTime: number;
  };
}
export interface MidiMessageEvent extends CustomEvent {
  detail: {
    message: Message<MessageData>;
    deltaTime: number;
  };
}
and used it like this : 
  dispatchEvent(
    new CustomEvent<MidiMessageEvent>("midi.message", {
      detail: { message: decodeMessage(msg_data), deltaTime: deltaTime },
    }),
  );
  dispatchEvent(
    new CustomEvent<MidiMessageEvent>("midi.message", {
      detail: { message: decodeMessage(msg_data), deltaTime: deltaTime },
    }),
  );
But the linter is not happy about it :
Type '{ message: Message<MessageData>; deltaTime: number; }' is not assignable to type 'MidiMessageEvent'. Object literal may only specify known properties, and 'message' does not exist in type 'MidiMessageEvent'.deno-ts(2322)
despite having the propriety 🤷‍♂️ If I do this instead, it's ok... 
dispatchEvent(
  new CustomEvent("midi.message", {
    detail: { message: decodeMessage(msg_data), deltaTime: deltaTime },
  }) as MidiMessageEvent,
);
dispatchEvent(
  new CustomEvent("midi.message", {
    detail: { message: decodeMessage(msg_data), deltaTime: deltaTime },
  }) as MidiMessageEvent,
);
But I still get an error when trying to add an event listener : 
addEventListener("midi.message", (e: CustomEvent<midi.MidiMessageEvent>) => {
  console.log(e.detail);
});
addEventListener("midi.message", (e: CustomEvent<midi.MidiMessageEvent>) => {
  console.log(e.detail);
});
And btw the linter is not happy but the code runs anyway, I receive the events and can print them without a problem
NDH
NDH•15mo ago
I have a very simple, strongly typed eventBus that work great for this. I use it my Deno-Desktop framework. https://github.com/nhrones/dwm_gui/blob/main/src/coms/eventBus.ts
Stéphane
StéphaneOP•15mo ago
Looks cool, thank you for the suggestion 🙂 I managed to understand the right syntax for my above issues, but I'll try to go further Reading in more details now, and this actually makes more sense to use something like you did in my use case, that's nice 🙂 I'll try to implement this tomorrow, thanks for sharing!
NDH
NDH•15mo ago
I have some better examples of it's usage if you need.
Stéphane
StéphaneOP•15mo ago
Hi again 🙂 Thanks again for the help @ndh3193 I implemented something very similar to what you shared (you can check here : https://github.com/stfufane/deno-midi/blob/main/lib/events.ts for the types and https://github.com/stfufane/deno-midi/blob/main/lib/midi.ts#L258 for on/off/emit) Now I tried some things to take it a bit further, but with no success, and wonder if you'd have any extra knowledge on how to achieve this (maybe it's juste not possible...) Basically, with this API, the user can do this : 
midi_in.on("message", ({ message, deltaTime }) => {
  console.log("message callback at ", deltaTime);
  if (message instanceof midi.NoteOn) {
    console.log(message.data.note, message.data.velocity);
  }
});
midi_in.on("message", ({ message, deltaTime }) => {
  console.log("message callback at ", deltaTime);
  if (message instanceof midi.NoteOn) {
    console.log(message.data.note, message.data.velocity);
  }
});
which is nice, but I'd like a way to automatically infer the type of the message variable in the callback, and using generics I just did not find how to do that. My idea is something like this : 
export interface MessageEventData<T extends Message<MessageData>> {
  message: T;
  deltaTime?: number;
}

export type MessageHandler<T extends Message<MessageData>> = (
  data: MessageEventData<T>,
) => void;

on<T extends Message<MessageData>>(
  handler: MessageHandler<T>,
): void {
  this.handlers.set(T, handler); // <-- HERE, how to link the type of the generic with the map ?_?
}

private emit<T extends Message<MessageData>>(
  data: MessageEventData<T>,
): void {
  const handler = this.handlers.get(T); // <-- HERE, use the same trick
  if (handler) {
    handler(data);
  }
}
export interface MessageEventData<T extends Message<MessageData>> {
  message: T;
  deltaTime?: number;
}

export type MessageHandler<T extends Message<MessageData>> = (
  data: MessageEventData<T>,
) => void;

on<T extends Message<MessageData>>(
  handler: MessageHandler<T>,
): void {
  this.handlers.set(T, handler); // <-- HERE, how to link the type of the generic with the map ?_?
}

private emit<T extends Message<MessageData>>(
  data: MessageEventData<T>,
): void {
  const handler = this.handlers.get(T); // <-- HERE, use the same trick
  if (handler) {
    handler(data);
  }
}
So that the user would be able to do : 
midi_in.on<midi.NoteOn>(({ message, deltaTime }) => {
  // The message is automatically detected as NoteOn yay
});
midi_in.on<midi.NoteOn>(({ message, deltaTime }) => {
  // The message is automatically detected as NoteOn yay
});
I think it'd make a clearer usage with autocompletion and everything running smoothly. But T is a type, not a value, typeof T and T.prototype.constructor don't seem to work either...
NDH
NDH•15mo ago
If you use the interface/typing below, you'll get autocomplete. https://gist.github.com/nhrones/409268659347ba17a8562a680a0663b2 For more context; Please look at coreEevents: https://gist.github.com/nhrones/468b3b9b8ad7518189b1989d173640e8 eventBus: https://gist.github.com/nhrones/1fa6c27ed5f8e565a5f2ed1670755f20 example: eventTypes: in coreEvents.ts 
export type CoreEvents = {
   /** hide \<Popup\> command event */
   HidePopup: null,
   
   /** PopupReset */
   PopupReset: null,
   
   /** \<Popup\> view focus command event */
   FocusPopup: any,
   
   /** Show \<Popup\> view event */
   ShowPopup: {
      title: string,
      msg: string
   }
}
export type CoreEvents = {
   /** hide \<Popup\> command event */
   HidePopup: null,
   
   /** PopupReset */
   PopupReset: null,
   
   /** \<Popup\> view focus command event */
   FocusPopup: any,
   
   /** Show \<Popup\> view event */
   ShowPopup: {
      title: string,
      msg: string
   }
}
   // in game-ts  we fire our event

   // this is strongly typed --
   // you'll only be allowed to enter an event type from above
   // you'll get an error if the payload does not match this type
   send('ShowPopup', "", { title: 'Game Over!', msg: 'You Won!' })

   // in popup.ts  we've subscribed to this event

   // this is strongly typed --
   // you'll get an error for ent event name not typed above
   // you'll get an error if the params of this callback don't match the type
   when('ShowPopup', "", (data: { title: string, msg: string }) => {
      this.title = data.title
      this.show(data.msg)
   })

   // the app viewmodel watches for a popup touch event and fires this event
   // this `HidePopup` event type has no payload in the type.  
   // you'll get an error if you add any params in this callback
   when('HidePopup', "", () => {
      this.hide()
   })
   // in game-ts  we fire our event

   // this is strongly typed --
   // you'll only be allowed to enter an event type from above
   // you'll get an error if the payload does not match this type
   send('ShowPopup', "", { title: 'Game Over!', msg: 'You Won!' })

   // in popup.ts  we've subscribed to this event

   // this is strongly typed --
   // you'll get an error for ent event name not typed above
   // you'll get an error if the params of this callback don't match the type
   when('ShowPopup', "", (data: { title: string, msg: string }) => {
      this.title = data.title
      this.show(data.msg)
   })

   // the app viewmodel watches for a popup touch event and fires this event
   // this `HidePopup` event type has no payload in the type.  
   // you'll get an error if you add any params in this callback
   when('HidePopup', "", () => {
      this.hide()
   })
Stéphane
StéphaneOP•15mo ago
Oh, that looks great, very close to what I'm trying to achieve, I'll give it a try 🙂 Thanks a lot I didn't know this kind of syntax was possible : 
      when<EventName extends keyof T>(
         eventName: EventName,
         id: string,
         handler: EventHandler<T[EventName]>
      when<EventName extends keyof T>(
         eventName: EventName,
         id: string,
         handler: EventHandler<T[EventName]>
NDH
NDH•15mo ago
It's called a type-contract! I have a much richer example I can show if you need. For simple apps, I now prefer the function name 'on' rather than 'when' and 'fire' rather than 'send'.
Stéphane
StéphaneOP•15mo ago
Yeah, I agree with on 🙂 I went with emit for fire haha, matters of taste I guess 😄
NDH
NDH•15mo ago
Naming can help self document the semantics! like type-contract Your app sounds exciting. will you use an AudioContext to play notes? You might want to take a look at my 'comms' lib. There I call this typedEmmiter.ts https://github.com/nhrones/comms
Stéphane
StéphaneOP•15mo ago
Thanks 🙂 it's not really an app, it's meant to be used server side. It's a port of this npm package : https://github.com/justinlatimer/node-midi The idea behind it (for my use case at least) is to create remote midi controllers for basically anything 🙂
NDH
NDH•15mo ago
Stéphane
StéphaneOP•15mo ago
I used the npm version a few years ago for a digital arts project where many people could connect to my computer via their phone to control music in a live context, all at the same time, using websockets. It was pretty fun
NDH
NDH•15mo ago
If you get a chance watch that video. It's amazing.
Stéphane
StéphaneOP•15mo ago
And now I want to use Deno to do something similar but it doesn't have a midi module, so I decided to make it 😄 Thanks ! Looks pretty interesting indeed, I'll give it a go this weekend 🙂
NDH
NDH•15mo ago
Rust is the future of UI.
Stéphane
StéphaneOP•15mo ago
this kind of project gives hope about it, cause so far it looks like UI is the complicated part when using Rust
NDH
NDH•15mo ago
From the makepad vid.
No description
NDH
NDH•15mo ago
^^^