examples of Deno+Llamacpp
Are there examples of Deno and Ollama/Llamacpp setups in typescript?
Its should t be hard since it just an api call to the instance with Docker.
Essentially I want to demonstrate if we can have a CUDA or MPS enabled image of llamacpp or ollama, as an api over the internet, and query it directly via a deno/fresh app running on Deno deploy or railway
1 Reply
I wasn't able to find the answer to your question in the official Deno documentation. However, based on your message, I can provide some general advice.
Firstly, it seems there might be a typo in your message. The correct spelling might be "LlamaCPP" or "OllamaCPP" if you're referring to a specific library or framework. It's important to use the correct names when searching for documentation or examples.
Secondly, you mentioned "Its should t be hard since it just an api call to the instance with Docker." This sentence seems to have a typo. It might be meant to say, "It shouldn't be hard since it's just an API call to the instance with Docker." Ensuring clarity in your descriptions can help when seeking assistance or examples from others.
Please note that this advice is not based on the Deno documentation, and I would advise you to double-check the accuracy of the advice given.
If you're looking to set up an API with Deno that communicates with a Dockerized service that utilizes CUDA or MPS, you would typically need to ensure that the Docker container is set up correctly with the necessary dependencies and that it exposes an API endpoint that your Deno application can communicate with. Your Deno application would then make HTTP requests to this API endpoint.
For a Deno/Fresh application, you would write TypeScript code to make these API calls and handle the responses accordingly. Since this is a specific use case involving technologies like CUDA, MPS, and potentially LlamaCPP or OllamaCPP, you might need to look for examples or documentation specific to those technologies or reach out to their respective communities for more targeted assistance.