D
Deno

help

Caching results from API

APA person3/15/2024
Hi, I'm currently working on a project where I get data from an API (https://www.robotevents.com/api/v2), process the data, and return it to the browser. I was wondering about how I could cache the results from the API because if the website has let's say 5000 users then the API won't be able to handle all of those, so I want to cache the data somewhere and update it every 24 hours or so. Do I use something like Deno KV for this and then when the server get's a request it checks if data is already in that database and returns it. Also, how do I call the API every 24 hours? Thank you for your help.
DAHDeno AI Helper3/15/2024
Hello! Caching results from an API is a great way to improve performance and reduce the load on the API server. In Deno, you can use the Cache API to cache Request/Response objects, which can be particularly useful when you're generating responses dynamically. The documentation provides an example of how to use the Cache API:
import { serve } from "https://deno.land/std@0.158.0/http/server.ts";

const CACHE = await caches.open("v1");

serve(async (req: Request) => {
const res = await CACHE.match(req);
if (res) {
res.headers.set("x-cache-hit", "true");
return res;
}

const { searchParams } = new URL(req.url);
const name = searchParams.get("name");
const response = new Response(`Hello ${name}!`);

await CACHE.put(req, response.clone());
return response;
});
import { serve } from "https://deno.land/std@0.158.0/http/server.ts";

const CACHE = await caches.open("v1");

serve(async (req: Request) => {
const res = await CACHE.match(req);
if (res) {
res.headers.set("x-cache-hit", "true");
return res;
}

const { searchParams } = new URL(req.url);
const name = searchParams.get("name");
const response = new Response(`Hello ${name}!`);

await CACHE.put(req, response.clone());
return response;
});
In this example, the cache is persisted to the file system, so restarting Deno will not flush the cache. You can use similar logic to cache the responses from the API you're calling. When a request comes in, you can check if the data is already in the cache and return it if available. If not, you can fetch the data from the API, cache it, and then return it to the user. As for updating the cache every 24 hours, you can use a simple interval to trigger the cache update. Here's a general idea of how you might set up such an interval:
// Function to update the cache
async function updateCache() {
// Logic to call the API and update the cache
}

// Call `updateCache` immediately to populate the cache
updateCache();

// Set an interval to call `updateCache` every 24 hours
setInterval(updateCache, 24 * 60 * 60 * 1000);
// Function to update the cache
async function updateCache() {
// Logic to call the API and update the cache
}

// Call `updateCache` immediately to populate the cache
updateCache();

// Set an interval to call `updateCache` every 24 hours
setInterval(updateCache, 24 * 60 * 60 * 1000);
Please note that the above code is a general guide and not directly from the documentation. You'll need to adapt it to fit your specific use
DAHDeno AI Helper3/15/2024
case, including handling the API calls and processing the data as needed. Regarding Deno KV, it's a globally replicated database that you can use to add state to your serverless functions or applications. It's very easy to use, as shown in the documentation:
const kv = await Deno.openKv();
const kv = await Deno.openKv();
You could use Deno KV to store the processed data from the API if you need a more persistent and globally replicated solution. However, for simple caching purposes, the Cache API might be sufficient. Remember to handle the cache invalidation logic according to your needs, such as setting a TTL (time to live) for the cache entries or manually clearing them after 24 hours. 🦕
APA person3/15/2024
Can I do the same with Deno Web Workers?
EEsente3/15/2024
There is no Deno Web Workers. And you can't use the Cache API on Deno Deploy, yet You can also use Deno.cron to fetch the API every 24h
Rraunioroo3/15/2024
Do you just mean there is no Deno-branded version of Web Workers? Because Web Workers in Deno definitely is a thing. (not sure if cache api is usable inside web workers, but would be slightly surprised if it wasn't.)
EEsente3/15/2024
Ah, sorry I was wrong. I was basing off my previous encounter with errors not having Worker available, but that was probably on Deploy.
APA person3/15/2024
I probably is cause you can just put it inside the function that calls the worker
Ccknight3/15/2024
I'd use KV to store the cached entry (assuming data is always less than 64kb). When a request gets the data from KV, if it isn't there, then call the API and store the value. You can use Deno cron to call the API every 24 hours:
Deno.cron("Call API every day at 1am", "0 1 * * *", async () => {
const result = await callApi();
await kv.set(["myAPICache"], result);
});
Deno.cron("Call API every day at 1am", "0 1 * * *", async () => {
const result = await callApi();
await kv.set(["myAPICache"], result);
});

Looking for more? Join the community!

Recommended Posts
INTERNAL_SERVER_ERROR_2 / 500 response codeSeveral clients concurrently reported INTERNAL_SERVER_ERROR_2 with a 500 error on requests to a deplMy site seems to be under attackI receive a large amount of request last night. Does this mean my site is under attack...?Deno drizzle-kit migrate:pgI don't know if its drizzle error or Deno error... I saw path resolution problem, it is said `C:UseDoes V8 engine eventloop/threading model conflict with rust?hi, I am trying to embed deno's rusty_v8 into rust application. I am confused about the eventloop/thHow installing Silverbullet on denoOn the officle website of silverbullet they say we can install silver bullet on deno but i dont knowimplementing swagger in deno typescript with deno_swagger_doc on deno.land. it throws error.error: Uncaught (in promise) Error: Cannot find module './isomorphic.node' var e = new Error("Cannnpm install -D equivalentI want to use drizzle-kit. `npm install -D drizzle-kit` but don't know how to get it work in Deno.Kill deno process treeHow would you kill a Deno process _tree_ as in, force kill of Deno process _and any of it's subproceWhat to do if they are attacking a project of mine?Hello!, Yesterday I suffered an attack on one of my projects, they made 6 million requests in a veryError Installing Deno using PowerShellHello, may I please ask for assistance in regards to installing Deno using PowerShell on VS Code. AnSvelte/SvelteKit support in 2024What’s your experience?Subprocesscan someone help me how do i get to show Hello world in my terminal by using Deno.Commands () using Problems using @apple/app-store-server-library, how do I debug?Hi, I'm attempting to build out the backend of an IOS app on Supabase and want to be able to verifyIs there an easy way to read tls certificate and key from a filesystem?I do the following: ```js if (Deno.env.has("TLS_CERT_PATH") && Deno.env.has("TLS_PKEY_PATH")) { coPartials in a redirect ???Does anyone have any idea how i can redirect after login to a route with initialised partial: ExamHow to troubleshoot runaway (100% CPU) deno processDeno has been running a couple of websites rock solid for many months. But during last week or so I'Import SCSS in TS filesHey, I am currently building a template for web development and I want to use ESBuild as a bundler. How to modify HTTP cache folderHello, I'm using Astro framework with Deno and Directus, but after a while of the website running onDeno deployment can't connect to the Mongo Cloud, due to the Mongo Cloud Network Access Rules.Is there a way to add the Deno Deploy IP to Mongo Cloud? The Deno Depoy Regions only region names fIs there a way to pull a module from x in favour of jsr?I have ported my module `@vicary/fresh-graphql` to JSR and I would like to redirect my users go JSR