RedBean•2mo ago

Is deno's cache philosophy coherent for local files ?

I'm new to deno and surprised that it will cache local files that are imported and not update the cache when those files change. This is particularly painful when tying to use deno in Jupyter. I assume that a good cache should be transparent to the user (developer in this case). If I want the cache to be further optimized so it no longer behaves like a normal cache but require manually tracking dependencies then I should have to enable an option or do something explicit. But maybe I am not understanding the design philosophy of deno ? It would be nice if there were an easy way to stop deno caching local files during development of jupyter notebooks. It would be even better if deno could check the date of the files and maintain a coherent cache for local files.

9 Replies

bartlomieju•2mo ago

This is how ES modules work, browsers behave the same way. If you need to import a file multiple times and have changes by reflected you can append a random string at the end of module like so:

const foo = await import(`./my_file.ts?cachebust=${new Date()}`);

const foo = await import(`./my_file.ts?cachebust=${new Date()}`);

Keep in mind that this solution effectively leaks memory as it just loads new modules all the time and the "old" modules are never unloaded

RedBeanOP•2mo ago

I would need to add that during dev and then remove it for prod. Would it make more sense to have a way of telling deno not to cache files with a URL and globing ? I mention it because it is a nasty surprise when first trying to pick up deno.

bondsandbarrels•2mo ago

I used to work around this by simply restarting the kernel. Is that an option? Not saying this is optimal, simply offering another quick workaround.

bartlomieju•2mo ago

We might consider that, but as it stands at the moment we don't have infrastructure to provide such functionaity. Feel free to open an issue with a feat request though

RedBeanOP•2mo ago

I am new to deno but it seems the cache can persist across restarts. It makes sense that it should for files that are not under development.

jeff.hykin•2mo ago

(I'm not part of the deno team btw) AFAIK NodeJS does this as well when ES imports are enabled. Its not a deno behavior, its an ES import vs common js behavior. It is annoying in the repl and juypter. Another painful edgecase (for files not repl) is execution order:

console.log("1")
import "hello_world.js"

console.log("1")
import "hello_world.js"

Will print "Hello world" then "1" Because static imports get hoisted to the top It definitely shoudnt persist across restarts though. That would 100% be a bug for local-file-path imports. The "cache" is only in ram.

RedBeanOP•2mo ago

I am unsure of my understanding but there is certainly a cache directory https://denolib.gitbook.io/guide/advanced/deno_dir-code-fetch-and-cache If I recall I was seeing files in the gen directory that were associated with imports of TS local files but I am less sure if the issue was there with a restart. I was trying to allow for updating a file without a restart - as we can imagine being many cells into a Jupyter notebook and then wanting to modify a local import that would require re-running the entire notebook again!

DENO_DIR, Code Fetch and Cache | A Guide to Deno Core

jeff.hykin•2mo ago

Ah, for TS that is true, that's my bad for assuming the local import was JavaScript. Should still 100% be a bug across restarts though. I agree its annoying in Jupyter. To be fair though, python reimports also don't work in Jupyter. I mean python has some force-reload stuff but its about the same amount of work as force reloading in deno, and is equally not-good-for-prod. You might want to add something like this at the top:

const load = (path)=>import(
    path+(
        Deno.env.get("PROD") ? "" : `${Math.random()}`
    )
  )

const load = (path)=>import(
    path+(
        Deno.env.get("PROD") ? "" : `${Math.random()}`
    )
  )

Then use that instead of imports:

// var is the only one that let's the cell run twice without error
var utils = load("./utils.ts")

// var is the only one that let's the cell run twice without error
var utils = load("./utils.ts")

RedBeanOP•2mo ago

That is clever, thanks.