NobbZ
NobbZ9mo ago

How to use vendoring with node_modules

As trying to recreate a DENO_DIR via deno cache creates different results on different machines, I have to assume that the lockfile is not enough. So I wanted to try to vendor instead, so I set "vendor": true in my deno.jsonc and ran a single build, which creates a vendor and a node_modules folder. Even though I copied both, the vendor and node_modules to my git repository, the remote (airgapped) builder fails like this:
Download https://registry.npmjs.org/wrap-ansi
Download https://registry.npmjs.org/wrappy
Download https://registry.npmjs.org/yallist
Download https://registry.npmjs.org/yaml
Download https://registry.npmjs.org/zwitch
error: failed reading lockfile '/build/source/deno.lock'

Caused by:
Error getting response at https://registry.npmjs.org/@alloc/quick-lru for package "@alloc/quick-lru": error sending request for url (https://registry.npmjs.org/@alloc/quick-lru): error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution: error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution: dns error: failed to lookup address information: Temporary failure in name resolution: failed to lookup address information: Temporary failure in name resolution
Download https://registry.npmjs.org/wrap-ansi
Download https://registry.npmjs.org/wrappy
Download https://registry.npmjs.org/yallist
Download https://registry.npmjs.org/yaml
Download https://registry.npmjs.org/zwitch
error: failed reading lockfile '/build/source/deno.lock'

Caused by:
Error getting response at https://registry.npmjs.org/@alloc/quick-lru for package "@alloc/quick-lru": error sending request for url (https://registry.npmjs.org/@alloc/quick-lru): error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution: error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution: dns error: failed to lookup address information: Temporary failure in name resolution: failed to lookup address information: Temporary failure in name resolution
(there are much of the Download https://registry.npmjs.org/… lines, all refer to npmjs of course)
6 Replies
NobbZ
NobbZOP9mo ago
Also weird, other tools can properly read the lockfile, like cat and jqing just works…
Deno AI Helper
Deno AI Helper9mo ago
I wasn't able to find the answer to your question in the official Deno documentation. However, based on the information you've provided, there are a couple of things to note: 1. Deno does not use node_modules by design. The presence of a node_modules folder is not a standard part of Deno's workflow, as Deno uses URLs to manage dependencies. 2. The error message you're seeing suggests that there is a network issue on your remote (airgapped) builder. It seems like it's trying to reach out to the npm registry, which should not be the case if you're using Deno and have vendored all your dependencies correctly. 3. The deno.lock file is meant to ensure that the exact same dependencies are used every time you run deno cache. If the lockfile is not being respected, or if there are network calls being made when you expect everything to be vendored, there might be an issue with how the vendoring was set up. 4. Make sure that when you're copying the vendor directory to your git repository, you're also including the deno.lock file, as it is crucial for ensuring that the correct versions of your dependencies are used. Please note that this advice is not based on the Deno documentation, and you should double-check the accuracy of the advice given. It's also possible that there might be a misunderstanding of how Deno's vendoring works, so I would recommend reviewing the official documentation on dependency management and vendoring in Deno, and ensuring that you're following the recommended practices.
NobbZ
NobbZOP9mo ago
"airgapped" means, without internet, yes. Any failing internet access is by design. And I have ensured by hashing mechanisms, that the vendor as well as the node_modules on the builder are the same as on my machine. After looking closer at the build output, it seems that deno task resolves correctly, and then the actual error happens as part of the resolved task. On the dev host, I also can observe that fetching of dependencies happens "in stages". I first see download progress, then the following lines get printed, then another batch of dependencies is downloaded.
Task build deno task lume
Task lume echo "import 'lume/cli.ts'" | deno run -A -
Task build deno task lume
Task lume echo "import 'lume/cli.ts'" | deno run -A -
It also appears that on the dev host only the second batch of downloads appears to create the node_modules. So the issue might actually be in lume, rather than deno.
Leokuma
Leokuma9mo ago
I think I've seen an open issue describing problems to cache npm deps. Might be a bug
NobbZ
NobbZOP9mo ago
Can you link the issue? I'd be happy to provide more information or reproducers if possible
Leokuma
Leokuma9mo ago
Actually it was not an issue. It was question in this channel that I thought might be related: https://discord.com/channels/684898665143206084/1212752847364431923/1212768217798279178 Not sure though