D
Deno

help

Serving multiple static files to an HTTP request

Zzidan29998/10/2023
If a client sends a request for an array of static files names like so.
["component01.js", "component01.css", "component02.js", "component02.css"]
["component01.js", "component01.css", "component02.js", "component02.css"]
Using a Deno.serve server, is it possible to serve all files in response to that one request, instead of having the client request them one by one? Would I need something like HTTP/2 or HTTP/3? And if so, are any of those supported in Deno?
Ppyrote8/10/2023
What is your use case? Browser to server or API client to server? Is returning a ZIP file with all the requested files an option?
Gguest2713148/12/2023
One way to do that is serve the files as a single JSON response.
Zzidan29998/13/2023
I don't know how to do that. Can you illustrate or link an example? My use case: I'm building an SPA. and have JS files for UI components and their corresponding CSS files. A function that renders a form needs 4 UI components for different field, buttons and so on. The form rendering function checks if those 4 UI components are available before attempting to use them. If not found, it requests them with an import statement one by one, then requests the corresponding CSS files one by one too. A ZIP file with all the requested files sounds good. But I don't know how to do that. Can you illustrate or link an example?
Ccyracrimes8/13/2023
It sounds like you're already doing the right thing. Putting the files into some kind of zip/json package wouldn't work well here because you'd need to pile on a couple of additional hacks to then still be able to actually import the scripts/stylesheets. Importing the resources individually isn't necessarily harmful at all - is your question motivated by a measured performance issue? Have you maybe looked into bundling your application in a build step using something like webpack?
Zzidan29998/13/2023
Not a performance issue, no. It's about efficiency really. A request and a response per file sounds very wasteful to me. So I wondered if I can send multiple responses per request if the request already has an array of the needed files. @pyrote mentioned a zip file with all the requested files. So that would be 1 request and 1 response per several files, which sounded pretty good to me 😄. @guest271314 mentioned a single JSON response with the files. No idea how that works 😕.
Ccyracrimes8/13/2023
Both ideas won't work easily because you want the browser to load the files as scripts and stylesheets. They would be potential solutions if the files were just used programmatically by your app, but you're importing them, which is different. You also can't send multiple "responses" per request as far as HTTP is concerned. But: Browsers are good at doing requests. Importing multiple things at the same time isn't generally less efficient than importing only one thing, unless your logic blocks the loading of some of the resources until others are loaded
Zzidan29998/13/2023
I see.. Well, I'm using Promise.all() at the moment. That shouldn't be blocking so I'll just keep using that. Thanks for the clarification.
Ccyracrimes8/13/2023
That sounds fine to me :) If you run into something like this again where you really do want to avoid many individual imports, especially if they're nested, I'd encourage you to look into module bundling with tools like webpack
Gguest2713148/13/2023
Simple {"0":[0, 255...], "1":[0, 255...]}. See https://plnkr.co/plunk/gCjYSt. Since Deno supports full duplex streaming you can stream all the files or whatever else you want using a single fetch() request, e.g., https://github.com/guest271314/native-messaging-deno/tree/fetch-duplex.
Jjoshcantcode8/13/2023
You could try opening a WebSocket, sending each file at a time and have the user and server notify each other when a file is received / sent, may be too much overhead though
Gguest2713148/13/2023
Technically you can just serve the entire combination of N files as a single Uint8Array as long as you send the offsets, too. Here I combine a Uint32Array that includes the length of the following JSON configuration, which can optionally include images, album, artist, track information to display with Media Session API in global media controls, and offsets of the following discrete ArrayBuffer s which are Opus encoded audio output by WebCodecs AudioEncoder into a single file, where I extract the images and other media information and offsets of the discrete media chunks, and play back the file in the browser https://github.com/guest271314/WebCodecsOpusRecorder, largely inspired by the Native Messaging protocol which encodes the length of a message then the message. Bonus: The resulting file is less size than Opus audio encoded in WebM container. Alternatively, include multiple files in a .tar.gz or .zip file then extract the files from archive using Compression Streams, e.g., here I extract the node executable from the Node.js nightly release and get rid of everything else - in the browser https://github.com/guest271314/download-node-nightly-executable. This is not a particularly difficult case to solve. Just choose your approach and stick to that on the fron-end and back-end.
Gguest2713148/13/2023
That will work. Create the script elements and set textContent. With CSSOM we can set CSS Rules directly in a stylesheet, see Modify element :before CSS rules programmatically in React and Empty style element with working CSS rules?.
Stack Overflow
Modify element :before CSS rules programmatically in React
I have an element whose :before style has to be modified based on calculations. export class Content extends React.Component { render() { return ( <div className="ring-b...
Ccyracrimes8/13/2023
I fully agree that you can do this, I just think it's an unnecessary hack in this case. In fairness of course, much of the technology that makes the modern web work could be described this way :P Well, I guess it's often necessary hacks. I don't think this particular thing is worth doing in this particular case
Gguest2713148/13/2023
Somebody said in some article Emscripten started off as a hack. Now there's WebAssembly and Bytecode Alliance. It's not really a hack. No more than YouTube using Media Source Extensions to serve multiple files for media playback. Again, at the extreme side there are Web Bundles https://github.com/GoogleChromeLabs/webbundle-plugins/blob/main/packages/webbundle-webpack-plugin/README.md, though Crromium/Chrome did remove support for navigation to Web Bundles. Choose your approach; JSON, Streams, whatever. They each work, as I demostrate in the linked working examples. Nothing is stopping you from encoding you data as JSON then using a single import assertion to import all of the files at once. Here's another example of piping multiple files to the same stream https://github.com/guest271314/AudioWorkletStream, something like
let port;
onmessage = async e => {
'use strict';
if (!port) {
[port] = e.ports;
port.onmessage = event => postMessage(event.data);
}
const { urls } = e.data;
// https://github.com/whatwg/streams/blob/master/transferable-streams-explainer.md
const { readable, writable } = new TransformStream();
(async _ => {
for await (const _ of (async function* stream() {
while (urls.length) {
yield (await fetch(urls.shift(), {cache: 'no-store'})).body.pipeTo(writable, {
preventClose: !!urls.length,
});
}
})());
})();
port.postMessage(
{
readable,
},
[readable]
);
};
let port;
onmessage = async e => {
'use strict';
if (!port) {
[port] = e.ports;
port.onmessage = event => postMessage(event.data);
}
const { urls } = e.data;
// https://github.com/whatwg/streams/blob/master/transferable-streams-explainer.md
const { readable, writable } = new TransformStream();
(async _ => {
for await (const _ of (async function* stream() {
while (urls.length) {
yield (await fetch(urls.shift(), {cache: 'no-store'})).body.pipeTo(writable, {
preventClose: !!urls.length,
});
}
})());
})();
port.postMessage(
{
readable,
},
[readable]
);
};
Gguest2713148/13/2023
Just for completeness here is an example of using signed Web Bundles in an Isolated Web App - with a Deno and txiki.js TCP server https://github.com/GoogleChromeLabs/telnet-client/pull/18. For some reason Direct Sockets have been gated behind Progressive Web Apps and now Isolated Web Apps. Somebody thinks that leads to greater "security", however I have already incorporated browser extension code into the signed Web Bundle which provides a means to communicate between an arbitrary Web page and the IWA for the purpose of controlling a TCPSocket() connection from any arbitrary Web site I choose.
GitHub
Build software better, together
GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 330 million projects.
Gguest2713148/13/2023
Note, for standardized approach you can serve a FormData object from the server and just read the data as File objects in the browser/client, see https://stackoverflow.com/a/47067787.
Gguest2713148/13/2023
Here you go https://plnkr.co/edit/jluHMgfqdPD1z6Y1?open=lib%2Fscript.js Server
onfetch = (e) => {
if (e.request.url.includes('files')) {
const fd = new FormData();
fd.append(
'0',
new Blob([`console.log('script 0')`], { type: 'text/javascript' }),
'0.js'
);

fd.append(
'1',
new Blob([`console.log('script 1')`], { type: 'text/javascript' }),
'1.js'
);

fd.append(
'2',
new Blob([`body {color:blue}`], { type: 'text/css' }),
'2.css'
);
let client = clients.get(e.clientId);
// console.log(e.request.headers.get('content-type'));
e.respondWith(
new Response(fd, {
headers: {},
})
);
}
};
onfetch = (e) => {
if (e.request.url.includes('files')) {
const fd = new FormData();
fd.append(
'0',
new Blob([`console.log('script 0')`], { type: 'text/javascript' }),
'0.js'
);

fd.append(
'1',
new Blob([`console.log('script 1')`], { type: 'text/javascript' }),
'1.js'
);

fd.append(
'2',
new Blob([`body {color:blue}`], { type: 'text/css' }),
'2.css'
);
let client = clients.get(e.clientId);
// console.log(e.request.headers.get('content-type'));
e.respondWith(
new Response(fd, {
headers: {},
})
);
}
};
Browser
async function processFormData() {
fetch('./?files')
.then(async (r) => {
console.log(r.headers.get('content-type'));
/*
return new Response(await r.text(), {
headers: { 'content-type': r.headers.get('content-type') },
})
*/
return r.formData();
})
.then(async (fd) => {
console.log(fd);
for (const [, file] of fd) {
switch (file.type) {
case 'text/javascript':
const script = document.createElement('script');
script.textContent = await file.text();
document.body.appendChild(script);
break;
case 'text/css':
const style = document.createElement('style');
style.textContent = await file.text();
document.head.appendChild(style);
break;
default:
console.log(file.type);
}
}
})
.catch(console.warn);
}
async function processFormData() {
fetch('./?files')
.then(async (r) => {
console.log(r.headers.get('content-type'));
/*
return new Response(await r.text(), {
headers: { 'content-type': r.headers.get('content-type') },
})
*/
return r.formData();
})
.then(async (fd) => {
console.log(fd);
for (const [, file] of fd) {
switch (file.type) {
case 'text/javascript':
const script = document.createElement('script');
script.textContent = await file.text();
document.body.appendChild(script);
break;
case 'text/css':
const style = document.createElement('style');
style.textContent = await file.text();
document.head.appendChild(style);
break;
default:
console.log(file.type);
}
}
})
.catch(console.warn);
}
LLeokuma8/15/2023
The browser will download the files only once in a lifetime and then cache them for future access. Some of the solutions proposed here will prevent the browser from caching them, forcing a redownload of all the files everytime your SPA is opened. Also I believe the cost of downloading and processing a Zip file is higher than downloading the files separately compressed as gzip or Brotli which are web standards
Zzidan29998/15/2023
Wow, hack/worth or not, I did not know about any of the approachs you mentioned. Thanks a lot. Now, I did some reading on FormData and Blobs. And wondering if it's possible to use the JS/CSS files from a FormData Blob without creating/appending script tags like when using import? Or is this the only way for that approach? Like auto caching? Man, do I have a lot of reading to do xD Thanks for the heads up
LLeokuma8/15/2023
Yes browsers cache files automatically, so only the first time the user accesses your SPA the files will be actually downloaded. On the following times when you do import() the browser will retrieve from cache
Zzidan29998/15/2023
In my novice mind. A user sends a sign in request with their credentials, if successful, I would respond with body like this:
{ user: { id: '', name: '', phone: '' }, files: [ file/blob, file/blob ] }
{ user: { id: '', name: '', phone: '' }, files: [ file/blob, file/blob ] }
and the files would work without the need to create and append script elements like when done using import. Instead of responding with just user data + files names so that a pre existing function can import them one by one.
Gguest2713148/24/2023
Now, I did some reading on FormData and Blobs. And wondering if it's possible to use the JS/CSS files from a FormData Blob without creating/appending script tags like when using import? Or is this the only way for that approach?
Technically you can create a Blob URL and use import(). What is the use case?
Zzidan29998/24/2023
I have divided my SPA's UI into what I named components, modules and events. A component is function that returns an HTML element. A module is function that uses components to build pages, forms, popups Etc. An event is a function that is called by a component. All of the above is loaded on demand only, so each module function has an array for component dependencies that are checked for availability (are X, Y, Z already loaded?) before the module attempts to use them. And if they are available, the availability function (which returns a promise) just fulfills. And if one or more of the component dependencies are missing, it runs a Promise.all() on an array of import() statements to request the component files and their corresponding CSS files one by one, then fulfills or rejects accordingly. And this is working just fine for now, but I wanted to see if I can request all files at once instead of one by one. So I wanted to try HTTP/2 server push in Deno but then found that its no longer supported. Then you suggested using the FormData object which would allow me to send a list of files like I wanted to achieve, expect that it involves creating and appending <script> and <link> HTML tags to the DOM for each file, which I would like to avoid.
AAapoAlas8/25/2023
Web bundles is also a thing that was promising this exact feature but it was apparently sort of abandoned this year.
AAapoAlas8/25/2023
Chrome Developers
Get started with Web Bundles - Chrome Developers
Web Bundles enable you to share websites as a single file over Bluetooth and run them offline in your origin's context.
Gguest2713148/26/2023
Web Bundles was not abandoned.
AAapoAlas8/26/2023
I was going to link the Chromium bug where they say this:
The code for navigation to Web Bundles has been behind a flag for a long time, but we have no plans to ship it.
but indeed I'd misunderstood, it's not the full Web Bundles but just "navigation to Web Bundles", using something like file:///home/user/web_bundle to load a page.
Our focus on WebBundle has shifted to subresource loading and Isolated Web Apps use cases.
So yeah, still going but with a somewhat limited scope. My bad.
Gguest2713148/26/2023
Chromium did remove the ability to navigate directly to a (Signed) Web Bundle. It is still possible to create a (Signed) Web Bundle including whatever files you want https://github.com/GoogleChromeLabs/telnet-client/issues/25 (although I don't think it is possible to register a ServiceWorker in isolated-app: protocol; at least I have not been able to do so using the Webpack and Web Bundle plugins here https://github.com/GoogleChromeLabs/telnet-client/).

Looking for more? Join the community!

Recommended Posts
Kv encodes integer as doubleAnyone know why Deno.kv encodes an integer in a mutli-part key as a double? ``` const key = ["int",Looking for caching API ResponsesHello there, I'm looking for a package/or examples to cache API responses for a fresh project whicUsing ZeroMQ from NPMAnyone got fixed the same error as this issue? https://github.com/denoland/deno/issues/18345Trying to pass parameters to runtime from rustHi, I'm launching a deno runtime from my project's code. I'm trying to pass some dynamic json variconsole.log in a worker - capture or hide output?Running under `deno repl` with no permissions, this prints 'yee'. In a browser's dev tools it would Deno debugging configuration is missing?```json "deno.enable": true, "deno.unstable": true, "deno.enablePaths": ["./api"], "deno.docIs there a way to show permissions used on start up during runtime?For example: If I was to run an App with `permissions: --allow-net=${hostname}:${port}` is there a wrestrict path applied for import mapshi, is it possible to restrict importmap application to only specific path? the rationale is: - vscoBest way to pass backend data to SSR rendered view?Looking for the "proper" way to pass data to an SSR rendered view. My first idea is setting custom pImport mapHi I have this in deno.json: ``` "imports": { "./app.controller": "./app.controller.ts", ``` butWhat should I specify to pass a "char *" in Deno.dlopen?For example, this is how you would use it in Python: ```python import ctypes lib = ctypes.cdll.LoadOverride NPM package dependencyRelating to my question yesterday (https://canary.discord.com/channels/684898665143206084/1137532072VS Code's debugger terminates before finishedI notice that occasionally the debugger terminates at the middle of the running, or at least doesn'tI need some guidance regarding interprocess communication with DenoWhat are good cross-platform (Windows, Linux) options for interprocess communication / data sharing What is the Deno class?What is Deno in Deno.cwd() ? Can someone link me to documentation? Why doesn't the Deno class need tGetIP + Ctx from middleware to a route not passedHi, I have test to retrieve IP visitor Code taken from Discord is always returning "localhost" (wheerror: No config file found in VS CodeI'm new in Deno and in coding, and need help with the following issue. I have installed Deno throughDeclaration merging for external packagesHi! I'm using unoCSS with Deno Fresh and have been attempting to use the attributify preset (https:Deno.serve AbortController onError. How?When I run my new Deno.serve() updated implementation I get a type error. Perhaps I'm doing it wrongLocalStorage OverrideI'm using a library (@supabase/supabase-js), and I'm compiling my Deno app into an executable, howev