D
Deno

help

Puppeteer: "BadResource: Bad resource ID" on Ubuntu

DDNA3/2/2024
Hello, i just switched my server os over to ubuntu server 23.10. When executing my script, it immediately throws an error an exits. Before, on my windows machine, the script worked completely fine. Error:
error: Uncaught (in promise) BadResource: Bad resource ID
const result = await reader.read(inspectArr);
^
at read (ext:deno_io/12_io.js:116:28)
at FsFile.read (ext:deno_fs/30_fs.js:706:12)
at readDelim (https://deno.land/std@0.93.0/io/bufio.ts:652:33)
at readDelim.next (<anonymous>)
at readStringDelim (https://deno.land/std@0.93.0/io/bufio.ts:702:20)
at readStringDelim.next (<anonymous>)
at readLines (https://deno.land/std@0.93.0/io/bufio.ts:711:18)
at readLines.next (<anonymous>)
at waitForWSEndpoint (https://deno.land/x/puppeteer@16.2.0/src/deno/BrowserRunner.ts:168:20)
at eventLoopTick (ext:core/01_core.js:166:7)
error: Uncaught (in promise) BadResource: Bad resource ID
const result = await reader.read(inspectArr);
^
at read (ext:deno_io/12_io.js:116:28)
at FsFile.read (ext:deno_fs/30_fs.js:706:12)
at readDelim (https://deno.land/std@0.93.0/io/bufio.ts:652:33)
at readDelim.next (<anonymous>)
at readStringDelim (https://deno.land/std@0.93.0/io/bufio.ts:702:20)
at readStringDelim.next (<anonymous>)
at readLines (https://deno.land/std@0.93.0/io/bufio.ts:711:18)
at readLines.next (<anonymous>)
at waitForWSEndpoint (https://deno.land/x/puppeteer@16.2.0/src/deno/BrowserRunner.ts:168:20)
at eventLoopTick (ext:core/01_core.js:166:7)
Relevant code:
import puppeteer from "https://deno.land/x/puppeteer@16.2.0/mod.ts";
import * as path from "https://deno.land/std@0.197.0/path/mod.ts";

const browser = await puppeteer.launch({
headless: true,
product: "chrome",
defaultViewport: {width: 1280, height: 720},
args: [
//
`--disable-extensions-except=${path.join(Deno.cwd(), "extensions", "cjpalhdlnbpafiamejdnhcphjbkeiagm", "1.50.0_0")}`,
`--load-extension=${path.join(Deno.cwd(), "extensions", "cjpalhdlnbpafiamejdnhcphjbkeiagm", "1.50.0_0")}`,
"--mute-audio",
],
});
import puppeteer from "https://deno.land/x/puppeteer@16.2.0/mod.ts";
import * as path from "https://deno.land/std@0.197.0/path/mod.ts";

const browser = await puppeteer.launch({
headless: true,
product: "chrome",
defaultViewport: {width: 1280, height: 720},
args: [
//
`--disable-extensions-except=${path.join(Deno.cwd(), "extensions", "cjpalhdlnbpafiamejdnhcphjbkeiagm", "1.50.0_0")}`,
`--load-extension=${path.join(Deno.cwd(), "extensions", "cjpalhdlnbpafiamejdnhcphjbkeiagm", "1.50.0_0")}`,
"--mute-audio",
],
});
Troubleshooting steps taken: - Re-ran installation script - Cleared cache - Rebooted machine uname -a:
Linux dnarpi 6.5.0-1011-raspi #14-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 9 14:06:28 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
deno --version:
deno 1.41.1 (release, aarch64-unknown-linux-gnu) v8 12.1.285.27 typescript 5.3.3
No description
Ccknight3/2/2024
This doesn't answer your question I'm afraid, but if you're not deeply invested in Puppeteer, you could consider using the excellent and Deno native Astral https://astral.deno.dev/
Introduction | Astral
A simple introduction to Astral
DDNA3/2/2024
I'll take a look at that real quick, give me a few minutes
Ccknight3/2/2024
But my best guess as to your problem is the really old version of std being imported (like 120 versions or so old) which I'm guessing is not compatible with the latest Deno.
DDNA3/2/2024
No description
DDNA3/2/2024
The problem is just, that puppeteer for deno hasnt been updated in 2 years Maybe i'd try the node version?
Ccknight3/2/2024
That was going to be my next suggestion. @lino-levan any thoughts?
IioB3/2/2024
Yeah, I mean, Astral will work for arm64, but you'd need to supply a browser binary yourself Everything after that will work just fine I have no idea why chrome for testing doesn't ship linux arm64 binaries In the medium term I'm adding support for Firefox but FF is even more undocumented than Chromium is when it comes to browser automation.
DDNA3/2/2024
I just installed/downloaded chrome using npx @puppeteer/browsers install chrome@stable, but that returned me the same error
No description
DDNA3/2/2024
That was pretty much the only source of the chrome binary i found online
IioB3/2/2024
chrome doesn't ship an arm linux binary chromium does iirc I believe you should be able to sudo apt-get install chromium-browser or something
DDNA3/2/2024
That worked! Thank you so much
DDNA3/2/2024
Using Astral, how can i use request interception?
No description
IioB3/2/2024
what are you trying to do? I'd recommend looking at the celestial bindings you can get them on a page using https://deno.land/x/astral@0.3.5/mod.ts?s=Page&p=prototype.unsafelyGetCelestialBindings.
const celestial = page.unsafelyGetCelestialBindings();
const celestial = page.unsafelyGetCelestialBindings();
that opens up access to the raw chrome devtools protocol which gives you a ton of control it's fully typed too which is quite nice
IioB3/2/2024
the docs for the chrome devtools protocol can be found https://chromedevtools.github.io/devtools-protocol/
Chrome DevTools Protocol
Chrome DevTools Protocol - version tot
IioB3/2/2024
depending on what you're trying to do, you should look at https://chromedevtools.github.io/devtools-protocol/tot/Fetch/
Chrome DevTools Protocol
Chrome DevTools Protocol - version tot - Fetch domain
IioB3/2/2024
something like
const celestial = page.unsafelyGetCelestialBindings();
await celestial.Fetch.enable();
celestial.addEventListener("Fetch.requestPaused", (event)=>{
console.log("got event", event);
// run await celestial.Fetch.continueRequest({ requestId }); to continue request
});
const celestial = page.unsafelyGetCelestialBindings();
await celestial.Fetch.enable();
celestial.addEventListener("Fetch.requestPaused", (event)=>{
console.log("got event", event);
// run await celestial.Fetch.continueRequest({ requestId }); to continue request
});
might be a good start
DDNA3/2/2024
Code:
const videoPage = await browser.newPage();

const endTimeMin = Date.now() + 5000;
// await videoPage.setRequestInterception(true);
const celestial = page.unsafelyGetCelestialBindings();

await celestial.Fetch.enable({});
const videoPage = await browser.newPage();

const endTimeMin = Date.now() + 5000;
// await videoPage.setRequestInterception(true);
const celestial = page.unsafelyGetCelestialBindings();

await celestial.Fetch.enable({});
Error:
error: Uncaught (in promise) InvalidStateError: readyState not OPEN
this.ws.send(JSON.stringify({
^
at WebSocket.send (ext:deno_websocket/01_websocket.js:326:13)
at Celestial.#sendReq (https://deno.land/x/astral@0.3.5/bindings/celestial.ts:13261:13)
at Object.enable (https://deno.land/x/astral@0.3.5/bindings/celestial.ts:21634:33)
at file:///E:/asdfasdfasdfasdf/download-account.ts:141:26
at eventLoopTick (ext:core/01_core.js:166:7)
error: Uncaught (in promise) InvalidStateError: readyState not OPEN
this.ws.send(JSON.stringify({
^
at WebSocket.send (ext:deno_websocket/01_websocket.js:326:13)
at Celestial.#sendReq (https://deno.land/x/astral@0.3.5/bindings/celestial.ts:13261:13)
at Object.enable (https://deno.land/x/astral@0.3.5/bindings/celestial.ts:21634:33)
at file:///E:/asdfasdfasdfasdf/download-account.ts:141:26
at eventLoopTick (ext:core/01_core.js:166:7)
No description
IioB3/2/2024
perhaps unsafelyGetCelestialBindings should wait for the websocket to be open hm
DDNA3/2/2024
I cant use await on page.unsafelyGetCelestialBindings() however
IioB3/2/2024
okay, try adding
export async function websocketReady(ws: WebSocket) {
await new Promise<void>((res) => {
ws.onopen = () => {
res();
};
});
}
export async function websocketReady(ws: WebSocket) {
await new Promise<void>((res) => {
ws.onopen = () => {
res();
};
});
}
and then doing
const videoPage = await browser.newPage();

const endTimeMin = Date.now() + 5000;
// await videoPage.setRequestInterception(true);
const celestial = page.unsafelyGetCelestialBindings();
await websocketReady(celestial.ws);
await celestial.Fetch.enable({});
const videoPage = await browser.newPage();

const endTimeMin = Date.now() + 5000;
// await videoPage.setRequestInterception(true);
const celestial = page.unsafelyGetCelestialBindings();
await websocketReady(celestial.ws);
await celestial.Fetch.enable({});
this will be fixed in the next release of astral
DDNA3/2/2024
I did implement this a bit differently than you did, but the result should be the same anyway. However, for some reason, its stuck after "done 0", basically at the Promise Code:
const celestial = videoPage.unsafelyGetCelestialBindings();
console.log("done 0");
await new Promise<void>((resolve) => (celestial.ws.onopen = () => resolve()));
console.log("done 1");
await celestial.Fetch.enable({});
console.log("done 2");

celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event);
});

console.log("done 3");
await videoPage.goto(video);
console.log("done 4");
const celestial = videoPage.unsafelyGetCelestialBindings();
console.log("done 0");
await new Promise<void>((resolve) => (celestial.ws.onopen = () => resolve()));
console.log("done 1");
await celestial.Fetch.enable({});
console.log("done 2");

celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event);
});

console.log("done 3");
await videoPage.goto(video);
console.log("done 4");
IioB3/2/2024
you have to do somethign with the request basically you "caught" the request, and you have to decide what to do with it
DDNA3/2/2024
But wouldnt the promise get resolved anyway, even if the request isnt handled yet? Im not getting "deno 1", which is right after the promise
IioB3/2/2024
well by default goto waits for network activity to stop you can change that behavior by using the waitUntil option
await videoPage.goto(video, { waitUntil: "load" });
await videoPage.goto(video, { waitUntil: "load" });
or something oh wait hold on I misread your message okay well the issue is the socket opened by the time you set up the promise so it's hanging
DDNA3/2/2024
ws.onopen is never called
IioB3/2/2024
something like
export async function websocketReady(ws: WebSocket) {
await new Promise<void>((res) => {
if(ws.readyState === 1) {
res();
} else {
ws.onopen = () => {
res();
};
}
});
}
export async function websocketReady(ws: WebSocket) {
await new Promise<void>((res) => {
if(ws.readyState === 1) {
res();
} else {
ws.onopen = () => {
res();
};
}
});
}
should fix it? if not I have another guess
DDNA3/2/2024
Yeah that worked Thank you so much for helping me with my problems
IioB3/2/2024
No worries, that's what I'm here for.
DDNA3/2/2024
I still got a small question? How do i pass the request?
celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event);
});
celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event);
});
IioB3/2/2024
If you always want to pass it, you can simply call
await celestial.Fetch.continueRequest({ requestId });
await celestial.Fetch.continueRequest({ requestId });
where requestId is somewhere in the event, I'd look at the console.log to see where maybe event.data.requestId? not sure
DDNA3/2/2024
Thanks once again, this worked
celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event.detail.request.url);
celestial.Fetch.continueRequest({requestId: event.detail.requestId});
});
celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event.detail.request.url);
celestial.Fetch.continueRequest({requestId: event.detail.requestId});
});
IioB3/2/2024
👍
DDNA3/3/2024
So i just ran into the next problem. When intercepting the requests, on windows, the request for master.m3u8 comes in and is also console.logged, but on my linux machine, the request never shows up. Does anyone have an idea, why it is like that? Also, the screenshot taken right before the while loop looks just the same on both machines Code:
const videoPage = await browser.newPage();
await videoPage.setViewportSize({width: 1920, height: 1080});
console.log("Video page prepared");

const videoPageCelestial = videoPage.unsafelyGetCelestialBindings();
await new Promise<void>((resolve) => (videoPageCelestial.ws.readyState === 1 ? resolve() : (videoPageCelestial.ws.onopen = () => resolve())));
await videoPageCelestial.Fetch.enable({});

console.log("Enabled request interception");

videoPageCelestial.addEventListener("Fetch.requestPaused", (event) => {
console.log("\n" + event.detail.request.url);
if (event.detail.request.url.includes("master.m3u8")) data.master = event.detail.request.url;

if (event.detail.request.url.includes(".png") || event.detail.request.url.includes(".jpeg") || event.detail.request.url.includes(".jpg") || event.detail.request.url.includes(".css") || event.detail.request.url.includes(".ico") || event.detail.request.url.includes(".gif") || event.detail.request.url.includes("data:image") || event.detail.request.url.includes("data:font")) return videoPageCelestial.Fetch.failRequest({requestId: event.detail.requestId, errorReason: "Aborted"});
else videoPageCelestial.Fetch.continueRequest({requestId: event.detail.requestId});
});

console.log("Going to video page: " + video);
await videoPage.goto(video);
console.log("Video page loaded, waiting for master.m3u8");

Deno.writeFileSync("screenshot.png", await videoPage.screenshot({format: "png"}));

while (!data.master) null; // <-- Stuck here on ubuntu server 23.10, proceeds after a few seconds on my main machine (windows 11) though
const videoPage = await browser.newPage();
await videoPage.setViewportSize({width: 1920, height: 1080});
console.log("Video page prepared");

const videoPageCelestial = videoPage.unsafelyGetCelestialBindings();
await new Promise<void>((resolve) => (videoPageCelestial.ws.readyState === 1 ? resolve() : (videoPageCelestial.ws.onopen = () => resolve())));
await videoPageCelestial.Fetch.enable({});

console.log("Enabled request interception");

videoPageCelestial.addEventListener("Fetch.requestPaused", (event) => {
console.log("\n" + event.detail.request.url);
if (event.detail.request.url.includes("master.m3u8")) data.master = event.detail.request.url;

if (event.detail.request.url.includes(".png") || event.detail.request.url.includes(".jpeg") || event.detail.request.url.includes(".jpg") || event.detail.request.url.includes(".css") || event.detail.request.url.includes(".ico") || event.detail.request.url.includes(".gif") || event.detail.request.url.includes("data:image") || event.detail.request.url.includes("data:font")) return videoPageCelestial.Fetch.failRequest({requestId: event.detail.requestId, errorReason: "Aborted"});
else videoPageCelestial.Fetch.continueRequest({requestId: event.detail.requestId});
});

console.log("Going to video page: " + video);
await videoPage.goto(video);
console.log("Video page loaded, waiting for master.m3u8");

Deno.writeFileSync("screenshot.png", await videoPage.screenshot({format: "png"}));

while (!data.master) null; // <-- Stuck here on ubuntu server 23.10, proceeds after a few seconds on my main machine (windows 11) though
IioB3/5/2024
Hey, sorry for getting back to you so late. I suspect this is site-to-site variance. This site might avoid loading videos for someone on linux.
DDNA3/6/2024
Is it possible to manually set the useragent?
IioB3/7/2024
Yes, it is it's just Celestial.Emulation.setUserAgentOverride

Looking for more? Join the community!

Recommended Posts
Unable to use Deno.createHttpClient without deprecated flag --unstableI expected it to be behind the newer `--unstable-net` flagMultiple JsRuntimes crashe if first is not dropped before a second is created and ranIn my application, I want an `op2` to take in a function which will be stored in a struct as a `GlobStructure microservicesHi, how would you structure a Deno microservice repository with a good developer experience (good dGet image from automated downloadI have this page which displays a render using threejs and renders it into a png and downloads it. iGet image data from automated downloadI have this page which displays a render using threejs and renders it into a png and downloads it. iWait until Deno.Command process has exitedHow would i stop code execution until the process has exited? since its not a promise i cant just doWhy JSX is not working ?// @deno-types="@types/react" import React from "react"; const A = (): React.ReactElement => <div />Running with --cached-only not workingFirst I cache all dependencies of my typescript file: `deno cache --lock=deno.lock --lock-write serSubhosting DeploymentsI've been playing around with deno subhosting. I get the concept of a project and deployments in a pThe new and updated version for using socket.io handler with Deno.serve()Hey there! am getting a error when I pass socket.io handler to Deno.serve(), Here is what am doing :Sorting tailwindcss class names with prettier plugin?When developing in Fresh, I would like to have my class names automatically sorted with the [officiaHow to validate types in API Requests and Responses?Hello, I'm wondering how to validate and handle the types in my app's API. I'll use the example founDeno LSP + React + TypeScript + Vite (--node-modules-dir)Does the Deno LSP works with the --node-modules-dir option? did ```sh deno run --allow-env --allow-deno_runtime NpmResolver like the CLI?As far as I can tell, if I want to embed Deno in my Rust application using deno_core and deno_runtimIs it possible to share the same privateLink in Deno Subhosting?Recently we are trying to implement a "custom js function" module in Deno Subhosting, our concern isSource generation for dependenciesHello, I'm working on a source gen for a library (Node, Deno, Bun) and currently my idea was to overFetch errors outFailed to serve connection: hyper::Error(BodyWrite, Os { code: 55, kind: Uncategorized, message: "NoDeno Jupyter Notebook - monorepo supportHi all! We have a typescript-based monorepo that uses Turbo/Pnpm/etc Our primary reason for coming workspace importI have a vsc root-level settings.json file ```json { "deno.enable": true, "deno.enablePathsHow Enable HTTPS localhost DENO?I want to run Deno Oak over Https in local. I use OpenSsl/mkcert to create cert and key. But no work