DNA
DNA8mo ago

Puppeteer: "BadResource: Bad resource ID" on Ubuntu

Hello, i just switched my server os over to ubuntu server 23.10. When executing my script, it immediately throws an error an exits. Before, on my windows machine, the script worked completely fine. Error:
error: Uncaught (in promise) BadResource: Bad resource ID
const result = await reader.read(inspectArr);
^
at read (ext:deno_io/12_io.js:116:28)
at FsFile.read (ext:deno_fs/30_fs.js:706:12)
at readDelim (https://deno.land/std@0.93.0/io/bufio.ts:652:33)
at readDelim.next (<anonymous>)
at readStringDelim (https://deno.land/std@0.93.0/io/bufio.ts:702:20)
at readStringDelim.next (<anonymous>)
at readLines (https://deno.land/std@0.93.0/io/bufio.ts:711:18)
at readLines.next (<anonymous>)
at waitForWSEndpoint (https://deno.land/x/puppeteer@16.2.0/src/deno/BrowserRunner.ts:168:20)
at eventLoopTick (ext:core/01_core.js:166:7)
error: Uncaught (in promise) BadResource: Bad resource ID
const result = await reader.read(inspectArr);
^
at read (ext:deno_io/12_io.js:116:28)
at FsFile.read (ext:deno_fs/30_fs.js:706:12)
at readDelim (https://deno.land/std@0.93.0/io/bufio.ts:652:33)
at readDelim.next (<anonymous>)
at readStringDelim (https://deno.land/std@0.93.0/io/bufio.ts:702:20)
at readStringDelim.next (<anonymous>)
at readLines (https://deno.land/std@0.93.0/io/bufio.ts:711:18)
at readLines.next (<anonymous>)
at waitForWSEndpoint (https://deno.land/x/puppeteer@16.2.0/src/deno/BrowserRunner.ts:168:20)
at eventLoopTick (ext:core/01_core.js:166:7)
Relevant code:
import puppeteer from "https://deno.land/x/puppeteer@16.2.0/mod.ts";
import * as path from "https://deno.land/std@0.197.0/path/mod.ts";

const browser = await puppeteer.launch({
headless: true,
product: "chrome",
defaultViewport: {width: 1280, height: 720},
args: [
//
`--disable-extensions-except=${path.join(Deno.cwd(), "extensions", "cjpalhdlnbpafiamejdnhcphjbkeiagm", "1.50.0_0")}`,
`--load-extension=${path.join(Deno.cwd(), "extensions", "cjpalhdlnbpafiamejdnhcphjbkeiagm", "1.50.0_0")}`,
"--mute-audio",
],
});
import puppeteer from "https://deno.land/x/puppeteer@16.2.0/mod.ts";
import * as path from "https://deno.land/std@0.197.0/path/mod.ts";

const browser = await puppeteer.launch({
headless: true,
product: "chrome",
defaultViewport: {width: 1280, height: 720},
args: [
//
`--disable-extensions-except=${path.join(Deno.cwd(), "extensions", "cjpalhdlnbpafiamejdnhcphjbkeiagm", "1.50.0_0")}`,
`--load-extension=${path.join(Deno.cwd(), "extensions", "cjpalhdlnbpafiamejdnhcphjbkeiagm", "1.50.0_0")}`,
"--mute-audio",
],
});
Troubleshooting steps taken: - Re-ran installation script - Cleared cache - Rebooted machine uname -a:
Linux dnarpi 6.5.0-1011-raspi #14-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 9 14:06:28 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
deno --version:
deno 1.41.1 (release, aarch64-unknown-linux-gnu) v8 12.1.285.27 typescript 5.3.3
No description
36 Replies
cknight
cknight8mo ago
This doesn't answer your question I'm afraid, but if you're not deeply invested in Puppeteer, you could consider using the excellent and Deno native Astral https://astral.deno.dev/
Introduction | Astral
A simple introduction to Astral
DNA
DNA8mo ago
I'll take a look at that real quick, give me a few minutes
cknight
cknight8mo ago
But my best guess as to your problem is the really old version of std being imported (like 120 versions or so old) which I'm guessing is not compatible with the latest Deno.
DNA
DNA8mo ago
No description
DNA
DNA8mo ago
The problem is just, that puppeteer for deno hasnt been updated in 2 years Maybe i'd try the node version?
cknight
cknight8mo ago
That was going to be my next suggestion. @lino-levan any thoughts?
ioB
ioB8mo ago
Yeah, I mean, Astral will work for arm64, but you'd need to supply a browser binary yourself Everything after that will work just fine I have no idea why chrome for testing doesn't ship linux arm64 binaries In the medium term I'm adding support for Firefox but FF is even more undocumented than Chromium is when it comes to browser automation.
DNA
DNA8mo ago
I just installed/downloaded chrome using npx @puppeteer/browsers install chrome@stable, but that returned me the same error
No description
DNA
DNA8mo ago
That was pretty much the only source of the chrome binary i found online
ioB
ioB8mo ago
chrome doesn't ship an arm linux binary chromium does iirc I believe you should be able to sudo apt-get install chromium-browser or something
DNA
DNA8mo ago
That worked! Thank you so much
DNA
DNA8mo ago
Using Astral, how can i use request interception?
No description
ioB
ioB8mo ago
what are you trying to do? I'd recommend looking at the celestial bindings you can get them on a page using https://deno.land/x/astral@0.3.5/mod.ts?s=Page&p=prototype.unsafelyGetCelestialBindings.
const celestial = page.unsafelyGetCelestialBindings();
const celestial = page.unsafelyGetCelestialBindings();
that opens up access to the raw chrome devtools protocol which gives you a ton of control it's fully typed too which is quite nice
ioB
ioB8mo ago
the docs for the chrome devtools protocol can be found https://chromedevtools.github.io/devtools-protocol/
Chrome DevTools Protocol
Chrome DevTools Protocol - version tot
ioB
ioB8mo ago
depending on what you're trying to do, you should look at https://chromedevtools.github.io/devtools-protocol/tot/Fetch/
Chrome DevTools Protocol
Chrome DevTools Protocol - version tot - Fetch domain
ioB
ioB8mo ago
something like
const celestial = page.unsafelyGetCelestialBindings();
await celestial.Fetch.enable();
celestial.addEventListener("Fetch.requestPaused", (event)=>{
console.log("got event", event);
// run await celestial.Fetch.continueRequest({ requestId }); to continue request
});
const celestial = page.unsafelyGetCelestialBindings();
await celestial.Fetch.enable();
celestial.addEventListener("Fetch.requestPaused", (event)=>{
console.log("got event", event);
// run await celestial.Fetch.continueRequest({ requestId }); to continue request
});
might be a good start
DNA
DNA8mo ago
Code:
const videoPage = await browser.newPage();

const endTimeMin = Date.now() + 5000;
// await videoPage.setRequestInterception(true);
const celestial = page.unsafelyGetCelestialBindings();

await celestial.Fetch.enable({});
const videoPage = await browser.newPage();

const endTimeMin = Date.now() + 5000;
// await videoPage.setRequestInterception(true);
const celestial = page.unsafelyGetCelestialBindings();

await celestial.Fetch.enable({});
Error:
error: Uncaught (in promise) InvalidStateError: readyState not OPEN
this.ws.send(JSON.stringify({
^
at WebSocket.send (ext:deno_websocket/01_websocket.js:326:13)
at Celestial.#sendReq (https://deno.land/x/astral@0.3.5/bindings/celestial.ts:13261:13)
at Object.enable (https://deno.land/x/astral@0.3.5/bindings/celestial.ts:21634:33)
at file:///E:/asdfasdfasdfasdf/download-account.ts:141:26
at eventLoopTick (ext:core/01_core.js:166:7)
error: Uncaught (in promise) InvalidStateError: readyState not OPEN
this.ws.send(JSON.stringify({
^
at WebSocket.send (ext:deno_websocket/01_websocket.js:326:13)
at Celestial.#sendReq (https://deno.land/x/astral@0.3.5/bindings/celestial.ts:13261:13)
at Object.enable (https://deno.land/x/astral@0.3.5/bindings/celestial.ts:21634:33)
at file:///E:/asdfasdfasdfasdf/download-account.ts:141:26
at eventLoopTick (ext:core/01_core.js:166:7)
No description
ioB
ioB8mo ago
perhaps unsafelyGetCelestialBindings should wait for the websocket to be open hm
DNA
DNA8mo ago
I cant use await on page.unsafelyGetCelestialBindings() however
ioB
ioB8mo ago
okay, try adding
export async function websocketReady(ws: WebSocket) {
await new Promise<void>((res) => {
ws.onopen = () => {
res();
};
});
}
export async function websocketReady(ws: WebSocket) {
await new Promise<void>((res) => {
ws.onopen = () => {
res();
};
});
}
and then doing
const videoPage = await browser.newPage();

const endTimeMin = Date.now() + 5000;
// await videoPage.setRequestInterception(true);
const celestial = page.unsafelyGetCelestialBindings();
await websocketReady(celestial.ws);
await celestial.Fetch.enable({});
const videoPage = await browser.newPage();

const endTimeMin = Date.now() + 5000;
// await videoPage.setRequestInterception(true);
const celestial = page.unsafelyGetCelestialBindings();
await websocketReady(celestial.ws);
await celestial.Fetch.enable({});
this will be fixed in the next release of astral
DNA
DNA8mo ago
I did implement this a bit differently than you did, but the result should be the same anyway. However, for some reason, its stuck after "done 0", basically at the Promise Code:
const celestial = videoPage.unsafelyGetCelestialBindings();
console.log("done 0");
await new Promise<void>((resolve) => (celestial.ws.onopen = () => resolve()));
console.log("done 1");
await celestial.Fetch.enable({});
console.log("done 2");

celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event);
});

console.log("done 3");
await videoPage.goto(video);
console.log("done 4");
const celestial = videoPage.unsafelyGetCelestialBindings();
console.log("done 0");
await new Promise<void>((resolve) => (celestial.ws.onopen = () => resolve()));
console.log("done 1");
await celestial.Fetch.enable({});
console.log("done 2");

celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event);
});

console.log("done 3");
await videoPage.goto(video);
console.log("done 4");
ioB
ioB8mo ago
you have to do somethign with the request basically you "caught" the request, and you have to decide what to do with it
DNA
DNA8mo ago
But wouldnt the promise get resolved anyway, even if the request isnt handled yet? Im not getting "deno 1", which is right after the promise
ioB
ioB8mo ago
well by default goto waits for network activity to stop you can change that behavior by using the waitUntil option
await videoPage.goto(video, { waitUntil: "load" });
await videoPage.goto(video, { waitUntil: "load" });
or something oh wait hold on I misread your message okay well the issue is the socket opened by the time you set up the promise so it's hanging
DNA
DNA8mo ago
ws.onopen is never called
ioB
ioB8mo ago
something like
export async function websocketReady(ws: WebSocket) {
await new Promise<void>((res) => {
if(ws.readyState === 1) {
res();
} else {
ws.onopen = () => {
res();
};
}
});
}
export async function websocketReady(ws: WebSocket) {
await new Promise<void>((res) => {
if(ws.readyState === 1) {
res();
} else {
ws.onopen = () => {
res();
};
}
});
}
should fix it? if not I have another guess
DNA
DNA8mo ago
Yeah that worked Thank you so much for helping me with my problems
ioB
ioB8mo ago
No worries, that's what I'm here for.
DNA
DNA8mo ago
I still got a small question? How do i pass the request?
celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event);
});
celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event);
});
ioB
ioB8mo ago
If you always want to pass it, you can simply call
await celestial.Fetch.continueRequest({ requestId });
await celestial.Fetch.continueRequest({ requestId });
where requestId is somewhere in the event, I'd look at the console.log to see where maybe event.data.requestId? not sure
DNA
DNA8mo ago
Thanks once again, this worked
celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event.detail.request.url);
celestial.Fetch.continueRequest({requestId: event.detail.requestId});
});
celestial.addEventListener("Fetch.requestPaused", async (event) => {
console.log(event.detail.request.url);
celestial.Fetch.continueRequest({requestId: event.detail.requestId});
});
ioB
ioB8mo ago
👍
DNA
DNA8mo ago
So i just ran into the next problem. When intercepting the requests, on windows, the request for master.m3u8 comes in and is also console.logged, but on my linux machine, the request never shows up. Does anyone have an idea, why it is like that? Also, the screenshot taken right before the while loop looks just the same on both machines Code:
const videoPage = await browser.newPage();
await videoPage.setViewportSize({width: 1920, height: 1080});
console.log("Video page prepared");

const videoPageCelestial = videoPage.unsafelyGetCelestialBindings();
await new Promise<void>((resolve) => (videoPageCelestial.ws.readyState === 1 ? resolve() : (videoPageCelestial.ws.onopen = () => resolve())));
await videoPageCelestial.Fetch.enable({});

console.log("Enabled request interception");

videoPageCelestial.addEventListener("Fetch.requestPaused", (event) => {
console.log("\n" + event.detail.request.url);
if (event.detail.request.url.includes("master.m3u8")) data.master = event.detail.request.url;

if (event.detail.request.url.includes(".png") || event.detail.request.url.includes(".jpeg") || event.detail.request.url.includes(".jpg") || event.detail.request.url.includes(".css") || event.detail.request.url.includes(".ico") || event.detail.request.url.includes(".gif") || event.detail.request.url.includes("data:image") || event.detail.request.url.includes("data:font")) return videoPageCelestial.Fetch.failRequest({requestId: event.detail.requestId, errorReason: "Aborted"});
else videoPageCelestial.Fetch.continueRequest({requestId: event.detail.requestId});
});

console.log("Going to video page: " + video);
await videoPage.goto(video);
console.log("Video page loaded, waiting for master.m3u8");

Deno.writeFileSync("screenshot.png", await videoPage.screenshot({format: "png"}));

while (!data.master) null; // <-- Stuck here on ubuntu server 23.10, proceeds after a few seconds on my main machine (windows 11) though
const videoPage = await browser.newPage();
await videoPage.setViewportSize({width: 1920, height: 1080});
console.log("Video page prepared");

const videoPageCelestial = videoPage.unsafelyGetCelestialBindings();
await new Promise<void>((resolve) => (videoPageCelestial.ws.readyState === 1 ? resolve() : (videoPageCelestial.ws.onopen = () => resolve())));
await videoPageCelestial.Fetch.enable({});

console.log("Enabled request interception");

videoPageCelestial.addEventListener("Fetch.requestPaused", (event) => {
console.log("\n" + event.detail.request.url);
if (event.detail.request.url.includes("master.m3u8")) data.master = event.detail.request.url;

if (event.detail.request.url.includes(".png") || event.detail.request.url.includes(".jpeg") || event.detail.request.url.includes(".jpg") || event.detail.request.url.includes(".css") || event.detail.request.url.includes(".ico") || event.detail.request.url.includes(".gif") || event.detail.request.url.includes("data:image") || event.detail.request.url.includes("data:font")) return videoPageCelestial.Fetch.failRequest({requestId: event.detail.requestId, errorReason: "Aborted"});
else videoPageCelestial.Fetch.continueRequest({requestId: event.detail.requestId});
});

console.log("Going to video page: " + video);
await videoPage.goto(video);
console.log("Video page loaded, waiting for master.m3u8");

Deno.writeFileSync("screenshot.png", await videoPage.screenshot({format: "png"}));

while (!data.master) null; // <-- Stuck here on ubuntu server 23.10, proceeds after a few seconds on my main machine (windows 11) though
ioB
ioB8mo ago
Hey, sorry for getting back to you so late. I suspect this is site-to-site variance. This site might avoid loading videos for someone on linux.
DNA
DNA8mo ago
Is it possible to manually set the useragent?
ioB
ioB8mo ago
Yes, it is it's just Celestial.Emulation.setUserAgentOverride