D
Deno

help

puppeteer makes script to hang for a while at the end

Rrnbguy8/22/2023
I am using npm:puppeteer in my script. Everything is working fine. But often the script hangs at the end of the exit. Is this normal with puppeteer? I think that some async op is leaking. But I am not sure if I made any mistake or it is coming from puppeteer. I do call browser.close() in a finally block.
IioB8/22/2023
Could you share some code?
Jjeff.hykin8/22/2023
There's also a deno puppeteer which might fix your problems
IioB8/22/2023
there's also https://deno.land/x/astral if you just need a browser automation library
Rrnbguy8/22/2023
@lino-levan here is a code reproducing the behavior on my machine.
import puppeteer from "npm:puppeteer";

const browser = await puppeteer.launch({
executablePath: "google-chrome-stable",
headless: "new",
});

const page = await browser.newPage();

try {
await page.goto("https://www.google.com");
} finally {
page.close();
browser.close();
}

console.log("bye")
import puppeteer from "npm:puppeteer";

const browser = await puppeteer.launch({
executablePath: "google-chrome-stable",
headless: "new",
});

const page = await browser.newPage();

try {
await page.goto("https://www.google.com");
} finally {
page.close();
browser.close();
}

console.log("bye")
I am actually using puppeteer-extra for fortified browser. the same behavior persists.
import "npm:puppeteer";
import { default as puppeteer } from "npm:puppeteer-extra";
import StealthPlugin from "npm:puppeteer-extra-plugin-stealth";
puppeteer.use(StealthPlugin());

const browser = await puppeteer.launch({
executablePath: "google-chrome-stable",
headless: "new",
});

const page = await browser.newPage();

try {
await page.goto("https://www.google.com");
} finally {
page.close();
browser.close();
}

console.log("bye")
import "npm:puppeteer";
import { default as puppeteer } from "npm:puppeteer-extra";
import StealthPlugin from "npm:puppeteer-extra-plugin-stealth";
puppeteer.use(StealthPlugin());

const browser = await puppeteer.launch({
executablePath: "google-chrome-stable",
headless: "new",
});

const page = await browser.newPage();

try {
await page.goto("https://www.google.com");
} finally {
page.close();
browser.close();
}

console.log("bye")
@jeff.hykin I can confirm import puppeteer from "https://deno.land/x/puppeteer@16.2.0/mod.ts"; solves the issue. 馃檪 but I need to use puppeteer-extra for fortification. 馃槮 hey @lino-levan thanks for suggesting astral. looks like it is fortified by default. 馃檪 https://bot.sannysoft.com tests are passed but astral is not customizable enough like puppeteer 馃槮 I need to start the browser with proxy and send page.evaluate() with function arguments. but nonetheless, it looks solid. maybe I will come back to it later 馃檪
IioB8/22/2023
probably worth opening an issue! Have you tried setting the HTTP_PROXY env var? It works for me in Astral. Released a new update with function arguments to page.evaluate(). Let me know if you run into any other blockers.
Rrnbguy8/22/2023
ooh awesome! @lino-levan if you're the maintainer, let me look into it more and open issues 馃檪
IioB8/22/2023
Would love feedback on the project. I think browser automation libraries should be hardened by default, no configuration neccessary.
Rrnbguy8/22/2023
btw, I will try with HTTP_PROXY, but can you please add support for argument passing for browser executable? like for chrome, I can just pass --proxy-server=${proxyUrl}. no pressure 馃檪
IioB8/22/2023
This isn't portable unfortunately. I'm hoping to get FF/Safari support going pretty soon and they don't have equivalents. HTTP_PROXY seems to be portable accross both chrome and FF at least
Rrnbguy8/22/2023
I meant, passing argument to browser executable in general. I am fine with passing proxy-server argument by myself. like, this can have another field called browserArgs?: string[].
Rrnbguy8/22/2023
GitHub
astral/src/browser.ts at main 路 lino-levan/astral
A high-level puppeteer/playwright-like library for Deno - lino-levan/astral
IioB8/22/2023
Would probably be in LaunchOptions, but that sounds more than reasonable. I'll open a PR.
IioB8/22/2023
Any other immediate feedback? Would love to roll this out into a new release.
S脫scar8/22/2023
I have the same issue with puppeteer. In fact I filed an issue here: https://github.com/denoland/deno/issues/20179
Rrnbguy8/22/2023
@lino-levan awesome! with your latest commit, I managed to completely move from puppeteer to astral 馃嵒 only thing I missed is waitUntil in Page.goto() ref. https://github.com/puppeteer/puppeteer/blob/2fee2b0adc7fd5f26c7b1d75a3efa29c2fe12dab/packages/puppeteer-core/src/common/Frame.ts#L109
GitHub
puppeteer/packages/puppeteer-core/src/common/Frame.ts at 2fee2b0adc...
Node.js API for Chrome . Contribute to puppeteer/puppeteer development by creating an account on GitHub.
Rrnbguy8/22/2023
also, I was using .$$eval before - which is not available in astral. but I worked it around by just using .evaluate. so it's not a biggie 馃檪
IioB8/22/2023
I believe page.goto takes in a GoToOptions which includes a waitUntil (of type "load" | "networkidle0" | "networkidle2").
Rrnbguy8/22/2023
ah. I was getting errors because I was using "domcontentloaded". all good in that case 馃檪

Looking for more? Join the community!

Recommended Posts
In Fresh, how do I return a JSON response in a middleware?In Fresh, how do I return a JSON response in a middleware?get length for KV list via prefix?Is there a fast way to get just the count of items for a prefix?Property 'params' does not exist on type 'Context<State, Record<string, any>> ... what to do?I want to access params in oak, but typescript complains. What can I do about that? Thanks.What is the std library for making a guessing game like shown in rust?```rust use std::io; fn main() { println!("Guess the number!"); println!("Please input youData must be valid JSONA simple project, after successful deployment, reports an error "Data must be valid JSON".FFI deno_bindgen, how to see build logs?Hi, using deno_bindgen for rust, is there anyway i can view the build logs? --verbose or -v do not sTypeError: str.split is not a functionHey Gang. Im following along this tutorial: https://deno.com/blog/build-a-blog-with-fresh But I am Can't sign up/sign in to Deno DeployI'm tired of this error. Does anybody know the reason or at least how to solve it?CSP Middleware for OAK?Just wondering what the best module is for this? I noticed SNELM but that hasn't been updated in abExternal dependencies with npm specifier in combination with vite does not work.Hi, I am new to Deno and want to build an app with deno, vite and react. Unfortunately the dependencQuestion about for loopsWhy does the following code log numbers from 0 to 9 instead of 1 to 9? `i` is assigned as zero in thDockerfile saaskit deploymentI created an entry in `deno.json` starting a script named `db:init` to initialize my KV DB with an uDeno TypesWhat is the Deno.ServeHandler type? ```No overload matches this call. Overload 1 of 3, '(handler: I use Deno KV for my application, and I'm planning to use Deno Deploy. Can I allow deploy to use...I use Deno KV for my application, and I'm planning to use Deno Deploy. Can I allow deploy to use KVbad resource id with node:crypto```ts function test() { const key = randomBytes(32); const iv = randomBytes(16); const cipher Close a Deno.serve() connection?Is it possible to close a Deno.serve() connection?Relative import path "preact" not prefixed with / or ./ or ../ at https://deno.land/x/fresh@1.4.Please help. ``` deno 1.36.1 (release, aarch64-apple-darwin) v8 11.6.189.12 typescript 5.1.6 ```Problems with Deno KV```ts import InputLoop from "https://deno.land/x/input@2.0.3/index.ts"; const kv = await Deno.openKFile differs only by case errorI'm getting a strange error from TypeScript. For some reason it thinks the case of my file name is Possible to have multiple http servers listening to different ports in the same module?I'm porting a service mesh platform we wrote for NodeJS over to Deno and it is based on the idea of