There is a built-in option for that: await page.goto(‘https://www.digg.com/’, {“waitUntil” : “networkidle0”}); networkidle0 – consider navigation to be finished when there are no more than 0 network connections for at least 500 ms networkidle2 – consider navigation to be finished when there are no more than 2 network connections for at least 500 ms. Of … Read more
You can use exposeFunction to expose a local function: await page.exposeFunction(‘getItem’, function(a) { console.log(a); }); Then you can use page.evaluate to create an observer and listen to new nodes created inside a parent node. This example scrapes (it’s just an idea, not a final work) the python chat in Stack Overflow, and prints new items … Read more
Check the following Dockerfile, read the comments. It should run properly. # First, we need to make sure all dependencies are there. If you are using docker, then the important dependencies are already present on most node images. FROM node:8 # To run Headful mode, you will need to have a display, which is not … Read more
I think document would only be available within page.evaluate (according to puppeteer documentation ) Try: async function gallery(page) { await page.waitFor(3000); await page.evaluate(() => { document.querySelector(‘div.image’).click(); }) }
I know this is 9 months late but I had the same use case at work on Window machines but you can make it work with Mac. The trick is to use puppeteer-web https://github.com/puppeteer/puppeteer/tree/master/utils/browser#bundling-for-web-browsers Bundle the repository and place it in your chrome extension folder and then reference it in your popup.html with something like … Read more
One option is to do the following: page.on(‘response’, response => { if (response.url().endsWith(“your/match”)) console.log(“response code: “, response.status()); // do something here }); This still catches all requests, but allows you to filter and act on the event emitter. https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#event-response
Short answer: It’s not possible Chrome only allows to either start the browser in headless or non-headless mode. You have to specify it when you launch the browser and it is not possible to switch during runtime. What is possible, is to launch a second browser and reuse cookies (and any other data) from the … Read more
If anyone need in future for the same problem. Using puppeteer-extra I have tested the code on a server. On 2nd run there is google Captcha. You can solve it your self and restart the bot or use a Captcha solving service. I did run the code more than 10 times there is no ip … Read more
What “Target closed” means When you launch a browser via puppeteer.launch it will start a browser and connect to it. From there on any function you execute on your opened browser (like page.goto) will be send via the Chrome DevTools Protocol to the browser. A target means a tab in this context. The Target closed … Read more
The function page.evaluate() can only return a serializable value, so it is not possible to return an element or NodeList back from the page environment using this method. You can use page.$$() instead to obtain an ElementHandle array: const nodes = await page.$$(`${selector} > *`); // selector children If the length of the constant nodes … Read more