You can use exposeFunction to expose a local function: await page.exposeFunction(‘getItem’, function(a) { console.log(a); }); Then you can use page.evaluate to create an observer and listen to new nodes created inside a parent node. This example scrapes (it’s just an idea, not a final work) the python chat in Stack Overflow, and prints new items … Read more
Welcome to SO. There are different option to get the associated events. Let’s take the stackoverflow search box (the one which is on the top with s-input js-search-field class) 1) Using getEventListeners Go to chrome console in the dev tools and then use getEventListeners(element). getEventListeners(document.querySelector(‘.s-input.js-search-field ‘)) 2) Using Dev Tools Event Listner Go to chrome … Read more
Maybe you want to use twill. It’s quite easy to use and should be able to do what you want. It will look like the following: from twill.commands import * go(‘http://example.org’) fv(“1”, “email-email”, “blabla.com”) fv(“1”, “password-clear”, “testpass”) submit(‘0’) You can use showforms() to list all forms once you used go… to browse to the site … Read more
Here are a list of headless browsers that I know about: HtmlUnit – Java. Custom browser engine. Limited JavaScript support/DOM emulated. Open source. Ghost – Python only. WebKit-based. Full JavaScript support. Open source. Twill – Python/command line. Custom browser engine. No JavaScript. Open source. PhantomJS – Command line/all platforms. WebKit-based. Full JavaScript support. Open source. … Read more