How can I make sure if some HTML elements are loaded for Selenium + Python?

by

As per your code trials you are using ChromeDriver and Chrome Browser to Automate the steps. As you have configured set_page_load_timeout(10) the timeout exception is raised as the page havn’t completely loaded within the timeframe configured through set_page_load_timeout(). But as you have invoked print(self.page_source) the elements rendered within the partially rendered HTML DOM are retrieved.

Now about your individual queries :

  • How can I make sure some elements are already loaded? : An ideal testcase would have a definite step e.g. to validate the presence of an element, to validate the visibility of an element or to validate the interactability (while clicking) of element. From this perspective verifying the elements are already loaded may not include the desired element. Hence instead of such a broader search criteria you need to narrow down your search criteria to some thing definite e.g.

    • Page Title
    • Page Heading
    • Presence of an Alert
    • Attribute of an element
    • Presence of an element
    • Presence of a group of elements
    • Visibility of an element
    • Visibility of a group of elements
    • Clickablity of an element
    • StalenessOf of an element
    • FrameToBeAvailableAndSwitchToIt

Implementing these narrowed down search criteria can save a lot of execution time with the help of WebDriverWait inconjunction with expected_conditions.

  • How can I make sure all elements are loaded? : Again, our tests should be focused only on the element/elements with which we need to interact with and leaveout verifying the status/condition of other elements which are not of our interest.

  • Now, following the two points mentioned above these are the 3 most commonly used usecases :

    • presence_of_element_located : An expectation for checking that an element is present on the DOM of a page. This does not necessarily mean that the element is visible.
    • visibility_of_element_located : An expectation for checking that an element is present on the DOM of a page and visible. Visibility means that the element is not only displayed but also has a height and width that is greater than 0.
    • element_to_be_clickable : An Expectation for checking an element is visible and enabled such that you can click it.

  • As per the usecase you mentioned you can make a List of all the <th> elements visible within the DOM Tree whilst waiting for a configurable amount of time inducing WebDriverWait as follows :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC 

headerList = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//table[@class="fth1_" and @id='fth1_']/thead/tr//th")))

Note : The xpath used in this illustration is a sample xpath used only for demonstration purpose.

More Related Contents:

  1. How can I select a html element no matter what frame it is in in selenium?
  2. How to simulate HTML5 Drag and Drop in Selenium Webdriver?
  3. How to get text of an element in Selenium WebDriver, without including child element text?
  4. StaleElementReferenceException on Python Selenium
  5. Does anybody know how to identify shadow dom web elements using selenium webdriver?
  6. How can I get text of an element in Selenium WebDriver, without including child element text?
  7. How to sleep Selenium WebDriver in Python for milliseconds
  8. Running Selenium Webdriver with a proxy in Python
  9. How can I download a file on a click event using selenium?
  10. Python Selenium accessing HTML source
  11. Log into gmail using Selenium in Python
  12. Select -> option abstraction
  13. How to set the timeout of ‘driver.get’ for python selenium 3.8.0?
  14. in selenium web driver how to choose the correct iframe
  15. selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities with Firefox 46 through Selenium
  16. How to set Selenium Python WebDriver default timeout?
  17. selenium.common.exceptions.WebDriverException: Message: ‘geckodriver’ executable needs to be in PATH with GeckoDriver Selenium Firefox
  18. Mouse scroll wheel with selenium webdriver, on element without scrollbar?
  19. Setting path to firefox binary on windows with selenium webdriver
  20. Undetected Chromedriver not loading correctly
  21. Is there a way to perform a mouseover (hover over an element) using Selenium and Python bindings?
  22. How to solve ‘Getting Default Adapter failed’ error when launching Chrome and try to access a webpage using the ChromeDriver using Selenium
  23. Checking if an element exists with Python Selenium
  24. How to find element by part of its id name in selenium with python
  25. How to get an attribute of an element from Selenium
  26. Selenium webdriver: How do I find ALL of an element’s attributes?
  27. selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities with GeckoDriver, Selenium and Firefox
  28. Chrome browser initiated through ChromeDriver gets detected
  29. WebDriverException: Service U:/Scraping/chromedriver.exe unexpectedly exited. Status code was: 1 while working with Chrome and Python
  30. How do I get current URL in Selenium Webdriver 2 Python?

Leave a Comment