Scraping Data from a website which uses Power BI – retrieving data from Power BI on a website

by

Putting the scroll part and the JSON aside, I managed to read the data. The key is to read all of the elements inside the parent (which is done in the question):

parent = driver.find_element_by_xpath('//*[@id="pvExplorationHost"]/div/div/div/div[2]/div/div[2]/div[2]/visual-container[4]/div/div[3]/visual/div')
children = parent.find_elements_by_xpath('.//*')

Then sort them using their location:

x = [child.location['x'] for child in children]
y = [child.location['y'] for child in children]
index = np.lexsort((x,y))

To sort what we have read in different lines, this code may help:

rows = []
row = []
last_line = y[index[0]]
for i in index:
    if last_line != y[i]:
        row.append[children[i].get_attribute('title')]
    else:
        rows.append(row)
        row = list([children[i].get_attribute('title')]
rows.append(row)

More Related Contents:

  1. how to scrape web page that is not written directly using HTML, but is auto-generated using JavaScript? [closed]
  2. WebDriverWait not working as expected
  3. Difference between text and innerHTML using Selenium
  4. selenium with scrapy for dynamic page
  5. How can I download a file on a click event using selenium?
  6. Python: find_element_by_css_selector
  7. ‘list’ object has no attribute ‘get_attribute’ while iterating through WebElements
  8. Scrape website with dynamic mouseover event
  9. Dynamic dropdown doesn’t populate with auto suggestions on https://www.nseindia.com/ when values are passed using Selenium and Python
  10. How to click a link by text with No Text in Python
  11. Selenium Python: How to web scrape the element text
  12. Selenium-Debugging: Element is not clickable at point (X,Y)
  13. unable to call firefox from selenium in python on AWS machine
  14. Fetch all href link using selenium in python
  15. Python selenium multiprocessing
  16. Scraping dynamic data selenium – Unable to locate element
  17. Python selenium error when trying to launch firefox
  18. Speeding up beautifulsoup
  19. Python: Disable images in Selenium Google ChromeDriver
  20. “Failed to decode response from marionette” message in Python/Firefox headless scraping script
  21. Capture AJAX response with selenium and python
  22. Python Selenium accessing HTML source
  23. WebDriverException: Message: Service /usr/lib/chromium-browser/chromedriver unexpectedly exited on Raspberry-Pi with ChromeDriver and Selenium
  24. How to set Selenium Python WebDriver default timeout?
  25. How to get past a cookie agreement page using Python and Selenium?
  26. selenium.common.exceptions.ElementNotVisibleException: Message: element not visible while trying to access an element with Python + Selenium
  27. Selenium can’t open a second page
  28. How to get the localStorage with Python and Selenium WebDriver
  29. Selenium Python – Finding Elements by Class Name With Spaces
  30. How to handle IncompleteRead: in python

Leave a Comment