How to bypass being rate limited ..HTML Error 1015 using Python

by

If you are randomly greeted with…

rateLimited

…implies that the site owner implemented Rate Limiting that affects your visitor traffic.

rate-limiting reason

Cloudflare can rate-limit the the visitor traffic trying to counter a possible Dictionary attack.

rate-limit thresholds

In generic cases Cloudflare rate-limits the visitor when the visitor traffic crosses the rate-limit thresholds which is calculated by, dividing 24 hours of uncached website requests by the unique visitors for the same 24 hours. Then, divide by the estimated average minutes of a visit. Finally, multiply by 4 (or larger) to establish an estimated threshold per minute for your website. A value higher than 4 is fine since most attacks are an order of magnitude above typical traffic rates.

Solution

In these cases the a potential solution would be to use the undetected-chromedriver to initialize the Chrome Browsing Context.

undetected-chromedriver is an optimized Selenium Chromedriver patch which does not trigger anti-bot services like Distill Network / Imperva / DataDome / Botprotect.io. It automatically downloads the driver binary and patches it.

  • Code Block:

    import undetected_chromedriver as uc
from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
driver = uc.Chrome(options=options)
driver.get('https://bet365.com')

References

You can find a couple of relevant detailed discussions in:

More Related Contents:

  1. How can I reconnect to the browser opened by webdriver with selenium?
  2. selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable when clicking on an element using Selenium Python
  3. Invalid selector: Compound class names not permitted error using Selenium
  4. DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  5. webdriver.FirefoxProfile(): Is it possible to use a profile without making a copy of it?
  6. SessionNotCreatedException: Message: session not created from disconnected: unable to connect to renderer with ChromeDriver 2.45 Chrome v71
  7. Selenium “Unable to find a matching set of capabilities” despite driver being in /usr/local/bin
  8. AttributeError: ‘WebDriver’ object has no attribute ‘find_element_by_xpath’
  9. selenium.common.exceptions.WebDriverException: Message: unknown error: unable to discover open pages using ChromeDriver through Selenium
  10. how to scrape web page that is not written directly using HTML, but is auto-generated using JavaScript? [closed]
  11. How to retrieve the text of a WebElement using Selenium – Python
  12. How to make Firefox headless programmatically in Selenium with Python?
  13. Python : no module named selenium
  14. Of the many findElement(s)/By functions in Selenium, when would you use one over the other?
  15. ElementNotVisibleException: Message: element not interactable error while trying to click a button through Selenium and Python
  16. How to deal with certificates using Selenium?
  17. Scrolling to element using webdriver?
  18. in selenium web driver how to choose the correct iframe
  19. UnicodeEncodeError: ‘charmap’ codec can’t encode character ‘\u2010’: character maps to [duplicate]
  20. how to set proxy with authentication in selenium chromedriver python?
  21. SyntaxError: Failed to execute ‘evaluate’ on ‘Document’: The string ‘//img[contains(‘1236548597′)]’ is not a valid XPath expression
  22. How to use Tor with Chrome browser through Selenium
  23. Sending selenium chrome instance to the background using Python
  24. selenium.common.exceptions.SessionNotCreatedException: Message: session not created from tab crashed using ChromeDriver Chrome Selenium Python
  25. Failed to load resource: the server responded with a status of 429 (Too Many Requests) and 404 (Not Found) with ChromeDriver Chrome through Selenium
  26. “Eager” Page Load Strategy workaround for Chromedriver Selenium in Python
  27. ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
  28. How to click on the anchor element when the spinner obscures it using Selenium and Python?
  29. Handle notifications in Python + Selenium Chrome WebDriver
  30. How to suppress console error/warning/info messages when executing selenium python scripts using chrome canary

Leave a Comment