selenium.common.exceptions.WebDriverException: Message: invalid session id using Selenium with ChromeDriver and Chrome through Python

I'm writing some code using Selenium, and at one point I make 7 requests, all to different websites. For the first one, this works fine. However, for others, I get a session ID error. I think that my browser is configured correctly, as I do get results from the first website. I have tried to put a WebDriverWait in between the requests, but to no avail. I think the websites might be blocking my requests. Does anyone have any idea how to solve this problem?

I'm sorry if this is something stupid or if I'm doing anything wrong, I'm quite new ^^

Thanks in advance!

Traceback (most recent call last):
  File "/home/cena/PycharmProjects/Frikandelbroodje/main.py", line 56, in <module>
    dirk_price = get_price(dirk_url, dirk_classname)
  File "/home/cena/PycharmProjects/Frikandelbroodje/main.py", line 44, in get_price
    browser.get(url)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: invalid session id
  (Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}),platform=Linux 4.15.0-50-generic x86_64)

invalid session id

The invalid session ID error is a WebDriver error that occurs when the server does not recognize the unique session identifier. This happens if the session has been deleted or if the session ID is invalid.

A WebDriver session can be deleted through either of the following ways:

• Explicit session deletion: A WebDriver session is explicitly deleted when explicitly invoking the quit() method as follows:

  • Code Block:

    from selenium import webdriver
    from selenium.common.exceptions import InvalidSessionIdException
    
    driver = webdriver.Chrome(executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
    print("Current session is {}".format(driver.session_id))
    driver.quit()
    try:
        driver.get("https://www.google.com/")
    except Exception as e:
        print(e.message)
    

  • Console Output:

    Current session is a9272550-c4e5-450f-883d-553d337eed48
    No active session with ID a9272550-c4e5-450f-883d-553d337eed48
    

• Implicit session deletion: A WebDriver session is implicitly deleted when you close the last window or tab invoking close() method as follows:

  • Code Block:

    driver = webdriver.Chrome(executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
    print("Current session is {}".format(driver.session_id))
    # closes current window/tab
    driver.close()
    try:
        driver.get("https://www.google.com/")
    except Exception as e:
        print(e.message)
    

  • Console Output:

    Current session is a9272550-c4e5-450f-883d-553d337eed48
    No active session with ID a9272550-c4e5-450f-883d-553d337eed48
    

Conclusion

As the first one request works fine but for others you get a session ID error most possibly the WebDriver controled Web Browser is getting detected and hence blocking the next requests.

There are different reasons for the WebDriver controled Web Browser to get detected and simultaneously get blocked. You can find a couple of detailed discussion in:

• How does recaptcha 3 know I'm using selenium/chromedriver?
• Selenium and non-headless browser keeps asking for Captcha

Browser page crash may leads to InvalidSessionIdException. Selenium says to us: session deleted because of page crash. Check if your browser page still exists when you got your errors.

Here an example of a traceback of this case:

[2021-06-28 15:05:43,787: ERROR/ForkPoolWorker-2] Message: invalid session id
Traceback (most recent call last):
  ...
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from tab crashed
  (Session info: chrome=83.0.4103.61)


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  ...
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 580, in find_elements_by_class_name
    return self.find_elements(by=By.CLASS_NAME, value=name)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 1007, in find_elements
    'value': value})['value'] or []
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSessionIdException: Message: invalid session id

If you want some technical details, take a look at Chromium sources where you can find string session deleted because of page crash.