Selenium Python - How do I handle the exceptions on popups regardless of the order and handle a null or empty search

python selenium

I have a search that is looping through a list. I've added a try for the situations where the search requires additional clicks on the popups. However, the order of the nested try, except causes an issue depending on which order the popups come in. With the current code the application is getttng stuck on the optionModalContent popup. Additionally, if a search is completed that is null or empty, the application will timeout. What is the best way to handle the empty/null search? Example would be searching C06 or 0.

Below is my code with a slimmed down version of the application.

from selenium import webdriver
import pandas as pd
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait, TimeoutException
import time
appended_data  = []
classlist = ['C',
            'C01',
            'BETA BLOCKING AGENTS',  #optionModalContent
            'ANTIADRENERGIC AGENTS', #spellingModal
            'C02', 
            'C06',        #no search results timesout need to know how to handle this
            'C07',
            'Vitamins' #handled by optionModalContentTable    
            ]
time.sleep(1)
with webdriver.Chrome('C:\Program Files\Chrome Driver\chromedriver.exe') as driver:   ## uses the content manager by using with webdriver.Chrome, improves performance

    for search in classlist:
            page = f"https://mor.nlm.nih.gov/RxClass/search?query={search}"
            driver = webdriver.Chrome('C:\Program Files\Chrome Driver\chromedriver.exe')
            driver.get(page)

            modal_wait = WebDriverWait(driver, 4)
            try:
                modal_el = modal_wait.until(EC.element_to_be_clickable((By.ID, 'synModalContent')))  ## handles last popup if the search isn't exact
                modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
                

                try:
                    modal_el = modal_wait.until(EC.element_to_be_clickable((By.ID, 'optionModalContent')))  ## handles popup that requires you to select a line item
                    modal_el.find_element(By.CSS_SELECTOR, 'ul.uloption').click()   
                    
                except:
  
                    try:
                        modal_el = modal_wait.until(EC.element_to_be_clickable((By.ID, 'spellingModal')))  ## handles popup to suggest spelling
                        modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
                         

                    except:
                        modal_el = modal_wait.until(EC.element_to_be_clickable((By.ID, 'optionModalContentTable')))   ##handles first popup window on search if there are multiple classes for vitamin
                        modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
                        pass 
   
            except TimeoutException:
                pass        
            table = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, 'tr.dbsearch')))
            classid = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.table-responsive div.propText strong:nth-child(2)"))).text
            classname = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.table-responsive div.propText strong:nth-child(1)"))).text
            classtype = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.table-responsive div.propText strong:nth-child(3)"))).text
            filename = classname[0:30]+'.csv'
            df = pd.read_html(driver.page_source)[1].iloc[:,:-1].assign(ClassID=classid, ClassName=classname, ClassType=classtype)
            appended_data.append(df)
            driver.quit()
    appended_data = pd.concat(appended_data)
    appended_data.to_csv('testcategorization.csv')

I know my problem is going to be in my nested try except. I've moved things around and added passes, but regardless of the order the application will eventually get stuck on one of the searches. What is the best way to handle all of the popups regardless of the order?

Perhaps there is a better way to go about the try and except. Additionally if the search returns no results, I'm not sure how to handle that.

The end result I'm looking for is to get my application to run without getting stuck on a popup that is in the list regardless of the order it's ran in.


For now, this is what got me going:

Fetch the search key from current url, and subject the code to if elif else under try block. You may still improve it by using try/except in each if elif else statements.

title = driver.current_url
        print(title)
    try:
        if "BETA%20BLOCKING%20AGENTS" in title:
            modal_el = modal_wait.until(EC.element_to_be_clickable(
                (By.ID, 'optionModalContent')))  ## handles popup that requires you to select a line item
            modal_el.find_element(By.CSS_SELECTOR, 'ul.uloption').click()
        elif "ANTIADRENERGIC%20AGENTS" in title:
            modal_el = modal_wait.until(
                EC.element_to_be_clickable((By.ID, 'spellingModal')))  ## handles popup to suggest spelling
            modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
            modal_wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='optionModalContent']//li"))).click()
        elif "Vitamins" in title:
            modal_el = modal_wait.until(EC.element_to_be_clickable((By.ID,
                                                                    'optionModalContentTable')))  ##handles first popup window on search if there are multiple classes for vitamin
            modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
            modal_wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='optionModalContent']//li"))).click()
        else:
            modal_el = modal_wait.until(
                EC.element_to_be_clickable((By.ID, 'synModalContent')))  ## handles last popup if the search isn't exact
            modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()

Still checking how to handle the no classes found condition. If I crack it, will post it here as update. Will check in my free time. I would want to appreciate for the code that you wrote. It is very lean except that an additional browser instance opens, but I would say this is a very good one. I learned something today from you.

UPDATE: I tried some alternatives and it worked. Please check this below code. It manages to get the result by search each item from the search list, while handling the C06.

I used a lot of print statements which I used for debug purposes. Feel free to remove them. Also, I used time.sleep in few occasions to have the alert handled. Try to reduce it and see if it works.

for search in classlist:
    page = f"https://mor.nlm.nih.gov/RxClass/search?query={search}"
    driver = webdriver.Chrome(ChromeDriverManager().install())
    driver.get(page)
    time.sleep(5)
    try:
        WebDriverWait(driver, 3).until(EC.alert_is_present())
        alert = driver.switch_to.alert
        alert.accept()
        print("alert accepted")
    except TimeoutException:
        print("no alert")
    title = driver.current_url
    print(title)
    modal_wait = WebDriverWait(driver, 4)
    try:
        if "BETA%20BLOCKING%20AGENTS" in title:
            print("betablocker")
            modal_el = modal_wait.until(EC.element_to_be_clickable(
                (By.ID, 'optionModalContent')))  ## handles popup that       requires you to select a line item
            modal_el.find_element(By.CSS_SELECTOR, 'ul.uloption').click()
        elif "ANTIADRENERGIC%20AGENTS" in title:
            print("diuretic")
            modal_el = modal_wait.until(
                EC.element_to_be_clickable((By.ID, 'spellingModal')))  ## handles popup to suggest spelling
            modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
            modal_wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='optionModalContent']//li"))).click()
        elif "Vitamins" in title:
            print("vitamin")
            modal_el = modal_wait.until(EC.element_to_be_clickable((By.ID,
                                                                    'optionModalContentTable')))  ##handles first popup window on search if there are multiple classes for vitamin
            modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
            modal_wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='optionModalContent']//li"))).click()
        else:
            print("others")
            modal_el = modal_wait.until(
                EC.element_to_be_clickable((By.ID, 'synModalContent')))  ## handles last popup if the search isn't exact
            modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
    except Exception as e:
        print(f"No tables found in {search}")
        continue

Console Output (End result):

Process finished with exit code 0

UPDATE 2: TO HANDLE TRY/EXCEPT per the comment of query creator.

for search in classlist:
    page = f"https://mor.nlm.nih.gov/RxClass/search?query={search}"
    driver = webdriver.Chrome(ChromeDriverManager().install())
    driver.get(page)
    time.sleep(2)
    try:
        WebDriverWait(driver, 3).until(EC.alert_is_present())
        alert = driver.switch_to.alert
        alert.accept()
        print("alert accepted")
        print(f"No tables found in {search}")
        continue
    except TimeoutException:
        print("no alert")
    modal_wait = WebDriverWait(driver, 4)
    try:
        modal_el = modal_wait.until(
            EC.element_to_be_clickable((By.ID, 'synModalContent')))  ## handles last popup if the search isn't exact
        modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
    except:
        try:
            modal_el = modal_wait.until(EC.element_to_be_clickable(
                (By.ID, 'optionModalContent')))  ## handles popup that requires you to select a line item
            modal_el.find_element(By.CSS_SELECTOR, 'ul.uloption').click()
        except:
            try:
                modal_el = modal_wait.until(
                    EC.element_to_be_clickable((By.ID, 'spellingModal')))  ## handles popup to suggest spelling
                modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
            except:
                modal_el = modal_wait.until(EC.element_to_be_clickable((By.ID,
                                                                        'optionModalContentTable')))  ##handles first popup window on search if there are multiple classes for vitamin
                # modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
                modal_el.find_element(By.CSS_SELECTOR, 'div.nameEntry').click()
                pass

Related

Matplotlib pie chart wedges using color gradient
Export P7b file with all the certificate chain into CER file
JS map return object
How to detect if front camera is available in android?
I want to display a popup message of date contains in text file
Close polygons using linestrings and selection of polygons created by area
Sign text color: Can't enter the § symbol
I've accidentally set too much TNT explosions and destroyed my hard worked on builds, what do I do to reverse the damage quicker?
What does it mean when I hear giant gates and chains while mining?
How do I lock any doors in minecraft using command blocks?
How do I give players a potion effect when they reach a certain XP level?
How to map certain Kega Fusion functions to a gamepad?