Setup Guidelines for Automating Web Interactions Using Selenium 4, Custom View Bot

Well this will be simple but custom view bot, that finds a element and then click it, and since it is overlayed by iframe, we tend to remove it through java script code which we will inject at runtime.

If you have not installed python you can do this via following this tutorial.

Automating web interactions can significantly enhance productivity and streamline processes, especially for repetitive tasks. This guide outlines how to set up a Selenium-based bot to interact with web elements, including overcoming obstructions such as iframes. By following these guidelines, you can configure a robust automation script using Python.

Prerequisites

Before diving into the setup, ensure you have the following installed:

• Python: The programming language for scripting.
• Selenium: A powerful library for web browser automation.
• Webdriver Manager: To manage browser driver binaries.

Step-by-Step Setup

1. Install Required Libraries

First, install Selenium and Webdriver Manager using pip:

pip install selenium webdriver-manager

2. Import Necessary Modules

In your Python script, import the required modules:

from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.webdriver import WebDriver
from webdriver_manager.chrome import ChromeDriverManager
import time

3. Define Constants

Set up the URL, XPath of the target element, and iframe ID:

url = "https://play.unity.com/webgl/5f5e9d70-473e-405b-ba09-4a94e559e8fd?screenshot=true&userID=63d3f6dac774962608d65d56"
xpath = r"/html/body/div[1]/div/div/div[1]/div[4]/div[1]/a"
iframe_id = "fullscreeniframe"

4. Function to Click Element

Create a function to find and click the element, handling iframes and retries:

def If_element_is_here_do_click(xpath1):
    global driver
    executed = 0
    time_inwhile = 0
    while executed == 0:
        try:
            print("x" + xpath1)
            iframe = driver.find_element(By.ID, iframe_id)
            driver.execute_script("arguments[0].parentNode.removeChild(arguments[0]);", iframe)
            driver.find_element(By.XPATH, xpath1).click()
            executed = 1
            print("Clicked")
            time.sleep(0.5)
            driver.refresh()
            If_element_is_here_do_click(xpath1)
        except:
            print("No" + str(time_inwhile))
            time_inwhile += 0.5
            if time_inwhile > 2:
                print("x is greater than 10")
                driver.close()
                driver = launchBrowser()
                driver.refresh()
                If_element_is_here_do_click(xpath1)
            else:
                print("time_inwhile is less than or equal to 4")
                time.sleep(0.5)

5. Function to Remove Iframe

Define a helper function to remove the iframe:

def removeIframe():
    global driver
    global iframe_id
    iframe = driver.find_element(By.ID, iframe_id)
    driver.execute_script("arguments[0].parentNode.removeChild(arguments[0]);", iframe)

6. Function to Launch Browser

Create a function to configure and launch the Chrome browser:

def launchBrowser():
    global driver
    chrome_options = Options()
    chrome_options.add_argument("--pageLoadStrategy=eager")
    service = Service(ChromeDriverManager().install())
    driver = WebDriver(service=service, options=chrome_options)
    return driver

7. Initialize and Run

Finally, initialize the browser and start the process:

driver = launchBrowser()
driver.get(url)
If_element_is_here_do_click(xpath)

Explanation of the Workflow

1. Launch Browser: The launchBrowser() function sets up Chrome options, installs the ChromeDriver, and initializes a Chrome browser instance configured to use an “eager” page load strategy.
2. Navigate to URL: The bot navigates to the specified URL (url).
3. Interact with Web Page: The If_element_is_here_do_click(xpath1) function continuously tries to find and click the element specified by the XPath. If an iframe obstructs the element, it removes the iframe and retries clicking the element, refreshing the page if necessary. If it fails within a specific timeframe, it re-launches the browser and retries the entire process.

Full Code

from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.webdriver import WebDriver
from webdriver_manager.chrome import ChromeDriverManager
import time

url = "https://play.unity.com/webgl/5f5e9d70-473e-405b-ba09-4a94e559e8fd?screenshot=true&userID=63d3f6dac774962608d65d56"
xpath = r"/html/body/div[1]/div/div/div[1]/div[4]/div[1]/a"
iframe_id = "fullscreeniframe"

def If_element_is_here_do_click(xpath1):
    global driver
    executed = 0
    time_inwhile = 0
    while executed == 0:
        try:
            print("x" + xpath1)
            iframe = driver.find_element(By.ID, iframe_id)
            driver.execute_script("arguments[0].parentNode.removeChild(arguments[0]);", iframe)#window.stop();
            driver.find_element(By.XPATH, xpath1).click()
            executed = 1
            print("Clicked")
            time.sleep(0.5)
            driver.refresh()
            If_element_is_here_do_click(xpath1)
        except:
            print("No"+str(time_inwhile))
            time_inwhile += 0.5
            #time_inwhile = 5
            if time_inwhile > 2:
                print("x is greater than 10")
                driver.close()
                driver = launchBrowser()
                driver.refresh()
                If_element_is_here_do_click(xpath1)
            else:
                print("time_inwhile is less than or equal to 4")
                time.sleep(0.5)

def removeIframe():
    global driver
    global iframe_id
    iframe = driver.find_element(By.ID, iframe_id)
    driver.execute_script("arguments[0].parentNode.removeChild(arguments[0]);", iframe)

def launchBrowser():
    global driver
    chrome_options = Options()
    #1 10 23chrome_options.add_argument("--incognito")
    chrome_options.add_argument("--pageLoadStrategy=eager")
    # chrome_options.add_argument('--headless')
    service = Service(ChromeDriverManager().install())
    driver = WebDriver(service=service, options=chrome_options)
    return driver

driver = launchBrowser()
driver.get(url)
If_element_is_here_do_click(xpath)

Conclusion

By following these setup guidelines, you can create a Selenium-based automation bot to interact with web elements efficiently. This approach is particularly useful for scenarios where dynamic web content requires repeated interactions, such as testing web applications or scraping data. With the ability to handle obstructions like iframes and implement retries, this bot ensures robust and reliable automation.