Javascript using Python Selenium WebDriver

In this tutorial, let’s analyze and see different ways to execute Javascript statements/commands through Python Selenium WebDriver. It may so happen in some real time projects, Selenium WebDriver is not able to perform some action on a particular web element and interacting with web elements through Javascript was the only possible solution.

For example, since WebDriver simulates end-user interaction, it is natural that it will refuse to click on an element that is not visible to end user (sometimes it happens though the web element is visible on the page). There can be several other similar reasons or scenarios. In these cases, we can rely on JavaScript to click or perform some actions on that web element and this javascript statement can be executed through WebDriver.

You can do everything that the WebElement interface does and more with Javascript.

What is JavaScript?

JavaScript is a scripting language that runs on client side, i.e on the browser and does some magic things when you surf through web pages. For more details visit here.

How do we do this?

Python selenium WebDriver provides an in-built method

driver.execute_script("some javascript code here")

There are two ways through which we can execute JavaScript within the browser.

Executing JavaScript at document root level:

In this case we capture the element, that we want to work with, using javascript provided methods, then declare some actions on it and execute this javascript using WebDriver.

Example:

javaScript = "document.getElementsByName('username')[0].click();"
driver.execute_script(javaScript)

What are we doing here ?

Step 1: We are inspecting and fetching the element by one of its property ‘name’ (Also id and class properties can be used) using javascript.

Step 2: Declare perform click action on an element using javascript.

Step 3: Call execute_script() method and pass the javascript that we created as String value

Notice [0] in getElementsByName(‘username’)[0] statement. JavaScript functions getElementsByName, getElementsByClassName etc return all matching elements as an array and in our case we need to act on first matching element which can be accessed via index [0]. If you know what you are doing i.e. if you know index of element that you want to operate, you can directly use the index like getElementsByName(‘username’)[2]

Whereas if you are using JavaScript function ‘getElementById’, you no need to use any indexing as it will return only one element (‘id’ should be unique).

While on execution, WebDriver will inject the JavaScript statement into the browser and the script will perform the job; In our example perform click operation on the target element. This javascript has its own namespace and does not interfere with javascript in actual web page.

Executing JavaScript at element level:

In this case we capture the element that we want to work with using webdriver, then we declare some actions on it using javascript and execute this javascript using WebDriver by passing web element as an argument to the javascript. Is it confusing? Lets break down

Example:

userName = driver.find_element_by_xpath("//button[@name='username']")
driver.execute_script("arguments[0].click();", userName)

Step 1: Inspect and capture the element using webdriver provided methods like ‘find_element_by_xpath’

userName = driver.find_element_by_xpath("//button[@name='username']")

Step 2: Declare perform click action on the element using javascript.

arguments[0].click()

Step 3: Call execute_script() method with the javascript statement that we created as String value and webelement captured using webdriver as arguments

driver.execute_script("arguments[0].click();", userName)

The above two lines of code can be shortened to below format where we find element using webdriver, declare some javascript functions and execute the javascript using webdriver.

driver.execute_script("arguments[0].click();", driver.find_element_by_xpath("//button[@name='username']"))

Another more frequently faced issue is scrolling to the bottom of the web page and you can perform this operation in a single line of code

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

Also you can have more than one javascript actions in your statement.

For example,

userName = driver.find_element_by_xpath("//button[@name='username']")
password = driver.find_element_by_xpath("//button[@name='password']")
driver.execute_script("arguments[0].click();arguments[1].click();", userName, password)

In this case, usage of order of web elements does not matters but index matters. Accessing index with [0] anywhere inside javascript statement will retrieve first web element passed.

driver.execute_script("arguments[1].click();arguments[0].click();", userName, password)

How to return values?

Other important aspect of javascript executor is, it can be used to fetch values from web element; That means execute_script() method can return values.

For example

text = driver.execute_script('return document.getElementById("fsr").innerText')
print(text)

Note that if you want something returned by javascript code, you need to use return. Also elements can be located with selenium and passed into the script.

What happens when element not found?

When javascript unable to find element to operate on, it throws WebDriverException with respective error message.

Scenario 1: Trying to read property using ‘print driver.execute_script(‘return document.getElementById(“fsr”).innerText’)‘ but no such element available in web page. We get following message in exception trace.

selenium.common.exceptions.WebDriverException: Message: unknown error: Cannot read property 'innerText' of null

Scenario 2: Trying to use invalid operation or error function name inside javascript like ‘print driver.execute_script(‘document.getElementById(“fsr”).clic();’)‘. Note the spelling mistake in click() method name.

selenium.common.exceptions.WebDriverException: Message: unknown error: document.getElementById(...).clic is not a function

Summary:
Here is a summary of all actions, but not limited to, possible using javascript.

• get an elements text or attribute
• find an element
• do some operation on element like click()
• change attributes of an element
• scroll to an element or location of a web page
• wait until the page is loaded

Basic knowledge of Javascript helps a lot when handling the DOM with Selenium.

Please let us know your thoughts in comments. Happy coding!!!