+91 9790664230
kumar.arun211@gmail.com

Selenium-Docker: Parallel execution made easy

Blog to learn python and selenium

Selenium-Docker: Parallel execution made easy

Selenium Grid supports distributed test execution and allows running tests on different machines against different browsers in parallel. This reduces execution time from days to hours. However Selenium Grid setup and maintenance requires effort and has its own limitations.

  1. User needs to install browsers manually as required
  2. User needs to ensure proper selenium libraries are in place in hub and node machines
  3. User needs to ensure proper browser drivers are in place in node machines
  4. In case of same browser with different versions are used, User need to setup all versions manually and ensure selenium node setup command is proper.
  5. Scaling of browsers available is not easy; Node has to be brought down to increase browser instances and session which is not possible when tests are already running
  6. Requires constant maintenance as the machines had to be kept up and running at all times
  7. Setup cannot be changed from one machine to other machine easily

Selenium Docker helps in setting up test labs with grid in very easy and simple steps by removing all complexities. This has vastly improved the efficiency of automation and provides an ideal environment for distributed testing by addressing above pain points. It allows us to spin up the grid/nodes and tear them down all in simple commands. By end of this article, we will get solution for all the limitations mentioned above.

Please visit https://www.docker.com/ for more information on Docker. This article requires docker installation as prerequisite – https://docs.docker.com/engine/installation/

You can verify docker installation by running any of below commands

docker ps -a

docker version

docker info

Deploying Selenium Grid on a Standalone Docker Container:

Selenium provides standalone grid server Docker images with Chrome and Firefox browsers already installed on it.

docker run -d -P -p "4444:4444" --name standalone_grid_chrome selenium/standalone-chrome

With this command, the Selenium standalone Chrome image will be downloaded and the server will be started on your local machine on port 4444.

Similarly Firefox standalone server also can be started.

docker run -d -P -p "4444:4444" --name standalone_grid_firefox selenium/standalone-firefox

Verify whether the running container is running or not using the following command or visiting http://[your machine IP]:4444/grid/console url.

docker ps -a

Please note that the execution happens on these images is headless using Xvfb.

If you want to see your execution or debug your script step by step, you can use the selenium/standalone-chrome-debug or Firefox images. These images have a VNC server installed so you can use VNC viewer or any other VNC client.

docker run -d -P -p "4444:4444" --name standalone_grid_firefox selenium/standalone-firefox-debug

docker run -d -P -p "4444:4444" --name standalone_grid_chrome selenium/standalone-chrome-debug

By default, when there is no image tag specified, latest image with the latest browser version is used. Specific version of the browser can also be used to create standalone grid.

You should use docker image available with that browser version. For example, if you want to use Firefox 51.0, you can use the tag 3.5 tag next to the image name.

docker run -d -P -p "5554:4444" --name firefox_3.0 --link hub:hub selenium/node-firefox:3.0

All images are maintained with the tag on the Selenium Docker Hub – https://hub.docker.com/r/selenium/

Deploying Selenium Grid on Multiple Containers Using Docker Compose:

Docker Compose is the tool that lets you deploy Selenium Grid in multiple containers. Docker Compose uses YAML files to configure application services like a hub. Chrome and Firefox will be the services in this case.

Create a docker-compose.yml file on your machine.


version: "2"
services:
 hub:
   image: selenium/hub
   ports:
    - 4444:4444
   environment:
    GRID_MAX_SESSION: 10
 firefox:
   image: selenium/node-firefox
   depends_on:
    - hub
   environment:
    HUB_HOST: hub
    NODE_MAX_INSTANCES: 5
    NODE_MAX_SESSION: 5
 chrome:
   image: selenium/node-chrome
   depends_on:
    - hub
   environment:
    HUB_HOST: hub
    NODE_MAX_INSTANCES: 5
    NODE_MAX_SESSION: 5

image – Selenium docker image name

environment – used to configure environment variables for the hub and nodes.

depends_on – used to maintain start up order

HUB_HOST – used to connect nodes to the hub with its service name.

NODE_MAX_INSTANCES – number of instances of same version of browser that can run in node.

NODE_MAX_SESSION – number of browsers (Any browser and version) that can run in parallel at a time in node.

Note: NODE_MAX_SESSION overrides max instances settings by restricting the number of browser instances that can run in parallel.

Save the docker-compose.yml file and start the grid using the command:

docker-compose up -d

This will pull and run the images as hub, and nodes. The nodes will register and start communicating with the hub. You can check whether everything is running using below command.

docker ps -a

Also you can verify it in browser – http://[your machine IP]:4444/grid/console

If you want to increase the number of browser containers, you can use ‘scale‘ option. This will create specified number of chrome containers and number of containers includes already running containers. Same ‘scale’ option can be used to reduce browser containers, just need to provide lesser numbers than existing number of containers(For example, to remove 3 from 5 containers, use scale=2).

docker-compose scale chrome=2

To stop all containers created using compose file:

docker-compose stop

To start all containers created using compose file:

docker-compose start

To start/stop all containers created using compose file:

docker-compose start #containerID

docker-compose stop #containerID

You can get container ID using ‘docker ps -a’ command.

To stop and remove containers:

docker-compose down

Deploying Selenium Grid on Multiple Containers with different browsers and versions:

docker run -d -P -p "4444:4444" --name hub selenium/hub


docker run -d -P -p "5551:4444" --name chrome_latest --link hub:hub selenium/node-chrome

docker run -d -P -p "5552:4444" --name chrome_3.4.0 --link hub:hub selenium/node-chrome:3.4.0

docker run -d -P -p "5553:4444" --name chrome_3.0 --link hub:hub selenium/node-chrome:3.0



docker run -d -P -p "5556:4444" --name firefox_latest --link hub:hub selenium/node-firefox

docker run -d -P -p "5555:4444" --name firefox_3.4.0 --link hub:hub selenium/node-firefox:3.4.0

docker run -d -P -p "5554:4444" --name firefox_3.0 --link hub:hub selenium/node-firefox:3.0

After running above commands, a grid will be setup with 3 different versions of chrome and firefox each.

Difference between images and containers:

Images –

The file system and configuration of application which are used to create containers.

Containers –

It is running instance of Docker images

You can see all tagged versions of selenium images on docker hub. For example, standalone-chrome tags can be found in https://hub.docker.com/r/selenium/standalone-chrome/tags/

Please note the difference between different image names

standalone-firefox : Image to create standalone grid

standalone-firefox-debug : Image to create standalone grid with debugging capability

node-firefox : Image to create selenium node that can be registered to hub

Hope we have got solutions for all limitations mentioned.

4 Responses

  1. […] I hope that you found this useful in solving the limitations mentioned. You can also find this post on my blog. […]

  2. Mislav says:

    nice post.

    I tried to run multiplier browsers in parallel for web scraping, but never couldn’t get more than 3 stable processes. When I ran say 5 browsers in parallel, there were always some errors appeared.

    can you also write a post about how to implement this docker hub for parallel testing, with real code in python on some other language?

  3. Lolo says:

    By following the docker-compose.yml, it does not works on docker version 18.06
    ERROR: In file ‘./docker-compose.yml’, service ‘image’ must be a mapping not a string.
    How to fix this ?

    • admin says:

      docker-compose.yml is a YAML file which follows certain format and indentation. The issue you faced is due to incorrect indentation. I updated post. Please try and let me know.

Leave a Reply

Your email address will not be published. Required fields are marked *

© 2018 Copyrights. All Rights Reserved. Arunkumar Velusamy