Take screenshot of full page with Selenium Python with chromedriver
This answer improves upon prior answers by am05mhz and Javed Karim.
It assumes headless mode, and that a window-size option was not initially set. Before calling this function, ensure the page has loaded fully or sufficiently.
It attempts to set the width and height both to what is necessary. The screenshot of the entire page can sometimes include a needless vertical scrollbar. One way to generally avoid the scrollbar is by taking a screenshot of the body element instead. After saving a screenshot, it reverts the size to what it was originally, failing which the size for the next screenshot may not set correctly.
Ultimately this technique may still not work perfectly well for some examples.
from selenium import webdriver
def save_screenshot(driver: webdriver.Chrome, path: str = '/tmp/screenshot.png') -> None:
# Ref: https://stackoverflow.com/a/52572919/
original_size = driver.get_window_size()
required_width = driver.execute_script('return document.body.parentNode.scrollWidth')
required_height = driver.execute_script('return document.body.parentNode.scrollHeight')
driver.set_window_size(required_width, required_height)
# driver.save_screenshot(path) # has scrollbar
driver.find_element_by_tag_name('body').screenshot(path) # avoids scrollbar
driver.set_window_size(original_size['width'], original_size['height'])
If using Python older than 3.6, remove the type annotations from the function definition.
Screenshots are limited to the viewport but you can get around this by capturing the body
element, as the webdriver will capture the entire element even if it is larger than the viewport. This will save you having to deal with scrolling and stitching images, however you might see problems with footer position (like in the screenshot below).
Tested on Windows 8 and Mac High Sierra with Chrome Driver.
from selenium import webdriver
url = 'https://stackoverflow.com/'
path = '/path/to/save/in/scrape.png'
driver = webdriver.Chrome()
driver.get(url)
el = driver.find_element_by_tag_name('body')
el.screenshot(path)
driver.quit()
Returns: (full size: https://i.stack.imgur.com/ppDiI.png)
How it works: set browser height as longest as you can...
#coding=utf-8
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
def test_fullpage_screenshot(self):
chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--start-maximized')
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get("yoururlxxx")
time.sleep(2)
#the element with longest height on page
ele=driver.find_element("xpath", '//div[@class="react-grid-layout layout"]')
total_height = ele.size["height"]+1000
driver.set_window_size(1920, total_height) #the trick
time.sleep(2)
driver.save_screenshot("screenshot1.png")
driver.quit()
if __name__ == "__main__":
test_fullpage_screenshot()
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('https://developer.mozilla.org/')
element = driver.find_element_by_tag_name('body')
element_png = element.screenshot_as_png
with open("test2.png", "wb") as file:
file.write(element_png)
This works for me. It saves the entire page as screenshot. For more information you can read up the api docs: http://selenium-python.readthedocs.io/api.html
The key is to turn on the headless
mode!
No stitching required and no need for loading the page twice.
Full working code:
URL = 'http://www.w3schools.com/js/default.asp'
options = webdriver.ChromeOptions()
options.headless = True
driver = webdriver.Chrome(options=options)
driver.get(URL)
S = lambda X: driver.execute_script('return document.body.parentNode.scroll'+X)
driver.set_window_size(S('Width'),S('Height')) # May need manual adjustment
driver.find_element_by_tag_name('body').screenshot('web_screenshot.png')
driver.quit()
This is practically the same code as posted by @Acumenus with slight improvements.
Summary of my findings
I decided to post this anyway because I did not find an explanation about what is happening when the headless
mode is turned off (the browser is displayed) for screenshot taking purposes.
As I tested (with Chrome WebDriver), if the headless
mode is turned on, the screenshot is saved as desired. However, if the headless
mode is turned off, the saved screenshot has approximately the correct width and height, but the outcome varies case-by-case. Usually, the upper part of the page which is visible by the screen is saved, but the rest of the image is just plain white. There was also a case with trying to save this Stack Overflow thread by using the above link; even the upper part was not saved which interestingly now was transparent while the rest still white. The last case I noticed was only once with the given W3Schools link; there where no white parts but the upper part of the page repeated until the end, including the header.
I hope this will help for many of those who for some reason are not getting the expected result as I did not see anyone explicitly explaining about the requirement of headless
mode with this simple approach.
Only when I discovered the solution to this problem myself, I found a post by @vc2279 mentioning that the window of a headless browser can be set to any size (which seems to be true for the opposite case too). Although, the solution in my post improves upon that that it does not require repeated browser/driver opening or page reloading.
Further suggestions
If for some pages it does not work for you, I suggest trying to add time.sleep(seconds)
before getting the size of the page. Another case would be if the page requires scrolling until the bottom to load further content, which can be solved by the scheight
method from this post:
scheight = .1
while scheight < 9.9:
driver.execute_script("window.scrollTo(0, document.body.scrollHeight/%s);" % scheight)
scheight += .01
Also, note that for some pages the content may not be in any of the top-level HTML tags like <html>
or <body>
, for example, YouTube uses <ytd-app>
tag.
As a last note, I found one page that "returned" a screenshot still with the horizontal scrollbar, the size of the window needed manual adjustment, i.e., the image width needed to be increased by 18 pixels, like so: S('Width')+18
.