visual_testing.yml
The visual_testing.yml file defines how visual regression testing is performed using Test Evolve.
enabled
enabled: true
Enable visual testing checks during a test run.
| option | description |
|---|---|
| true | performs a visual check when specified during the test run |
| false | ignores any visual checks during the test run |
source
source: local
Specify which visual testing integration will be used.
| language | source | description |
|---|---|---|
 | local | performs visual check comparisons with a saved baseline locally |
 | Applitools | uploads screenshots for comparison in Applitools |
| Percy | uploads DOM snapshots for comparison in Percy |
local
rubyIf you have configured source to be local, then the local section defines the configuration for the visual regression checks.
baseline_directory
rubybaseline_directory: visual_testing/baseline
Specify the directory to store baseline images in.
fail_fast
rubyfail_fast: false
Fails the functional test scenario on the first visual check that identifies a difference from the baseline.
| option | description |
|---|---|
| true | fails the test scenario on the first visual regression |
| false | performs all visual checks in the scenario before determining scenario status |
baseline_directory
rubywait_before_screenshots: 0
Specify the number of seconds before taking the screenshot for a visual check.
applitools
nodejsrubyIf you have configured source to be applitools, then the applitools section defines the configuration for the visual regression checks.
In order to use the Test Evolve Applitools integration you will need an account with Applitools. In order to perform visual regression tests using the following configuration you will also need to add a valid api key to the accounts.yml configuration file.
branch
nodejsrubybranch: default
Applitools supports a branching mechanism. Branches in Applitools contain visual baselines. The branching mechanism in Applitools allows the user to create, maintain and merge branches easily.
match_level
nodejsrubymatch_level: Layout
When writing an automated visual test, sometimes we will want to change the comparison method between our test and its baseline, especially when dealing with applications that consist of dynamic content. Applitools Eyes provides you the option to set different match levels within the test setup.
| option | description |
|---|---|
| None | With this level Eyes won't do any matching at all, whatever the content is, will be considered to be matched. |
| Layout | This level is useful for validating that the page layout is consistent. |
| Layout2 | This level is useful for validating that the page layout is consistent. |
| Content | This match level is similar to the strict match level but ignores differences in color. |
| Strict | This is the recommended level when you want to validate the precise content of the page. |
| Exact | This match level is not recommended for ordinary validation purposes. With this match level, Eyes does a pixel to pixel comparison of the two images. |
stitch_mode
nodejsruby stitch_mode: css # css, scroll
Use this method to set the type of stitching used for full page screenshots.
| option | description |
|---|---|
| css | Using CSS transition is a good method for pages with fixed position elements (i.e: fixed headers/floating bars), and prevents the page from capturing those elements in it's viewport while scrolling the screen in force-to fullscreen mode. |
| scroll | Using standard scrolling in cases that involve fixed position elements can be problematic: the fixed position element will always appear in the viewport, which means that every "part" of the entire page will include that element, even though it should only appear at the top of the page. |
fullpage_screenshot
nodejsruby fullpage_screenshot: true or false
Most browsers don't automatically get a full page screenshot: the browser grabs only what is visible in the viewport of the browser's window while the test is running. In order to get a full page screenshot, we have to enable it with Applitools Eyes.
save_new_tests
nodejsruby save_new_tests: true or false
The above function only applies on new tests - when a new test is run (according to the parameters that define the baseline), the new baseline will automatically be saved. If you set this method to False, you will be able to review your new baseline before saving it.
save_failed_tests
nodejsruby save_failed_tests: true or false
The above function only applies to failed tests, and saves them automatically as baselines. This feature can be used for maintenance, and allows you to avoid the previous baseline, and set the current screens as the new one automatically. Note: It's important to understand that this feature holds a risk since the changes are not being reviewed before they become your new baseline, and might include visual bugs.
hide_scrollbars
nodejsruby hide_scrollbars: true or false
Use this method to set whether Eyes hides the scrollbars before capturing screenshots or not. This is useful for eliminating false mismatches that are due to differences in the scrollbar position when the checkpoint is captured.
scale_ratio
nodejsruby scale_ratio: 1 # can be left blank or set to a number
Use this property to set and retrieve the scale ratio to be applied to images before matching. Typically used to compensate for different vertical and horizontal pixels density on some devices.
match_timeout
nodejsruby match_timeout: 5 # must be an integer and set >= 5s(Ruby) or 500ms(JavaScript)
Use this method to set the maximum time Eyes will try to perform a match on the fully captured image. Since a browser can take time to render a page (because it is complex, or because of slow network speeds), if Eyes detects mismatches, it will initially assume that the mismatch is because the render has not completed yet, and it will retry the match after a short wait. You can use this method to determine how much time Eyes spends retrying the matching before declaring a mismatch.
wait_before_screenshots
nodejsruby wait_before_screenshots: 5 # required - must be an integer > 0 - seconds in Ruby or Milliseconds in JavaScript
Use this property to set and retrieve the amount of time in milliseconds that Eyes will wait before capturing a screenshot. When a large image is captured with multiple sub-images using scrolling and stitching, Eyes will wait the amount of time specified by this method before capturing each sub-image.
app_name
nodejsruby app_name: Your Application or website
Use this method to set the value of the application name property. The application name is one of the 5 properties that define the baseline.
viewport_size
nodejsrubywidth: 800
height: 600
Set the default viewport size property. Eyes uses this if a subsequent call to open is done without a viewport size.
percy
nodejsrubyIf you have configured source to be percy, then the percy section defines the configuration for the visual regression checks.
In order to use the Test Evolve Percy integration you will need an account with Percy. In order to perform visual regression tests using the following configuration you will also need to add a valid api key to the accounts.yml configuration file.
branch
nodejsrubybranch: master
The branch in Percy to upload the DOM snapshot to.
widths
nodejsrubywidths: [800, 1200]
Specify the widths that each browser configured in Percy will be resized to for visual comparison.
debug
nodejsrubydebug: true
Turn on debug mode in Percy to get extra logs to the console.
Example visual_testing.yml
- Ruby
- JavaScript
&defaults_visual_testing
enabled: false
source: local
local:
baseline_directory: visual_testing/baseline
fail_fast: false
wait_before_screenshots: 0
applitools:
branch: default
match_level: Layout2
stitch_mode: css
fullpage_screenshot: true
save_new_tests: false
save_failed_tests: false
hide_scrollbars: true
scale_ratio:
match_timeout:
wait_before_screenshots:
app_name: Your Application or website
viewport_size:
width: 800
height: 600
percy:
branch: master
widths: [800, 1200]
debug: false
enabled: false
source: applitools
applitools:
branch: branch
match_level: Layout2
stitch_mode: css
fullpage_screenshot: true
save_new_tests: false
save_failed_tests: false
hide_scrollbars: true
scale_ratio:
match_timeout: 500
wait_before_screenshots:
app_name: Your Application or website
viewport_size:
width: 800
height: 600
percy:
branch: test
widths: [800, 1200]
debug: false