Synthetic Monitoring with Playwright

This tutorial demonstrates how Playwright automation tests can be turned into full synthetic monitoring using Step.

Get Step SaaS for free to follow this tutorial
Illustration for Synthetic Monitoring with Playwright

Synthetic monitoring, also known as active monitoring, is a method of monitoring your applications by simulating users, and directing the path they might take through the application. This provides information on the availability and performance of your critical business transactions.

This tutorial will demonstrate how Playwright automation tests can be turned into full synthetic monitoring in just a few minutes by leveraging the Step platform. At the end of the tutorial, you will be able to setup end-to-end synthetic monitoring for a web application (such as our e-shop OpenCart demo application) by:

  • Integrating your Playwright test case with Step
  • Designing and configuring your synthetic monitoring
  • Visualizing the performance and availability dashboard
  • Setting up alerts in order to get notified in case of errors

Because the reusability of your automation artifacts is one of the main advantages of Step, you can directly jump to the definition of the scenario if you already followed our tutorial for Playwright load testing.

Prerequisites

  • Step: You can either install Step Enterprise or OS following the quick setup guide or get a SaaS instance up and running in minutes. For most users the SaaS option is recommended, you can get it easily up and running on stepcloud.ch.
  • Git
  • Maven

Preparing the sample project

We have prepared a sample project that will be used throughout this tutorial which uses Playwright to automate interactions with a test environment called OpenCart that is provided by us.

1. Get the code

The project is available on github; to clone it using git, run:

  git clone https://github.com/exense/step-samples.git
  

2. (optional) Run JUnit tests locally

To see the interactions that are performed, run the JUnit tests:

  cd step-samples/keywords/java/demo-playwright-keyword
mvn clean compile test
  

The test cases automatically navigate through the “Desktop” and “Components” categories of the website, as defined in the unit tests.

3. Create a keyword package

  mvn clean package -DskipTests
  

This creates a jar file, normally named demo-playwright-keyword-0.0.0.jar, in the “target” folder of the project directory. This is the file that we will use in the subsequent steps.

Hint: -DskipTests will skip the local execution of the tests, if you did the optional step 2 successfully adding -DskipTests is not required.

Switch to your Step instance

If you have access to an existing Step instance (on-premise or SaaS), as described in the prerequisites, access it and log in.

1. Import the keyword package

Step has a special view for managing Keywords. This screenshot shows the Keywords view with the Upload Package dialog open as a reference for the next steps.

  1. Navigate to Keywords
  2. Click the Upload Package button
  3. Select the demo-playwright-keyword-0.0.0.jar as the package file
  4. Click Save to do the import

We can now use our automation code to design and perform synthetic monitoring on our OpenCart applications with Step

2. Create a plan

In Step, the plan is used to create automation scenarios; it allows you to design the user paths and to use controls for defining functional tests, performance tests, RPA, or synthetic monitoring. For this tutorial, we will keep the plan simple and use the scheduling of executions to perform synthetic monitoring.‍

  1. Navigate to Plans
  2. Click + to create a new plan
  3. Enter a name
  4. Select “TestCase” as a Template
  5. Click Save and edit

3. Visual plan editor

The visual plan editor is the main tool in to create and edit plans in Step. We will use this to create the TestCase, so get familiar with the UI explained in this screenshot:‍

4. Add a keyword to the plan

Our first keyword Opencart Testcase as Keyword used for this tutorial already represents the full user path to buy a product; we will simply add it to our plan and execute it once for validation.

Note: For production monitoring you would create different plans for features to be monitored, this tutorial will showcase this with only one plan.

Synthetic Monitoring Setup

Now that we have shown how to integrate any Playwright automation code with Step, we will demonstrate how to use them to define a test scenario and configure the synthetic monitoring.

In Step, the “test plan” is the entity used to create automation scenarios as it allows you to design the user paths and to use controls for defining either functional tests, performance tests, RPA or synthetic monitoring. In the context of synthetic monitoring, we will keep the test plan simple and use the scheduling of executions to perform synthetic monitoring.

  1. To create a plan in Step, go to the “Plans” view, click on “New plan” and keep the default type and template:
  2. Add the first Keyword (Opencart Testcase as Keyword) to the test plan and execute it once for validation. This Keyword already represents the full user path to buy a product.
  3. Create another plan using the individual Keywords.
  4. This plan will be used for synthetic monitoring by scheduling its execution at a given frequency, as well as grant the user access to more performance metrics and flexibility.
  5. Click the “run” button and choose to schedule the executions with the frequency of your choice. This will begin the synthetic monitoring process.
Note: The plans are also available in the git repository and can be directly imported in Step.

From here, the synthetic monitoring is in place, and you will see in the next sections how to monitor the performance and availability of your system, as well as how to create notifications in order to be automatically alerted in case of an error

Synthetic monitoring

Step contains a powerful scheduler module that allows us to execute plans periodically.

1. Setup the synthetic monitoring

To actually perform the synthetic monitoring, we will use the scheduling feature of Step. Simply click the “run” button and schedule the executions with your choice of frequency:

Now that the synthetic monitoring is in place, you will see in the next sections how to monitor the performance and availability of your system, as well as how to create notifications in order to be automatically alerted in case of error.

2. Monitor your scheduled executions

On the Monitoring view you can follow the latest status of your scheduled tasks. You can do this by clicking on the last execution to see the related execution reports, or by clicking on the chart icon to jump to the performance dashboard.

3. Analyze monitoring

On the performance dashboard, you can easily follow the overall performance and success rate of your synthetic monitoring, as well as the detailed statistics for individual user interaction with the monitored system.

4. Notifications for the monitoring

Pre-requisite: You will need to configure your notification gateway (email or webhooks).

While you can actively check the status and performance using the monitoring and dashboard views, for synthetic monitoring it is always recommended to be automatically alerted in case of errors. Setting up alerts for a given plan can be done directly from an execution or from the Notification Subscription menu entry.

Select Alerting → Notifications in the menu and create a new subscription. Select the plan you want to receive notifications about, the event that should trigger them and a Gateway where they should be delivered on.

Here we set up notifications for failed executions in our Synthetic Monitoring plan

Detect slow response times

To realize the full potential of synthetic monitoring, nonfunctional assertion should be added to the plan in order to detect response times above SLA. You can easily do that by adding performance assertions to your plan.

  1. add PerformanceAssert from the Controls tab
  2. type the exact name of the keyword that should be measured
  3. specify “lower than” (default)
  4. select the SLA (3000 is default)
Illustration for Using Step with Grafana
Using Step with Grafana

This article demonstrates how to connect Grafana to data generated by Step.

Illustration for Setting up system monitoring with a Step agent
Setting up system monitoring with a Step agent

This article demonstrates how to set up distributed system monitoring using Keyword executions, and analyze the results as measurements.

Illustration for NET tutorials: Microsoft Office automation with Step
NET tutorials: Microsoft Office automation with Step

This tutorial demonstrates how to automate interaction with Microsoft Office applications using the Office Interop Assembly.

Illustration for JUnit Plan Runner
JUnit Plan Runner

This article provides documentation for how to integrate JUnit tests into Step.

Illustration for How to monitor services availability and performance
How to monitor services availability and performance

This tutorial demonstrates how Step can be used to monitor services, availability and performance metrics.

Illustration for .NET tutorials: AutoIt with Step
.NET tutorials: AutoIt with Step

This tutorial demonstrates how to utilize the AutoIt C# binding to automate interactions with Windows applications.

Illustration for Android Testing using Step and Appium
Android Testing using Step and Appium

This article demonstrates the automation of mobile applications on Android using the Appium framework.

Illustration for Browser-based automation with Step and Selenium
Browser-based automation with Step and Selenium

This article defines three Keywords which will be used in browser-based automation scenarios, using Step and Selenium, as general drivers.

Illustration for Load Testing with Cypress
Load Testing with Cypress - advanced

This tutorial shows you how to efficiently set up a browser-based load test using existing Cypress tests in the Step automation platform.

Illustration for Adding and Configuring New Agents
Adding and Configuring New Agents

In this short tutorial, we show how to quickly implement a simple browser-based load test based on Cypress scripts in Step.

Illustration for Load Testing with Playwright
Load Testing with Playwright - advanced

This tutorial shows you how to efficiently set up a browser-based load test using existing Playwright tests in the Step automation platform.

Illustration for Basic Keyword Development
Basic Keyword Development

This article explains Keywords in Step and demonstrates how to create simple ones.

Illustration for Designing functional tests
Designing functional tests

This tutorial demonstrates the design, execution, and analysis of functional tests using the web interface of Step.

Illustration for Robotic Process Automation (RPA) with Selenium
Robotic Process Automation (RPA) with Selenium

This tutorial will demonstrate how to use Step and Selenium to automate various browser tasks.

Illustration for Robotic Process Automation (RPA) with Cypress
Robotic Process Automation (RPA) with Cypress

This tutorial demonstrates how to use Step and Cypress to automate various browser tasks.

Illustration for Synthetic Monitoring with Selenium
Synthetic Monitoring with Selenium

This tutorial demonstrates how Selenium automation tests can be turned into full synthetic monitoring using Step.

Illustration for Load Testing with Cypress
Load Testing with Cypress

In this tutorial, you'll learn how to reuse existing Cypress tests to quickly set up and run a browser-based load test using the automation as code approach.

Illustration for Load Testing with Cypress
Load Testing with Cypress using the Step UI

In this tutorial, you'll learn how to reuse existing Cypress tests to quickly set up and run a browser-based load test using the Step UI.

Illustration for Load Testing with Selenium
Load Testing with Selenium

This tutorial shows you how to efficiently set up a browser-based load test using existing Selenium tests in the Step automation platform.

Illustration for Synthetic Monitoring with Cypress
Synthetic Monitoring with Cypress

This tutorial demonstrates how Cypress automation tests can be turned into full synthetic monitoring using Step.

Illustration for Robotic Process Automation (RPA) with Playwright
Robotic Process Automation (RPA) with Playwright

This tutorial will demonstrate how to use Step and Playwright to automate various browser tasks.

Illustration for Load Testing with Playwright
Load Testing with Playwright

In this tutorial, we'll show you how to easily set up and run a browser-based load test with Step's wizard, using your existing Playwright tests, on the Step automation platform.

Illustration grafana devops tutorial
Continuous load testing with K6

Quickly integrate K6 based load-tests in your DevOps workflow

Illustration for playwright synthetic monitoring in a devops workflow
DevOps Synthetic Monitoring with Playwright - Advanced

This tutorial demonstrates how Playwright tests can be reused for synthetic monitoring of a productive environment in a DevOps workflow

Illustration for playwright synthetic monitoring in a devops workflow
DevOps Synthetic Monitoring with Playwright

This tutorial demonstrates how Playwright tests can be reused for synthetic monitoring of a productive environment in a DevOps workflow

Illustration for okhttp devops
Protocol-based load testing with okhttp

In this tutorial you'll learn how to quickly set up a protocol-based load test with okhttp

Illustration for playwright devops
Continuous end-to-end testing

Learn how to set up continuous end-to-end testing across several applications based on Playwright tests in your DevOps pipeline using Step

Illustration for playwright devops
Continuous load testing with Playwright

Learn how to quickly set up continuous browser-based load testing using Playwright tests in your DevOps pipeline

Want to hear our latest updates about automation?

Don't miss out on our regular blog posts - Subscribe now!

Image of a laptop device to incentivize users to subscribe