Michał Kuncio
Michał Kuncio
March 06, 2023

Recording and replaying network requests with Playwright

Created by human

Playwright is a new Cypress

For many years, Cypress was industry-standard tool for end-to-end tests, especially in JavaScript world. I loved that tool for its great API and amazing docs. The interactive UI was great and made debugging tests fun and easy. But there is a new cool kid on the block - Playwright. It has many benefits over Cypress but since there are already many articles about that, we will not cover it inside this article. Instead, we will cover not so popular mocking technique - recording and replying requests.

Standard API mocking

Standard API mocking will generally intercept a real API call and provide mock JSON passed to route.fulfill(). You can do it easily with Playwright using the following code:

await page.route('https://testapi.com/api/football-players', async route => {
  const json = {
    [
        'Kylian Mbappe',
        'Erling Haaland',
        'Arturo Vidal',
        'Kamil Grosicki'
    ]
  };
  await route.fulfill({ json });
});

This method is perfectly fine and would be the best option for most cases. But in some scenarios, we might want to record requests and run tests based on that.

Why would I want to record requests?

Let's imagine the following situation. We have an e-commerce website with many, many, many filters. Some filters are isolated, some filters depend on other filters. The payload may consist of 200 query parameters. How would you test it with single mocked JSON? It's impossible to test hundreds of combinations that way. That's why it might be a good idea to run the test, record all API requests, store the whole information about the requests inside a single file, and then use that data to run the tests in the future. Why not use real API instead of mocking? Well, it would be silly to perform requests like POST, PUT, PATH or DELETE which are mutable by nature on real DB just to run tests. What about simple GET queries? Sometimes the API has some limits or its pricing is based on a number of requests. There are multiple reasons why we might want to avoid using real API inside our tests. Let's take a quick look at how we can achieve it with Playwright.

Example app

I have built a simple Vue app which fetches the data about sunset and sunrise using sunsetsunrise API.

Here's the code:

<template>
<div class="sun-hours">
    <h1 data-testid="sun-hours">Sun hours</h1>
    <input type="text" placeholder="latitude" v-model="lat">
    <input type="text" placeholder="longitude" v-model="lng">
    <button @click="getSunHours" data-testid="search-btn">Search</button>

    <div v-if="result">
        <h2 data-testid="results-header">Results:</h2>
        <div>
            <span>Sunrise: </span>
            <span>{{result.sunrise}}</span>
        </div>

        <div>
            <span>Sunset: </span>
            <span>{{result.sunset}}</span>
        </div>

        <div>
            <span>Golden hour: </span>
            <span>{{result.golden_hour}}</span>
        </div>
    </div>
</div>
</template>

<script setup lang="ts">
import {onMounted, ref} from "vue";
const lat = ref('38.907192');
const lng = ref('-77.036873');
const result = ref<SunHoursResponse | null>(null);

interface SunHoursResponse {
    sunrise: string,
    sunset: string,
    golden_hour: string,
}

async function getSunHours() {
    const response = await fetch(`https://api.sunrisesunset.io/json?lat=${lat.value}&lng=${lng.value}&timezone=UTC&date=today`);
    result.value =  (await response.json()).results;
}
</script>

<style scoped>
.sun-hours {
    display: flex;
    flex-direction: column;
    gap: 20px;
}
</style>

Simple test

The test simply fills out the form with latitude and longitude and clicks the button to get the data from API. After that, we perform some checks and that's it:

//sunhours.spec.ts
import { test, expect } from '@playwright/test';

test('checks if API returns result', async ({browser }) => {
    await page.goto('http://localhost:5173/');

    await expect(page.getByTestId('sun-hours')).toHaveText('Sun hours');

    await page.getByTestId('search-btn').click();
    await expect(page.getByTestId('results-header')).toHaveText('Results:');
});

Recording requests

Let's assume that this API is a paid product and there are much more tests like this one which checks different combinations of parameters and edge cases. Let's configure our test to record the initial requests by adding the following lines at the beginning of the test:

const context = await browser.newContext({
        recordHar: { path: 'requests.har', mode: 'full', urlFilter: '**/api.sunrisesunset.io/**' }
    });

const page = await context.newPage();

and close the context at the end of test file:

await context.close();

We are creating the new browser context with the setting to record requests and store them inside HAR files. We are passing an options object to newContext constructor with an option called recordHar.

  • path - the path to the file storing all recordings
  • mode - it can be set to 'full' or 'minimal'. The 'full' option will store every single detail about the request, while 'minimal' is more concise.
  • urlFilter - we can specify the pattern on which URLs recordings should be made

The whole test file looks like this:

import { test, expect } from '@playwright/test';

test('checks if API returns result', async ({browser }) => {

    const context = await browser.newContext({
        recordHar: { path: 'requests.har', mode: 'full', urlFilter: '**/api.sunrisesunset.io/**' }
    });

    const page = await context.newPage();

    await page.goto('http://localhost:5173/');

    await expect(page.getByTestId('sun-hours')).toHaveText('Sun hours');

    await page.getByTestId('search-btn').click();
    await expect(page.getByTestId('results-header')).toHaveText('Results:');
    await context.close();
});

Once we run the test by the following command,

npx playwright test sunhours.spec.ts

we can see that a new file requests.har is created. Here's what this file may look like:

{
  "log": {
    "version": "1.2",
    "creator": {
      "name": "Playwright",
      "version": "1.31.1"
    },
    "browser": {
      "name": "firefox",
      "version": "109.0"
    },
    "pages": [
      {
        "startedDateTime": "2023-03-05T21:38:57.390Z",
        "id": "page@e9d0544dbdcdaf2e7ef5e43c0e00e4eb",
        "title": "Vite + Vue + TS",
        "pageTimings": {
          "onContentLoad": 164,
          "onLoad": 176
        }
      }
    ],
    "entries": [
      {
        "startedDateTime": "2023-03-05T21:38:57.654Z",
        "time": 182.55100000000002,
        "request": {
          "method": "GET",
          "url": "https://api.sunrisesunset.io/json?lat=38.907192&lng=-77.036873&timezone=UTC&date=today",
          "httpVersion": "HTTP/2.0",
          "cookies": [],
          "headers": [
            { "name": "Host", "value": "api.sunrisesunset.io" },
            { "name": "User-Agent", "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/109.0" },
            { "name": "Accept", "value": "*/*" },
            { "name": "Accept-Language", "value": "en-US" },
            { "name": "Accept-Encoding", "value": "gzip, deflate, br" },
            { "name": "Referer", "value": "http://localhost:5173/" },
            { "name": "Origin", "value": "http://localhost:5173" },
            { "name": "Connection", "value": "keep-alive" },
            { "name": "Sec-Fetch-Dest", "value": "empty" },
            { "name": "Sec-Fetch-Mode", "value": "cors" },
            { "name": "Sec-Fetch-Site", "value": "cross-site" }
          ],
          "queryString": [
            {
              "name": "lat",
              "value": "38.907192"
            },
            {
              "name": "lng",
              "value": "-77.036873"
            },
            {
              "name": "timezone",
              "value": "UTC"
            },
            {
              "name": "date",
              "value": "today"
            }
          ],
          "headersSize": 376,
          "bodySize": 0
        },
        "response": {
          "status": 200,
          "statusText": "OK",
          "httpVersion": "HTTP/2.0",
          "cookies": [],
          "headers": [
            { "name": "date", "value": "Sun" },
            { "name": "date", "value": "05 Mar 2023 21:38:57 GMT" },
            { "name": "content-type", "value": "application/json" },
            { "name": "cf-ray", "value": "7a358247496cb90f-AMS" },
            { "name": "access-control-allow-origin", "value": "*" },
            { "name": "age", "value": "6553" },
            { "name": "cache-control", "value": "s-max-age=1320" },
            { "name": "cache-control", "value": "max-age=1320" },
            { "name": "last-modified", "value": "Sun" },
            { "name": "last-modified", "value": "05 Mar 2023 17:59:18 GMT" },
            { "name": "link", "value": "<https://sunrisesunset.io/wp-json/>; rel=\"https://api.w.org/\"" },
            { "name": "strict-transport-security", "value": "max-age=15552000" },
            { "name": "vary", "value": "Accept-Encoding" },
            { "name": "cf-cache-status", "value": "HIT" },
            { "name": "fastcgi-cache", "value": "BYPASS" },
            { "name": "x-content-type-options", "value": "nosniff" },
            { "name": "x-frame-options", "value": "SAMEORIGIN" },
            { "name": "x-robots-tag", "value": "noindex" },
            { "name": "x-ua-compatible", "value": "IE=edge" },
            { "name": "x-xss-protection", "value": "1; mode=block" },
            { "name": "report-to", "value": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=vnbhiEqRfzhoBgtaOEfESVjShLW%2B3OMVpAeGJ%2FxwC7RLPvxSvjPNaaFdQTtgAtl3Jw6E%2FkXC43DvmB8oyCxrok7%2BRa2M6JYR0l2Ri9el79wOg3DQEtwC%2BwKjpRR7iP4CrZDM64Pr3g%3D%3D\"}]" },
            { "name": "report-to", "value": "\"group\":\"cf-nel\"" },
            { "name": "report-to", "value": "\"max_age\":604800}" },
            { "name": "nel", "value": "{\"success_fraction\":0" },
            { "name": "nel", "value": "\"report_to\":\"cf-nel\"" },
            { "name": "nel", "value": "\"max_age\":604800}" },
            { "name": "server", "value": "cloudflare" },
            { "name": "content-encoding", "value": "br" },
            { "name": "alt-svc", "value": "h3=\":443\"; ma=86400" },
            { "name": "alt-svc", "value": "h3-29=\":443\"; ma=86400" },
            { "name": "X-Firefox-Spdy", "value": "h2" }
          ],
          "content": {
            "size": 266,
            "mimeType": "application/json",
            "compression": 80,
            "text": "{\"results\":{\"sunrise\":\"11:38:10 AM\",\"sunset\":\"11:04:37 PM\",\"first_light\":\"10:09:42 AM\",\"last_light\":\"12:33:05 AM\",\"dawn\":\"11:11:29 AM\",\"dusk\":\"11:31:19 PM\",\"solar_noon\":\"5:21:24 PM\",\"golden_hour\":\"10:28:53 PM\",\"day_length\":\"11:26:27\",\"timezone\":\"UTC\"},\"status\":\"OK\"}"
          },
          "headersSize": 1114,
          "bodySize": 186,
          "redirectURL": "",
          "_transferSize": 1214
        },
        "cache": {},
        "timings": { "dns": 0.207, "connect": 76.133, "ssl": 46.679, "send": 0, "wait": 59.464, "receive": 0.068 },
        "pageref": "page@e9d0544dbdcdaf2e7ef5e43c0e00e4eb",
        "serverIPAddress": "188.114.97.13",
        "_serverPort": 443,
        "_securityDetails": {
          "protocol": "TLS 1.3",
          "subjectName": "*.sunrisesunset.io",
          "issuer": "GTS CA 1P5",
          "validFrom": 1677028638,
          "validTo": 1684804637
        }
      }
    ]
  }
}

It stores all pieces of information about the requests and responses. If the test is performing 100 different API calls then 100 requests will be recorded into HAR file.

Replaying requests

If we want to use the recorded requests next time the test will be run, all we have to do is add the following line:

await page.routeFromHAR('requests.har');

This line of code will examine every possible API call and if the params match one of the requests recorded into requests.har file, the request to real API will be replaced by our request from har file.

Full code:

import { test, expect } from '@playwright/test';

test('checks if API returns result', async ({page, browser }) => {
    await page.goto('http://localhost:5173/');

    await page.routeFromHAR('requests.har');

    await expect(page.getByTestId('sun-hours')).toHaveText('Sun hours');

    await page.getByTestId('search-btn').click();
    await expect(page.getByTestId('results-header')).toHaveText('Results:');
});

Summary

This technique is a more flexible way of standard mocking, especially if we want to test many responses based on different parameters. Playwright provides a convenient way to record and replay requests using HAR files. For more detailed information, you can check the dedicated docs section here.

© Copyright 2024 Michał Kuncio