Intercepting & Modifying Network Requests in Web Automation frameworks like Selenium and Playwright

Requestly is a powerful browser extension that allows developers and QAs to intercept and modify HTTP requests. It offers features such as request and response body overrides, URL redirects, HTTP header modifications, and the ability to inject custom CSS and JavaScript. However, integrating Requestly with web automation frameworks like Selenium and Playwright isn’t straightforward.

In this guide, we will show you how you can use Requestly in an e2e web testing framework. This guide covers integration with Playwright but It can be easily adapted to other web automation frameworks in a similar manner.

Steps to use Requestly in an automation environment

1. Configure Rules in a Team Workspace and Get API Key

Start by creating the rules you want to use in a new team workspace. Once the rules are set up, request an API key for the workspace by filling out this form. The rules in this workspace will be automatically fetched and applied during the end-to-end (E2E) testing in the browser instance.

2. Install Requestly Extension in automated env

To use Requestly in an automated environment, install the Requestly extension in the browser instance. Some frameworks require the extension's CRX file for installation. You can download the CRX file for any Chrome extension following this guide.

To learn how to install requestly in various testing frameworks, refer to the following resources:

• Selenium

• Playwright

• Cypress

• Puppeteer

3. Close the Requestly Dashboard Page

Requestly automatically opens the dashboard upon installation but you can safely close it.

4. Open the Selenium Importer Page

Use your automation framework to open the Requestly e2e testing page:

Copyhttps://app.requestly.io/selenium-importer?apiKey=<API_KEY>

This URL uses an API key to fetch rules from a workspace.

5. Click "Load Requestly Rules"

After loading the page, you can automate or click the "Load Requestly Rules" button manually to start importing rules.

6. Listen for the Completion Message

Once the import process completes, the page posts a message:

Copywindow.postMessage({ type: "RQ_IMPORT", message: "ALL_STEPS_DONE" }, "*");

You can listen for this event in your setup to confirm that rules have been successfully imported.

7. Ready to Use

At this point, Requestly rules are imported and will be applied to the network requests triggered further in your browser instance.

Example Source Code with Playwright

Here’s an example source code using Playwright, you can automate Requestly rule imports, first extract the crx file in a folder and run the following script:

Copyconst { chromium } = require('playwright');

(async () => {
    const extensionPath = './extensions/requestly'; // Change this to your extension path
    const userDataDir = './user-data'; // Used for persistence

    // Launch browser with extension
    const browser = await chromium.launchPersistentContext(userDataDir, {
        headless: false, // Extensions only work in headed mode
        args: [
            `--disable-extensions-except=${extensionPath}`,
            `--load-extension=${extensionPath}`
        ]
    });

    // Open a new page
    const page = await browser.newPage();

    // Navigate to the Selenium Importer page
    await page.goto('<https://app.requestly.io/selenium-importer?apiKey=><API_KEY>');

    console.log('🌍 Page Loaded!');

    // Click the button 
    await page.waitForSelector('button'); 
    await page.click('button');

    console.log('🖱️ Button Clicked! Waiting for postMessage...');

    // Expose a function to capture messages from window.postMessage
    await page.exposeFunction('handlePostMessage', (eventData) => {
        console.log('Received message:', eventData);
        
        if (eventData.type === "RQ_IMPORT" && eventData.message === "ALL_STEPS_DONE") {
            console.log("✅ Import process completed!");
        }
    });

    // Listen for messages from window.postMessage
    await page.evaluate(() => {
        window.addEventListener("message", (event) => {
            if (event.data && event.data.type === "RQ_IMPORT") {
                window.handlePostMessage(event.data);
            }
        });
    });

    // Keep the browser open for debugging
    await new Promise(() => {});

})();

Using API Keys for Different Workspaces

The Secret API Key is tied to a specific workspace and automatically fetches all rules associated with that workspace. If you need to import rules from a different workspace, you must generate a new Secret API Key for that workspace by filling out a new request form. Once obtained, update the Selenium Importer URL accordingly.