End-to-end testing an Astro project with Playwright

Setting up the project

In an existing Astro project, setting up the necessary configuration for Playwright end-to-end tests can be done with a single command. The command guides you through a set of questions to decide what you want to cover with your tests and ensures you will have the required settings properly configured.

pnpm create playwright

Afterwards, you can extend your package manager scripts to include a handy trigger for your end-to-end tests. This way, you can run your tests using pnpm test:e2e.

{
  "scripts": {
    "test:e2e": "pnpm exec playwright test"
  }
}

Configuring the testing environment

You can set some options to optimise your end-to-end testing setup for your local environment and a CI environment.

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './e2e',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: 'html',
  use: {
    baseURL: 'http://localhost:4321',
    trace: 'on-first-retry',
  },
  projects: [
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome'],
      },
    },
    {
      name: 'firefox',
      use: {
        ...devices['Desktop Firefox'],
      },
    },
    {
      name: 'webkit',
      use: {
        ...devices['Desktop Safari'],
      },
    },
  ],
  webServer: {
    command: 'pnpm preview',
    url: 'http://localhost:4321',
    timeout: 120 * 1000,
    reuseExistingServer: !process.env.CI,
  },
});

This configuration ensures some aspects:

  • In your local environment, the tests attempt to reuse an existing dev server (e.g. from pnpm dev).
  • CI pipelines retry failing tests and use a single worker to avoid concurrency issues.
  • You can configure browsers to test. In my configuration, I have Chrome, Firefox, and Safari.

If you want to have more specific testing environments, you can configure the browsers even further.

{
  name: 'chromium',
  use: {
    ...devices['Desktop Chrome'],
    // Simulate dark mode
    colorScheme: 'dark',
    // Simulate locale
    locale: 'de-CH',
    // Simulate mobile or desktop viewports
    viewport: { width: 480, height: 720 },
  },
}

Writing your tests

You can write your tests using familiar helpers like test, test.describe, and exepct from playwright.

import { expect, test } from '@playwright/test';

test('has a link to the projects overview', async ({ page }) => {
  await page.goto('/');

  const link = page.getByRole('link', { name: 'Discover more projects' });

  await expect(link).toHaveAttribute('href', '/projects/');

  const [response] = await Promise.all([
    page.waitForResponse('/projects/'),
    link.click(),
  ]);

  expect(response?.status()).not.toBe(404);
});

Testing content collections

For your project it might make sense to run tests for your whole content collections. In my case, I have a handful of articles and project case studies, so running end-to-end tests for all of them does not take a substantial amount of time.

const projectsDir = fileURLToPath(
  new URL('../src/content/projects', import.meta.url),
);

// My projects are structured like /src/content/projects/[slug]/index.md
const slugs = readdirSync(projectsDir, { withFileTypes: true })
  .filter((entry) => entry.isDirectory())
  .map((entry) => entry.name);

for (const slug of slugs) {
  test(`"${slug}" has a title, a teaser, a hero image, and content`, async ({
    page,
  }) => {
    await page.goto(`/projects/${slug}/`);

    await expect(page.getByRole('heading', { level: 1 })).not.toBeEmpty();
    await expect(page.locator('header p').first()).not.toBeEmpty();
    await expect(page.locator('.hero-image img')).toBeVisible();
    await expect(page.locator('.content p').first()).toBeVisible();
  });
}

I also like to ensure that content collections handle 404 errors properly.

test('returns a 404 for an invalid project', async ({ page }) => {
  const response = await page.goto('/projects/this-project-does-not-exist/');

  expect(response?.status()).toBe(404);
});

Testing accessibility

You can also automate accessibility checks using playwright. These tests check for common accessibility violations and report any findings.

pnpm add --save-dev @axe-core/playwright

For my tests, I want to cover WCAG standards from levels A and AA in the iterations 2, 2.1, and 2.2. I want to ensure my main pages do not have any accessibility violations. This can also be extended to cover all content collections. For the accessibility tests, I check both the light and dark themes to verify that both themes do not have any contrast violations.

import AxeBuilder from '@axe-core/playwright';
import test, { expect } from '@playwright/test';

const items = [
  { name: 'Home', href: '/' },
  { name: 'About', href: '/about/' },
  { name: 'Articles', href: '/articles/' },
  { name: 'Projects', href: '/projects/' },
];

for (const item of items) {
  test(`"${item.name}" should not have any accessibility issues`, async ({
    page,
  }) => {
    const axeBuilder = new AxeBuilder({ page }).withTags([
      'wcag2a',
      'wcag2aa',
      'wcag21a',
      'wcag21aa',
      'wcag22a',
      'wcag22aa',
    ]);

    await page.emulateMedia({ colorScheme: 'light' });
    await page.goto(item.href, { waitUntil: 'networkidle' });

    expect((await axeBuilder.analyze()).violations).toEqual([]);

    await page.emulateMedia({ colorScheme: 'dark' });
    await page.goto(item.href, { waitUntil: 'networkidle' });

    expect((await axeBuilder.analyze()).violations).toEqual([]);
  });
}

Automating tests on GitHub Actions

In my case, I want to run my end-to-end tests as part of the pull request pipeline. Like this, I can enforce that tests are running without errors before any pull request is merged. This could also be changed to a periodic test run or purely manual execution.

name: Run end-to-end tests

# Trigger tests automatically when creating a pull request to the main branch
on:
  pull_request:
    branches: [main]

# Cancel existing runs when a new commit is pushed to a pull request branch
concurrency:
  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
  cancel-in-progress: true

permissions:
  contents: read

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      # Set up the pipeline environment
      - name: Checkout repository
        uses: actions/checkout@v7
      - name: Setup pnpm
        uses: pnpm/setup@v2
        with:
          cache: true
      # Set up playwright and all necessary browsers for testing
      - name: Install Playwright browsers
        run: pnpm exec playwright install --with-deps
      # The project needs to be built explicitly before serving a preview of the build
      - name: Build site
        run: pnpm build
      # Run end-to-end tests and upload the test report, accessible through the GitHub UI
      - name: Run Playwright tests
        run: pnpm test:e2e
      - uses: actions/upload-artifact@v7
        if: ${{ !cancelled() }}
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30

For comments and questions, contact me through e-mail.

Read more articles