Integration testing with Playwright and Testcontainers in ASP.NET
Testing plays a crucial role in software development and we strive to test as much as possible using isolated unittests. However, there are times when these unittests fail to capture issues that arise when the entire application is tested. Such issues may include discrepancies between front-end and back-end implementation of APIs, or differences in behavior between real and mocked databases. These problems can be easily identified by running a series of scenarios that test the entire application, including the front-end, back-end, and database. Many developers resort to manual testing to achieve this, but automated testing offers greater repeatability and the ability to integrate checks into the development pipeline.
This article demonstrates how to easily automate the testing process using Playwright .NET and Testcontainers for .NET. By doing so, developers can eliminate the need for manual testing and streamline the testing process without relying on deployments.
The application under test
For the purpose of this article I reused a blazor example contact application and replaced SQLite with SQL Server.
The result is a straight forward setup. A SQL Server database, a ASP.NET application and a Blazor frontend.
Use a real database
While local installation of a database is always an option there are better ways. Using the Testcontainers for .NET project you can easily setup a database from scratch for each test or testrun. You don’t need to have a custom script or a preinstalled database somewwhere when you have Docker.
Let’s start with the assuption that a CustomWebApplicationFactory is is present.
In our CustomWebApplicationFactory we will add the definition of a real database.
internal class CustomWebApplicationFactory : WebApplicationFactory<Program>
{
public readonly MsSqlContainer DatabaseContainer;
public FullIntegrationWebApplicationFactory()
{
try
{
DatabaseContainer = new MsSqlBuilder()
.Build();
}
catch (ArgumentException ae)
{
throw new XunitException($"Is docker installed and running? {ae.Message}.");
}
}
protected override IHost CreateHost(IHostBuilder builder)
{
....
}
}
This code will start a docker container with MS SQL Server 2019 (the default at the tie of writing) container. The connectionstring is exposed the via DatabaseContainer.GetConnectionString(). That allows you to do thing like overriding DbContext as part of ConfigureTestServices as shown below.
services.AddScoped((sp) =>
{
var options = new DbContextOptionsBuilder<DatabaseContext>()
.UseSqlServer(DatabaseContainer.ConnectionString + ";TrustServerCertificate=True")
.Options;
var db = new DbContext(options);
db.Database.Migrate();
return db;
});
});
Then to actually start the container you can use a xUnit class fixture:
public class TestFixture : IAsyncLifetime
{
private readonly CustomWebApplicationFactory _factory;
public TestFixture()
{
_factory = new FullIntegrationWebApplicationFactory();
}
public async Task InitializeAsync()
{
await _factory.DatabaseContainer.StartAsync();
}
public async Task DisposeAsync()
{
await _factory.DatabaseContainer.StopAsync();
}
Full example on GitHub: https://github.com/netwatwezoeken/full-integration-testing
Automate browser interaction
To automate a test we must be able to do whatever the user is doing on the browser. Playwright works very well in this scenario because it is fully integratable in you IDE workflow (Visual Studio or Rider). Tests can be written in C# and run through xUnit, NUnit or MSTest.
Playwright can be added through a xUnit ClassFixture. Let’s add it to TestFixture from the previous part.
public class TestFixture : IAsyncLifetime
{
private readonly FullIntegrationWebApplicationFactory _factory;
private IPlaywright _playwright;
private IBrowser _browser;
private IBrowserContext _context;
public TestFixture()
{
_factory = new FullIntegrationWebApplicationFactory();
}
public async Task InitializeAsync()
{
await _factory.DatabaseContainer.StartAsync();
_playwright = await Playwright.CreateAsync();
_browser = await _playwright.Chromium.LaunchAsync();
_context = await _browser.NewContextAsync();
Page = await _context.NewPageAsync();
_factory.CreateClient();
}
public IPage Page { get; set; }
public async Task DisposeAsync()
{
await _factory.DatabaseContainer.StopAsync();
await _context.DisposeAsync();
_playwright.Dispose();
}
}
By default the WebApplicationFactory does not listen on a port by default. But we want to Playright to start a real browser that will need to connect to something. To do so you must override CreateHost and add a line to within ConfigureTestServices:
protected override IHost CreateHost(IHostBuilder builder)
{
// Create a plain host that we can return.
var dummyHost = builder.Build();
// Configure and start the actual host.
builder.ConfigureWebHost(webHostBuilder => webHostBuilder.UseKestrel());
var host = builder.Build();
host.Start();
return dummyHost;
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseUrls("https://localhost:7048");
....
}
With that fixture and modification to the CustomWebApplicationFactory in place we can create a test.
public class Contacts : IClassFixture<TestFixture>
{
private readonly IPage _page;
public EditContact(TestFixture fixture)
{
_page = fixture.Page;
}
[Fact]
public async Task RemoveContact()
{
// Go to the page and wait for the loading to complete
await _page.GotoAsync("https://localhost:7048/");
await _page.WaitForLoadStateAsync(LoadState.NetworkIdle);
await _page.Locator(".contact-detail")
.IsVisibleAsync();
// check that there's one contact
Assert.Equal(1, await _page.Locator(".contact-detail")
.CountAsync());
// Delete the contact
await _page.Locator(".clickable").ClickAsync();
await _page.GetByRole(
AriaRole.Button,
new() { NameString = "Confirm" }).ClickAsync();
// Check if the contact is deleted
await _page.GetByText("Page 1 of 0: showing 0 of 0 items. Previous Next")
.ClickAsync();
Assert.Equal(0, await _page.Locator(".contact-detail").CountAsync());
}
}
Full example on GitHub: https://github.com/netwatwezoeken/full-integration-testing
Please note that you need Powershell 7 to be installed and follow the intruction given by Playwright during the first run. Yes, Powershel 7, also when on MacOS or Linux
With this in place you can use the xUnit ClassFixtures to either run test in isolation (slower) or start the DB and browser once or a couple of times (faster). Thanks to the usage of the xUnit test runner tests can now be easily run from within your IDE.
Pipeline integration
Having these test in the IDE is great. Nice and quick feedback before check-in of code. A build pipeline can be fairly simple. But there are some caveats.
The first thing is .NET 7. At the time of writing .NET 7 is out and suported under standard support (STS). The official playwright docker image only supports .NET 6 (LTS). Either nwwz/playwright-dotnet Or build your own image. Added .NET 7 to the official playwright image is fairly easy as can be seen in this Dockerfile
Next up are the certificates for https. Certificates are not present by default and also not shared with the browser.
Certificates can be generated using the dev-certs command:
dotnet dev-certs https --clean
dotnet dev-certs https --trust
Then in the TestsFixture we can tell the browser to ignore https errors:
_context = await _browser.NewContextAsync(new BrowserNewContextOptions()
{
IgnoreHTTPSErrors = true
});
And lastly the Ryuk container. This is a helper container started by testcontainers to cleanup container when not in use anymore. Typically a cloud hosted environment already lceans up after and does not allow any containers to be run in priviledged mode. Simply disable the Ryuk container by setting TESTCONTAINERS_RYUK_DISABLED to true.
All of the above in a Bitbucket pipeline would look somthing like this:
- step:
name: End to end testing
image: nwwz/playwright-dotnet:v1.30.0-focal
script:
- dotnet dev-certs https --clean
- dotnet dev-certs https --trust
- export TESTCONTAINERS_RYUK_DISABLED=true
- dotnet restore
- cd tests/End2EndTests
- dotnet test
artifacts:
- tests/End2EndTests/traces/**
services:
- docker
Finetuning and caveats
Other databases or SQL Sever versions
The Testcontainers projects allows for a lot more than just MS SQL Server containers. The project comes with some useful builders to setup a variety of services you might need. And then there is always the option to specify your own specific image, ports, environment variables, etc.
Use the following if you specifically want to specify the version of SQL Server:
DatabaseContainer = new MsSqlBuilder()
.WithImage("mcr.microsoft.com/mssql/server:2022-latest")
.Build();
Turn off Slowmo and set Headless to true
The example project has Headless disabled and slowmo set to 2000ms per action. This is great to see what is going on. On enviroments like the Bitbucket cloud hosted runners this will fail due to the lack a of screen. Be sure to either comment or completely remove these 2 lines:
_browser = await _playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
{
//Headless = false,
//SlowMo = 2000
});
More dependencies than just the database
Of course a real application migth have more that just it’s own database to talk to. Keep in mind that Testcontainers for .NET has helpers for many services. Redis, Minio, Mongo, RabbitMq and even Kafka. Even without a helper available it is really easy to spin up any other docker image to be used in testing.
Conclusion
Playwright .NET and Testcontainers for .NET within a .NET solution provide a nice and easy way have E2E integration tests at both IDE an pipeline level.
I help organisations and developers to build better software faster. Feel free to reach out through the my socials if you want to know more, need help, have questions or remarks.