Commissioning test automation means commissioning a second codebase that you will own and maintain for as long as the first one. Most disappointments here are not about the tests being wrong; they are about nobody agreeing in advance who keeps them working.
A big suite is not the goal
Vendors are incentivised to deliver test count because it is visible and billable. What you want is a small, fast, reliable suite covering the journeys that would genuinely hurt if they broke, and the discipline to add carefully. A thousand slow tests that fail intermittently is worse than fifty that never lie, because a suite people stop believing has stopped working while continuing to cost money.
Flakiness needs to be a contract term
A test that sometimes fails for no reason teaches everyone to rerun rather than investigate, and the real failure then goes unnoticed. Agree an acceptable flakiness rate, who fixes an intermittent test and within what period, and require that a test which cannot be made reliable is deleted rather than retried. An unreliable test has negative value and should be treated that way explicitly.
Runtime is a design constraint
If the suite takes an hour, developers stop waiting for it and it stops preventing defects. Set a maximum acceptable runtime as a requirement at the start and make the vendor design within it: parallelisation, sensible layering with more fast tests and fewer slow end to end ones, and honest decisions about what does not need automating. A runtime target shapes the architecture, which is why it must be a requirement rather than a hope.
Ownership, location and handover
Tests belong in your repository beside the application code, running in your pipeline, not in a vendor platform you lose at the end of the contract. Require documentation of how to run the suite locally, how test data is created and how to add a test, written for someone who has never met the vendor. Automation you cannot run or extend yourself is a subscription rather than an asset.
Questions people ask about qa automation company
How should we judge a QA automation proposal?
On suite health rather than size: a maximum runtime, an agreed flakiness threshold with named ownership, tests in your repository and pipeline, and documentation written for someone who has never met the vendor.
What do we do about flaky tests?
Make them a contract term with an acceptable rate, named ownership and a fix period, and require that a test which cannot be made reliable is deleted. Intermittent failures teach people to ignore real ones.
Where should the tests live?
In your repository beside the application code, running in your pipeline. Tests held in a vendor platform are a subscription you lose at exactly the moment the relationship ends.