In the last 12 months, Penetration Testing as a Service has exploded in popularity. Is that because it’s a better business model for penetration testing vendors? Or because there’s more security and business value in the practice for companies? That’s what I set out to explore in this article.
Below, I’ve outlined what exactly PTaaS is, how it is different from traditional pentesting, the key benefits, the costs, and ultimately, what sort of companies PTaaS is right for.
(NOTE: If you’re considering a pentest (as a service or otherwise) our free tool below matches your firm with a top-rated vendor that suits your needs and budget)
What Exactly is Penetration Testing as a Service?
To understand exactly what PTaaS is, I think it’s worth looking at it in the context of three similar security services. Those services are as follows:
- Vulnerability Scanning – This is typically contracted directly with a software vendor but can also be completed by a consulting company. This service is highly automated with the goal of enumerating as many easily identifiable vulnerabilities as possible. Scanners are available for network, application, cloud, and container technologies.
- Penetration Testing – This is typically a service but also can be completed by an internal FTE with proper background. Usually, it is a highly manual process that relies on a vulnerability scanner up front and is followed by a time-boxed period of manual testing to identify additional vulnerabilities or configuration issues.
- Bug Bounty Program – This can be run through a third party managing the program or through a responsible disclosure program by the company. This service has the goal of allowing ethical hackers to probe, identify, and validate vulnerabilities at any time without having a direct contract with the ethical hacker. These programs pay a bounty based on difficulty to the individual that has disclosed the vulnerability to the company.
PTaaS is not a direct like-for-like replacement for vulnerability scanning, penetration testing, or bug bounty programs but a unique approach that combines elements of all three of these.
A typical PtaaS arrangement combines heavily automated testing with manual validation testing on a recurring basis. Automated testing will be utilized on a recurring basis to identify any changes in infrastructure, application, or services and to validate the remediation of previous vulnerabilities. When changes are identified by automated testing, manual testing, and validation may be utilized to provide additional value and to remove false positives.
PTaaS is usually contracted for a yearly term, like engaging a software vendor. This service is not meant to be a short-term, single test but is meant to be used for ongoing management of the in-scope systems. It is a service built to provide ongoing feedback and validation that combines both automated and manual testing in a blended approach.
How is it Different to Regular Pentesting?
Compared to a vulnerability scanning solution, penetration testing as a service will add validation and false positive removal from the scans but may be slower to provide reports. Some services can provide a report or output of findings daily, but this may come with additional costs. Unlike running a vulnerability scanning tool internally, where you can run ad-hoc tests as needed, PTaaS will often require a scheduled assessment. Additional emphasis will be placed on scheduled assessments or tests by the vendor to help manage costs. While it may not be a direct replacement for vulnerability scanning, the service can still displace internal vulnerability scanning, especially for a company that does not have the internal expertise.
Similarly, PtaaS is not a pure penetration test. It should not be assumed that this service can fully replace a penetration test, as a dedicated, highly manual penetration test will most likely identify more complex vulnerabilities, simply due to the time devoted to manual testing. PTaaS typically devotes small sections of time for manual testing (1-2 days per test), with heavy reliance on automated testing; whereas manual penetration testing is a time-boxed approach with most of the time dedicated to manual testing. A traditional penetration test can take a long lead time to identify vulnerabilities, as most companies do not contract for these services on more than a yearly basis, due to cost.
PtaaS may be less thorough than a standard penetration test as it devotes less time to manual testing, but purports to overcome this limitation by testing more frequently.
A PTaaS contract is a great way to close the gap between changes, identification of new vulnerabilities, and validation of remediation, as testing is conducted throughout the course of the contract. This allows the service to align with Agile/DevOps companies that are making frequent changes to their services.
What are the Benefits of PtaaS?
PTaaS providers will have invested in modern web applications to provide results and automated integrations to streamline the management of findings. These applications often have integrations into collaboration tools, ticket management services, and other internal data management platforms to allow for automation. It is not uncommon for these tools to allow for the opening of Jira stories or GitHub issues for a development team to prioritize work. As mentioned in the previous section, this allows for closing the lead time for remediation for organizations, due to the ability to integrate into the agile methodology.
Additionally, as it is contracted for a long-term ongoing engagement, PTaaS can also test changes as soon as they go into a testable environment. This can be used to identify any new vulnerabilities that may have been introduced, allowing the company to resolve those issues sooner. The other benefit is that remediation testing is built into the service and can be engaged as needed for no additional cost. Being able to validate the proper remediation before going to production and before closing an issue can really lower the cost of remediation.
What’s the Process?
Contract & Scoping: The first step in the process is to contract. As part of this step, this service will require the definition of the scope or services that are to be tested. Just like penetration testing or vulnerability scanning, there are PTaaS that are dedicated to networks or applications, so defining which type is needed will be the first step.
Loading Scoped Assets into the PTaaS application: Once the scope has been defined and contracted for, loading the assets into the application will be requested. This could take place during a kickoff or onboarding call, but in some cases, the customer will be able to do this on their own. This may include the IP address ranges, URLs for applications, and any credentials that are required for testing. This will provide the scanners and manual testers with the information they need to be able to conduct the testing.
Settings & Integrations: After the information has been loaded, most providers will supply settings and integrations that will need to be set up. Some of the settings for the service may be the frequency of scanning and manual testing, the permitted testing window, and appropriate points of contact. The integrations will depend on the provider’s application, but often they include the ability to automatically create a Jira story, GitLab/GitHub issue, or other notification to a Slack channel of findings. In some cases, the integrations with the ticket management solution will be bidirectional, allowing the provider to validate remediation once a ticket/issue has been closed on the customer side.
Test Schedule Begins: Beyond the setup and configuration of the service, testing will be conducted on the set schedule. Findings will be added to the site, ad-hoc reports can be generated as needed, and findings will be updated based on the previous test. If the customer wants to validate the remediation of a finding off-cycle, there will be a way to request validation on the specific vulnerability that was identified.
A Note on Reporting with PTaaS
For most PTaaS service providers, it is still possible to receive a traditional formal report at the end of the test. Before getting started, it is worth confirming with your PTaaS provider exactly what their formal reports will contain. Reporting is a big differentiator between pentest providers and PTaaS is no different. If for example, your company had really valued the remediation advice in past pentest reports, you’ll want to confirm that the proposed PTaaS provider’s reports are equivalent.
Where these platforms really differentiate from traditional pen testing services is that they often provide real-time findings, similar to scanning tools, allowing your organization to better prioritize findings. In fact, many of the platforms are now built with integrations into tools like GitHub, GitLab, Jira or other work management platforms. By allowing direct integration into a company’s work management platform, as findings are identified, they can be opened within the company’s work management platform to be prioritized. It’s important to note that this increases the speed of reporting, but not necessarily the depth.
Even though many of the providers offer continuous testing and live results, it is still possible to generate a detailed penetration testing report as needed. The report will be able to be requested or generated as a point-in-time status of the application or network. As just with other penetration testing service providers, the report will include details like scope, exclusions, executive summary, and detailed results.
How Much Does Penetration Testing as a Service Cost?
Penetration testing as a service can cost anywhere between $15k to well into the six figures annually depending on scope, complexity and contract length.
The two most common PtaaS services offered are network (infrastructure) and application services. The network pentest service will typically involve internal and/or external tests over a range of IP addresses provided. Application PTaaS will be based on the number of unique applications, or URI, that need to be tested.
For both services, providers will offer black-box and white-box testing. White-box testing will require credentials or appropriate user permissions to be provided; whereas black-box testing will be focused on what is accessible without credentials.
When it comes to impacts on pricing, both services will be based on the scope. For the network, it will be impacted by the total number of IP addresses or systems in scope. For applications, it will be a bit more complex, as it will be based on the number of user roles, pages with forms, and the number of APIs that are accessible for each individual application. These items relate to the time it takes for the automated scanner to run and how much time will be devoted to manual penetration testing.
Beyond the scoping details, the final component that will impact pricing is the contract term. In general, the default for this service is a year, but some providers may offer up a shorter period of time. Keep in mind, if your company is looking to contract for a shorter period than one year, you will pay a premium for it. Of course, there are options to go for longer-term contracts, which will help lower the overall cost of the service.
Just like penetration testing, PTaaS has a very broad range of costs.
In the current market, this service will typically be a bit more expensive than a single penetration test, but if you look at it through the lens of continuous testing over the course of a year, there is extreme value to be had for an organization.
Which Firms Is PtaaS Right For?
The short answer is any company that would like to do more than a single point-in-time penetration test and is planning for frequent changes over the course of a year should consider PTaaS. With that in mind, I see value coming for any company practicing Agile or DevOps in application development with frequent changes. Any company with a SaaS app, for example, would be a good candidate for PTaaS. The service is positioned well to allow for more in-depth testing of those changes and the generation of reports, trends, and other data points over the course of the year.
Beyond companies that are developing software, the other ideal companies for this service are those (such as small businesses) that may not have the internal security expertise to run vulnerability scanning tools. This service can provide a great blend between vulnerability scanning and penetration testing: allowing for better allocation of resources, reduction in false positives, and direct access to consultants to ask questions. For a company with limited or no internal security resources, this can provide the ability to gain access to expertise, while also further securing systems and assets.