What really matters when it comes to pentesting deliverables

With so many penetration testing providers in the market, evaluating which one is right for your organization can be difficult.

Beyond getting to know the people who will do the test and understanding how much time they will spend on it, one of the best ways to see who stands out from the crowd is by looking at what types of deliverables the firm provides.

There are right and wrong ways to write a penetration test report, so asking for a sample in advance can help you weed out those that don’t provide the information that really matters.

To help you identify what a good report looks like (and just as importantly, what a bad report looks like), we break down exactly what pentesting deliverables should and should not include.

What pentesting deliverables should include

Before you hire a pentesting firm, verify that their reports include the following:

1. Detailed steps for remediation

One of the most important things a good pentest report should include is an explanation of how to fix the issues that were found. If you’re only provided with descriptions of the vulnerabilities, it’s not going to help you dive in and remediate them.

In the instance where the pentester does not know the specific implementation details for your environment, they should instead provide best practices for fixing the vulnerability. For example, with a server-side request forgery (SSRF) vulnerability, the pentester may suggest that you implement an allowlist that only permits verified servers to be requested.

In addition, a good pentester will understand that you as the client may not have the ability to immediately make changes to code. When that is the case, they should suggest workarounds or methods to restrict access to vulnerable functionality until there is a permanent solution.

2. Clear explanations of the practical risks associated with issues

Vulnerability severity levels often lack the context that you need to understand the true risk to your organization. Some vulnerabilities that are scored with a high severity rating may actually have little to no real-world impact given the application’s environment, configuration, or access required. Conversely, some vulnerabilities with a low or medium severity rating may actually have more severe consequences than described.

A good pentest report will explain why the finding was scored the way it was while also describing the bottom line risk to your organization. For example, if a cross-site request forgery (CSRF) vulnerability is found with no SameSite attribute set, the report should describe how this affects different web browsers and should explain the practicality of a successful exploit.

This additional context can help you prioritize which issues to fix first, especially when many issues are found.

3. Complete inventory of in-scope assets

Your pentest report should include a detailed inventory of the in-scope assets so that you can see the test has been performed thoroughly and accurately, and so that you can understand the state of the network at the time of testing.

The inventory should list hosts, IP addresses, and geolocation information. For assets hosted on a content delivery network, additional information that is pertinent to the test should be provided as well. For example, if a domain resolves to a Cloudflare IP address, it’s probable that some form of web application firewall (WAF) is in use.

The inventory should include any extra information that’s specific to the asset and vulnerability. For instance, if the finding involves a web server, the report should mention the port, path, and specific parameters it uses.

4. Links to references

For some vulnerabilities, you might want to go beyond the information that’s provided in your pentest report and dig deeper into the technical details and remediation recommendations. A good pentest report will include reference links to trusted third-party sources like OWASP or NIST so that you can investigate further.

5. Images and command line output

Images can add additional context to the findings in your pentest report and help you follow along with what is being reported. They’re also useful in cases when an issue is not reproducible.

In some cases, issues can be sufficiently explained without images. For instance, if the finding is verifiable through a command line tool, that should be included along with the output.

But in other instances, it's useful to have a visual of what’s happening in a browser or graphical testing tool. This helps to quickly draw attention to what’s being described.

What pentesting deliverables should NOT include

If you receive a pentest report that contains any of the following, consider working with a different pentesting firm.

1. Unexplained technical jargon

As a rule of thumb, a good pentest report should be written with the assumption that it will be read by someone who is not familiar with information security. As such, it should not contain technical jargon without a proper explanation in plain language. If you can’t understand the findings written about in your report, you won’t be able to understand the risks your organization is facing.

2. Output from automated scanners

If your pentest report includes copied and pasted output from automated scanners, it’s a big red flag. This could be a signal that a manual penetration test was not actually performed.

Pentesters can use automated scanners to complement a manual penetration test, but it should not replace one.

3. Inconsistencies between the summary and findings

You want to hire pentesters that pay close attention to detail. If there are inconsistencies between the report summary and the findings, that’s another red flag and could be a sign that they’ve copied and pasted from a previous report. In addition, if the summary orders the findings by severity/CVSS score, the findings section should do so as well.

Wrapping up

While it can sometimes take trial and error to find the right pentesting provider that fits your needs, taking the time to review sample reports upfront can be beneficial in the long run.

To complement our attack surface management solution, Halo Security offers penetration testing services that strictly follow these reporting guidelines. If you’re interested in learning more about our security testing services, book a meeting with our team and we will be happy to share a sample report.

Editor's note (August 2022): This article was originally posted on the TrustedSite blog in April 2022. It has been updated for the Halo Security blog.