There are many important aspects to consider in any given penetration test. I have talked at length in other blog posts about many of these considerations. There is one important aspect I have not written much about. It is critically important to determine the amount of foreknowledge that the tester should get. This aspect has a plethora of names but is almost always referred to with the “box” descriptor. In college, I was taught white box, gray box, and black box as the three levels of disclosure related to a penetration test. Many, including CynergisTek, use the term “crystal” in place of “white”. Really, the names are just descriptors – the concept remains the same and that is what’s most crucial.
Black Box Testing: A Dark View of the Engagement
Let us begin with, in my experience, the least common type of assessment, known as the black box penetration test. During a black box penetration test, the penetration tester has only been given the bare minimum information on the in-scope systems. More commonly, these limitations are used during large internal penetration tests. If the black box method is used at all it generally leads to longer engagements, since this method requires the tester to spend a significant amount of time on the initial discovery phase of the test. As is clearly defined in all of the current industry accepted penetration testing frameworks, the discovery phase can easily take half or more of a penetration test’s allotted time.
This approach adds another wrench to the works: the tester has to be “slow and quiet” to not be discovered by the defenses and defenders. While it seems logical that an offensive assessment may want to remain under the radar, this is not the most effective approach. Edge protection and security monitoring services force the tester to perform the bare minimum of testing to find an entry point, often causing the test to miss a plethora of other weaknesses it could have uncovered.
Gray Box Testing: Partly Cloudy
The gray box penetration testing method is somewhat more common. In my experience, almost every engagement I have ever been involved in that began as a black box test has turned into, at a minimum, a gray box level of foreknowledge. Gray box tests are a mixture of the zero-knowledge black box and the full-disclosure crystal box. The gray box method is an attempt to get the benefits of both the black and crystal box methods in one assessment. While gray box assessments are more effective and less costly than the black box assessment above, they are also less effective and more costly than the following crystal box method.
Crystal Box Testing: I Can See Clearly Now
Crystal box tests are by far the most common type of penetration test. Why? Because penetration tests are not free, so the tester has limited time. A real-world attack against an organization is a long and drawn-out affair, as malicious attackers do not have time limits, whether they are foreign criminals, nation-state sponsored attackers, or even the stereotype of a lonely hacker living in their mom’s basement.
These hackers will be slow, careful, and will often sit inside of a compromised network for extended periods of time building up their knowledge of its inner workings. It is often said that by the time a malicious attacker is ready to begin the exfiltration of the data in the target network, they often gain a better understanding of the network and its inner workings than the owners could ever hope to have.
The crystal box method gives the penetration testers the ability to skip past the extended information gathering phase and move right into finding vulnerabilities and avenues of attack. This is the great advantage that we ethical hackers have over the bad guys, and this collaboration is a crucial piece that makes penetration testing so effective.
Another aspect of the crystal box method that ensures the assessor can best use the time at their disposal is whitelisting. When a malicious attacker is after an organization they will spend time slowly and quietly gaining access to the systems. Once access has been gained, the bad actor will lie in wait until they have an opportunity to attack and escape or attack without being noticed. When a penetration test is performed, it is important for the system owners to whitelist the penetration tester’s IP so that they do not get blocked during their limited testing window.
Overall the crystal box is the most effective as it gives the tester the most effective access possible. Black and gray box approaches have their place, be it a small one. However, in general it is highly recommended that for penetration testing to effectively find the vulnerabilities present, it should be performed in a manner that allows the tester to access the systems unfettered by DMZ (demilitarized zone) and other perimeter protections.