FFIEC Information Security Booklet

Penetration Test

Frequency of penetration testing

The following is an excerpt about penetration testing from the FFIEC information Security Booklet.  

INDEPENDENT DIAGNOSTIC TESTS

Independent diagnostic tests include penetration tests, audits, and assessments. Independence provides credibility to the test results.  To be considered independent, testing personnel should not be responsible for the design, installation, maintenance, and operation of the tested system, as well as the policies and procedures that guide its operation.  The reports generated from the tests should be prepared by individuals who also are independent of the design, installation, maintenance, and operation of the tested system.

Penetration tests, audits, and assessments can use the same set of tools in their methodologies.  The nature of the tests, however, is decidedly different.  Additionally, the definitions of penetration test and assessment, in particular, are not universally held and have changed over time.

Penetration Tests. A penetration test subjects a system to the real-world attacks selected and conducted by the testing personnel. The benefit of a penetration test is to identify the extent to which a system can be compromised before the attack is identified and assess the response mechanism's effectiveness. Penetration tests generally are not a comprehensive test of the system's security and should be combined with other independent diagnostic tests to validate the effectiveness of the security process.

Audits. Auditing compares current practices against a set of standards. Industry groups or institution management may create those standards. Institution management is responsible for demonstrating that the standards they adopt are appropriate for their institution.

Assessments. An assessment is a study to locate security vulnerabilities and identify corrective actions. An assessment differs from an audit by not having a set of standards to test against.  It differs from a penetration test by providing the tester with full access to the systems being tested.  Assessments may be focused on the security process or the information system.  They may also focus on different aspects of the information system, such as one or more hosts or networks.

KEY FACTORS

Management is responsible for considering the following key factors in developing and implementing independent diagnostic tests:

Personnel.  Technical testing is frequently only as good as the personnel performing and supervising the test.  Management is responsible for reviewing the qualifications of the testing personnel to satisfy themselves that the capabilities of the testing personnel are adequate to support the test objectives.

Scope.  The tests and methods utilized should be sufficient to validate the effectiveness of the security process in identifying and appropriately controlling security risks.

Notifications.  Management is responsible for considering whom to inform within the institution about the timing and nature of the tests.  The need for protection of institution systems and the potential for disruptive false alarms must be balanced against the need to test personnel reactions to unexpected activities.

Controls Over Testing.  Certain testing can adversely affect data integrity, confidentiality, and availability. Management is expected to limit those risks by appropriately crafting test protocols.  Examples of issues to address include the specific systems to be tested, threats to be simulated, testing times, the extent of security compromise allowed, situations in which testing will be suspended, and the logging of test activity.  Management is responsible for exercising oversight commensurate with the risk posed by the testing.

Frequency.  The frequency of testing should be determined by the institution's risk assessment. High-risk systems should be subject to an independent diagnostic test at least once a year.  Additionally, firewall policies and other policies addressing access control between the financial institution's network and other networks should be audited and verified at least quarterly.  Factors that may increase the frequency of testing include the extent of changes to network configuration, significant changes in potential attacker profiles and techniques, and the results of other testing.

Proxy Testing.  Independent diagnostic testing of a proxy system is generally not effective in validating the effectiveness of a security process.  Proxy testing, by its nature, does not test the operational system's policies and procedures, or its integration with other systems. It also does not test the reaction of personnel to unusual events.  Proxy testing may be the best choice, however, when management is unable to test the operational system without creating excessive risk.

FFIEC information Security Booklet