Code review and application security testing

The fourth Domain 8 objective asks you to assess the effectiveness of software security. Effectiveness is assessed in two ways: by examining the software itself through review and testing, and by examining the process that produces it through audits, metrics and maturity assessments. This lesson covers the first. It explains the main review and testing techniques, where each fits in the life cycle, what each finds and misses, and how a manager turns a stream of tool findings into risk reduction rather than noise.

CISSP-115 Lesson 8 introduced security testing in the software life cycle as part of an organization-wide assessment and testing program, and its Lessons 3 to 5 covered vulnerability scanning, scoring with CVSS, EPSS and KEV, and penetration testing methodology and rules of engagement. This lesson does not repeat those topics. It looks at the same techniques from inside the development organization: how they are built into daily work, how they are tuned and how their results are used.

Peer code review

Code review, in which one or more developers examine a change before it is merged, is the oldest and still one of the most effective quality and security practices. Reviews catch logic errors, missing authorization checks, misuse of security functions and violations of the coding standard that tools miss, and they spread knowledge across the team. They also provide separation of duties: a malicious or careless change needs a second person to approve it. Lesson 4 described how branch protection enforces review.

Reviews are only as good as the reviewers' attention and knowledge. Large changes receive superficial reviews, so teams should keep pull requests small. Checklists focused on the organization's common weaknesses help reviewers look for the right things. Sensitive code, such as authentication, authorization, payment logic and cryptography, should be assigned to designated code owners with security expertise. Formal inspections, a more structured technique with defined roles and meetings, are used for safety-critical and high-assurance software. Reviews of infrastructure-as-code and pipeline definitions matter as much as reviews of application code, because a single changed line in a cloud configuration can expose a database.

Static application security testing

Static application security testing (SAST) analyses source code, bytecode or binaries without running them, looking for patterns that indicate weaknesses, such as data flowing from an untrusted input to a database query without parameterization. Because it needs no running application, SAST can run in the developer's IDE and on every pull request, giving feedback within minutes. It covers all code paths, including rarely executed ones. Its weaknesses are false positives, which waste time and erode trust, and false negatives for weaknesses that depend on configuration, runtime behaviour or business rules. SAST results improve markedly when rules are tuned to the organization's frameworks and when developers can mark findings as false positives with a recorded justification that the security team can review.

Dynamic, interactive and runtime testing

Dynamic application security testing (DAST) examines a running application from the outside, sending requests and analysing responses, much as an external user would. It finds problems that only appear at runtime, such as misconfigured servers, missing security headers, weak session handling and some injection flaws, and its findings are usually real. It cannot see inside the code, may miss pages it cannot reach, and needs a working environment, so it runs later, typically against staging and sometimes against production with care.

Interactive application security testing (IAST) places sensors inside the running application, usually through an agent in the runtime, and observes data flows while functional tests or users exercise the application. It combines some of the code visibility of SAST with the runtime accuracy of DAST and produces fewer false positives, but it depends on test coverage and supports only certain languages. Runtime application self-protection (RASP) uses similar instrumentation in production to detect and block attacks as they happen. RASP is a protective control rather than a testing technique, and like a web application firewall it should be treated as a compensating layer, not a substitute for fixing defects.

Software composition analysis

Modern applications are mostly made of third-party components. Software composition analysis (SCA) identifies the open source and commercial components in an application, including transitive dependencies that the developers never chose directly, and reports known vulnerabilities and licence obligations for each. Good SCA tools also judge reachability, that is, whether the application actually calls the vulnerable code, which helps teams prioritize. SCA is the foundation of the software bill of materials discussed in Lesson 10.

Fuzzing

Fuzz testing feeds a program large volumes of automatically generated, malformed or unexpected input to find crashes, hangs and assertion failures that indicate defects such as memory safety errors. Coverage-guided fuzzers observe which code paths each input exercises and evolve inputs to reach new paths, which makes them remarkably effective at finding deep bugs in parsers, file format handlers and network protocol code. Fuzzing belongs in the developer's own test suite and in continuous integration for components that handle untrusted input; Google's OSS-Fuzz service has found thousands of vulnerabilities in widely used open source projects this way. For a manager, the governance points are that fuzzing is aimed at the organization's own code in its own test environments, and that crashes it finds are treated as potential security defects until shown otherwise.

Security unit tests, misuse tests and test coverage

Functional tests check that software does what it should. Security tests also check that it refuses what it should refuse. Every security requirement from Lesson 1 should have at least one test: a refund larger than the payment is rejected, a viewer cannot create refunds, a request for another merchant's object returns not found, a session expires after the idle timeout. These negative tests run in the pipeline with every change, which turns the security requirements into permanent regression checks. Test coverage metrics show how much of the code the tests exercise; high coverage does not prove security, but very low coverage of security-sensitive code is a warning sign worth acting on.

Turning findings into risk reduction

Running tools is easy; using their output well is hard. A new SAST deployment on a large code base can report thousands of findings, most of them low risk or false positives. If every finding blocks the pipeline, teams revolt; if findings are simply dumped into a backlog, nothing improves. The usual approach is to establish a baseline of existing findings, block only on new high-confidence, high-severity findings in changed code, triage the backlog by risk, and measure the precision of each rule so that noisy rules are tuned or disabled.

import random
random.seed(11)
# Triage results for SAST rules over one quarter (synthetic data)
rules = {   # rule: (findings raised, probability a finding is a true positive)
    'sql-string-concat':     (48, 0.80),
    'xss-unescaped-output':  (35, 0.55),
    'weak-hash-function':    (60, 0.25),
    'hardcoded-credential':  (22, 0.70),
    'generic-taint-flow':    (310, 0.06),
}
total_raised = total_true = 0
print(f"{'rule':22s} {'raised':>6s} {'true':>5s} {'precision':>9s}  action")
for rule, (n, p) in rules.items():
    tp = sum(random.random() < p for _ in range(n))
    total_raised += n; total_true += tp
    prec = tp / n
    action = 'block on new' if prec >= 0.5 else ('report only' if prec >= 0.2 else 'tune or disable')
    print(f'{rule:22s} {n:6d} {tp:5d} {prec:9.0%}  {action}')
print(f'Overall precision {total_true / total_raised:.0%}; '
      f'{rules["generic-taint-flow"][0] / total_raised:.0%} of findings come from one noisy rule')

rule                   raised  true precision  action
sql-string-concat          48    39       81%  block on new
xss-unescaped-output       35    21       60%  block on new
weak-hash-function         60    21       35%  report only
hardcoded-credential       22    15       68%  block on new
generic-taint-flow        310    15        5%  tune or disable
Overall precision 23%; 65% of findings come from one noisy rule

In this synthetic quarter, one generic rule produced most of the findings while finding very little. Disabling it outright might lose its few real detections, so the better course is to tune it for Ridgeline's frameworks, for example by recognizing the approved query library as safe. Precision-based policies like this keep developer trust, which is the scarcest resource in an application security program.

Findings must flow into the same defect tracking system that teams use for other work, with severity, owner and due date. Remediation deadlines are usually tied to severity and exposure, such as 7 days for a critical issue in an internet-facing service and 90 days for a medium issue in an internal one. Findings that cannot be fixed in time go through the risk acceptance process described in Lesson 1. Lesson 12 covers the metrics a program uses to report on all of this.

Independent testing, bug bounties and disclosure programs

Internal testing is complemented by independent perspectives. Penetration tests by qualified internal or external testers, run under written authorization and rules of engagement as CISSP-115 explains, are appropriate before major releases, after significant architectural changes and periodically for high-risk applications. A vulnerability disclosure program (VDP) publishes a policy and a channel through which anyone can report security issues in good faith, with a commitment to respond and not to pursue legal action against good-faith researchers who follow the rules. A bug bounty program adds payment for valid findings, usually run through a platform that manages researchers and triage. Both require a mature internal process to receive, verify and fix reports quickly; launching a bounty before that process exists invites frustration on all sides. Government guidance in both Canada and the United States encourages organizations to publish disclosure policies, and the EU Cyber Resilience Act will require manufacturers of products sold in the EU to have coordinated vulnerability disclosure policies.