CI/CD Automation Tutorial 0/46 lessons ~6 min read Lesson 33

    SAST Security Scanning

    SAST (Static Application Security Testing) analyzes source code without executing it — finding SQL injection sinks, hardcoded secrets, unsafe deserialization, and auth bypass pa…

    Course progress0%
    Focus
    20 guided sections
    Practice signal
    Examples included
    Career prep
    Interview Q&A included

    Introduction

    SAST (Static Application Security Testing) analyzes source code without executing it — finding SQL injection sinks, hardcoded secrets, unsafe deserialization, and auth bypass patterns before merge. In CI, SAST is a shift-left gate: Semgrep for fast custom rules, CodeQL for deep inter-procedural taint analysis. Staff engineers spend as much time on false positive tuning as on enabling rules — a noisy SAST gate gets disabled within a quarter.

    The story

    A neobank's manual security review missed a string-concatenated SQL query in a rush hotfix. Semgrep's sql-injection rule flagged it in CI on the PR — developer dismissed it as "false positive" twice before security champion traced the data flow: user input reached the query unsanitized. After tuning, they added a custom Semgrep rule for their ORM bypass pattern and integrated CodeQL on nightly for deeper paths. Zero SQLi escapes in 14 months; mean time to fix findings dropped from 11 days to 2 because findings appeared on PR, not pentest.

    Understanding the topic

    SAST in the pipeline runs on every PR against the diff or full tree. Two dominant OSS/commercial patterns: Semgrep (fast, YAML rules, great for custom org policies) and CodeQL (GitHub-native, query language, strong on Java/C/JavaScript taint tracking).

    • Semgrep: run semgrep ci with rule packs (p/security-audit, p/owasp-top-ten); custom rules in .semgrep/ for org-specific anti-patterns.
    • CodeQL: github/codeql-action — builds DB from compiled language, runs queries; best on main languages; slower but deeper.
    • When to run: Semgrep on every PR (1–3 min); CodeQL on PR + weekly full scan for new queries.
    • False positive tuning: inline nosemgrep/codeql suppress with ticket ID; baseline files for legacy; severity thresholds (ERROR fails, WARNING comments).
    • Output: SARIF uploaded to GitHub Security tab or DefectDojo for triage workflow.

    Internal architecture

    SAST stage placement — parallel with lint and unit tests; blocks merge on configured severity.

    text
    PR opened / updated
    Checkout + language setup
    ┌─────────────┬──────────────┐
    │ Semgrep CI │ CodeQL init │ ← parallel
    │ (all langs) │ (compile DB) │
    └──────┬──────┴──────┬───────┘
    ↓ ↓
    SARIF upload Query run
    ↓ ↓
    Policy gate: ERROR → fail PR
    WARNING → review required
    Merge blocked until fixed or waived

    Visual explanation

    Two diagrams show where SAST Security Scanning lives in the delivery path and how teams implement it in production.

    SAST Security Scanning — system view
    Trigger
    Git event
    Pipeline
    Stages
    Artifact
    Immutable
    Deploy
    Gated
    Where this topic sits in the delivery path.
    SAST Security Scanning — execution flow
    Plan
    Design
    Build
    Verify
    Release
    Promote
    Observe
    Metrics
    Follow this loop when designing or reviewing pipelines.

    Step-by-step explanation

    1. Enable Semgrep: add .semgrep.yml extending p/ci and p/security-audit rule packs.
    2. Run baseline scan on main; export findings — suppress legacy with documented tickets or .semgrepignore.
    3. Add CI job: semgrep ci --sarif --output semgrep.sarif; upload SARIF; fail on ERROR severity.
    4. Enable CodeQL for primary languages via GitHub Advanced Security or self-hosted runner with codeql CLI.
    5. Define waiver process: security team approves nosemgrep/codeql-disable with expiry date and linked JIRA.
    6. Review rule noise monthly — disable rules with >50% false positive rate until rewritten.

    Production implementation

    GitHub Actions — Semgrep + CodeQL with SARIF and severity gate:

    yaml
    jobs:
    semgrep:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - run: pip install semgrep
    - run: semgrep ci --config p/ci --config p/security-audit --sarif --output semgrep.sarif
    - uses: github/codeql-action/upload-sarif@v3
    if: always()
    with: { sarif_file: semgrep.sarif }
    - run: |
    semgrep ci --config p/ci --json | jq -e '.results | map(select(.extra.severity=="ERROR")) | length == 0'
    codeql:
    permissions: { security-events: write, contents: read }
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: github/codeql-action/init@v3
    with: { languages: javascript, python }
    - uses: github/codeql-action/autobuild@v3
    - uses: github/codeql-action/analyze@v3
    with: { category: /language:${{ matrix.language }} }

    Execution workflow

    1SAST rollout with false positive tuning
    1 / 5

    Baseline scan

    Run Semgrep + CodeQL on main; export SARIF inventory.

    Classify true vs false positive.

    Real-world use

    Microsoft, GitHub, and Shopify use CodeQL at scale — GitHub Advanced Security democratized SAST for open source. Semgrep (r2c) powers custom policy-as-code at Dropbox, Figma, and Snowflake. OWASP Benchmark scores tools differently per language — staff engineers pick Semgrep for velocity, CodeQL for depth, commercial tools when compliance mandates vendor support.

    Enterprise use cases

    Regulated financial services: Semgrep for PR gate (fast); Checkmarx or Fortify for release branch (policy); CodeQL for GitHub repos. Findings flow to DefectDojo → JIRA with SLA: Critical 24h, High 7d. Waivers require CISO delegate approval with compensating control documented.

    • Monorepo: scan only affected paths via Semgrep --baseline-commit on PR.
    • Polyglot: Semgrep covers 30+ languages; CodeQL per-language jobs in matrix.
    • Insider risk: rule changes and suppressions require security team CODEOWNERS review.

    Production case study

    B2B API platform (SOC2 Type II): pentest found 3 SQLi issues that SAST would have caught — audit finding required automated SAST on every merge.

    • Challenge: initial Semgrep run: 1,400 findings, 70% false positives — devs demanded disable.
    • Decision: baseline suppress legacy; ERROR-only gate on diff; security office hours for triage training.
    • Tuning: disabled 12 noisy rules; added 4 custom rules for internal frameworks.
    • Outcome: PR scan avg 90 sec; 40 true positives fixed in 60 days; passed next SOC2 audit with CI log evidence.

    Trade-offs

    • Semgrep: fast, easy custom rules; less deep inter-procedural analysis than CodeQL on complex Java.
    • CodeQL: deep taint tracking; requires build step; slower; GitHub-centric workflow.
    • Fail on all findings: blocks delivery on legacy debt — baseline/waiver required.
    • Warn only: findings ignored — change to fail on new ERROR in diff only (Semgrep diff scan).
    • Commercial SAST: better support and compliance mapping; cost and pipeline time increase.

    Security implications

    SAST itself introduces pipeline considerations:

    • CodeQL DB and SARIF may contain code snippets — restrict artifact retention and access.
    • Custom rules must not log secrets matched in patterns — use masked output.
    • Fork PRs: run Semgrep in sandbox without secrets; CodeQL may need pull_request_target carefully (security risk if misconfigured).
    • Suppress comments (nosemgrep) are audit artifacts — track in GRC system.

    Scalability analysis

    SAST at monorepo scale:

    • Full CodeQL on 2M LOC exceeds 30 min — diff-scoped analysis or affected modules only on PR.
    • Semgrep with --baseline-commit scans only changed files + dependencies — 10× faster on large PRs.
    • Central rule registry (semgrep registry submodule) versioned — avoid each repo forking rules.
    • SARIF aggregation across 100 repos needs DefectDojo or GitHub org-level security overview.

    Staff engineer insights

    • A SAST gate without a waiver process will be removed — budget security champion time for triage, not just tool install.
    • Diff-scoped gating ("no new ERRORs") is how you ship SAST on legacy without stopping the world.
    • Semgrep custom rules are code — review them in PRs; a bad rule blocks all merges or misses real bugs.
    • CodeQL query updates can surface thousands of "new" findings — pin query packs; test in nightly before promoting to PR gate.

    Best practices

    • Upload SARIF to centralized dashboard — developers triage in one place.
    • Run SAST parallel to unit tests — don't serialize pipeline on security.
    • Require ticket ID in every nosemgrep/codeql-disable comment.
    • Pin Semgrep rule pack versions — auto-updates can break CI overnight.
    • Train developers on top 5 finding types — reduces "it's a false positive" reflex.

    Common mistakes

    • Enabling all rules day one on legacy codebase — gate disabled in week 2.
    • Using pull_request_target to run CodeQL on fork PRs with secrets — supply chain attack vector.
    • No owner for waiver requests — suppressions accumulate without expiry.
    • SAST only on release branch — bug found after 50 merges; fix cost 10× higher.

    Advanced interview questions

    Interview Prep

    Practice concise answers, then expand each card for the explanation.

    5 questions
    1AdvancedQuestionSemgrep vs CodeQL — when do you choose each in CI?+

    Answer

    Semgrep: fast PR feedback, custom YAML rules, polyglot, diff scans in minutes. CodeQL: deep taint analysis on supported languages, GitHub integration, slower build+analyze. Use Semgrep as primary PR gate; CodeQL for depth on critical repos or nightly. Many teams run both in parallel.

    Follow-up

    How do you handle Semgrep false positives at scale?
    2AdvancedQuestionHow do you tune false positives without disabling SAST entirely?+

    Answer

    Baseline legacy findings with suppress + ticket. Gate on new ERROR in diff only. Disable rules with >50% FP rate until rewritten. Severity: ERROR fails, WARNING notifies. Monthly rule review. Waiver process with expiry and security approval.

    Follow-up

    Diff scan vs full scan trade-offs?
    3IntermediateQuestionWhere does SAST run in your pipeline relative to build and deploy?+

    Answer

    Parallel with lint and unit tests on PR — before merge, before artifact publish. Never only post-deploy. Optional second full scan on main nightly for new CodeQL queries. Blocks merge on policy severity; does not replace DAST or pentest.

    Follow-up

    SAST vs SCA vs DAST positioning?
    4IntermediateQuestionDeveloper adds nosemgrep on every finding. Policy?+

    Answer

    Require linked security ticket, justification, compensating control, expiry date. CODEOWNERS on suppress files. Track suppress count per team — high count triggers architecture review. Unapproved suppress fails CI via linter for suppress comments.

    Follow-up

    Automated suppress detection?
    5AdvancedQuestionDesign SAST for a 500k LOC Java monorepo with 15 min CI budget.+

    Answer

    Semgrep diff scan on PR (~2 min). CodeQL affected-modules or scheduled nightly full scan. Cache CodeQL DB. Parallel job with tests. Fail PR on new Semgrep ERROR. Full CodeQL promotes to release gate only.

    Follow-up

    Checkmarx when OSS isn't enough?

    Hands-on exercise

    Lab: Introduce intentional SQL injection in a sample API. Configure Semgrep p/security-audit to catch it. Add a false positive case (safe parameterized query) and document why it doesn't fire. Create waiver template with expiry.

    • Upload SARIF to GitHub Security tab.
    • Write one custom Semgrep rule for a project-specific pattern.
    • Configure fail-on-ERROR in CI.

    Summary

    You can position SAST in CI: Semgrep for speed and custom policy, CodeQL for depth, SARIF for triage, and diff-scoped ERROR gates for legacy coexistence. Explain false positive tuning as the difference between a gate teams trust and one they disable.

    Ready to mark this lesson complete?Track your journey across the entire course.