CI/CD Automation Tutorial 0/46 lessons ~6 min read Lesson 33

    SAST Security Scanning

    SAST (Static Application Security Testing) analyzes source code without executing it — finding SQL injection sinks, hardcoded secrets, unsafe deserialization, and auth bypass pa…

    Course progress0%
    Focus
    20 guided sections
    Practice signal
    Examples included
    Career prep
    Interview Q&A included

    Introduction

    SAST (Static Application Security Testing) analyzes source code without executing it — finding SQL injection sinks, hardcoded secrets, unsafe deserialization, and auth bypass patterns before merge. In CI, SAST is a shift-left gate: Semgrep for fast custom rules, CodeQL for deep inter-procedural taint analysis. Staff engineers spend as much time on false positive tuning as on enabling rules — a noisy SAST gate gets disabled within a quarter.

    The story

    A neobank's manual security review missed a string-concatenated SQL query in a rush hotfix. Semgrep's sql-injection rule flagged it in CI on the PR — developer dismissed it as "false positive" twice before security champion traced the data flow: user input reached the query unsanitized. After tuning, they added a custom Semgrep rule for their ORM bypass pattern and integrated CodeQL on nightly for deeper paths. Zero SQLi escapes in 14 months; mean time to fix findings dropped from 11 days to 2 because findings appeared on PR, not pentest.

    Understanding the topic

    SAST in the pipeline runs on every PR against the diff or full tree. Two dominant OSS/commercial patterns: Semgrep (fast, YAML rules, great for custom org policies) and CodeQL (GitHub-native, query language, strong on Java/C/JavaScript taint tracking).

    • Semgrep: run semgrep ci with rule packs (p/security-audit, p/owasp-top-ten); custom rules in .semgrep/ for org-specific anti-patterns.
    • CodeQL: github/codeql-action — builds DB from compiled language, runs queries; best on main languages; slower but deeper.
    • When to run: Semgrep on every PR (1–3 min); CodeQL on PR + weekly full scan for new queries.
    • False positive tuning: inline nosemgrep/codeql suppress with ticket ID; baseline files for legacy; severity thresholds (ERROR fails, WARNING comments).
    • Output: SARIF uploaded to GitHub Security tab or DefectDojo for triage workflow.

    Internal architecture

    SAST stage placement — parallel with lint and unit tests; blocks merge on configured severity.

    text
    PR opened / updated
    ↓
    Checkout + language setup
    ↓
    ┌─────────────┬──────────────┐
    │ Semgrep CI │ CodeQL init │ ← parallel
    │ (all langs) │ (compile DB) │
    └──────┬──────┴──────┬───────┘
    ↓ ↓
    SARIF upload Query run
    ↓ ↓
    Policy gate: ERROR → fail PR
    WARNING → review required
    ↓
    Merge blocked until fixed or waived

    Visual explanation

    Two diagrams show where SAST Security Scanning lives in the delivery path and how teams implement it in production.

    SAST Security Scanning — system view
    Trigger
    Git event
    Pipeline
    Stages
    Artifact
    Immutable
    Deploy
    Gated
    Where this topic sits in the delivery path.
    SAST Security Scanning — execution flow
    Plan
    Design
    Build
    Verify
    Release
    Promote
    Observe
    Metrics
    Follow this loop when designing or reviewing pipelines.

    Step-by-step explanation

    1. Enable Semgrep: add .semgrep.yml extending p/ci and p/security-audit rule packs.
    2. Run baseline scan on main; export findings — suppress legacy with documented tickets or .semgrepignore.
    3. Add CI job: semgrep ci --sarif --output semgrep.sarif; upload SARIF; fail on ERROR severity.
    4. Enable CodeQL for primary languages via GitHub Advanced Security or self-hosted runner with codeql CLI.
    5. Define waiver process: security team approves nosemgrep/codeql-disable with expiry date and linked JIRA.
    6. Review rule noise monthly — disable rules with >50% false positive rate until rewritten.

    Production implementation

    GitHub Actions — Semgrep + CodeQL with SARIF and severity gate:

    yaml
    jobs:
    semgrep:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - run: pip install semgrep
    - run: semgrep ci --config p/ci --config p/security-audit --sarif --output semgrep.sarif
    - uses: github/codeql-action/upload-sarif@v3
    if: always()
    with: { sarif_file: semgrep.sarif }
    - run: |
    semgrep ci --config p/ci --json | jq -e '.results | map(select(.extra.severity=="ERROR")) | length == 0'
    codeql:
    permissions: { security-events: write, contents: read }
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: github/codeql-action/init@v3
    with: { languages: javascript, python }
    - uses: github/codeql-action/autobuild@v3
    - uses: github/codeql-action/analyze@v3
    with: { category: /language:${{ matrix.language }} }

    Execution workflow

    1SAST rollout with false positive tuning
    1 / 5

    Baseline scan

    Run Semgrep + CodeQL on main; export SARIF inventory.

    Classify true vs false positive.

    Real-world use

    Microsoft, GitHub, and Shopify use CodeQL at scale — GitHub Advanced Security democratized SAST for open source. Semgrep (r2c) powers custom policy-as-code at Dropbox, Figma, and Snowflake. OWASP Benchmark scores tools differently per language — staff engineers pick Semgrep for velocity, CodeQL for depth, commercial tools when compliance mandates vendor support.

    Enterprise use cases

    Regulated financial services: Semgrep for PR gate (fast); Checkmarx or Fortify for release branch (policy); CodeQL for GitHub repos. Findings flow to DefectDojo → JIRA with SLA: Critical 24h, High 7d. Waivers require CISO delegate approval with compensating control documented.

    • Monorepo: scan only affected paths via Semgrep --baseline-commit on PR.
    • Polyglot: Semgrep covers 30+ languages; CodeQL per-language jobs in matrix.
    • Insider risk: rule changes and suppressions require security team CODEOWNERS review.

    Production case study

    B2B API platform (SOC2 Type II): pentest found 3 SQLi issues that SAST would have caught — audit finding required automated SAST on every merge.

    • Challenge: initial Semgrep run: 1,400 findings, 70% false positives — devs demanded disable.
    • Decision: baseline suppress legacy; ERROR-only gate on diff; security office hours for triage training.
    • Tuning: disabled 12 noisy rules; added 4 custom rules for internal frameworks.
    • Outcome: PR scan avg 90 sec; 40 true positives fixed in 60 days; passed next SOC2 audit with CI log evidence.

    Trade-offs

    • Semgrep: fast, easy custom rules; less deep inter-procedural analysis than CodeQL on complex Java.
    • CodeQL: deep taint tracking; requires build step; slower; GitHub-centric workflow.
    • Fail on all findings: blocks delivery on legacy debt — baseline/waiver required.
    • Warn only: findings ignored — change to fail on new ERROR in diff only (Semgrep diff scan).
    • Commercial SAST: better support and compliance mapping; cost and pipeline time increase.

    Security implications

    SAST itself introduces pipeline considerations:

    • CodeQL DB and SARIF may contain code snippets — restrict artifact retention and access.
    • Custom rules must not log secrets matched in patterns — use masked output.
    • Fork PRs: run Semgrep in sandbox without secrets; CodeQL may need pull_request_target carefully (security risk if misconfigured).
    • Suppress comments (nosemgrep) are audit artifacts — track in GRC system.

    Scalability analysis

    SAST at monorepo scale:

    • Full CodeQL on 2M LOC exceeds 30 min — diff-scoped analysis or affected modules only on PR.
    • Semgrep with --baseline-commit scans only changed files + dependencies — 10× faster on large PRs.
    • Central rule registry (semgrep registry submodule) versioned — avoid each repo forking rules.
    • SARIF aggregation across 100 repos needs DefectDojo or GitHub org-level security overview.

    Staff engineer insights

    • A SAST gate without a waiver process will be removed — budget security champion time for triage, not just tool install.
    • Diff-scoped gating ("no new ERRORs") is how you ship SAST on legacy without stopping the world.
    • Semgrep custom rules are code — review them in PRs; a bad rule blocks all merges or misses real bugs.
    • CodeQL query updates can surface thousands of "new" findings — pin query packs; test in nightly before promoting to PR gate.

    Best practices

    • Upload SARIF to centralized dashboard — developers triage in one place.
    • Run SAST parallel to unit tests — don't serialize pipeline on security.
    • Require ticket ID in every nosemgrep/codeql-disable comment.
    • Pin Semgrep rule pack versions — auto-updates can break CI overnight.
    • Train developers on top 5 finding types — reduces "it's a false positive" reflex.

    Common mistakes

    • Enabling all rules day one on legacy codebase — gate disabled in week 2.
    • Using pull_request_target to run CodeQL on fork PRs with secrets — supply chain attack vector.
    • No owner for waiver requests — suppressions accumulate without expiry.
    • SAST only on release branch — bug found after 50 merges; fix cost 10× higher.

    Advanced interview questions

    Interview Prep

    Practice concise answers, then expand each card for the explanation.

    5 questions
    1AdvancedQuestionSemgrep vs CodeQL — when do you choose each in CI?+

    Answer

    Semgrep: fast PR feedback, custom YAML rules, polyglot, diff scans in minutes. CodeQL: deep taint analysis on supported languages, GitHub integration, slower build+analyze. Use Semgrep as primary PR gate; CodeQL for depth on critical repos or nightly. Many teams run both in parallel.

    Follow-up

    How do you handle Semgrep false positives at scale?
    2AdvancedQuestionHow do you tune false positives without disabling SAST entirely?+

    Answer

    Baseline legacy findings with suppress + ticket. Gate on new ERROR in diff only. Disable rules with >50% FP rate until rewritten. Severity: ERROR fails, WARNING notifies. Monthly rule review. Waiver process with expiry and security approval.

    Follow-up

    Diff scan vs full scan trade-offs?
    3IntermediateQuestionWhere does SAST run in your pipeline relative to build and deploy?+

    Answer

    Parallel with lint and unit tests on PR — before merge, before artifact publish. Never only post-deploy. Optional second full scan on main nightly for new CodeQL queries. Blocks merge on policy severity; does not replace DAST or pentest.

    Follow-up

    SAST vs SCA vs DAST positioning?
    4IntermediateQuestionDeveloper adds nosemgrep on every finding. Policy?+

    Answer

    Require linked security ticket, justification, compensating control, expiry date. CODEOWNERS on suppress files. Track suppress count per team — high count triggers architecture review. Unapproved suppress fails CI via linter for suppress comments.

    Follow-up

    Automated suppress detection?
    5AdvancedQuestionDesign SAST for a 500k LOC Java monorepo with 15 min CI budget.+

    Answer

    Semgrep diff scan on PR (~2 min). CodeQL affected-modules or scheduled nightly full scan. Cache CodeQL DB. Parallel job with tests. Fail PR on new Semgrep ERROR. Full CodeQL promotes to release gate only.

    Follow-up

    Checkmarx when OSS isn't enough?

    Hands-on exercise

    Lab: Introduce intentional SQL injection in a sample API. Configure Semgrep p/security-audit to catch it. Add a false positive case (safe parameterized query) and document why it doesn't fire. Create waiver template with expiry.

    • Upload SARIF to GitHub Security tab.
    • Write one custom Semgrep rule for a project-specific pattern.
    • Configure fail-on-ERROR in CI.

    Summary

    You can position SAST in CI: Semgrep for speed and custom policy, CodeQL for depth, SARIF for triage, and diff-scoped ERROR gates for legacy coexistence. Explain false positive tuning as the difference between a gate teams trust and one they disable.

    Ready to mark this lesson complete?Track your journey across the entire course.