Security Considerations

When using Bop, your code is sent to LLM providers for analysis. This page explains how data flows, what protections are in place, and recommendations for different repository types.

How Data Flows

When you run a Bop review:

1. Diff extraction: Bop extracts the diff between your branches locally
2. Context gathering: Relevant context files may be included (ARCHITECTURE.md, related code)
3. Transmission: The diff and context are sent to configured LLM providers via HTTPS
4. Processing: The LLM analyzes the code and returns findings
5. Output: Results are written locally and optionally posted to GitHub

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

┌─────────────┐     HTTPS      ┌─────────────────┐
│  Your Code  │ ───────────────▶│  LLM Provider   │
│   (diff)    │                │ (Anthropic,     │
└─────────────┘                │  OpenAI, etc.)  │
                               └─────────────────┘
                                       │
                                       ▼
                               ┌─────────────────┐
                               │    Findings     │
                               └─────────────────┘

What Gets Sent

Included:

• Code diff (added, modified, deleted lines)
• Context from surrounding code
• ARCHITECTURE.md (unless --no-architecture)
• Custom context files (if specified with --context)
• Review instructions

Not included:

• Files matching redaction patterns
• Git history beyond the diff
• Environment variables
• Local filesystem outside the repository

Secret Redaction

Bop includes built-in redaction to prevent sensitive files from being sent to LLMs.

Default Redaction Patterns

1
2
3
4
5
6
7
8

redaction:
  enabled: true
  denyGlobs:
    - "**/*.env"
    - "**/*.pem"
    - "**/*.key"
    - "**/secrets.*"
    - "**/credentials.*"

Custom Redaction

Add patterns in your bop.yaml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

redaction:
  enabled: true
  denyGlobs:
    - "**/*.env"
    - "**/*.pem"
    - "**/*.key"
    - "**/secrets/**"
    - "**/*secret*"
    - "**/config/production.*"
  allowGlobs:
    - "**/*.env.example"
    - "**/secrets.example.yaml"

Verification

Review what will be sent before running:

1
2
3
4
5

# Preview the diff
git diff main..HEAD

# Check for sensitive patterns
git diff main..HEAD --name-only | grep -E '\.(env|pem|key)$'

Provider Data Policies

Each LLM provider has different data retention and usage policies.

Anthropic (Claude)

• API data retention: Not used for training
• Logging: Requests logged temporarily for abuse prevention
• SOC 2 compliant: Yes
• Privacy policy: https://www.anthropic.com/privacy

OpenAI

• API data retention: Not used for training (by default)
• Opt-out available: For data usage
• SOC 2 compliant: Yes
• Privacy policy: https://openai.com/policies/privacy-policy

Google (Gemini)

• API data retention: Varies by service tier
• Enterprise options: Available for stricter policies
• Privacy policy: https://policies.google.com/privacy

Ollama (Local)

• No external transmission: Code stays on your machine
• Full control: You manage the model and data
• Ideal for: Highly sensitive code

Recommendations by Repository Type

Open Source Projects

Risk level: Low - code is already public

Recommendations:

• Standard configuration works well
• Enable redaction for any secrets in the repo
• Consider cost optimization with faster models

Private Commercial Projects

Risk level: Medium - code is confidential

Recommendations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

redaction:
  enabled: true
  denyGlobs:
    - "**/*.env"
    - "**/*.pem"
    - "**/secrets/**"
    - "**/config/production.*"

verification:
  enabled: true  # Reduce false positives

• Review provider data policies
• Consider enterprise agreements with providers
• Use OIDC in CI/CD for better token security

Highly Sensitive Code (Finance, Healthcare, Government)

Risk level: High - strict compliance requirements

Recommendations:

1. Use local models only:

   1 2 3 4 5 6 7 8 9 10

   providers: anthropic: enabled: false openai: enabled: false gemini: enabled: false ollama: enabled: true defaultModel: codellama

2. Or use enterprise agreements:

   • Anthropic Enterprise
   • OpenAI Enterprise
   • Google Cloud with appropriate data handling agreements

3. Additional controls:

   • Air-gapped environments for Ollama
   • VPN/private network for API calls
   • Audit logging enabled
   • Strict redaction patterns

GitHub Integration Security

Token Scopes

Use minimum required scopes:

GitHub Actions OIDC

OIDC is more secure than long-lived tokens:

1
2
3
4

permissions:
  id-token: write  # Enable OIDC

# No need to store long-lived tokens as secrets

Fork Pull Requests

Be cautious with workflows on fork PRs:

1
2
3
4
5
6

# Only run on internal PRs by default
on:
  pull_request:
    types: [opened, synchronize]

# For fork PRs, use pull_request_target with care

Audit and Compliance

Logging

Enable structured logging for audit trails:

1
2
3
4
5

observability:
  logging:
    enabled: true
    format: json
    redactAPIKeys: true

Review History

Enable the store for local review history:

1
2
3

store:
  enabled: true
  path: ~/.config/bop/bop.db

Cost Tracking

Monitor API usage through provider dashboards and Bop’s built-in cost tracking.

Security Checklist

Before deploying Bop in your organization:

• Review and configure redaction patterns
• Understand provider data policies
• Use minimum required GitHub token scopes
• Enable OIDC for GitHub Actions where possible
• Configure appropriate logging
• For sensitive repos, consider local models (Ollama)
• Train team on what code gets sent externally
• Review and approve custom reviewer personas
• Set up monitoring for unexpected API usage

Incident Response

If sensitive code was accidentally sent to an LLM provider:

1. Rotate any exposed secrets immediately
2. Review audit logs to understand scope
3. Contact the provider if needed (especially for enterprise agreements)
4. Update redaction patterns to prevent recurrence
5. Document the incident per your organization’s policy