AI-Assisted Code Review: How to Integrate LLM Linters into Your GitHub Actions Pipeline

Traditional linters are excellent at what they can express as rules, formatting, unused variables, known-bad patterns. They are blind to everything else, a retry loop that can duplicate a payment, a query that's correct but will scan a million rows, a function whose name promises one thing and does another.

Large language models can read a diff the way a reviewer does and comment on intent, edge cases, and risk. Wired into a pull request workflow, that gives every PR a fast first review before a human opens it. Done carelessly, it also gives you noisy comments, surprise bills, and a new attack surface.

This guide shows how to add LLM-based review to GitHub Actions deliberately, where it fits in your pipeline, a working workflow, how to write prompts that produce useful comments, and the security and cost controls that make it safe to run on every PR.

Where LLM Review Fits in the Pipeline

LLM review is a complement to deterministic checks, not a replacement. Layer them by cost and reliability:

Two principles follow. Run the cheap, deterministic layers first, so the LLM doesn't spend tokens flagging a missing semicolon. And keep LLM review advisory, its output is probabilistic, so it should inform reviewers, not gate merges. (Our guides on SonarQube quality gates and dependency auditing cover layers 2 and the supply-chain side.)

Step 1 - Set Up Access

Anthropic's official option is the anthropics/claude-code-action GitHub Action, which runs Claude Code on a GitHub-hosted runner. Setup has two parts:

1. Install the Claude GitHub App on the repository (or run /install-github-app from Claude Code, which opens a PR with starter workflows).
2. Add your API key as a repository secret named ANTHROPIC_API_KEY under Settings → Secrets and variables → Actions.

You can also build your own step by calling an LLM API from a script. The action saves you the plumbing for fetching the diff, posting comments, and handling repository context, so start there and drop to custom code only if you need it.

Step 2 - A Review Workflow

# .github/workflows/ai-review.yml
name: AI Code Review

on:
  pull_request:
    types: [opened, synchronize, ready_for_review]
    paths-ignore:
      - '**.md'
      - 'docs/**'

# Cancel stale runs when new commits are pushed
concurrency:
  group: ai-review-${{ github.event.pull_request.number }}
  cancel-in-progress: true

jobs:
  review:
    if: github.event.pull_request.draft == false
    runs-on: ubuntu-latest
    timeout-minutes: 10
    permissions:
      contents: read
      pull-requests: write

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 1

      - uses: anthropics/claude-code-action@v1
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          claude_args: "--max-turns 8"
          prompt: |
            REPO: ${{ github.repository }}
            PR NUMBER: ${{ github.event.pull_request.number }}

            Review this pull request following the guidelines in CLAUDE.md.
            Focus on correctness, security, and missing tests.
            Only comment when you are confident there is a real issue.
            Post findings as review comments, and one short summary comment.

A few details worth noting. The concurrency block avoids paying for reviews of commits that are already outdated. paths-ignore and the draft check skip PRs that don't need review. timeout-minutes and --max-turns cap runaway sessions. Input names can change between action versions, so confirm them against the action's README for the version you pin.

Step 3 - Give the Reviewer Your Standards

A generic reviewer gives generic comments. The action reads a CLAUDE.md file from your repository, so encode your team's real conventions there:

# CLAUDE.md - review guidelines

## What matters here
- Money handling must be idempotent, flag any retry path without an idempotency key.
- Database access goes through the repository layer, flag raw SQL in controllers.
- New endpoints require authorization checks and tests for the denied case.

## What to ignore
- Formatting and naming style (handled by ESLint and Prettier).
- Generated files under /src/generated.

## Comment style
- Be specific: cite the line and explain the failure scenario.
- Skip praise and restating the diff.
- Label each finding: [blocker], [suggestion], or [question].

The "what to ignore" section is as important as the first. It's the cheapest way to cut noise, because it stops the model from duplicating what your linters already enforce.

Step 4 - Prompts That Produce Signal

Whether you use the action's prompt input or a custom script, the same principles improve output:

Ask for evidence, not opinions. Require each finding to name the file, the scenario that fails, and a suggested fix. Vague comments like "consider error handling" are noise.

Set a confidence threshold. Instruct the model to stay quiet when unsure. A reviewer that comments on everything trains people to ignore it.

Limit the scope. A prompt that asks for security, performance, style, and architecture at once produces shallow coverage of all four. Run separate, focused jobs (for example, a security-focused review on changes touching auth code) rather than one catch-all prompt.

Give it the context a human would want. The PR description, linked issue, and nearby code make comments more accurate than the diff alone.

Step 5 - Security Considerations

Running an LLM with repository access on untrusted input needs the same care as any CI job that touches secrets.

Fork PRs don't receive secrets. On pull_request events from forks, GitHub withholds repository secrets, so the review step will fail. That's the safe behavior. Resist the temptation to switch to pull_request_target to make it work: that trigger runs with secrets in the context of the base repository, which is dangerous when combined with checking out untrusted code.

Treat PR content as untrusted input. Titles, descriptions, comments, and code can contain text written to manipulate the model (prompt injection). Never interpolate PR fields into shell commands, and don't give the review job abilities beyond reading code and posting comments.

Grant least privilege. The workflow above requests only contents: read and pull-requests: write. Don't add permissions the job doesn't need, and keep it away from deployment credentials.

Pin the action. Pin third-party actions to a full commit SHA rather than a mutable tag, and let Dependabot propose updates, as we covered in the supply-chain security guide.

Know your data flow. The diff and surrounding files are sent to the model provider. Confirm that matches your organization's policy for source code, and exclude paths containing sensitive material.

Step 6 - Control Cost and Noise

Every run consumes GitHub Actions minutes and API tokens, so costs scale with PR volume.

• Trigger selectively. Skip drafts, docs-only changes, and bot PRs. Consider a label like ai-review for opt-in on large changes.
• Cap diff size. For very large PRs, review only high-risk paths or ask authors to split the change.
• Cancel superseded runs with the concurrency setting shown above.
• Set budgets. Use your provider's usage limits and alerts so a misconfigured workflow can't run up an unexpected bill.

Step 7 - Measure Whether It Helps

Without measurement, you won't know if the reviewer is an asset or background noise. Track a few simple signals:

• Acceptance rate: the share of AI comments that lead to a code change or a resolved thread.
• Dismissal rate: comments marked outdated or resolved without action. A high number means the prompt or guidelines need tightening.
• Time to first review: whether authors address issues earlier in the cycle.
• Escaped defects: whether bugs the reviewer should have caught still reach production.

Review these monthly, adjust CLAUDE.md and prompts based on the dismissed comments, and expand the scope only when acceptance is healthy.

Common Pitfalls

Making it a required check too early. A probabilistic reviewer that can fail builds will block merges on false positives and erode trust. Keep it advisory until you have data.

Replacing human review. Authors and reviewers remain accountable for what merges. The model is a first pass, not an approver.

Letting it duplicate your linters. If comments repeat what ESLint already reports, tighten the guidelines.

Skipping repository guidance. Without a CLAUDE.md or equivalent, comments reflect generic best practices rather than your system's real risks.

Ignoring prompt injection. Assume anyone who can open a PR can try to steer the reviewer, and design permissions accordingly.

Conclusion

AI-assisted review earns its place when it's treated like any other CI stage: scoped, observable, least-privileged, and measured. Layer it after deterministic linters and tests, keep it advisory, teach it your standards through repository guidance, and lock down the workflow against untrusted input.

Start with one repository and one narrow goal, such as flagging missing tests and risky error handling. Watch the acceptance rate for a few weeks, refine the guidance, then widen the scope. The goal isn't to automate judgment, it's to give your human reviewers a better-prepared starting point on every pull request.

Already running AI review in CI, or trialing it? Share what you've tuned, whether the prompt, the rules, or the trigger conditions, in the comments.

#AICodeReview #GitHubActions #LLM #CodeReview #DevOps