The CLI powers Flightline's Readiness and Rulebook. Most teams interact with the results in the dashboard.

POWER USERS // CLI

The command line for AI safety

For teams that want full control. The CLI powers everything, from local testing to CI/CD integration.

Read the Docs Get your AI assessment

SECTION 01

QUICK START

Up and running in 5 minutes

Install the CLI, point it at your codebase, and see what Flightline discovers. No configuration required to get started.

1Install with pip
2Discover AI operations in your code
3Generate test scenarios
4Run your first eval

terminal

# Install
pip install flightline

# Discover AI ops in your code
flightline discover

# Generate 100 test scenarios
flightline gen -n 100

# Run evaluation
flightline eval --scenarios scenarios.json

SECTION 02

COMMANDS

Core Commands

Six commands that give you complete control over AI testing and evaluation.

discover

Scan your codebase for AI operations

$ flightline discover

Finds LLM calls, agents, and AI workflows in your code

learn

Create profiles from example data

$ flightline learn examples.json

Builds a profile of your data schema and business rules

generate

Create synthetic test scenarios

$ flightline gen -n 100

Generates edge cases, happy paths, and adversarial inputs

fltrace

Capture execution traces

$ flightline fltrace your-script.py

Records inputs, outputs, and intermediate steps

eval

Run evaluations against scenarios

$ flightline eval --scenarios test.json

Tests your AI against generated scenarios

check

Ship gate for CI integration

$ flightline check

Pass/fail gate based on Readiness score

SECTION 03

CI/CD INTEGRATION

Integrate with your pipeline

Add Flightline to GitHub Actions, GitLab CI, or any CI system. Block bad merges automatically.

GitHub Actions

.github/workflows/flightline.yml

name: Flightline Check
on: [pull_request]

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install Flightline
        run: pip install flightline

      - name: Run Check
        run: flightline check
        env:
          FLIGHTLINE_API_KEY: ${{ secrets.FL_KEY }}

GitLab CI

.gitlab-ci.yml

flightline:
  stage: test
  image: python:3.11
  script:
    - pip install flightline
    - flightline check
  variables:
    FLIGHTLINE_API_KEY: $FL_KEY
  rules:
    - if: $CI_MERGE_REQUEST_ID

REMEMBER

The CLI feeds the assessment

Every command you run contributes to your Readiness score and populates your Rulebook. The CLI is the engine, but the insight lives in the platform.

Explore Readiness →Explore Rulebook →

Ready to get started?

Install the CLI and see what Flightline discovers in your codebase.

Read the Docs Talk to the team