CLI Reference

Key commands available in the Omnivore CLI.

Commands

crawl

Start a web crawl from one or more seed URLs.

BASH
COPY
001# Basic crawl with default settings
002omnivore crawl https://example.com
003
004# Advanced crawl with options
005omnivore crawl https://example.com --workers 5 --depth 3 --output results.json

git

Extract and analyze code from Git repositories with intelligent filtering.

BASH
COPY
001# Analyze a GitHub repository
002omnivore git https://github.com/rust-lang/cargo --output cargo-analysis.txt
003
004# Analyze local repository
005omnivore git . --output project-code.txt
006
007# Extract specific file types
008omnivore git ./my-project --only "*.rs,*.toml" --output rust-files.txt

parse

Parse HTML content and extract structured data.

BASH
COPY
001omnivore parse index.html --rules parser-rules.yaml

graph

Build knowledge graphs from crawled data (under development).

BASH
COPY
001omnivore graph results.json --output knowledge-graph.db