Usage

Commands

metapod init                         # Generate config.yaml template
metapod list-probes                  # Show all 155 probes
metapod run "oracle.*"               # Extract metadata → ./output/
metapod run "oracle.*" --delta       # Only changed records
metapod diff                         # Compare two extractions
metapod serve --port 9090            # Daemon mode (pull via HTTP)
metapod schedule "oracle.*" --cron "0 6 * * *"
metapod push --target URL            # Optional: send to catalog platform
metapod provision azure              # Auto-create Azure test resources

Basic workflow

1. Configure — create config.yaml with your source credentials:

metapod init   # generates a template

Edit to match your source:

# config.yaml
source:
  type: oracle
  host: prod-db.example.com
  port: 1521
  service_name: ORCLPDB1
  username: reader
  password: "${ORACLE_PASSWORD}"
  schema_name: RISK_MGMT

2. Extract — run probes against your source:

metapod run "oracle.metadata.*" --config config.yaml

# Output:
# ./output/oracle/metadata/tables.json
# ./output/oracle/metadata/columns.json
# ./output/oracle/metadata/views.json
# ...

3. Use — the JSON files are yours:

# Read locally
cat output/oracle/metadata/tables.json | python3 -m json.tool

# Or push to a catalog platform (optional)
metapod push --target https://dataflix.example.com --project my-project

Glob patterns

metapod run "oracle.metadata.*"       # All Oracle metadata probes
metapod run "oracle.quality.*"        # All Oracle quality probes
metapod run "*.metadata.tables"       # Tables probe for every source
metapod run "azure_*.*"              # All Azure probes
metapod run "postgres.metadata.tables" # One specific probe

Environment variables

Credentials in config.yaml support ${ENV_VAR} substitution:

source:
  password: "${ORACLE_PASSWORD}"

metapod also loads .env files automatically (python-dotenv).

Common config examples

PostgreSQL:

source:
  type: postgres
  host: pg-host.example.com
  port: 5432
  service_name: mydb
  username: reader
  password: "${PG_PASSWORD}"
  schema_name: public

Azure SQL:

source:
  type: azure_sql
  host: myserver.database.windows.net
  service_name: mydb
  username: admin
  password: "${SQL_PASSWORD}"

SQLite (local, no credentials):

source:
  type: sqlite
  host: /path/to/database.db

File Scanner (local directory):

source:
  type: file_scanner
  host: /path/to/data/files

See Azure Walkthrough for Azure-specific configuration (ADLS, Blob, ADF, Purview).