Quickstart
Your first metadata extraction in under 5 minutes. No database credentials needed — we’ll use SQLite.
Step 1: Install
# From PyPI (released version)
pip install metapod-probes
# Or from local source (if you cloned the repo)
git clone https://github.com/mario-ambrosino/metapod.git
cd metapod && pip install -e "."
Step 2: Create a test database
python3 -c "
import sqlite3
conn = sqlite3.connect('demo.db')
conn.execute('CREATE TABLE customers (id INTEGER PRIMARY KEY, name TEXT, email TEXT)')
conn.execute('CREATE TABLE orders (id INTEGER PRIMARY KEY, customer_id INTEGER, amount REAL)')
conn.execute('INSERT INTO customers VALUES (1, \"Acme Corp\", \"acme@example.com\")')
conn.execute('INSERT INTO orders VALUES (1, 1, 150.00)')
conn.commit()
conn.close()
"
Step 3: Create config
cat > config.yaml << EOF
source:
type: sqlite
host: ./demo.db
EOF
Step 4: Extract
metapod run "sqlite.*" --config config.yaml
Output:
Running 2 probes matching 'sqlite.*'
sqlite.metadata.tables ━━━━━━━━━━━━━━━━━━━━ 2/2 0:00:00
┃ Probe ┃ Records ┃ Duration ┃ Status ┃
│ sqlite.metadata.columns │ 5 │ 1ms │ OK │
│ sqlite.metadata.tables │ 2 │ 0ms │ OK │
Step 5: Inspect
cat output/sqlite/metadata/tables.json
{
"envelope_version": "1.0",
"probe": "sqlite.metadata.tables",
"source": {"type": "sqlite", "id": "sqlite://./demo.db"},
"records": [
{"table_name": "customers", "type": "table"},
{"table_name": "orders", "type": "table"}
],
"record_count": 2,
"duration_ms": 0
}
That’s it. Every probe produces a JSON envelope with this structure.
Next steps
Usage — config.yaml reference, glob patterns, all CLI commands
Delta Engine — incremental extraction (only changed records)
Standalone Usage — output formats, pandas integration, CSV conversion
Platform Integrations — push to DataHub, OpenMetadata, Purview, Kafka
Probe Reference — browse all 197 probes across 49 sources