Delta Engine
============

The Delta Engine enables incremental metadata extraction. Instead of
re-extracting all records on every run, it compares the current state against
the previous scan and emits only new, modified, or deleted records.

Usage
-----

Add ``--delta`` to any ``metapod run`` command:

.. code-block:: bash

   # First run — extracts everything, saves state
   metapod run "oracle.metadata.*" --delta

   # Second run — only emits changed records
   metapod run "oracle.metadata.*" --delta

   # Force full re-extract (ignore saved state)
   metapod run "oracle.metadata.*" --delta --force

How it works
------------

On each run the DeltaEngine:

1. Loads previous state from ``output/.delta/<probe_name>.json``
2. Runs the probe normally (full extraction)
3. For each record, extracts a unique key and change signals
4. Compares against previous state to classify: new, modified, unchanged
5. Detects deletions (previous keys absent from current run)
6. Saves new state atomically
7. Returns only changed records

Change detection strategies
----------------------------

The engine auto-selects the strongest available signal per record:

.. list-table::
   :header-rows: 1

   * - Strategy
     - Fields detected
     - Use case
   * - **Hash**
     - sql_text, source_code, definition, view_definition
     - Procedure/view source changes
   * - **Mtime**
     - last_modified, modified, last_analyzed, create_date
     - DDL changes, file updates
   * - **Size**
     - row_count, num_rows, size_bytes
     - Data volume drift
   * - **Key only**
     - table_name, name, path, id
     - New/deleted detection when no other signals exist

Priority: hash > mtime > size. If all signals match, the record is unchanged.

State persistence
-----------------

State is stored as JSON in ``output/.delta/``:

.. code-block:: json

   {
     "probe_name": "oracle.metadata.tables",
     "last_scan": "2026-03-29T12:00:00+00:00",
     "scan_count": 3,
     "resources": {
       "CUSTOMERS": {"key": "CUSTOMERS", "mtime": 1711700000.0, "size": 1500, "hash": ""},
       "ORDERS": {"key": "ORDERS", "mtime": 1711700000.0, "size": 3200, "hash": ""}
     }
   }

Writes are atomic (write to ``.tmp``, then rename) to prevent corruption.

API
---

.. code-block:: python

   from core.delta import DeltaEngine

   engine = DeltaEngine(state_dir=Path("./output/.delta"))
   result, stats = engine.run(probe, connection, schema)

   # result.records — only changed records
   # stats.new, stats.modified, stats.deleted, stats.unchanged