Data Retention

2 probes

Checks data assets against GDPR retention policies. Flags datasets that have exceeded their maximum retention period or that lack a declared retention policy entirely. Policies are defined per-dataset or per-path pattern in YAML.

Configuration

sources:
  - type: retention
    storage_report: output/adls_gen2.metadata.paths.json
    policies: /etc/metapod/retention_policies.yaml

Policy file example:

# /etc/metapod/retention_policies.yaml
policies:
  - pattern: "landing/customers/**"
    max_retention_days: 90
  - pattern: "landing/orders/**"
    max_retention_days: 365
  - pattern: "archive/**"
    max_retention_days: 2555   # 7 years

Probes

Probe	Description
`retention.compliance.expired`	Flag datasets that exceed their declared retention period
`retention.compliance.no_policy`	Flag datasets with no matching retention policy

Usage

metapod run "retention.compliance.*" --config config.yaml

Output

{
  "probe": "retention.compliance.expired",
  "source": "retention",
  "collected_at": "2026-03-31T08:00:00Z",
  "results": [
    {
      "path": "landing/customers/2025-06-01/",
      "policy_max_days": 90,
      "actual_age_days": 303,
      "status": "fail",
      "recommendation": "Delete or archive data older than 90 days"
    }
  ]
}