Nullify
Secure. Transparent. Nullified.
Nullify is an open source, modular data retention, deletion, and lifecycle governance platform. It provides a centralized framework for discovering data, evaluating retention policies, executing controlled lifecycle actions, and maintaining verifiable audit records across distributed data environments.
Nullify is designed around a specification-first architecture. Core modules provide the foundational capabilities required for data lifecycle governance, while optional plugin modules extend Nullify with additional connectors, compliance frameworks, intelligence, integrations, storage systems, and deployment capabilities.
Specification
Nullify defines an open source architecture for centralized data lifecycle management.
The specification is built around several principles:
- Centralized policy coordination
- Distributed data source support
- Policy-driven retention and deletion
- Explicit authorization and approval
- Dry-run and validation workflows
- Immutable and verifiable auditing
- Data lineage and provenance
- Modular connectors and integrations
- Human oversight for destructive operations
- Secure-by-default execution
- Vendor-neutral architecture
- Local, cloud, hybrid, and federated deployment
- Extensible plugin architecture
- Transparent policy evaluation
- Reproducible lifecycle decisions
Nullify does not require organizations to migrate their data into a proprietary centralized repository. Instead, the system coordinates lifecycle policies and actions across existing data environments.
Architecture
Nullify is divided into two primary architectural layers:
- Core Modules
- Optional Plugin Modules
Core modules contain the foundational functionality required to operate Nullify. Optional plugins provide specialized functionality without making the base platform dependent on a particular database, cloud provider, compliance framework, AI system, notification platform, or infrastructure environment.
Core Architecture
The primary lifecycle flow is:
Discovery → Classification → Policy Evaluation → Approval → Scheduling → Execution → Verification → Audit
Each stage is represented by an independently maintainable core module.
Core Modules
1. Data Discovery Module
The Data Discovery Module identifies and inventories data resources managed by Nullify.
Features include:
- Data source registration
- Resource discovery
- Dataset and object inventories
- Metadata collection
- Data ownership metadata
- Creation and modification timestamps
- Access metadata
- Storage location tracking
- Resource status tracking
- Data source health monitoring
- Discovery scheduling
The module provides the inventory required by downstream lifecycle policies without requiring the underlying data to be copied into Nullify.
2. Data Classification Module
The Data Classification Module assigns structured metadata to discovered resources.
Features include:
- Data category assignment
- Sensitivity classification
- PII classification
- Financial data classification
- Health data classification
- Internal and public classification
- User-defined classifications
- Classification confidence
- Classification history
- Manual classification
- Classification overrides
Classification results become inputs to the policy evaluation process.
3. Policy Engine Module
The Policy Engine is the central decision-making component of Nullify.
Features include:
- Retention policies
- Deletion policies
- Archival policies
- Anonymization policies
- Legal hold rules
- Exception rules
- Policy priorities
- Policy inheritance
- Policy versioning
- Policy activation and expiration
- Policy simulation
- Policy conflict detection
- Policy validation
- Policy rollback
Policies should be declarative and machine-readable.
Nullify should support multiple policy formats while maintaining a normalized internal policy model.
4. Lifecycle Decision Module
The Lifecycle Decision Module converts policy evaluations into explicit lifecycle decisions.
Supported decisions include:
- Retain
- Review
- Archive
- Anonymize
- Delete
- Legal hold
- Exception
- Defer
Every decision should contain sufficient metadata to explain:
- What decision was made
- Which resource was affected
- Which policy produced the decision
- Which policy version was used
- When the decision was created
- When the decision should be executed
- Whether approval is required
5. Approval and Human Oversight Module
Nullify should not assume that every destructive operation can be fully automated.
The Approval Module provides controlled human oversight.
Features include:
- Approval queues
- Multi-person approval
- Role-based approval
- Approval delegation
- Approval expiration
- Rejection workflows
- Escalation workflows
- Emergency holds
- Manual overrides
- Approval history
Organizations can configure which actions require human approval and which may execute automatically.
6. Scheduling Module
The Scheduling Module manages when lifecycle actions occur.
Features include:
- Scheduled deletion
- Scheduled archival
- Scheduled anonymization
- Batch processing
- Priority queues
- Maintenance windows
- Resource-aware scheduling
- Retry scheduling
- Dependency-aware execution
- Workload balancing
- Execution throttling
Scheduling should separate the decision to perform an action from the actual execution of that action.
7. Action Execution Module
The Action Execution Module performs approved lifecycle operations against registered data sources.
Supported lifecycle actions include:
- Delete
- Archive
- Anonymize
- Redact
- Quarantine
- Move
- Expire
- Revoke access
Features include:
- Dry-run execution
- Pre-execution validation
- Execution confirmation
- Transaction-aware operations where supported
- Retry handling
- Failure detection
- Partial failure tracking
- Execution status
- Execution receipts
- Idempotent execution
- Safe execution controls
Destructive actions should require explicit authorization according to configured policy.
8. Verification Module
The Verification Module confirms whether lifecycle actions were successfully completed.
Features include:
- Deletion verification
- Archive verification
- Anonymization verification
- Source confirmation
- Replica verification
- Retry verification
- Failed-action detection
- Residual data detection
- Verification reports
Verification should distinguish between:
- Requested
- Authorized
- Scheduled
- Executed
- Verified
- Failed
- Partially completed
9. Audit and Evidence Module
The Audit and Evidence Module records the complete lifecycle of every important system action.
Features include:
- Immutable audit events
- Cryptographic event integrity
- Policy decision records
- Approval records
- Execution records
- Verification records
- User activity records
- Configuration history
- Policy history
- Audit export
- Evidence packages
- Chain-of-custody records
Audit records should make it possible to reconstruct why a lifecycle decision occurred and what happened afterward.
10. Data Lineage Module
The Data Lineage Module tracks relationships between data resources.
Features include:
- Source lineage
- Destination lineage
- Transformation lineage
- Copy relationships
- Replication relationships
- Derived-data relationships
- Parent-child relationships
- Data movement history
- Lifecycle propagation
Lineage allows Nullify to identify related resources that may also require retention, archival, anonymization, or deletion.
11. Access Control Module
The Access Control Module protects administrative and lifecycle operations.
Features include:
- Role-Based Access Control
- Attribute-Based Access Control
- Permission management
- Resource-level permissions
- Action-level permissions
- Approval permissions
- Administrative separation
- Session management
- Authentication integration
- Authorization auditing
Destructive operations should use least-privilege authorization.
12. Notification Module
The Notification Module provides system and lifecycle notifications.
Features include:
- Policy violation alerts
- Failed execution alerts
- Approval notifications
- Scheduled-action notifications
- Verification failures
- Data source failures
- Compliance alerts
- Administrative notifications
The core module should expose a notification interface while delivery mechanisms remain replaceable.
13. API Module
The API Module provides programmatic access to Nullify.
Features include:
- REST API
- GraphQL API
- Authentication
- Authorization
- Resource management
- Policy management
- Lifecycle management
- Audit queries
- Reporting
- Plugin management
- Administrative operations
APIs should expose stable versioned interfaces.
14. Dashboard Module
The Dashboard provides the primary administrative interface.
Features include:
- Data inventory
- Retention status
- Pending actions
- Policy status
- Approval queues
- Execution status
- Verification status
- Audit history
- Policy conflicts
- Compliance metrics
- System health
- Plugin status
The dashboard should provide visibility without requiring users to interact directly with underlying databases or execution systems.
15. Reporting Module
The Reporting Module converts lifecycle data into operational and compliance reports.
Features include:
- Retention reports
- Deletion reports
- Policy reports
- Audit reports
- Exception reports
- Legal hold reports
- Execution reports
- Verification reports
- Data inventory reports
- Compliance evidence packages
Supported export formats should include:
- JSON
- CSV
- PDF
- Structured machine-readable evidence formats
16. Multi-Tenant Module
The Multi-Tenant Module allows Nullify to operate across multiple organizational environments.
Features include:
- Organization isolation
- Tenant-specific policies
- Tenant-specific administrators
- Tenant-specific audit records
- Tenant-specific connectors
- Tenant-specific retention rules
- Tenant-specific reporting
- Tenant-level configuration
Tenant boundaries should be enforced at the authorization and data-access layers.
17. Federation Module
The Federation Module coordinates multiple Nullify installations.
Features include:
- Multi-cluster coordination
- Federated policies
- Distributed execution
- Regional lifecycle enforcement
- Cross-environment audit coordination
- Centralized visibility
- Local execution
- Federated verification
Federation should allow organizations to retain local control over their data while coordinating lifecycle governance centrally.
Optional Plugin Modules
Plugins extend Nullify without expanding the dependency requirements of the core platform.
Plugins should use documented interfaces and APIs and should be independently installable, upgradeable, enabled, and disabled.
Data Source Plugins
Optional connectors may include:
- PostgreSQL
- MySQL
- MariaDB
- Microsoft SQL Server
- Oracle Database
- MongoDB
- Redis
- Elasticsearch
- OpenSearch
- Snowflake
- BigQuery
- Databricks
- Apache Cassandra
- S3-compatible storage
- Google Cloud Storage
- Azure Blob Storage
- Network file systems
- Object storage systems
- Custom REST APIs
Cloud Provider Plugins
Optional integrations may include:
- AWS
- Microsoft Azure
- Google Cloud
- Cloudflare
- DigitalOcean
- Other S3-compatible infrastructure
Cloud plugins should remain optional so that Nullify remains vendor-neutral.
Compliance Plugins
Optional compliance policy packs may include:
- GDPR
- CCPA
- CPRA
- HIPAA
- GLBA
- FERPA
- PCI DSS
- SOX
- Regional privacy requirements
- Organization-specific compliance frameworks
Compliance plugins should provide policy templates and mappings rather than hard-code regulatory requirements into the core engine.
AI and Intelligence Plugins
AI functionality should remain optional.
Possible plugins include:
- Sensitive-data classification
- PII detection
- Document classification
- Entity recognition
- Retention recommendations
- Policy conflict analysis
- Policy optimization
- Anomaly detection
- Failed-deletion analysis
- Compliance assistance
- Natural-language policy creation
AI-generated recommendations should remain subject to policy controls and human oversight.
Workflow Plugins
Optional workflow integrations may include:
- Apache Airflow
- Dagster
- Temporal
- Kubernetes Jobs
- GitLab CI/CD
- Other workflow orchestration platforms
Event Bus Plugins
Optional event integrations may include:
- Apache Kafka
- RabbitMQ
- NATS
- Redis Streams
- MQTT
- Cloud event systems
Identity Plugins
Optional authentication and identity integrations may include:
- LDAP
- Active Directory
- OAuth
- OpenID Connect
- SAML
- Enterprise identity providers
Notification Plugins
Optional notification integrations may include:
- Email
- Slack
- Microsoft Teams
- Webhooks
- PagerDuty
- Other notification services
Storage Plugins
Nullify may support optional storage backends for audit records, evidence, metadata, and system state.
Possible plugins include:
- PostgreSQL
- SQLite
- MariaDB
- S3-compatible object storage
- MinIO
- Distributed databases
- Enterprise storage systems
Deployment Plugins
Optional deployment modules may provide:
- Docker
- Docker Compose
- Kubernetes
- Helm
- Terraform
- Ansible
- Cloud deployment templates
Security Architecture
Security is a core requirement rather than an optional plugin.
Nullify should provide:
- Encryption in transit
- Encryption at rest
- Least-privilege authorization
- Secure credential handling
- Secret management integration
- Authentication
- Authorization
- Audit logging
- Cryptographic audit integrity
- Rate limiting
- API security
- Administrative separation
- Secure plugin isolation
- Configuration validation
- Secure defaults
Nullify should never require plaintext credentials to be stored in application configuration.
Safe Deletion Architecture
Because deletion is potentially destructive, Nullify separates lifecycle decisions from execution.
The recommended lifecycle is:
- Discover the resource
- Classify the resource
- Evaluate applicable policies
- Generate a lifecycle decision
- Check exceptions and legal holds
- Request approval when required
- Schedule the action
- Execute the action
- Verify the result
- Record the evidence
- Update lifecycle state
- Report the result
Dry-run mode should allow organizations to evaluate the expected outcome before executing destructive actions.
Legal Holds and Exceptions
Nullify must support lifecycle exceptions that prevent automated deletion.
Examples include:
- Legal holds
- Investigations
- Active disputes
- Regulatory preservation requirements
- Security investigations
- Organizational exceptions
- Temporary retention extensions
A legal hold or approved exception should take precedence over ordinary deletion policies according to the configured policy hierarchy.
Policy Conflict Detection
Nullify should identify situations where policies produce conflicting lifecycle decisions.
Examples include:
- Delete versus retain
- Delete versus legal hold
- Archive versus delete
- Conflicting retention periods
- Conflicting organizational policies
- Conflicting jurisdictional policies
The system should explain the conflict and identify which policies contributed to it.
Transparency
Every important lifecycle decision should be explainable.
Nullify should provide a decision record containing:
- Resource
- Data classification
- Applicable policies
- Policy versions
- Policy evaluation
- Exceptions
- Approval requirements
- Final decision
- Execution status
- Verification status
- Relevant audit events
This creates an auditable chain from policy definition to lifecycle outcome.
Technology Architecture
Nullify is designed to remain technology-neutral at the specification level.
A reference implementation may use:
- Python
- FastAPI
- React
- PostgreSQL
- Open Policy Agent
- Apache Airflow
- Dagster
- Docker
- Kubernetes
- MinIO
- Apache Kafka
- RabbitMQ
- NATS
These technologies are implementation choices rather than mandatory requirements of the Nullify specification.
Modular Design
Nullify follows a modular architecture so organizations can deploy only the functionality they require.
The core platform should provide:
- Discovery
- Classification
- Policy evaluation
- Lifecycle decisions
- Approval
- Scheduling
- Execution
- Verification
- Auditing
- Lineage
- Access control
- Notifications
- APIs
- Dashboard
- Reporting
- Multi-tenancy
- Federation
Optional functionality should be delivered through plugins.
This architecture prevents the core platform from becoming tightly coupled to specific vendors, cloud providers, databases, AI systems, compliance frameworks, or infrastructure platforms.
Plugin Requirements
Plugins should:
- Use documented interfaces
- Maintain independent configuration
- Declare dependencies
- Provide health checks
- Support enable and disable operations
- Provide clear error reporting
- Respect Nullify authorization
- Emit appropriate audit events
- Avoid bypassing core policy evaluation
- Maintain compatibility with supported API versions
- Include documentation
- Include tests
Plugins must not circumvent lifecycle policies or authorization controls.
Observability
Nullify should expose operational telemetry for:
- Data discovery
- Policy evaluation
- Queue depth
- Scheduled actions
- Execution performance
- Execution failures
- Verification failures
- API performance
- Plugin health
- Data source health
- System health
Optional observability plugins may integrate with external monitoring and logging platforms.
Reliability
Nullify should support:
- Retries
- Idempotent actions
- Failure recovery
- Queue persistence
- Execution checkpoints
- Health checks
- Service recovery
- Backup and restoration
- Disaster recovery
- Partial failure handling
A failed deletion should never be silently reported as successful.
Deployment Models
Nullify should support:
- Local development
- Single-server deployment
- Docker deployment
- Kubernetes deployment
- On-premises deployment
- Cloud deployment
- Hybrid deployment
- Multi-region deployment
- Federated deployment
Organizations should be able to operate Nullify without depending on a proprietary hosted service.
Feature Roadmap
Core
Security
Governance
Reliability
Optional Plugins
Open Source Development
Nullify is intended to be developed as a community-driven open source project.
Contributors can participate by:
- Developing core modules
- Creating plugins
- Building connectors
- Writing policy packs
- Improving documentation
- Creating tests
- Reporting bugs
- Improving security
- Developing integrations
- Proposing specification improvements
The modular architecture allows contributors to extend Nullify without modifying the foundational lifecycle engine whenever a feature can be implemented as a plugin.
Design Goals
Nullify is designed to provide:
- Transparency over opaque lifecycle automation
- Policy-driven governance over manual processes
- Modular architecture over monolithic dependencies
- Vendor neutrality over platform lock-in
- Verifiable evidence over unverifiable claims
- Human oversight over uncontrolled automation
- Open source extensibility over proprietary integrations
- Centralized governance with distributed execution
- Secure deletion over uncontrolled destruction
Specification Branding License (SBL)
Standard
- Fully AGPL-3.0+ compliant system
- Copyleft enforced for network deployments
- Required attribution:
Optional
- Specification Branding License (SBL)
📄 License & Notice Requirements
Nullify is released under the GNU Affero General Public License v3.0 or later (AGPL-3.0+).
By contributing to this project, you agree that your contributions will also be released under this license.
Please note the following:
- All contributions must comply with the AGPL-3.0+ terms.
- Under Section 7 of the license, all redistributions, forks, and derivative works must preserve attribution to:
Roxanne Ardary and roxanneardary.com.
- Nullify specificiations are free to use with attribution. A Specification Branding License can be negotiated upon request.
- The project's notice.md file tracks attribution requirements and contributor acknowledgments.
Any update that adds new contributors or modifies attribution should also update notice.md.
- When submitting a pull request, ensure that any new files maintain the attribution headers where applicable.
- Network-deployed versions of this software must also remain fully AGPL-3.0+ compliant, including exposure of source code modifications when applicable under the license.
For full legal details, please refer to the AGPL-3.0+ license and the project's notice.md file.
Open Arsenal Hub
https://gitlab.com/Roxanne_Ardary/open-arsenal-specs
roxanne_ardary/nullify
