Implementing it in your organization
Adopt DevReplicate in stages: define the data boundary, prove the rules on an isolated replica, then make approved data part of developer onboarding.
Assign clear ownership
| Role | Owns |
|---|---|
| DBA / platform | Source access, isolated replicas, configuration deployment, init-sync, streaming, backfill, backup lineage, artifact publication and restore exercises. |
| Security | Sensitive-field inventory, transform review, hash-key custody, identity-provider scopes, coverage approval and review of residual plaintext. |
| Developers | Application and join checks, approved downloads, local restore recipes and reporting new sensitive fields or key patterns. |
These are adoption responsibilities for your team; they are not automated discovery or approval features.
Inventory the data and write rules
List sensitive SQL columns, Redis value selectors, nested JSON paths, key names and free-text fields. Include primary/foreign-key relationships and copies shared across engines. Decide which values the application needs, which may retain a suffix, and which can be removed.
Each product addresses a different surface, so the inventory has a different shape per engine. Write it down in the engine’s own terms:
| Product | A rule names |
|---|---|
| PostgreSQL / Patroni | Schema, table and column; json reaches paths inside a json, jsonb or text column. Configure a partitioned table at its root. |
| MySQL / MariaDB / Percona | Cluster, schema, table and column, so one configuration database can hold several source/replica pairs. |
| Redis / Valkey | A key glob, the container type, an optional field or entry selector, and an optional value_kind for typed canonicalisation. |
| MongoDB | Database and collection globs plus an exact field path, where . descends into a document and [*] into every array element. BSON carries the type, so no kind is declared. |
| Apache Kafka / Redpanda | A topic glob, which part of the record (value, key or header:<name>), how to read its bytes (json, text or bytes) and, for JSON, the leaf path. |
| Microsoft SQL Server | Schema, table and column. Which transforms a column accepts depends on its type, and the mismatch is refused before anything is written. |
| Oracle Database | Cluster, owner, table and column, spelled exactly as the catalog spells them — upper case unless the object was created quoted. A rule matching no column fails the baseline rather than protecting nothing quietly. |
| CouchDB | A database glob and a JSON field path. Reserved fields (_id, _rev, _attachments, _deleted) are refused at validation, and json is not a rule transform because the document already is JSON. |
-
hash: stable keyed values; agree on output length and input representation. -
mask: replace the value with a literal pattern plus an explicitly retained suffix; review the information that suffix reveals. -
fake: deterministic names, emails or addresses for readable development data; do not rely on fake names or addresses for uniqueness. -
null: remove values where the engine, target schema and application support it. SQL columns must allow NULL. -
json: target supported paths within documents; test absent keys, arrays and wrong shapes. -
pseudo_id: preserve integer joins with matching key, digit width and domain across every related field.
Keep hash keys in operator-controlled configuration or the documented key file/command mechanism, outside developer access. Keep tokens and keys out of repository files, container images and logs. Use the transform reference and the selected product’s configuration guide when writing rules; Redis selectors and SQL columns are different configuration surfaces.
Roll out in stages
- Pilot one bounded dataset. Choose an engine and an application workflow. Have platform and security review its inventory and transform rules before copying data.
-
Build the isolated replica. Configure all initial rules, run
init-sync, then runreplicate. Follow the engine-specific setup guide. Do not grant developer access merely because copying succeeded. - Validate coverage and behavior. Inspect every sensitive field and selector, test joins across engines, test invalid inputs and JSON shapes, and confirm that expected changes reach the replica. Review unconfigured fields, Redis key names and application exports.
- Approve developer delivery. Review backup contents, restore an artifact in isolation, and approve the dataset before issuing download scopes. Check artifact configuration version, capture time, engine compatibility and checksum.
- Expand with a change process. Repeat the review for each application, engine and newly sensitive field. Keep affected datasets restricted while pending rules or residual-plaintext cleanup remain.
Backfill after adding rules
For a replica already holding data, follow the product’s pending-rule workflow and run backfill or its documented re-baseline procedure. Forward-only streaming does not rewrite historical data merely because a rule was added. Pending fields and rules can continue to pass plaintext until the operation has completed and activated them.
Backfill completion is a logical-data check. It is not proof that older physical bytes, logs or exports have been erased.
Give developers an approved download workflow
With Enterprise, operators publish approved artifacts and configure serve to validate RS256 tokens from the organization’s OAuth2/OIDC identity provider. The service validates tokens; it does not issue them. Give developers per-database download scopes such as pa:download:orders, mya:download:app or ra:download:cache, rather than configuration-write permissions.
A database-scoped token cannot list all databases; give the developer the approved database name. Scope checks authorize the artifact route, not individual records inside a physical backup. Review the whole artifact’s contents before assigning access.
Use the product-specific download scope and restore instructions. Licences and keys are managed at account.devreplicate.com.
Wire the existing make dev-db recipe and Docker restore guidance into onboarding. Download a pinned artifact id, resume if needed, verify SHA-256 and size, then restore into a disposable local instance. Keep credentials out of command tracing and remove stale local datasets under your retention policy.
Review backups, logs and exports
A physical backup copies stored bytes; it does not rerun the anonymization policy. Backfilled plaintext can remain in dead tuples and archived WAL, and a bundle with WAL from before the backfill can reconstruct that plaintext during recovery.
Before distribution, follow the product’s clean-export procedure: rewrite affected tables or rebuild a fresh replica, reset the old backup/WAL lineage, and create a fresh base and subsequent bundles. Ordinary autovacuum and retention pruning are not substitutes for that cleanup.
PostgreSQL and MySQL physical backups can contain other databases in the same instance; MySQL physical backups also retain database credentials. Database-scoped download authorization does not filter those bytes. Review Redis snapshots and old persistence artifacts too, including unchanged key and field names.
Review operational logs, historical exports and previously downloaded copies for residual plaintext. Use isolated replica instances and explicit retention/deletion duties; a new rule does not retroactively sanitize files already distributed.
Review each product’s logs, checkpoints, change-capture storage and engine-native history as part of the data boundary. Re-baselining a target does not automatically remove old target files, source-side capture data or copies already distributed.
Keep the deployment and policy current
- Rotate hash keys deliberately. Because keyed output depends on the key, rotation requires coordinated re-anonymization from source data across related datasets. Plan a fresh replica where needed; applying a new hash to an old hash does not produce the new hash of the original value. Revalidate joins and retire old artifacts.
- Monitor the pipeline. Use Enterprise’s Prometheus endpoint to track lag, pending rules or fields, backfill outcomes and scrape failures.
- Exercise recovery. Restore approved bundles with the required engine version and tools. Observe the documented backup limits and Redis per-shard snapshot consistency limits.
- Review schema and policy changes together. Repeat the inventory when tables, columns, JSON shapes or Redis namespaces change. Use the transform reference together with the selected product’s supported configuration.
See API authentication and metrics for operational setup and pricing for the edition comparison.