Managed databases for your applications

A DAL connection lets AgentData track and manage a database's structure. Your application can use the published schema to build its own data access layer and connect directly with its own database credentials.

Current capabilities and planned schema governance

PostgreSQL managed changes support creating tables, adding nullable columns, adding unique/foreign-key/check constraints and creating indexes, with reviewed plans, drift checks and version history. Choose exact decimal precision/scale for money, timezone-aware instants, dates or UUIDs as appropriate. Identities, typed defaults and bounded arithmetic/comparison expressions for generated fields are supported. Validating a draft alone does not create database objects. The legacy generic numeric type remains floating point. Other database engines retain the narrower create-table/add-column support.

Per-connection schema design settings are available as described below. ERP and identity blueprints, automatic slowly changing dimensions, broad schema refactoring and workload-based index recommendations remain planned capabilities.

Schema export also does not make changing database engines automatic. Engine-specific types, constraints, application queries and existing data need a compatibility assessment and migration tests. Use the verified applied schema as the reference for your application; retain the difference between draft, database-applied and Git-published versions.

Connect an existing database

1. Select your tenant and project. Open Connectors → DAL · Managed databases → Add DAL connection.
2. Choose Existing write connection. Find the connection in the list and select Verify write permissions. The test creates a uniquely named probe table, checks insert/read/update/delete permissions and drops the probe table. It does not change your existing tables.
3. A successful test selects that connection automatically. To use another verified connection, select its row. Then open Database & verification. Read-only or unconfigured local connections cannot be selected as writable.
4. Choose Load available databases and select a database. Alternatively, select Create a database on this server and enter a new name. This requires database-creation permission on that server.
5. Leave DAL name empty to use provider_engine_database, or enter your own name. Select Verify writes and connect.

Use Local connector or Regular database connector to create a missing connection through the normal connector setup. Database credentials belong to that connection.

Configure cloud hosting

Open Settings & Monitoring → Hosting accounts → Add hosting account, or add one from the DAL setup. Each account is private to the selected tenant.

Hosting provider	Required configuration
Railway	Project ID, environment ID, project/account token type and API token
AWS	Region, access key ID, secret access key, and session token if using temporary credentials
Azure	Directory tenant, subscription, resource group, location and application client credentials
GCP	Project, region and service-account JSON

The name defaults to the provider name; change it if you keep multiple accounts. Select ⓘ beside any field for setup instructions in a wider dialog. Close it with × or Escape. Select Show beside a credential to inspect the value you are entering, then Hide to mask it again. You can use Test account access before saving, then Test access & discover databases on the saved account. This checks access to the configured scope. It does not yet prove SQL write permission. Credential fields are never filled back into the form; leave them blank on edit to keep the saved values. Changing hosting configuration requires a fresh test and review of affected DAL connections.

Choose Cloud hosting / new service in DAL setup, select the account, test access, and select a discovered database service. Railway resolves its database credentials from the service variables. Existing AWS, Azure and GCP services require a database username and password in addition to cloud credentials.

Find Railway project and environment IDs

An environment is a deployment scope inside a project, usually Production or Staging. Select the intended environment in Railway. Press Cmd+K on macOS or Ctrl+K on Windows/Linux and copy Project ID and Environment ID into their corresponding fields. These are UUIDs, shaped like xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, not an email address or environment name. The form starts with empty IDs; replace any value inserted by your browser with the copied ID. A service's RAILWAY_ENVIRONMENT_ID variable also contains its environment ID. A project token must cover the environment you selected. See Railway API authentication and copying IDs.

Select an AWS region

Enter the IAM access key ID and secret, plus the session token for temporary STS credentials. Select Test credentials & load regions. AgentData requests the regions enabled for that account and selects its configured default when available. Choose another region from the dropdown if needed. Changing credentials clears the list so you can load it again.

Region discovery requires ec2:DescribeRegions. Database discovery additionally requires rds:DescribeDBInstances. Each connector needs permissions for its own service; a successful region check does not prove S3 or SQL access. See AWS region discovery.

Reuse a hosting account for connectors

Open Connectors → Add connector, choose PostgreSQL, MySQL or Amazon Redshift, then select Saved hosting account. The account list shows providers supporting that database type. Select an account, test access, choose Use this service for a matching database, then test and save the connector. S3 account selection is in the S3 folder setup. Hosting accounts are managed separately under Settings & Monitoring.

Account	Connect existing services	Deploy new database services
AWS	S3 file stores, RDS PostgreSQL/MySQL, provisioned Redshift clusters	RDS PostgreSQL/MySQL
Railway	PostgreSQL/MySQL	PostgreSQL/MySQL
Azure	PostgreSQL Flexible Server	PostgreSQL Flexible Server
GCP	Cloud SQL PostgreSQL/MySQL	Cloud SQL PostgreSQL/MySQL

For S3, enter an existing bucket and optional prefix. You can also select Saved AWS hosting account in the S3 folder setup. The connector stores an account reference: rotating the saved AWS credentials applies automatically. S3 listing/reading/writing has separate bucket permissions. The saved-account option uses standard AWS endpoints and tenant-private stores. It does not create a bucket.

Redshift discovery needs redshift:DescribeClusters. Existing provisioned clusters are supported; Redshift Serverless and cluster creation are not part of this screen. Database connectors require database login credentials (Railway can resolve these from its service). SQL connector credentials are stored separately; reconnect the source after rotating its database password.

Create a cloud database service

In DAL setup, Add hosting sits beside Cloud hosting / new service. Select a hosting account, then Test access & discover databases. The result shows how many matching databases were found. A successful test with zero databases still enables Create database service. Choose Use this service for an existing database to continue to database selection and write verification.

After a successful hosting access test, choose Create database service to deploy a new service. Enter the service name and engine, then the required size and network settings. Review these settings and approve the billable resource before selecting Deploy database service.

Service names use 3–63 lowercase letters, numbers and hyphens, starting with a letter and ending with a letter or number (for example, test-db). Spaces, uppercase letters and underscores are not accepted. The form explains missing or invalid fields and can suggest a valid name. Choose Review deployment, check the account and service, confirm the billable resource, then select Deploy database service. Entering a name and choosing an engine alone does not submit a deployment. Select a specific project if the project filter is set to All projects.

• Railway uses an image-backed PostgreSQL or MySQL template in the selected project/environment. Choose Network access: Public TCP lets AgentData and external applications connect using the database password; Private works inside the same Railway project and environment. Public TCP is selected initially unless your deployment profile specifies Private. Review the selected access before deploying. It does not enable GitHub deployment hooks.
• AWS creates PostgreSQL or MySQL RDS instances in your configured subnet and security groups.
• Azure creates PostgreSQL Flexible Servers using your delegated subnet and private DNS zone.
• GCP creates PostgreSQL or MySQL Cloud SQL instances in your existing private network.

AWS, Azure and GCP services use private networking. AgentData must have a network route to the chosen endpoint. If AgentData runs outside that network, arrange connectivity before attempting SQL verification. Provisioning permission, database permission and network reachability are separate requirements.

Deployment history keeps the provider receipt. After submission, a status banner confirms that the request was accepted and the database list refreshes automatically. When the service appears, select it, then choose or create the database and verify writes. Automatic checks stop after a bounded interval; use Test access & discover databases to check again if deployment takes longer. A failed status check does not submit another deployment. A submitted deployment is not a verified DAL connection. If the result is uncertain after a timeout, inspect the provider console and refresh discovery before creating anything else; AgentData blocks automatic duplicate submissions.

AgentData creates resources only when you explicitly approve deployment. Removing an unused hosting account removes its saved configuration, not cloud resources. Accounts referenced by deployments, DAL connections or S3 stores cannot be removed.

Railway service is running but cannot connect

A running service can still have only a private address such as test-tal.railway.internal. That address is reachable from the same Railway project and environment. AgentData running in a different project needs a reachable endpoint.

If public database access is appropriate for your setup, open the database service in Railway, choose Settings → Networking → TCP Proxy, and enter its internal database port (PostgreSQL normally 5432, MySQL normally 3306). Railway supplies a proxy domain and port. Return to AgentData and select Test access & discover databases. AgentData discovers the existing proxy and resolves the database credentials server-side; you do not need to copy a connection string or add a DATABASE_PUBLIC_URL variable. PostgreSQL proxy connections require TLS. See Railway TCP proxy setup.

If the database must remain private, run AgentData in the same project/environment or arrange the required private connectivity. AgentData does not automatically expose an existing database to the internet.

Select Use this service → Database & verification → Verify writes and connect. The service name and database name are different: a service called test-tal may contain a database called railway. Leave the database selector on Configured database to use that existing database. Network, authentication, missing-database and permission failures now show separate explanations. A private-address error means the SQL permission check has not run yet.

Connect an external application

Open Connectors → DAL · Managed databases → Schema & migrations → Connect your application & export schema. Under External application connection, select Reveal credentials. An authorized tenant administrator can see and copy the full database connection string, including its username, password, host, port and database. Copy password copies the password separately. Railway connections use the public TCP endpoint; the endpoint must already exist.

For PostgreSQL the URL has this shape:

postgresql://USERNAME:PASSWORD@PUBLIC_HOST:PUBLIC_PORT/DATABASE?sslmode=require

Use Copy connection string to preserve the correct escaping for special characters in passwords. Put it in your external application's secret/environment settings. The database login determines that application's permissions; create a restricted database user when it needs less access than the DAL administrator.

Hide credentials clears the visible details. They also hide automatically after the displayed interval, when you leave the browser tab, or when you close the panel. Copying does not clear your clipboard. The disclosure is recorded in the audit history. Credentials remain excluded from normal connection lists, schema exports and Git files. Local connector passwords stay on the connector machine.

Use a local connector

The connector can serve folders, read-only databases and a write destination on the same machine. These are separate permissions.

In the installed AgentData Connector application, open database settings and configure Staging · write destination. Test the connection and start the connector. Then select that installation in AgentData. A read-only database configuration or an online heartbeat alone does not enable writes.

Database passwords remain on that machine. Select or change a local database in the connector application. PostgreSQL local connections support schema inspection and additive changes. Other local database engines may provide only partial schema metadata; AgentData shows that limitation rather than allowing unverified schema changes.

Inspect and download the schema

Open Schema & migrations on a managed connection. The panel shows the retained schema, table count, migration history and whether the live database has changed externally. Download schema YAML exports the current inspected schema contract. The retained schema remains available separately, with its original version history. Externally changed structures must be reviewed and accepted before they become the new baseline.

The schema download identifies its connection, version and inspection state, and uses stable IDs for tables and columns. Schema coverage shows which metadata was inspected and which native features need additional support. Exact numeric types, calendar dates, timezone-aware timestamps, defaults, generated columns, keys and relationships are retained. Views, expression indexes and other native features stay in the inventory with explicit limitations; this does not certify a cross-engine conversion. Inspection is scoped to the connection’s current schema; other visible schema names are discovery information. Older retained revisions keep their original format.

For PostgreSQL, review the exact SQL for table/column, constraint or index plans before selecting Apply reviewed SQL. One constraint or index can be added in its own plan. Direct PostgreSQL connections execute a multi-statement plan atomically. Local connectors accept one statement per plan; add a table and its index in separate reviewed plans. These additive plans do not rename or remove existing columns; see the reviewed phased schema-change workflow below. An uncertain execution outcome requires review before another change can run.

Connect your application & export schema shows the connection ID, authenticated schema API, and exact schema/migration paths under your configured repository directory. The application can download the YAML and build its own DAL. It uses its own database runtime credentials.

The database stores schema versions and migration history even without Git. If your tenant repository is configured, AgentData publishes schema YAML under schemas/databases/ and migration SQL under sql/migrations/. Connecting Git later publishes the retained history. Credentials and database row values are not included in these files.

Use a PostgreSQL reviewed plan for exact monetary precision, supported foreign keys/checks and indexes. Unsupported index expressions, arbitrary SQL functions and broader changes require a reviewed external migration followed by schema reconciliation. A native feature listed in Schema coverage is not automatically supported by the migration compiler.

Choose a deployment profile

When you create a new hosted database service, choose a Deployment profile if your organization has published profiles for that provider and database engine. The form shows the profile name, revision and image/version when available. A PostgreSQL profile may declare pgvector support; enabling that extension and verifying it in your database are separate from generating embeddings.

Profile defaults fill the service settings. Review the final values, confirm the billable service, then deploy. The operation records the profile revision and the settings submitted. A profile update affects new deployments; your existing database is not upgraded automatically. If a profile changes while you are reviewing the form, reload the available profiles and review the current revision before submitting.

Choose a schema design mode

Open Connectors → DAL · Managed databases, find your database connection and select Schema & migrations → Schema design policy. The policy belongs to that connection. Choose:

• Transactional / normalized for operational applications such as ERP, orders and customer management.
• Analytics / dimensional (DWH) for facts and dimensions used in reporting. Explain what one fact row represents and which changes need history.
• Mixed operational and analytics when the database needs both, with a role defined for each table.

Add a short business description, then Save. You can also set the management-table prefix, primary-key checks and audit timestamp column names under Design checks. The saved revision appears beside Save. Another administrator's concurrent edit produces a conflict; reload and review their changes before saving again.

Your connected assistant uses the same saved settings for the selected DAL connection. Tell it which connection to manage; different connections can use different modes. You can describe the application in ordinary language and ask it to recommend a mode. For example: “This is an order-management app with customer and order-line tables. Keep operational data normalized, store exact amounts and retain customer changes for reporting.” You do not need database terminology, but the assistant may ask what a row represents, which history matters and how tenants share data.

Changing the mode does not convert or rename existing tables. Schema changes still need a reviewed migration plan. A plan prepared before a policy change must be prepared and reviewed again. New policy errors block planning or applying a PostgreSQL change; resolve them or approve a reasoned exception for that exact finding. Existing schema findings remain visible without forcing unrelated renaming or cleanup.

Select Audit connected schema to inspect the actual database against the saved policy. The report shows errors, warnings and approved exceptions. Severity changes that rule across this connection. Approve exception requires a reason and applies only to the displayed rule and table/column path; Remove exception restores the finding. These changes create normal policy revisions with the administrator and timestamp retained. Save any unsaved policy changes before running the audit. Partial native coverage is shown separately.

Design audits report missing decisions and potential schema problems. They do not prove live permissions, timestamp triggers, SCD processing or application correctness. When a repository is connected, policy revisions are kept under schemas/databases/<connection_id>/policy.yaml and its policy/versions folder. Without Git, revisions remain in AgentData; connecting Git later publishes retained history.

Existing project settings are preserved as initial policies for existing DAL connections. New connections copy any saved defaults once. Changing one connection’s policy does not change other connections.

Test named data operations

Open a DAL connection's Schema & migrations → Non-production data tests. Choose a named contract prepared for a confirmed development or test database. Each contract pins its schema revision, permitted entities, operations and administrators. The environment label is an administrator's explicit declaration; it does not detect whether a database contains production data.

Opening the panel or selecting a contract does not read or change business rows. Choose a readable entity, enter an exact-match JSON filter such as {"id":"order-1"}, then select Read test data. Results are bounded; decimal values remain strings and instants are returned in UTC.

Expand Write a test event to supply an operations array. For example, if the selected contract defines an orders entity with these columns:

[
  {
    "entity": "orders",
    "operation": "upsert",
    "records": [{"id": "order-1", "total": "12.30"}]
  }
]

Use the names, columns and types declared by your own contract. insert, update and upsert need their corresponding grants. Keys must be supplied, non-null and backed by a supported unique constraint. Generated columns are read-only. Decimal strings with excess fractional digits are rejected rather than rounded. An update with no matching key fails explicitly.

Parent and child operations run in the supplied order, in one transaction together with a retry receipt. A failed child, timeout or schema mismatch rolls back the event. After a lost or uncertain response, retry the same event with the same request ID; changing its payload produces a conflict. Select New event ID for a new event. Upsert updates matching rows and does not remove child rows omitted from a later event.

The receipt table must first be created through a reviewed schema migration. Test operations never create or migrate destination tables. Current transaction support requires a direct PostgreSQL connection; local connectors and other engines report an unavailable transaction capability.

Configured document mappings use accepted pinned output tables, including their source references. Flow mappings use complete retained output from a specific request; truncated results and current-target reads cannot be ingested. CSV mappings use declared column types instead of guessing from cells; the default null marker is the literal \N, while an empty text cell remains an empty string. These sources use the same transaction and grant checks as record events.

Contracts and revisions remain in AgentData and are mirrored under schemas/databases/<connection-id>/ingestion/ when tenant Git publication is enabled. Test-event history stores counts, source references and digests rather than the submitted business values. This test surface is separate from a third-party application's direct production database access.

Pin an application to a schema version

Open Schema & migrations → Application schema pins to register the fields and operations an external app needs. Choose a retained schema version, identify the app and its development, staging or production environment, and enter its required entities. For example:

{
  "customers": {
    "columns": ["id", "company"],
    "operations": ["read"],
    "key": ["id"]
  }
}

Use the actual tables, columns and operations declared by your connection. The optional key must be backed by a supported non-null unique key. allow_additional_columns: false declares a strict result shape. strict_constraints defaults to true for write-compatibility checks. Registering an app records its expectations; it does not grant database permissions or certify its runtime behavior.

Download pinned contract returns that exact retained version with its checksum and no credentials. New schema versions do not silently upgrade an app's pin. Update the registered pin only after testing the application against its intended version.

The panel separates the retained schema, latest queued Git version and last published Git version. Check live compatibility inspects the actual database and reports drift against the retained schema. Without that check, compatibility refers to retained metadata. Findings cover declared requirements only; AgentData cannot infer every query an application may execute.

Reviewed migrations include registered consumer impact. Incompatible changes are blocked, and changing a consumer declaration after review requires a new migration plan. Enabled test-data contracts are identified as needing schema rebinding after a change. A migration with an uncertain outcome remains a separate recovery issue. Phased changes are described below. Registering a consumer does not perform a migration or establish that its production backups can be restored.

Change a PostgreSQL schema in stages

Open Connectors → DAL · Managed databases → Schema & migrations. Under Schema design policy → Deployment access, enable reviewed refactors, select the actual deployment environment, enter the database schema name and allow your account to deploy. Save. This permission is tied to the selected database; saving it changes no customer tables.

Open Phased schema changes → Plan a schema change. Choose a column rename, a type/nullability change, or a vertical table split/merge. Enter the source, target and reason. Direct PostgreSQL and a stable primary key are required. Dependencies or database features the operation cannot safely handle produce an explanation before changes run. Inherited or replicated tables, database DDL hooks, local connector transactions and other engines are not supported by this operation.

1. Review and run Expand schema. Both names remain available. Conflicting writes are rejected.
2. Review and run Copy existing values. Progress shows batches and rows copied. Pause copying stops after the current bounded batch; review the phase again to resume. Finish backfill before switching applications to the new column name. Until then, ambiguous unchanged alias writes are rejected rather than silently losing a value.
3. Review and run Verify data compatibility. Update your applications and their registered schema pins to use the new name. Pause dependent data contracts before removing the old name; rebind them to the final schema afterwards.
4. Review and run Remove replaced structure in development or staging. Declared incompatible consumers block removal. Production removal is unavailable until a verified restore-rehearsal path is supported.

An interrupted phase checks its database receipt before repeating work. Refresh progress and review the phase again; do not manually repeat its SQL or accept unrelated schema drift to clear an error. There is no automatic universal rollback. Plan backups and application compatibility before changing a populated database.

The new name retains the field's logical identifier. All four phase scripts and phase state are retained by AgentData. With Git configured, they are mirrored under sql/migrations/<connection>/<migration>/; schema versions remain under schemas/databases/<connection>/. A pending Git commit does not mean the database phase failed. Compatibility checks cover declared consumer requirements, not unknown application queries.

Type/nullability changes use a new column name so old and new applications can overlap. Supported casts stay within the integer, numeric or text family. While both names exist, values must round-trip without loss; a widened value that the old application cannot read is rejected. A NOT NULL target is enforced after all source values have been checked and corrected.

A table split moves selected plain fields to a one-to-one details table with the same single-column primary key and a cascading foreign key to the parent. A merge accepts that exact projection back into its parent. During overlap, writes synchronize both sides. Insert the parent before its details; delete through the parent. Many-to-many splits, arbitrary transformations, primary-key changes and unsupported native dependencies are rejected.

Analytics loading in development and test

A saved data contract can declare a fact grain, measure additivity and conformed dimensions. SCD1 updates the current attributes while retaining the surrogate key. SCD2 preserves half-open effective-time intervals and resolves fact references at the fact’s event time. Send effective timestamps with a timezone. Correcting the same business key and effective time updates that version; a late version splits the affected interval and rebinds matching facts within the configured repair budget.

Each fact needs a unique grain key and real foreign keys to its dimensions. An unknown member is explicitly rejected or stored as NULL in a nullable foreign-key column. Replaying the same event ID and payload returns its receipt; a changed payload requires a new event ID. Upserts at the same grain replace the row instead of adding another measure. Large history repairs stop and roll back the whole packet rather than partially updating it.

Use the existing named data-operation contract and test panel for this workflow. These loaders currently support direct PostgreSQL in development/test only. Measure additivity describes the data contract; it does not rewrite arbitrary SQL or make unsafe fact-to-fact joins valid. The external application can use its own DAL and the exported schema while AgentData is unavailable. Its own database permissions remain authoritative.