Data placement

Where your app's data lives

Every table your app owns has exactly one place it lives. There are two kinds of place, and a table belongs to one of them:

Three rules follow from "one place per table":

• No setting chooses between locations. You never configure "read from here or from there". The code reads and writes the table's place and nothing else.
• No fallback. If a place is missing, empty or unreachable, the app refuses to start or refuses the request. It does not quietly use another database. An app that answered every query with empty results because it was reading the wrong database is the failure this rule prevents.
• No cross-database joins. A query touches one database. When a route needs data from both places, it reads each and combines the results in code.

Transition note. The platform is moving first-party services to this model, and the app manifest still accepts older storage modes while that happens. See What the manifest accepts today before you pick a spec.tenancy.storage value.

Getting a session

Your app does not build engines or open connections for its own data. It takes a session from a dependency, hands it to the service that needs it, and opens the transaction in the route.

The dependencies are in egav_base.homes (egav-base 1.21.14 or later):

Import them as a module (from egav_base import homes). They are not re-exported from the top-level egav_base namespace.

A route that reads the catalog and writes organization data:

from egav_base import BaseResponse, async_transaction_context, homes
from fastapi import APIRouter, Depends
from sqlalchemy.ext.asyncio import AsyncSession

router = APIRouter()

@router.post("/runs", response_model=BaseResponse)
async def start_run(
    body: RunCreate,
    catalog: AsyncSession = Depends(homes.get_catalog_session),
    org: AsyncSession = Depends(homes.get_org_session),
):
    # Read the catalog: no transaction needed.
    tool = await ToolService(catalog).get_item(body.tool_guid)

    # Write to one home, inside a transaction opened here, in the route.
    async with async_transaction_context(org):
        run = await RunService(org).create_item(body, tool=tool)

    return BaseResponse(data=RunResponse.model_validate(run).model_dump())

The rules for this shape:

• The service takes its home's session in its constructor. ToolService(catalog), RunService(org). A service that needs both homes takes two sessions.
• The transaction lives in the route. Services and CRUD classes never commit, never open a transaction, never create an engine or session, and never cache a session.
• A transaction touches one home. If one action must change both, commit to one home and send the other change through the outbox so a failure half way cannot leave the two out of step. A session from get_catalog_session is marked as a catalog session, so with egav-outbox 0.30.2 or later an outbox row written on it wakes the catalog drain.
• Do not hold a connection while waiting on another from the same pool. Do not call your own API over HTTP from inside a request to read your own catalog data. The inner request waits for a connection that the outer request is still holding, and the service deadlocks. Use the catalog session you already have.
• Keep sessions short. Take the session, do the work, let the dependency close it.

Where the organization comes from

get_org_session takes the organization from the verified token, never from input. It has no organization argument. A header, query parameter, body field or path segment cannot select the database. Background workers that have no request bind the organization themselves from their own job row, using trusted_org_scope(org_guid) from egav_base.trusted_org_context.

It refuses the request, with no fallback to a shared database, when:

• there is no verified organization on the request;
• the organization has no active storage binding for your service in this region;
• the binding is in another region than the one the process serves, or the process declares no region (only the local development value skips the region check);
• the organization's database is not yet at your service's current schema version. The request gets a 503, and a background upgrade of that database starts so it heals without a restart.

Start-up check for the catalog

An empty catalog answers every query with a valid, empty result, and nothing fails. To catch it, call the readiness check from your app's lifespan, after any seeding, before the app takes traffic:

from egav_base.catalog_home import assert_catalog_home_ready_async

await assert_catalog_home_ready_async(
    catalog_url=settings.CATALOG_URL,
    service_url=settings.DATABASE_URI,
    required_tables=["tool", "provider"],   # tables your own seeding guarantees are non-empty
    engine=homes.get_catalog_engine(),
)

It raises CatalogHomeNotReady and the service does not start when:

• the catalog address is not configured;
• the catalog address names the same database as service_url;
• the catalog database has no row in its catalog_version table (its schema was never built);
• any table you listed in required_tables has no rows.

Error messages carry table names only, never a host, port, user or password. There is also a synchronous assert_catalog_home_ready for code without an event loop.

How the platform provisions each place

Catalog

Your manifest asks for a catalog database by declaring two variables in spec.required_env, both with source: db:

spec:
  required_env:
  - { name: CATALOG_URL, source: db, source_field: catalog_url, secret: true }
  - { name: CATALOG_HOME_REGION, source: db, source_field: catalog_home_region }

Declaring one without the other is refused when the platform is composed. The platform derives which services get a catalog database from these declarations; there is no separate list.

What the platform builds:

• In the home region, one catalog database and login role per declaring service, on the shared database instance. CATALOG_URL points at it. This is the only copy that accepts writes.
• In every other region, a read-only physical copy of the home catalog. CATALOG_URL there points at that region's copy. A regional stack never creates a catalog writer.
• CATALOG_HOME_REGION is compared with the region the process serves (PLATFORM_REGION_CODE) to decide whether a process may write the catalog. Whether a write is allowed is never a second database address.

The older CATALOG_WRITE_URL / CATALOG_READ_URL pair is retired. A manifest that still declares either is refused.

Behaviour you can rely on, from egav_base.catalog_db:

• A catalog write from outside the home region is refused with CatalogWriteRefused. It is not redirected to the home region.
• A process outside the home region reads its local copy and never opens a connection to the home region on the request path. If the local copy is unreachable, the read fails with a 503 rather than reaching across regions.
• Catalog migrations run only against the home writer. A migration pointed at a copy is refused before it connects.
• Every catalog edit bumps the single row in the catalog_version table in the same transaction, so a region can tell whether its copy has caught up.
• AsyncCatalogReader (egav_base.catalog_session) gives read-only catalog sessions for read paths. A write through it fails in Postgres.

Org

Organization databases are provisioned per organization, not per app instance. Each is recorded as an org storage binding in the Control Plane. When your service opens an org session, egav-base asks the Control Plane to resolve the binding for your service, the verified organization and the region. The answer names the database and a reference to its connection secret, never a password; the secret itself is read from the secrets store by your process.

Two properties matter when you design around it:

• A service can only resolve its own bindings. The Control Plane derives the service from the caller's registered key, so the Comments service cannot ask for the Audit service's database.
• If there is no active binding (not yet created, still provisioning, being migrated, scheduled for drop), the Control Plane answers 404 and your service refuses the request. It never falls back to a shared database.

In the manifest, spec.tenancy.storage: per_org_db means one database per organization per region for your service, with the organization's app instances separated by an app_instance_guid column. A per_org_db manifest that also composes a per-instance logical database is refused; drop the per-instance database isolation option.

Which services get org databases today. The Control Plane keeps a fixed list of services that are provisioned org databases: Audit, Comments, Notifications, Support, AI, AuthZ and the Control Plane itself. Declaring per_org_db in a manifest does not by itself add a service to that list.

Migrations: one chain per place

Each place has its own migration chain and its own version table, so the histories never mix.

How each chain is applied:

• The catalog chain runs at startup from the service's run_migrations.sh, only in the home region. The catalog env.py takes its address from CATALOG_URL through egav_base.catalog_db.catalog_write_url, then calls assert_catalog_migration_target, which refuses a copy or a read-only session before any DDL.
• The org chain is applied to every organization database by the startup fan-out (alembic-fanout --source org-storage, in egav_base.alembic_fanout_cli), which lists your service's bindings from the Control Plane. An organization created after startup is healed on its first request by the schema check described above, so a new organization does not wait for a restart.
• The org chain has no default URL. An unset URL is refused rather than pointed at the service database.

Write revisions with frozen SQL. A revision file must not import egav_* or app.* packages, because the revision has to mean the same schema change forever, however those packages change.

Reading another app's data

Your database is yours. Another app's data is reached through that app, never through its database:

1. If another service owns the data, call its API, or keep a local read model fed by its events. Never open a session on another service's database. Use the address the platform configures for it rather than hardcoding one. See Events for subscribing to what it emits.
2. If it is your own catalog data, use your catalog session. Do not call your own API to read it.
3. If it is your own organization data, use the org session.

Moving a table from one place to another

A move is one ordered change. It is never a runtime setting.

1. Copy the data to its new place with an idempotent tool, and verify the row counts.
2. Release once. In a single release, move the reads and the writes together, and add the start-up check so an empty or mismatched place stops the service.
3. Verify the feature on real data, then drop the old copy.

Do not ship a switch that flips readers first, dual-writes, or reads from "the old place if the new one is empty". A plan that contains a second path is the failure it is trying to avoid: if the reader moves and the writer does not, every query returns an empty, valid answer and nothing reports an error.

What the manifest accepts today

spec.tenancy.storage is validated against three values in the current app SDK schema:

shared_with_filter is still accepted by the schema, and several first-party services still declare it while their tables move to a catalog or org place. It does not give you either place, and it is not the direction of the platform. Do not choose it for new tables. Plan new data as catalog or org data, and see Tenancy & deployment for the isolation options each storage mode supports.

Optionally, spec.tenancy.data_graphs[] names a group of tables that is placed differently from storage. A table listed in a graph takes that graph's storage; every other table follows storage; a table can be in at most one graph.

Checklist

• Every table is either catalog or org data, and your code can say which.
• Routes take sessions from homes.get_catalog_session / homes.get_org_session; services take the session in their constructor; no service commits.
• No route writes to two databases in one transaction.
• No config value, environment variable or try/except picks a different database.
• spec.required_env declares CATALOG_URL and CATALOG_HOME_REGION if you have a catalog.
• The lifespan calls assert_catalog_home_ready_async with the tables your seeding fills.
• Catalog, org and service migrations are separate chains, and the catalog chain runs in the home region only.
• Other apps' data comes from their API or events.

For the manifest keys, see The app manifest and Tenancy & deployment. For the shared packages, see App SDKs.