Task — engineering-spec@1

"Tenant residency pinning (sg-1 / eu-1 / us-1 / vn-1) propagating to provider region selection"

doneTASK-AI-016
module ai · class product · priority p0 · created 2026-05-15 · shipped 2026-05-21
depends on TASK-AI-006 · blocks TASK-TEN-103, TASK-AI-104

§1 — Description (BCP-14 normative)

The AI Gateway service MUST enforce tenant residency at alias-resolution time. The enforcement and surrounding contract obey the following:

  1. MUST expose residency::matches(policy_residency: Residency, provider_region: &Region) -> bool. Returns true if and only if the provider's region is in the residency's acceptable-region set per the region_table.rs mapping.
  2. MUST define the residency → acceptable-region mapping in region_table.rs as a frozenset per residency; changes to the mapping require explicit task amendment:
  1. MUST be invoked by alias::resolve (TASK-AI-006 §1 #7). When residency::matches(policy.residency, &resolved_region) == false, alias::resolve returns Err(AliasError::ResidencyViolation { policy_residency, resolved_region, attempted_alias }).
  2. MUST treat tenants with a missing policy.ai_policy.residency field as fail-closed for PDPL-pinned tenants (any tenant with policy.tenant_jurisdiction == "VN"); for other tenants the missing-field default is Sg1 (Asia-Pacific consensus default for the CyberSkill home region). The default is documented in TASK-AI-005's schema; this task consumes the parsed value.
  3. MUST strip AZ suffix from the provider-returned region string before matching: "ap-southeast-1a""ap-southeast-1". The matcher operates on AWS region strings (no AZ); AZ-aware policies are out of scope at slice 4. The strip rule is ^(?P<region>[a-z]{2}-[a-z]+-\d+)[a-z]?$ — stripping a single trailing alpha character if present.
  4. MUST return false for Vn1 against ANY provider region in slice 4 (no VN provider integrated). Tenants pinning vn-1 MUST be refused at resolve time; the refusal carries a distinct error message vn1_no_provider_yet so the operator dashboard can distinguish "no VN provider" from "wrong region" failures. TASK-AI-104 (placeholder) will integrate Viettel Cloud + FPT Cloud and add their region strings to the Vn1 set.
  5. MUST emit an ai.residency_violation memory audit row when a request is refused due to residency. The row carries tenant_id, agent_persona, requested_alias, policy_residency, resolved_region, request_id, AND a vn1_no_provider boolean (true when residency is Vn1 and the failure is due to absence of a VN provider rather than wrong region).
  6. MUST propagate ResidencyViolation errors as HTTP 403 RESIDENCY_VIOLATION with body {"error":"residency_violation","policy_residency":"<r>","resolved_region":"<reg>","contact":"ops@cyberos.world"}. For Vn1 failures, the error code is residency_violation AND the body includes "reason":"no_vn_provider_yet" so client UIs can render an informative message.
  7. MUST be deterministic: same (Residency, Region) pair always returns the same boolean. The region_table.rs mapping is a LazyLock<HashMap<Residency, HashSet<&'static str>>>; no I/O, no time-dependent state, no env-var lookup.
  8. MUST integrate with TASK-AI-015 (ZDR enforcement) such that the precedence is: ZDR check first, then residency. A request that fails ZDR is refused with ZdrViolation; a request that passes ZDR but fails residency is refused with ResidencyViolation. Both checks run in alias::resolve; the order is fixed (ZDR before residency) and documented in TASK-AI-006's resolve function.
  9. MUST support a per-alias residency override (policy.ai_policy.residency_override) for tenants that need to pin a specific alias to a different residency than the tenant default. Schema: residency_override: { "<alias-glob>": "<residency>" }. Example: a SG tenant pinning chat.eu-customer-data to eu-1. The override is consulted BEFORE the tenant default; ambiguous overrides (multiple globs match) fail with OverrideAmbiguous.
  10. MUST NOT silently degrade vn-1 to a "closest available" region. Slice 4 explicitly refuses; the alternative (silent fallback to sg-1) is the failure mode that produces a Decree 53 violation on a tenant who pinned vn-1 precisely to avoid out-of-country routing.
  11. SHOULD emit OTel metrics:
  1. SHOULD log at WARN on every Vn1 refusal — vn1 residency refused; TASK-AI-104 Viettel integration needed for tenant=<id> so the operator dashboard can prioritise the VN-provider build.

§2 — Why this design (rationale for humans)

Why is residency the highest-precedence policy gate (after ZDR)? Residency violations are unrecoverable: once data crosses a border, no audit-trail or apology undoes the transfer. Decree 53/2022 (Vietnam) imposes data localisation for VN-resident PII; the fines are denominated in revenue percentage. GDPR Art. 44's cross-border-transfer principle, paired with the limited adequacy decisions (Art. 45), means EU data routed to non-adequate jurisdictions is presumed unlawful. Both regimes treat the violation as occurring at the moment of transfer; there is no remediation that reverses the breach. The cost of an extra HashMap lookup in alias::resolve (microseconds) is trivial vs. the cost of a single mis-routed call (potentially the contract).

Why fail-closed on missing residency for PDPL tenants (§1 #4)? A VN-jurisdiction tenant whose policy file omits residency: is in an indeterminate state — they didn't explicitly pin VN, but the regulator presumes VN data localisation applies. Defaulting to Sg1 (the closest acceptable AP region) is the conservative call: it avoids the Decree 53 risk without forcing every onboarding to explicitly set residency. Tenants who legitimately want EU or US routing must explicitly pin; the default is the safe-but-tight choice. Non-PDPL tenants get the same Sg1 default for operational consistency.

Why static enum (§1 #2) and not config-driven? The four residency values map to legal-jurisdiction categories that change rarely (AWS region launches don't add new residency tiers — they add new acceptable regions within an existing tier). Encoding them as a Rust enum + LazyLock map means the type system enforces "you can't add a residency without amending the task and recompiling." A config-driven approach invites operational drift ("operator added apac-2 to the YAML; nobody noticed it doesn't map to any AWS region"). The trade-off is loss of hot-reloadability for residency itself; given the rarity of changes, this is the right trade.

Why is Vn1 an empty set rather than mapped to ap-southeast-1 (the closest region)? Because mapping it to ap-southeast-1 would convert "Vietnam data residency" into "Singapore data residency" without telling anyone. A tenant pinning vn-1 is making a regulatory statement; satisfying that statement requires a VN-located provider, which we don't have at slice 4. The honest answer is "we can't serve you under this constraint yet" — the dishonest answer is "we'll route to Singapore and hope the regulator doesn't notice." Refusing and waiting for TASK-AI-104 (Viettel/FPT integration) preserves the regulatory contract.

Why strip AZ suffix (§1 #5)? Bedrock and Vertex sometimes return AZ-suffixed region strings (ap-southeast-1a) in error messages or routing metadata. The matcher operates at region granularity (the legal residency unit) — AZ is sub-region and irrelevant to localisation regulation. The single trailing [a-z]? strip handles every AWS AZ format and is unambiguous (no AWS region ends in a single letter; the ambiguity of ap-southeast-1a would only arise if AWS launched a region literally named ap-southeast-1a, which they won't because their convention treats trailing letters as AZ).

Why does residency precedence go AFTER ZDR (§1 #10)? Both are compliance gates; both refuse calls. Order matters because the operator dashboard sees the first-fired error. Putting ZDR first lets a tenant who fails BOTH gates see "ZDR violation" first (the more expensive/restrictive gate is more diagnostic — fixing ZDR often resolves residency, since ZDR-attested providers tend to publish region availability). Putting residency first would obscure ZDR failures behind region failures. The order is documented and tested; future tasks touching this precedence must justify a change.

Why per-alias residency override (§1 #11)? Real-world: a SG-default tenant has one workflow that processes EU customer data ("send EU customer support summary to model X"). They want most calls to route via SG (latency, cost), but THIS specific alias must route via EU. Without the override, the tenant has only two unsatisfactory choices: (a) flip the entire tenant to Eu1 (degrading every other call's latency), or (b) accept the GDPR risk. The override gives them surgical control. The ambiguity-rejection rule (OverrideAmbiguous) prevents the operational footgun of two glob patterns matching the same alias with different residencies.

Why a dedicated audit row (ai.residency_violation)? Same reasoning as ZDR's audit row (TASK-AI-015 §1 #6): a regulator's question "did you ever route VN data outside Vietnam" needs a positive answer (rows showing refusals) rather than absence-of-evidence. The vn1_no_provider field is informational — when a regulator asks "why did you refuse this Vn1 tenant," the row says "because we don't have a VN provider integrated yet" rather than "we refused for some reason."

Why does the metric set include ai_residency_vn1_refused_total{tenant_id}? Trending this metric tells us how badly we need TASK-AI-104 (the Viettel/FPT integration). If five VN tenants refuse 1000 calls/day each due to Vn1 not being mapped, the TASK-AI-104 prioritisation case writes itself. Without the per-tenant breakdown, the demand signal is invisible.

Why a property test (§5)? The matcher's correctness is "no cross-residency pairs return true." A unit test enumerating every (Residency × Region) pair has 4 × ~30 = ~120 cases; manually writing them is tedious and bug-prone. proptest generates random pairs and asserts the global property in 1000+ trials. The property test catches accidental aliasing (e.g., adding a region to Eu1 that's also in Us1 due to copy-paste error) that a happy-path test would miss.


§3 — API contract (formal spec for AI-agent implementers)

Type definitions

// services/ai-gateway/src/residency/mod.rs

use std::collections::{HashMap, HashSet};
use std::sync::LazyLock;
use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum Residency {
    #[serde(rename = "sg-1")] Sg1,
    #[serde(rename = "eu-1")] Eu1,
    #[serde(rename = "us-1")] Us1,
    #[serde(rename = "vn-1")] Vn1,
}

#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Region(String);   // newtype to prevent string-typing accidents

impl Region {
    /// Strip AZ suffix per §1 #5.
    pub fn from_provider_string(raw: &str) -> Result<Self, RegionParseError> {
        static RE: LazyLock<regex::Regex> = LazyLock::new(|| {
            regex::Regex::new(r"^(?P<region>[a-z]{2}-[a-z]+-\d+)[a-z]?$").unwrap()
        });
        let caps = RE.captures(raw).ok_or_else(|| RegionParseError::Invalid(raw.into()))?;
        Ok(Region(caps.name("region").unwrap().as_str().into()))
    }
    pub fn as_str(&self) -> &str { &self.0 }
}

pub fn matches(policy_residency: Residency, provider_region: &Region) -> bool {
    REGIONS_BY_RESIDENCY.get(&policy_residency)
        .map(|set| set.contains(provider_region.as_str()))
        .unwrap_or(false)
}

pub fn parse_residency(s: &str) -> Result<Residency, ResidencyParseError> {
    serde_yaml::from_str(s).map_err(|e| ResidencyParseError::Invalid(e.to_string()))
}

#[derive(Debug, thiserror::Error)]
pub enum ResidencyParseError {
    #[error("invalid residency value {0:?}; expected sg-1 | eu-1 | us-1 | vn-1")]
    Invalid(String),
}

#[derive(Debug, thiserror::Error)]
pub enum RegionParseError {
    #[error("invalid region string {0:?}; expected AWS region format")]
    Invalid(String),
}

// In TASK-AI-006 alias.rs (modified_files):
pub enum AliasError {
    // ... existing variants from TASK-AI-006 + TASK-AI-015 ...
    ResidencyViolation {
        policy_residency: Residency,
        resolved_region: Region,
        attempted_alias: String,
        vn1_no_provider: bool,
    },
}

Region table (single source of truth)

// services/ai-gateway/src/residency/region_table.rs

use std::collections::{HashMap, HashSet};
use std::sync::LazyLock;
use super::Residency;

/// §1 #2: residency → acceptable-region mapping. Changes require task amendment.
pub static REGIONS_BY_RESIDENCY: LazyLock<HashMap<Residency, HashSet<&'static str>>> =
    LazyLock::new(|| {
        let mut m = HashMap::new();
        m.insert(Residency::Sg1, {
            let mut s = HashSet::new();
            s.insert("ap-southeast-1");        // Singapore
            s
        });
        m.insert(Residency::Eu1, {
            let mut s = HashSet::new();
            s.insert("eu-central-1");          // Frankfurt
            s.insert("eu-west-1");             // Ireland
            s
        });
        m.insert(Residency::Us1, {
            let mut s = HashSet::new();
            s.insert("us-east-1");             // N. Virginia
            s.insert("us-east-2");             // Ohio
            s.insert("us-west-2");             // Oregon
            s
        });
        m.insert(Residency::Vn1, HashSet::new());   // §1 #6: empty until TASK-AI-104
        m
    });

Override parser

// services/ai-gateway/src/residency/parse.rs

use globset::{Glob, GlobMatcher};

pub struct ResidencyOverride {
    pub alias_pattern: GlobMatcher,
    pub residency: Residency,
}

#[derive(Debug, thiserror::Error)]
pub enum OverrideError {
    #[error("ambiguous override: aliases [{0:?}] both match alias {1!r}")]
    OverrideAmbiguous(Vec<String>, String),
    #[error("invalid glob pattern {0!r}: {1}")]
    InvalidGlob(String, String),
}

pub fn resolve_override(
    overrides: &[ResidencyOverride], alias: &str,
) -> Result<Option<Residency>, OverrideError> {
    let matches: Vec<_> = overrides.iter()
        .filter(|o| o.alias_pattern.is_match(alias))
        .collect();
    match matches.len() {
        0 => Ok(None),
        1 => Ok(Some(matches[0].residency)),
        _ => Err(OverrideError::OverrideAmbiguous(
            matches.iter().map(|o| o.alias_pattern.glob().to_string()).collect(),
            alias.into(),
        )),
    }
}

Tenant policy schema extension (TASK-AI-005)

# tenants/<tenant_id>/policy.yaml — additions
ai_policy:
  residency: sg-1                          # default residency for this tenant
  residency_override:                      # per-alias override (§1 #11)
    "chat.eu-customer-*":  eu-1            # any chat alias starting with eu-customer- pins EU
    "embeddings.gdpr-pii": eu-1            # specific embeddings alias pins EU

§4 — Acceptance criteria (testable, ordered, numbered)

  1. Sg1 → ap-southeast-1 matchesmatches(Sg1, &Region("ap-southeast-1".into())) returns true.
  2. Sg1 → us-east-1 mismatchesmatches(Sg1, &Region("us-east-1".into())) returns false.
  3. Eu1 → eu-central-1 AND eu-west-1 match — both regions return true.
  4. Us1 → all three US regions matchus-east-1, us-east-2, us-west-2 all return true.
  5. Vn1 → empty set always returns falsematches(Vn1, &Region("ap-southeast-1".into())) returns false.
  6. AZ-suffix strippedRegion::from_provider_string("ap-southeast-1a") returns Region("ap-southeast-1"); matches(Sg1, &that) returns true.
  7. Invalid region string rejectedRegion::from_provider_string("not-a-region") returns Err(RegionParseError::Invalid).
  8. Property test: no cross-residency leak — proptest 1000 trials over (any_residency, any_region): if matches(R, region) returns true, then region.as_str() is in REGIONS_BY_RESIDENCY[R]. No accidental aliasing between residencies.
  9. TASK-AI-006 integration: refusal on mismatch — Tenant policy residency: sg-1, alias resolves to bedrock us-east-1alias::resolve returns Err(AliasError::ResidencyViolation { policy_residency: Sg1, resolved_region: Region("us-east-1"), .. }).
  10. TASK-AI-006 integration: success on match — Tenant policy residency: sg-1, alias resolves to bedrock ap-southeast-1alias::resolve returns Ok((Bedrock, "claude-3-...", Region("ap-southeast-1"))).
  11. Vn1 refusal carries vn1_no_provider flag — Tenant policy residency: vn-1; any alias resolution returns Err(AliasError::ResidencyViolation { vn1_no_provider: true, .. }); OTel ai_residency_vn1_refused_total{tenant_id} increments.
  12. HTTP 403 RESIDENCY_VIOLATION on refusal — Handler converts AliasError to a 403 response with the documented body shape; Vn1 case includes "reason":"no_vn_provider_yet".
  13. Audit row emitted — Every refusal emits exactly one ai.residency_violation memory row with all required fields populated.
  14. Missing residency field defaults to Sg1 (non-PDPL tenant) — Tenant policy without residency field; TASK-AI-005 schema parser defaults to Sg1; alias::resolve enforces against Sg1.
  15. Missing residency field for PDPL tenant fails closed — Tenant with tenant_jurisdiction: VN but no residency field; TASK-AI-005 returns PolicyError::MissingResidencyForPdplTenant; HTTP 503 POLICY_INVALID.
  16. Per-alias override applies before tenant default — Policy residency: sg-1 + residency_override: { "chat.eu-customer-*": eu-1 }; resolving chat.eu-customer-summary enforces against Eu1, not Sg1.
  17. Ambiguous override rejected at parse — Two glob patterns match the same alias with different residencies; resolve_override returns Err(OverrideAmbiguous); TASK-AI-005 policy load fails with PolicyError::AmbiguousResidencyOverride.
  18. ZDR check fires before residency check — Tenant policy zdr_required: true + residency: sg-1; alias resolves to openai gpt-4o (non-ZDR, non-SG); alias::resolve returns Err(AliasError::ZdrViolation), NOT ResidencyViolation — ZDR is the first-fired error.

§5 — Verification

Happy + matcher tests

// services/ai-gateway/tests/residency_property_test.rs
use cyberos_ai_gateway::residency::{self, Residency, Region};

#[test]
fn sg1_accepts_apse1_only() {
    let region = Region::from_provider_string("ap-southeast-1").unwrap();
    assert!(residency::matches(Residency::Sg1, &region));
    let region = Region::from_provider_string("us-east-1").unwrap();
    assert!(!residency::matches(Residency::Sg1, &region));
}

#[test]
fn eu1_accepts_central_and_west() {
    for r in &["eu-central-1", "eu-west-1"] {
        assert!(residency::matches(Residency::Eu1, &Region::from_provider_string(r).unwrap()));
    }
}

#[test]
fn us1_accepts_all_three_us_regions() {
    for r in &["us-east-1", "us-east-2", "us-west-2"] {
        assert!(residency::matches(Residency::Us1, &Region::from_provider_string(r).unwrap()));
    }
}

#[test]
fn vn1_empty_set_always_returns_false() {
    for r in &["ap-southeast-1", "us-east-1", "eu-central-1"] {
        let region = Region::from_provider_string(r).unwrap();
        assert!(!residency::matches(Residency::Vn1, &region));
    }
}

#[test]
fn az_suffix_stripped() {
    let region = Region::from_provider_string("ap-southeast-1a").unwrap();
    assert_eq!(region.as_str(), "ap-southeast-1");
    assert!(residency::matches(Residency::Sg1, &region));
}

#[test]
fn invalid_region_string_rejected() {
    let err = Region::from_provider_string("not-a-region").expect_err("invalid");
    assert!(matches!(err, residency::RegionParseError::Invalid(_)));
}

#[test]
fn parse_residency_from_yaml() {
    assert_eq!(residency::parse_residency("sg-1").unwrap(), Residency::Sg1);
    assert_eq!(residency::parse_residency("vn-1").unwrap(), Residency::Vn1);
    assert!(residency::parse_residency("apac-2").is_err());
}

Property test

// services/ai-gateway/tests/residency_property_test.rs
use proptest::prelude::*;
use cyberos_ai_gateway::residency::{self, Residency, Region, region_table::REGIONS_BY_RESIDENCY};

fn any_residency() -> impl Strategy<Value = Residency> {
    prop_oneof![Just(Residency::Sg1), Just(Residency::Eu1), Just(Residency::Us1), Just(Residency::Vn1)]
}

fn any_region() -> impl Strategy<Value = Region> {
    let known_regions = [
        "ap-southeast-1", "ap-southeast-2", "ap-northeast-1",
        "eu-central-1", "eu-west-1", "eu-west-2", "eu-north-1",
        "us-east-1", "us-east-2", "us-west-1", "us-west-2",
        "ca-central-1", "sa-east-1",
    ];
    prop::sample::select(known_regions.to_vec())
        .prop_map(|r| Region::from_provider_string(r).unwrap())
}

proptest! {
    #![proptest_config(ProptestConfig::with_cases(1000))]

    #[test]
    fn no_cross_residency_leak(r in any_residency(), region in any_region()) {
        // AC #8: matches(R, region) → true ⇒ region in REGIONS_BY_RESIDENCY[R]
        if residency::matches(r, &region) {
            let allowed = REGIONS_BY_RESIDENCY.get(&r).unwrap();
            prop_assert!(allowed.contains(region.as_str()),
                        "cross-residency leak: {r:?} matched {region:?} but region not in allowed set");
        }
    }

    #[test]
    fn deterministic(r in any_residency(), region in any_region()) {
        // §1 #9: same pair → same result, run twice.
        let r1 = residency::matches(r, &region);
        let r2 = residency::matches(r, &region);
        prop_assert_eq!(r1, r2);
    }
}

Integration test (TASK-AI-006 wiring)

// services/ai-gateway/tests/residency_integration_test.rs
use cyberos_ai_gateway::{alias, residency::Residency};

#[tokio::test]
async fn alias_resolve_refuses_on_residency_mismatch() {
    let policy = test_policy_with_residency(Residency::Sg1);
    let result = alias::resolve("chat.us-only-alias", &policy).await;   // routes to us-east-1
    match result {
        Err(alias::AliasError::ResidencyViolation { policy_residency, resolved_region, vn1_no_provider, .. }) => {
            assert_eq!(policy_residency, Residency::Sg1);
            assert_eq!(resolved_region.as_str(), "us-east-1");
            assert!(!vn1_no_provider);
        }
        _ => panic!("expected ResidencyViolation"),
    }
}

#[tokio::test]
async fn alias_resolve_succeeds_on_residency_match() {
    let policy = test_policy_with_residency(Residency::Sg1);
    let (provider, model, region) = alias::resolve("chat.smart", &policy).await.unwrap();
    assert_eq!(region.as_str(), "ap-southeast-1");
}

#[tokio::test]
async fn vn1_carries_no_provider_flag() {
    let policy = test_policy_with_residency(Residency::Vn1);
    let result = alias::resolve("chat.smart", &policy).await;
    match result {
        Err(alias::AliasError::ResidencyViolation { vn1_no_provider: true, .. }) => {}
        _ => panic!("expected vn1_no_provider"),
    }

    let counter = otel_test_helper::counter_value(
        "ai_residency_vn1_refused_total",
        &[("tenant_id", &policy.tenant_id)],
    );
    assert!(counter >= 1);
}

#[tokio::test]
async fn audit_row_emitted_on_residency_refusal() {
    let request_id = "req_test_residency_001";
    let _ = handlers::chat::handle(test_request_with_residency(
        Residency::Sg1, request_id, "chat.us-only-alias",
    )).await;
    let rows = memory_test_helper::find_rows("ai.residency_violation", request_id);
    assert_eq!(rows.len(), 1);
    let p = &rows[0].payload;
    assert_eq!(p["policy_residency"], "sg-1");
    assert_eq!(p["resolved_region"], "us-east-1");
    assert_eq!(p["vn1_no_provider"], false);
}

#[tokio::test]
async fn zdr_check_fires_before_residency() {
    // §1 #10: ZDR before residency.
    let policy = test_policy_with_residency_and_zdr(Residency::Sg1, /* zdr_required */ true);
    let result = alias::resolve("chat.openai-us-only", &policy).await;   // openai gpt-4o, us-east-1, non-ZDR
    match result {
        Err(alias::AliasError::ZdrViolation { .. }) => {}   // ZDR fires first
        Err(alias::AliasError::ResidencyViolation { .. }) => panic!("residency fired before ZDR"),
        _ => panic!("expected error"),
    }
}

#[tokio::test]
async fn per_alias_override_wins_over_tenant_default() {
    let policy = test_policy_with_override(
        Residency::Sg1,   // default
        vec![("chat.eu-customer-*".into(), Residency::Eu1)],
    );
    let result = alias::resolve("chat.eu-customer-summary", &policy).await;
    // Without override, this would resolve to SG region; with override, must resolve to EU.
    let (_, _, region) = result.unwrap();
    assert!(region.as_str().starts_with("eu-"));
}

#[tokio::test]
async fn ambiguous_override_rejected_at_policy_load() {
    let yaml = r#"
        ai_policy:
          residency: sg-1
          residency_override:
            "chat.*-customer-*": eu-1
            "chat.eu-*":         us-1
    "#;
    let err = policy::parse(yaml).expect_err("expected AmbiguousResidencyOverride");
    assert!(matches!(err, policy::PolicyError::AmbiguousResidencyOverride { .. }));
}
cd services/ai-gateway
cargo test -p cyberos-ai-gateway residency
cargo test -p cyberos-ai-gateway --test residency_property_test

§6 — Implementation skeleton

See §3 for the type defs + region table + override parser. Integration into TASK-AI-006's resolve:

// services/ai-gateway/src/alias.rs (modified)

pub async fn resolve(alias: &str, policy: &TenantPolicy) -> Result<(ProviderKind, String, Region), AliasError> {
    // Resolve alias → (provider, model, region)
    let (provider, model, region) = ALIAS_MAP.get().unwrap().load().get(alias)
        .ok_or_else(|| AliasError::UnknownAlias(alias.into()))?;

    // §1 #10: ZDR before residency.
    if policy.ai_policy.zdr_required && !zdr::is_zdr(&provider, &model) {
        return Err(AliasError::ZdrViolation {
            resolved_provider: provider, resolved_model: model.clone(),
            attestation: zdr::attestation_for(&provider, &model),
        });
    }

    // §1 #11: per-alias override resolves first.
    let effective_residency = match parse::resolve_override(&policy.ai_policy.residency_override, alias) {
        Ok(Some(r)) => r,
        Ok(None) => policy.ai_policy.residency,   // tenant default
        Err(e) => return Err(AliasError::OverridePolicyInvalid(e.to_string())),
    };

    // §1 #1, §1 #5: matcher with AZ-strip.
    if !residency::matches(effective_residency, &region) {
        let vn1_no_provider = effective_residency == Residency::Vn1;
        return Err(AliasError::ResidencyViolation {
            policy_residency: effective_residency,
            resolved_region: region.clone(),
            attempted_alias: alias.into(),
            vn1_no_provider,
        });
    }

    Ok((provider, model.clone(), region.clone()))
}

canonical::residency_violation builder:

pub mod canonical {
    pub fn residency_violation(
        tenant_id: &str, agent_persona: &str, requested_alias: &str,
        policy_residency: Residency, resolved_region: &Region,
        vn1_no_provider: bool, request_id: &str,
    ) -> AuditRow {
        AuditRow {
            kind: "ai.residency_violation".into(),
            payload: serde_json::json!({
                "tenant_id": tenant_id,
                "agent_persona": agent_persona,
                "requested_alias": requested_alias,
                "policy_residency": serde_yaml::to_string(&policy_residency).unwrap().trim(),
                "resolved_region": resolved_region.as_str(),
                "vn1_no_provider": vn1_no_provider,
                "request_id": request_id,
            }),
            ..Default::default()
        }
    }
}

Handler refusal path:

// services/ai-gateway/src/handlers/chat.rs (modified)
match alias::resolve(&req.alias, &policy).await {
    Err(AliasError::ResidencyViolation { policy_residency, resolved_region, vn1_no_provider, attempted_alias }) => {
        memory_writer::emit(canonical::residency_violation(
            &policy.tenant_id, &req.agent_persona, &attempted_alias,
            policy_residency, &resolved_region, vn1_no_provider, &req.request_id,
        )).await?;
        metrics::residency_mismatch(policy_residency, &resolved_region);
        if vn1_no_provider {
            metrics::vn1_refused(&policy.tenant_id);
            tracing::warn!(tenant_id=%policy.tenant_id,
                          "vn1 residency refused; TASK-AI-104 Viettel integration needed");
        }
        let body = serde_json::json!({
            "error": "residency_violation",
            "policy_residency": serde_yaml::to_string(&policy_residency).unwrap().trim(),
            "resolved_region": resolved_region.as_str(),
            "reason": if vn1_no_provider { Some("no_vn_provider_yet") } else { None },
            "contact": "ops@cyberos.world",
        });
        return Err(ApiError::Forbidden(body));
    }
    // ... other error variants ...
}

§7 — Dependencies

Code dependencies (other tasks/modules)

Concept dependencies (shared types)

Operational / external


§8 — Example payloads

Tenant policy (with override)

# tenants/tenant_alpha/policy.yaml
tenant_id: tenant_alpha
tenant_jurisdiction: VN
ai_policy:
  residency: sg-1                          # default for this VN-jurisdiction tenant
  residency_override:
    "chat.eu-gdpr-*": eu-1                 # specific EU-data aliases pin EU
  zdr_required: true

Caller in TASK-AI-006 alias.rs

let (provider, model, region) = alias::resolve(&req.alias, &policy).await?;
// region.as_str() guaranteed in REGIONS_BY_RESIDENCY[policy.ai_policy.residency]
//                  OR in REGIONS_BY_RESIDENCY[override-resolved residency]

Audit row ai.residency_violation

{
  "kind": "ai.residency_violation",
  "ts_ns": 1747526400000000000,
  "payload": {
    "tenant_id": "tenant_alpha",
    "agent_persona": "cuo-cpo@0.4.1",
    "requested_alias": "chat.us-only-alias",
    "policy_residency": "sg-1",
    "resolved_region": "us-east-1",
    "vn1_no_provider": false,
    "request_id": "req_01HZK9R8M3X5C8Q4"
  }
}

HTTP refusal (generic mismatch)

HTTP/1.1 403 Forbidden
Content-Type: application/json

{
  "error": "residency_violation",
  "policy_residency": "sg-1",
  "resolved_region": "us-east-1",
  "contact": "ops@cyberos.world"
}

HTTP refusal (Vn1 no-provider)

HTTP/1.1 403 Forbidden
Content-Type: application/json

{
  "error": "residency_violation",
  "policy_residency": "vn-1",
  "resolved_region": "ap-southeast-1",
  "reason": "no_vn_provider_yet",
  "contact": "ops@cyberos.world"
}

Override-applied trace log

INFO  alias=chat.eu-gdpr-summary tenant_default=sg-1 override_matched=eu-1
      residency_override applied

Ambiguous-override policy load failure

ERROR policy file tenants/tenant_beta/policy.yaml load failed:
      AmbiguousResidencyOverride { aliases: ["chat.*-customer-*", "chat.eu-*"], for_alias: "chat.eu-customer-summary" }

§9 — Open questions

All resolved at authoring time. Items deferred to later tasks:


§10 — Failure modes inventory

FailureDetectionOutcomeRecovery
Tenant pins vn-1 but no VN providerVn1 empty set always returns falseErr(ResidencyViolation { vn1_no_provider: true })403 with reason: no_vn_provider_yetTenant chooses sg-1 (closest acceptable) OR waits for TASK-AI-104
New AWS region not in matcherRegion returns false (fail closed)Err(ResidencyViolation) for affected residenciesOperator adds region to region_table.rs; task amendment; redeploy
Property test detects cross-residency leakproptest panics in CIPR blockedFix region_table.rs (likely accidental aliasing across residencies)
Region string with AZ suffixRegion::from_provider_string strips suffixMatch proceeds against region-only stringBy design (§1 #5)
Region string in unknown formatRegionParseError::Invalid from regexErr propagates; alias resolve fails with RegionParseError (not residency violation)Operator investigates provider response; likely a provider API change
Tenant policy without residency field, non-PDPLTASK-AI-005 schema default → Sg1Enforce against Sg1Operator updates policy if non-default residency intended
Tenant policy without residency field, PDPL tenantTASK-AI-005 schema rejectsPolicyError::MissingResidencyForPdplTenant → 503 at policy loadOperator must explicitly pin residency in policy
Tenant policy with residency: apac-2 (invalid)parse_residency rejectsResidencyParseError::Invalid → policy load failsOperator fixes value to one of sg-1, eu-1, us-1, vn-1
Per-alias override glob matches no aliasesresolve_override returns None → fall back to defaultTenant default residency appliesBy design
Per-alias override glob matches multiple aliases ambiguouslyresolve_overrideOverrideAmbiguousPolicyError::AmbiguousResidencyOverride → 503 at policy loadOperator narrows glob patterns
Per-alias override glob has invalid syntaxGlob::new parse errorPolicyError::InvalidOverrideGlob → 503 at policy loadOperator fixes glob pattern
ZDR + residency both failPrecedence: ZDR fires first (§1 #10)Err(ZdrViolation) returned; residency check never runsBy design; operator sees ZDR error first
Audit row emit fails (memory bridge down)memory_writer::emit returns ErrRefusal still proceeds; sev-1 log "residency refused but audit row failed"Operator investigates memory; TASK-AI-003 §10 covers
Concurrent matches() callsLazyLock + immutable HashSetAll readers see same resultBy design (§1 #9 deterministic)
Region table mutation attempted at runtimeLazyLock prevents writesCompile error if attemptedBy design
TASK-AI-104 lands but Vn1 set still emptyTest test_vn1_set_populated_after_task_ai_104 failsCI blocked on TASK-AI-104 PRTASK-AI-104 must extend the table as part of its acceptance
Alias is in alias map but its region is invalid formatRegion::from_provider_string fails at alias-load timeAlias load fails; alias unavailableOperator fixes alias map entry
Tenant overrides residency to a value with no acceptable providerOverride resolves to (e.g.) Vn1 for an alias whose only region is us-east-1ResidencyViolation with override-resolved residencyOperator removes override OR waits for VN provider
Tenant onboarding skipped residency selectionTASK-AI-005 onboarding wizard enforces residencyCannot complete onboarding without selectionBy design
OTel metric label cardinality explosion (per-tenant_id × per-region)Cardinality monitoringMetric drop / sampleOperator scales OTel collector OR aggregates labels

§11 — Notes


End of TASK-AI-016. Status: draft (10/10 target).