SentinelAI — Tailoring & Learning (design)

How SentinelAI adapts to an organization and its departments. The model stays one shared, general-purpose SLM (Llama 3.2 3B today); adaptation is overwhelmingly config + prompt, with model training a rare, explicit last resort.

TL;DR — department differences = config/policy routing, not per-department models. "Learning from corrections" = feedback → few-shot (semi-automatic), not auto-fine-tuning.

1. The tailoring ladder (what changes behavior)

Rule of thumb: climb only as far as you must. Most org/department nuance is solved at the prompt policy layer; few-shot covers "it should learn from us"; fine-tuning is for scale we haven't hit.

2. Learning from analyst feedback (H3b) — no training required

Today the ✓/⚑ buttons store a label (/api/feedback); they do not change any decision. H3b closes that loop cheaply:

analyst marks a decision wrong  ─▶  stored (event snippet-hash + correct label + org/dept)
agent judge prompt  ─▶  "Here are recent corrections for THIS org:  <k examples>"  ─▶  better calls

• The agent (or the policy bundle it pulls) includes the org's last N corrected examples as few-shot guidance. Correct a decision on Monday → the model reflects it on Tuesday. No GPU, no training run, no weight changes — so it's safe and reversible.
• Fine-tuning (H5) only becomes worthwhile when few-shot examples outgrow the prompt window.

3. Department routing (H7) — identity → department → policy bundle

Different departments want different rules (Legal blocks contract text hard; Eng suppresses public code; Finance guards figures). One model, different injected policy.

logged-in user  ─▶  resolve DEPARTMENT  ─▶  select dept policy bundle  ─▶  inject + decide
                    (identity or config)     (context, markers,            (shared model + rules,
                                              thresholds, allow/deny)        dept-specific config)

3a. Where the department comes from (pick per deployment)

Interim vs target: ship per-endpoint SENTINEL_DEPARTMENT now (no auth needed); move to the SSO claim once auth lands, so the department can't be spoofed and follows the user across devices.

3b. Policy bundle shape (extends today's policy.json)

Key policy by org → department, with a department falling back to org default:

{
  "kshetra": {
    "_default":  { "context": "Kshetra Studio.", "proprietary_markers": ["orion","vega"] },
    "legal":     { "context": "Kshetra Legal.",  "proprietary_markers": ["orion","vega","msa","nda-2026"],
                   "thresholds": { "contract": 26 } },
    "finance":   { "context": "Kshetra Finance.","proprietary_markers": ["orion","vega","q3-forecast"],
                   "thresholds": { "financial": 26 } }
  }
}

thresholds overrides the risk bands per category for that department (e.g. Legal warns/blocks on contract language at a lower score). This is pure config — risk.py reads the dept's thresholds.

3c. Distribution (H2)

The endpoint agent fetches its bundle: GET /api/policy?org=<org>&dept=<dept> (cached, live-reload), so admins edit centrally and every endpoint in that department picks it up. Telemetry already carries org_id; add department so the dashboard can filter/aggregate by department too.

4. So: rules config, model training, or "depends"?

• Department-specific decisions → rules/policy config. One shared model; swap the dept bundle (context + markers + thresholds + lists). No per-department model.
• Getting smarter from corrections → prompt/few-shot (H3b) first; fine-tune (H5) only at scale.
• "Depends" applies only narrowly: structured detections are always rules; nuanced "is this proprietary to our team?" is handled by the department's injected context/examples, with fine-tuning as the rare escalation.

Dependencies: H7 (department routing) needs auth (#10) for trusted department claims — until then, per-endpoint SENTINEL_DEPARTMENT is the pragmatic stand-in. H3b/H5 need H3 feedback data (already being collected).