Skip to content

feat: support custom provider profile import and export #1081

Closed
#1170
Closed
feat: support custom provider profile import and export#1081
#1170
Labels
area:cliCLI-related workarea:gatewayGateway server and control-plane workarea:policyPolicy engine and policy lifecycle workfeature requestroadmapstate:agent-readyApproved for agent implementationstate:pr-openedPR has been opened for this issue

Description

@johntmyers
johntmyers
opened on Apr 30, 2026

Problem Statement

Issue #947 and PR #1037 added the first provider profile composition path using built-in YAML-backed profiles. The next incremental step from the broader provider v2 design in #896 is to let users inspect, export, customize, validate, register, and delete custom provider profiles without changing runtime credential injection yet.

Users need a practical way to start from OpenShell's built-in provider profiles, customize endpoints/deny rules/auth declarations, and register those custom profiles as new provider profile definitions. They also need profile import validation that reports all schema and semantic issues in one pass, instead of failing one error at a time.

Related: #896, #947, #1037

Current Baseline From #1037

  • Built-in provider profiles are authored as one YAML file per provider under the top-level providers/ directory.
  • The first provider profile browsing command is openshell provider list-profiles.
  • Profile categories are an enum: inference, agent, source_control, messaging, data, knowledge, and other.
  • Provider v2 behavior is gated by the global gateway setting providers_v2_enabled, which defaults to false.
  • Existing provider behavior remains the default when providers_v2_enabled is false.
  • Credential injection is unchanged for now. Legacy provider mode and provider v2 still use the existing credential resolution/injection path until profile-defined credential injection is implemented later.
  • Gateway global policy remains a hammer override: when a global policy is configured, it suppresses provider profile policy layers so operators can make rapid policy changes when needed. Effective policy composition must always use the sandbox's current provider attachment list, so detaching a provider removes that provider's profile-generated policy layer on the next JIT policy fetch.

Proposed Design

Add a provider profile registry and import/export UX on top of the built-in profile catalog.

Built-in profiles remain available as read-only defaults. Custom profiles are stored by the gateway/control plane in the existing objects table using a distinct object type; do not add profile-specific persistence tables or migrations. They are exposed through the same provider profile browsing APIs as built-ins. Custom profiles can also be deleted by id. Deletion applies only to custom profiles; built-in profiles remain immutable read-only defaults and attempts to delete them should return a clear error. Custom profile deletion should also be rejected while any sandbox is using the profile. With the current schema, usage is derived from SandboxSpec.providers provider names and the referenced Provider.type matching the profile id. In the first iteration, avoid silently overwriting built-in profiles; custom profiles should use distinct ids unless an explicit replace/override mode is added.

Built-in profile files should continue to use the YAML DTO shape introduced in #1037. That means import/export should preserve the human-authored profile format, including fields such as id, display_name, description, category, credentials, endpoints, binaries, and inference_capable. The DTO should continue mapping compact authoring forms, such as binaries: [/usr/bin/gh], into the richer internal policy structs.

Add CLI commands for profile inspection and file-based profile management. Exact command names can follow the existing provider command shape, but expected capabilities are:

openshell provider list-profiles
openshell provider profile export <id> -o yaml|json
openshell provider profile import -f profile.yaml
openshell provider profile import --from ./profiles/
openshell provider profile lint -f profile.yaml
openshell provider profile delete <id>

Support K8s-style output modes where useful:

-o table
-o yaml
-o json

Default output should remain human-readable table output for list/browse commands. Profile export should default to YAML because profiles are authored as YAML.

For import/lint, validate both schema-level and semantic constraints and return an aggregated diagnostic summary. The implementation should collect all discoverable issues before returning, including examples such as:

  • missing or empty id
  • unsupported category outside the current enum values
  • duplicate profile ids in a bulk import
  • duplicate credential names within a profile
  • duplicate credential environment variable names within a profile
  • invalid or incomplete auth declarations
  • missing header_name for header auth
  • missing query_param for query auth
  • endpoint port out of range
  • empty endpoint host
  • invalid deny rules
  • empty binary paths
  • unsupported output/import file format

Bulk import should support a directory of *.yaml, *.yml, and *.json files. Multi-document YAML support is desirable if it fits cleanly. Recursive directory import can be deferred or gated behind an explicit --recursive flag.

Definition of Done

  • Gateway stores custom provider profiles separately from built-in profiles using the existing objects table and a distinct object type; no new profile-specific storage tables are added.
  • Built-in profiles remain available from the top-level providers/ catalog and cannot be accidentally overwritten by import.
  • Profile list/get APIs return both built-in and registered custom profiles with clear precedence semantics.
  • Profile list/get APIs expose the current category enum values: inference, agent, source_control, messaging, data, knowledge, and other.
  • CLI preserves and extends the existing openshell provider list-profiles UX; do not introduce the older list-types naming.
  • CLI can export a profile as YAML and JSON.
  • CLI supports K8s-style output modes for profile browsing where appropriate.
  • CLI can import a single profile file.
  • CLI can bulk import profiles from a directory.
  • CLI can lint profile files without registering them.
  • CLI can delete registered custom profiles by id.
  • Custom profile deletion is rejected when any sandbox attaches a provider whose Provider.type matches the profile id.
  • Built-in profile deletion is rejected with a clear error.
  • Import/lint returns an aggregated summary of all validation issues found in the provided input set.
  • Tests cover successful single import, bulk import, export round trip, custom profile deletion, in-use deletion rejection, built-in overwrite/delete rejection, and multi-error validation output.
  • Existing provider behavior remains unchanged unless the global providers_v2_enabled setting is enabled.
  • Provider v2 import/export does not change runtime credential injection behavior yet.
  • Effective policy composition uses the current sandbox provider attachment list; detached providers no longer contribute profile-generated policy on the next JIT policy fetch.
  • Internal architecture notes are updated if implementation details change; published docs/* updates are out of scope for this issue and will be handled in a later docs sweep.

Non-Goals

  • Do not implement proxy-side credential injection from profile auth declarations in this issue.
  • Do not implement OAuth2 credential refresh in this issue.
  • Do not implement provider attach/detach for running sandboxes in this issue.
    • If attach/detach exists or is added elsewhere, this issue should preserve the invariant that JIT policy composition reflects the current provider list, not a cached profile layer.
  • Do not change the default behavior of existing providers.
  • Do not add new profile-specific persistence tables; use the existing object store/objects table.
  • Do not delete or hide built-in provider profiles.
  • Do not cascade custom profile deletion into existing provider records or sandboxes.
  • Do not introduce a new sandbox attached-profile field in this issue; infer profile usage through existing sandbox providers and provider types.
  • Do not rename openshell provider list-profiles back to list-types.

Agent Investigation

Reviewed #896 after #1037 merged. This is the next low-risk chunk because it extends profile management and validation without changing sandbox runtime injection behavior. It prepares the registry surface needed by later provider v2 work: profile-backed credential injection, attach/detach, inference automation, verification, and refresh.

Updated after #1037 to reflect the merged command and setting names, top-level built-in provider YAML location, category enum, global policy override behavior, and current credential injection boundary.

Activity

  1. johntmyers
    added
    area:cliCLI-related work
    area:policyPolicy engine and policy lifecycle work
    area:gatewayGateway server and control-plane work
    roadmap
    feature request
    on Apr 30, 2026

  2. johntmyers commented on May 5, 2026

    @johntmyers
    johntmyers
    on May 5, 2026
    CollaboratorAuthor

    🏗️ build-plan

    Implementation Plan

    Issue type: feat
    Complexity: High
    Confidence: Medium — clear baseline from #1037, but custom profile persistence and import/export/delete API shape need careful compatibility choices

    Summary

    Add custom provider profile import/export/lint support on top of the YAML-backed built-in profile catalog. The implementation will keep built-in profiles read-only, store custom profiles separately in the gateway, expose a merged built-in + custom profile view through the provider profile APIs, and add CLI commands for list/export/import/lint/delete with aggregated validation diagnostics.

    Runtime credential injection remains unchanged in this issue. Effective policy composition should remain just-in-time over the current sandbox provider attachment list, so detached providers stop contributing profile-generated policy on the next policy fetch. The new registry prepares provider v2 profile management without moving credential injection into profile-defined auth yet.

    Scope

    • proto/openshell.proto: add provider profile management RPCs/messages for importing, linting, and returning diagnostics, and deleting custom profiles; preserve existing ListProviderProfiles and GetProviderProfile behavior for clients.
    • crates/openshell-providers/src/profiles.rs: extend the YAML DTO layer with YAML/JSON serialization, file/directory parsing helpers, category formatting/parsing, and aggregated validation diagnostics instead of first-error-only validation for import/lint paths.
    • crates/openshell-server/src/grpc/provider.rs: add custom profile persistence helpers, built-in overwrite rejection, merged built-in/custom list/get semantics, and handlers for import/lint/export/delete-style API calls.
    • crates/openshell-server/src/persistence/* as needed: store custom profile records with the existing object store/objects table using a distinct object type. Do not add new profile-specific tables or migrations.
    • crates/openshell-cli/src/main.rs: add provider profile subcommands while preserving openshell provider list-profiles.
    • crates/openshell-cli/src/run.rs: implement CLI UX for profile export, profile import, profile import --from, profile lint, profile delete, and -o table|yaml|json output modes where appropriate.
    • crates/openshell-cli/tests/* and server/provider unit tests: cover successful import, bulk import, export round trip, built-in overwrite rejection, category enum visibility, and multi-error validation output.
    • architecture/sandbox-providers.md: update internal architecture notes if implementation details change. Published docs/* updates are intentionally out of scope and will be handled in a later docs sweep.

    Implementation Steps

    1. Extend the profile DTO and validation layer.

      • Add serialization helpers for YAML/JSON export.
      • Add validation that accumulates all schema/semantic issues discovered in a provided profile set.
      • Keep the built-in catalog loader strict and fail-fast at startup if built-ins are invalid, but reuse the richer diagnostics for import/lint UX.

    2. Add gateway custom profile storage and merged registry semantics.

      • Store custom profiles separately from built-ins in the existing objects table with a distinct object type.
      • Reject imports that attempt to overwrite built-in profile ids unless a future explicit override mode is added.
      • Make list/get return built-ins plus custom profiles with deterministic sorting and clear custom-vs-built-in semantics.
      • Keep provider profile policy composition derived from current sandbox provider attachments rather than persisted effective policy, so provider detach removes that profile layer automatically on the next JIT policy fetch.

    3. Add API messages and handlers for profile import/lint/export/delete support.

      • Import validates and persists only when diagnostics have no errors.
      • Lint validates without persisting.
      • Export/get can return a profile in the same authoring DTO shape used by YAML files.
      • Delete removes only custom profiles, rejects built-in profile ids with a clear error, and rejects deletion while any sandbox is using the profile. With today's schema, usage is inferred by reading sandboxes' spec.providers, resolving those provider names to Provider records, and checking whether any referenced Provider.type matches the custom profile id.

    4. Add CLI provider profile UX.

      • Preserve openshell provider list-profiles.
      • Add openshell provider profile export <id> -o yaml|json.
      • Add openshell provider profile import -f profile.yaml.
      • Add openshell provider profile import --from ./profiles/ for bulk import.
      • Add openshell provider profile lint -f profile.yaml.
      • Add openshell provider profile delete <id> for custom profiles.
      • Support -o table|yaml|json for browsing commands where it is useful.

    5. Add tests around the new behavior.

      • DTO validation tests for aggregated diagnostics.
      • Server tests for built-in/custom merge, overwrite rejection, import persistence, custom profile deletion, in-use delete rejection, built-in delete rejection, and lint no-op behavior.
      • CLI parsing/run-function tests for the new commands and output modes.

    6. Update internal architecture notes only if needed.

      • Keep architecture/sandbox-providers.md aligned with storage, merge, deletion, and JIT composition behavior if the implementation changes those internals.
      • Do not update published docs/* pages in this issue; user-facing docs will be swept separately.

    Test Plan

    • Unit tests: profile DTO parsing/serialization/validation, category parsing, bulk duplicate detection, built-in overwrite rejection, and custom profile registry merge behavior.
    • Integration tests: CLI/provider gRPC tests for import, bulk import, export round trip, lint failures with multiple diagnostics, and list-profiles including custom entries, custom profile deletion, in-use delete rejection, and built-in delete rejection.
    • E2E tests: N/A for this issue unless implementation touches sandbox runtime behavior. This is control-plane and CLI profile management only.
    • Verification: run mise run pre-commit; run targeted provider CLI/server tests during development.

    Risks & Open Questions

    • The current ProviderProfile proto does not include metadata or source fields. The implementation may need either additive fields or a small storage wrapper so custom profiles can be persisted and identified without breaking existing clients.
    • Import/export should preserve the human-authored DTO shape rather than exposing the richer internal NetworkBinary object shape where the YAML intentionally uses compact path strings.
    • Profile persistence must reuse the existing object store/objects table; no new profile-specific tables should be introduced.
    • Profile deletion should be blocked when any sandbox attaches a provider whose Provider.type matches the custom profile id.
    • Profile deletion should not cascade into existing provider records or sandboxes; deleting the profile only removes the custom profile definition.
    • Bulk import diagnostics need stable, user-readable source locations such as file path plus profile id so users can fix multiple errors in one pass.
    • Replacement/override semantics are intentionally out of scope; built-in ids should be protected for this iteration.

    Documentation Impact

    • Update architecture/sandbox-providers.md if needed for the registry model and API behavior.
    • Published docs/* updates are out of scope and will be handled separately.

    Revision 6 — removed published docs updates from scope
    Revision 5 — clarified provider detach removes profile policy layer via JIT composition
    Revision 4 — clarified in-use custom profiles cannot be deleted
    Revision 3 — clarified custom profile storage uses existing objects table
    Revision 2 — added custom profile deletion scope
    Revision 1 — initial plan

  3. johntmyers
    added
    state:review-readyReady for human review
    on May 5, 2026

  4. johntmyers commented on May 5, 2026

    @johntmyers
    johntmyers
    on May 5, 2026
    CollaboratorAuthor

    🏗️ build-from-issue-agent

    We should support profile deletion as well

    Agreed. I updated the issue and build plan to include openshell provider profile delete <id> plus API/server support for deleting custom profiles. The scope is custom-only: built-in profiles remain read-only and delete attempts against built-in ids should return a clear error. Deletion also should not cascade into existing provider records or sandboxes; it only removes the custom profile definition.

  5. johntmyers commented on May 5, 2026

    @johntmyers
    johntmyers
    on May 5, 2026
    CollaboratorAuthor

    🏗️ build-from-issue-agent

    Any storage for profiles needs to be in the objects table we use, no creating new tables as well

    Updated the issue and plan to make that a hard constraint. Custom provider profiles should be persisted through the existing object store/objects table with a distinct object type. No new profile-specific persistence tables or migrations should be added.

  6. johntmyers commented on May 5, 2026

    @johntmyers
    johntmyers
    on May 5, 2026
    CollaboratorAuthor

    🏗️ build-from-issue-agent

    A profile should not be able to be deleted if its in-use, I think we can check the sandboxes if they have attachemd profiles to check that with today's schema?

    Agreed, with one schema nuance: sandboxes do not currently attach profile ids directly. They attach provider names in SandboxSpec.providers; each referenced Provider has a type, and provider v2 resolves that type to the provider profile id. I updated the issue and plan so deleting a custom profile must scan sandboxes, resolve their attached provider names to provider records, and reject deletion if any referenced Provider.type matches the profile id.

    That keeps the check on today's schema without adding a new attached-profile field in this issue.

  7. johntmyers
    added
    state:agent-readyApproved for agent implementation
    state:in-progressWork is currently in progress
    state:pr-openedPR has been opened for this issue
    and removed
    state:in-progressWork is currently in progress
    state:review-readyReady for human review
    on May 5, 2026
  8. johntmyers
    closed this as completedin #1170on May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:cliCLI-related workarea:gatewayGateway server and control-plane workarea:policyPolicy engine and policy lifecycle workfeature requestroadmapstate:agent-readyApproved for agent implementationstate:pr-openedPR has been opened for this issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions