Make WordPress Core

Opened 7 months ago

Closed 3 months ago

Last modified 3 days ago

#64955 closed enhancement (fixed)

Improve JSON Schema compatibility when handling abilities

Reported by: gziolo Owned by: gziolo
Priority: normal Milestone: 7.1
Component: Abilities API Version: 6.9
Severity: normal Keywords: has-patch has-unit-tests has-dev-note
Cc: Focuses: rest-api

Description

The ​Abilities API's input_schema and output_schema use WordPress's draft-04-style JSON Schema conventions. When these schemas are consumed by AI agents — via the ​AI Client, the ​MCP adapter, or browser-based agents — they must conform to the constrained JSON Schema subset that AI providers accept for strict tool calling / structured outputs.

AI providers like ​OpenAI, ​Anthropic, and ​Google implement constrained decoding — they compile tool schemas into a grammar that restricts token generation at inference time. This means they only accept structural keywords (type, properties, required, enum, anyOf) and reject or strip validation keywords (minimum, maxLength, pattern, format). No provider conforms to any single JSON Schema draft; their supported subset is roughly "draft-04 structural keywords plus $defs and const from later drafts, minus all value-level assertion keywords."

WordPress's JSON Schema conventions diverge from this subset in six areas:

  1. required predominantly uses draft-03 per-property syntax ('required' => true); AI providers only understand the draft-04 array syntax. The client-side AJV validator also does not enforce draft-03 syntax.
  2. additionalProperties defaults to true when absent; OpenAI requires false on every object in strict mode.
  3. oneOf is used; OpenAI rejects it (only anyOf accepted), Anthropic normalizes it to anyOf.
  4. Validation keywords (minimum, maxLength, pattern, format) are stripped or rejected by AI providers, though they remain enforced by WordPress's validators.
  5. WordPress-only keywords (context, readonly, arg_options) are meaningless to AI consumers.
  6. definitions and $ref (draft-04 schema composition keywords) are not in the ​allowed keywords list, preventing their use in ability schemas.

Currently, no component in the WordPress AI stack transforms schemas for provider compatibility. The AI Client passes ability schemas verbatim to the ​PHP AI Client SDK, which treats them as opaque payloads. The three featured provider plugins (​OpenAI, ​Anthropic, ​Google) handle authentication and API formatting but do not transform schemas. This means ability schemas authored by plugin developers flow through to AI provider APIs unchanged, where they may be rejected or silently degraded.

Server-side abilities are validated on both client (AJV draft-04, with no type coercion) and server (rest_validate_value_from_schema, with type juggling), creating a natural strictness gradient: AI compiled schema (strictest) → client-side AJV (strict) → server-side PHP (most permissive). Because the compiler's transformations only narrow what's accepted, values conforming to the compiled schema automatically pass both existing validators without any changes to either validation layer.

Proposed solution

Introduce wp_compile_ability_schema_for_ai() — a public function that transforms a canonical ability schema into an AI-compatible variant at the output boundary:

  • Normalizes required from draft-03 per-property to draft-04 array syntax.
  • Sets additionalProperties: false on all object nodes.
  • Converts oneOf to anyOf.
  • Relocates validation keywords (minimum, maxLength, etc.) into property description text as soft guidance for the model, while WordPress's validators continue to enforce them.
  • Strips WordPress-only keywords (context, readonly, arg_options).
  • Maps definitions → $defs in output for AI providers that expect the modern keyword name.
  • Accepts a $target parameter for provider-specific behavior (openai, anthropic, google, default).
  • Is filterable via wp_ability_schema_compiled.

The compiler is a server-side concern, invoked at AI-facing boundaries: the AI Client (before PHP AI Client SDK hand-off), and optionally by standalone adapters like the MCP adapter. The canonical schema remains unchanged for REST API consumers. The compiled schema never reaches the client-side store.

Additionally:

  • Add a wp_rest_allowed_schema_keywords filter to ​rest_get_allowed_schema_keywords().
  • Add definitions and $ref to the allowed keywords list for output passthrough.
  • Integrate the compiler into the AI Client with the $target derived from the active provider plugin.

No changes to existing validation behavior on either client or server.

Change History (39)

#1 @gziolo
7 months ago

  • Owner set to gziolo
  • Status new → assigned

I did extensive research into how JSON schemas flow through the WordPress AI stack, what each AI provider actually accepts, and how the client-side validation layer interacts with the server-side one. The findings below informed the ticket description above. I'm sharing the full technical detail here so the community can verify the claims and correct anything I got wrong — particularly around the provider plugin internals, AI client, or MCP adapter.

How schemas flow today

No component in the WordPress AI stack transforms ability schemas for AI provider compatibility. Here's the path:

wp_register_ability( input_schema )        ← WordPress draft-04 style
        │
   WP_Abilities_Registry
        │
   ┌────┼──────────────────────┬───────────────────────┐
   │    │                      │                       │
   ▼    ▼                      ▼                       ▼
REST API                  WP AI Client             MCP Adapter
(/wp-abilities/v1/)       (Prompt Builder)         (/wp-json/mcp/...)
   │                          │                       │
   │                     PHP AI Client SDK        MCP tools/list
   │                     (opaque pass-through)    (limited fixes)
   │                          │                       │
   │                     Provider Plugin              │
   │                     (no schema transform)        │
   ▼                          ▼                       ▼
Canonical Schema          AI Provider API          MCP Client
(unchanged)               (may reject/strip)       (may reject/strip)

The WP AI Client converts ability names to provider-safe tool names (e.g., my-plugin/translate-content → my_plugin__translate_content) but passes input_schema verbatim to FunctionDeclaration in the ​PHP AI Client SDK. The SDK treats schemas as opaque payloads. The three provider plugins (​OpenAI, ​Anthropic, ​Google) extend the SDK's base model classes and handle API formatting but include no schema normalization logic. The ​MCP adapter performs limited structural fixes (empty schema normalization, non-object wrapping for MCP compliance) but no keyword-level transformation.

Incompatibility details

The ticket lists six incompatibilities. Here's the deeper technical context for each.

required keyword syntax

WordPress supports both draft-03 per-property required: true (predominant since WP 4.7) and ​draft-04 array syntax (added in WP 5.5). AI providers only understand v4 syntax. ​OpenAI further requires that all properties appear in the required array — optional fields must use "type": ["string", "null"] union types.

A schema using 'required' => true on individual properties passes WordPress server-side validation but the requirement is invisible to both AI providers and the client-side AJV validator (which only recognizes v4 array syntax). Abilities using draft-03 syntax effectively have no required-field enforcement on the client.

additionalProperties default

​JSON Schema and WordPress default additionalProperties to true when absent. ​OpenAI mandates additionalProperties: false on every object in strict mode. Anthropic strongly recommends it.

oneOf vs anyOf

WordPress supports both (#51025, WP 5.6). OpenAI rejects oneOf — only anyOf accepted. Anthropic normalizes oneOf to anyOf. Both providers ​reject oneOf, allOf, and anyOf at the root level of a tool's input_schema. Converting oneOf to anyOf is safe for generation since anyOf is semantically more permissive — the canonical validators still enforce oneOf semantics on both client and server.

Validation keywords

Keywords like minimum, maximum, minLength, maxLength, pattern, multipleOf, and most format values are fully supported by WordPress's validators but unsupported by AI providers' constrained decoding. ​Anthropic's SDK strips them and injects constraint descriptions into description. ​OpenAI rejects them in strict mode. ​Google's Gemini API has broader support including numeric constraints.

The compiler relocates these to description text as soft guidance. Enforcement remains with WordPress's validators — if an AI model produces a value outside the minimum/maximum range, the client-side or server-side validator catches it.

$ref / definitions gap

WordPress does not support $ref or definitions/$defs for schema composition. AI providers support internal $ref references and use $defs (draft 2019-09 keyword name), while draft-04 uses definitions. WordPress schemas are already fully inlined, so no $ref resolution is needed in the compiler — but these keywords aren't in the ​allowed keywords list and get silently stripped.

Canonical ability schemas should use definitions (draft-04) rather than $defs (draft 2019-09), because the client-side ajv-draft-04 validator recognizes definitions but not $defs. The compiler maps definitions → $defs in AI-facing output. Similarly, canonical schemas should use single-value enum rather than const (draft-06), since const doesn't exist in draft-04. AI providers accept both forms.

WordPress-only keywords

context (view/edit/embed array), readonly (lowercase — see #56152), and arg_options (escape hatch for custom callbacks, ​stripped by `get_public_item_schema()`). Meaningless to AI providers.

Client-side validation and the strictness gradient

The @wordpress/abilities package validates schemas client-side using ajv-draft-04 with coerceTypes: false — stricter than WordPress PHP, which performs type juggling. The ​header comment in validation.ts explicitly states the design intent:

Rules are configured to support the intersection of common rules between JSON Schema draft-04, WordPress (a subset of JSON Schema draft-04), and various providers like OpenAI and Anthropic.

Both server-side and client-side abilities are validated on the client. executeAbility() in api.ts validates input and output unconditionally when schemas are present. Server-side abilities fetched by @wordpress/core-abilities carry their schemas into the client-side store, creating a double-validation path: client-side AJV first, then server-side rest_validate_value_from_schema when the REST API call executes.

This creates a natural strictness gradient:

AI Provider Schema (strictest)     ← canonical schema compiled for AI consumption
        │
        │  AI model generates values conforming to this
        ▼
Client-side AJV draft-04 (strict)  ← validates against canonical schema
        │
        │  values pass because compiled schema is a strict subset
        ▼
Server-side PHP validator (most permissive)  ← validates against canonical schema
        │
        │  values pass because PHP is even more permissive than AJV
        ▼
Ability callback executes

Because values flow from the strictest layer to the most permissive, any value that satisfies the compiled AI schema automatically passes both validators. The compiler's transformations only narrow what's accepted — setting additionalProperties: false, adding all properties to required, converting oneOf to anyOf — so the compiled schema is inherently safe for the double-validation path. The canonical schema in the client-side store never needs to change.

Proposed compiler architecture

The compiler sits at every boundary where schemas leave WordPress for AI consumption:

wp_register_ability( input_schema )        ← WordPress draft-04 style
        │
   WP_Abilities_Registry
        │
   ┌────┼──────────────────────┬───────────────────────┐
   │    │                      │                       │
   ▼    ▼                      ▼                       ▼
REST API                  WP AI Client             MCP Adapter
(/wp-abilities/v1/)       (Prompt Builder)         (standalone plugin)
   │                          │                       │
   │                     compile_schema()         compile_schema()
   │                          │                       │
   ▼                          ▼                       ▼
Canonical Schema          AI-Strict Schema         AI-Strict Schema
(unchanged for REST       (before SDK hand-off)    (in tools/list)
API consumers)

Compiler transformations (in order)

  1. Normalize required: Walk all object nodes. Collect per-property required: true flags (draft-03) and merge with any existing required array (draft-04). For AI targets, add all remaining properties to required and convert their types to ["original_type", "null"] unions. Remove per-property required booleans.
  2. Set additionalProperties: false: Recursively on every object node, including nested objects inside properties, items, and anyOf sub-schemas.
  3. Convert oneOf to anyOf: Direct keyword replacement. Sub-schema array unchanged.
  4. Relocate validation keywords to descriptions: For each property with minimum, maximum, minLength, maxLength, pattern, multipleOf, or unsupported format values, append a human-readable constraint summary to the property's description. Then remove the keywords.
  5. Strip WordPress-only keywords: Remove context, readonly, arg_options, and any other WordPress-specific annotations.
  6. Map definitions to $defs: If the schema uses definitions (draft-04), rename it to $defs (draft 2019-09) in the output. AI providers expect the modern keyword name.
  7. Validate structural limits: Check for total properties, nesting levels, and enum values (​OpenAI limits). Emit _doing_it_wrong() warnings if exceeded.

Provider-specific $target behavior

  • OpenAI: Strictest. Strip all validation keywords, enforce additionalProperties: false, put all properties in required. Strip format entirely.
  • Anthropic: Moderately strict. Same structural requirements, but allows some optional properties (up to ​optional parameters, parameters with union types across all strict tools). May preserve some format values.
  • Google Gemini: Most permissive. Supports ​numeric constraints and does not require additionalProperties: false. The compiler can preserve more keywords.
  • default: Targets the most restrictive common subset (effectively OpenAI's constraints), or whatever else we decide as a better approach.

Why this approach over alternatives

Alternative A: Dual-mode validator in core. Add a strict flag to rest_validate_value_from_schema. Invasive — touches the most critical validation path in WordPress, risks regressions in every REST API endpoint, and couples AI provider requirements to core's validation logic. AI provider requirements change faster than WordPress release cycles. Rejected.

Alternative B: Require ability authors to write AI-compatible schemas directly. Shifts complexity to every plugin author, guarantees inconsistency, and doesn't help with required normalization (which requires semantic changes like null-union types). Existing core abilities would all need manual rewrites. Rejected.

Alternative C: Add schema transformation to the PHP AI Client SDK or provider plugins. Fixes the WP AI Client path but not MCP adapter, WebMCP, or any future adapter. Places WordPress-specific schema knowledge inside a ​provider-agnostic SDK that treats schemas as opaque payloads. Rejected.

Alternative D: Schema compiler as a shared core utility (proposed). Pure, testable, filterable function alongside the Abilities API. Changes no validation behavior. All AI-facing components call the same function. The strictness gradient guarantees compiled schemas are safe for the existing double-validation path. Accepted.

Implementation steps

Steps are ordered by dependency. Steps within the same group can be done in parallel.

Step 1: Ship the schema compiler function (PHP)

Introduce wp_compile_ability_schema_for_ai( $schema, $target = 'default' ) in WordPress core, alongside the Abilities API. Purely additive — transforms schemas on output, changes no validation behavior. Filterable via wp_ability_schema_compiled.

No dependencies. Enables all downstream work.

Step 2: Integrate compiler into the WP AI Client

The WP AI Client's Ability_Function_Resolver should call wp_compile_ability_schema_for_ai() on each ability's input_schema before passing it to the ​PHP AI Client SDK. The $target should be derived from the active provider plugin.

Highest-impact integration point — covers the primary path through which WordPress sites interact with AI providers.

Depends on Step 1.

Step 3: Add wp_rest_allowed_schema_keywords filter

Add a filter hook to ​rest_get_allowed_schema_keywords():

apply_filters( 'wp_rest_allowed_schema_keywords', $keywords, $context );

Minimal core change with broad utility beyond the Abilities API. Enables plugins to pass through keywords like definitions and $ref without patching core.

No dependencies. Parallel with Steps 1-2.

Step 4: Add definitions and $ref to the allowed keywords list

Add these to ​rest_get_allowed_schema_keywords() for output passthrough — not validation resolution. definitions is the draft-04 keyword compatible with the client-side ajv-draft-04 validator; the compiler maps it to $defs in AI-facing output. $ref is supported by AJV draft-04 natively.

AI providers also support $defs (draft 2019-09) and const (draft-06), but neither exists in draft-04 and the client-side validator does not recognize them. Canonical ability schemas should use definitions and single-value enum respectively.

Depends on Step 3, or can be done directly in core alongside it.

Step 5: Emit authoring guidance via _doing_it_wrong()

When abilities are registered, check schemas for patterns that cause AI compatibility issues:

  • oneOf used where anyOf would suffice → suggest anyOf.
  • additionalProperties absent on object schemas → suggest setting it explicitly.
  • Per-property required: true (draft-03) → suggest v4 array syntax. This syntax is also not enforced by the client-side AJV validator, so switching improves client-side validation coverage.
  • readonly used instead of readOnly → note the casing deviation (#56152).

Advisory notices, not validation errors.

Depends on Step 1.

Step 6: Introduce schema_version for opt-in strict authoring

Add an optional schema_version key to ability registration args. Abilities declaring schema_version: 2 opt into stricter conventions:

  • Draft-03 per-property required: true is rejected at registration time.
  • additionalProperties must be explicitly set on all object nodes.
  • oneOf is rejected (use anyOf instead).
  • The compiler can skip most transformation steps for v2 schemas.

Migration path: new abilities adopt v2, existing abilities continue on implicit v1.

Depends on Steps 1 and 5.

Adjacent: MCP adapter integration

The ​MCP adapter is a standalone plugin. Once Step 1 lands, it should call wp_compile_ability_schema_for_ai() in its tools/list handler. Does not require coordination with core releases.

Future considerations (not blocking)

  • General-purpose wp_validate_json_schema() function. rest_validate_value_from_schema now serves the Abilities API in contexts unrelated to the REST API. A wp_validate_json_schema( $value, $schema, $args ) wrapper would reflect this broader role and provide a natural home for strict-mode validation tied to schema_version. Initially a thin passthrough.
  • $ref resolution in the validator. Teaching the validator to resolve internal $ref references would unlock schema composition for validation, not just output passthrough. Significant change — should be its own tracked effort.
  • Client-side schema compilation for browser-agent protocols. Since executing any ability routes through the REST API where server-side compilation applies, a JavaScript compiler is not needed today. As ​WebMCP matures, it may need to expose client-side ability schemas directly to browser-based AI agents. The PHP compiler's test fixtures should be portable to JavaScript when this need arises.
  • Schema compilation in the PHP AI Client SDK. If the SDK eventually gains its own schema normalization, the WP AI Client could delegate to it. The SDK currently treats schemas as opaque payloads; changing this is a separate decision.
  • Structural limit warnings. OpenAI limits schemas for properties, nesting levels, and enum values. Worth checking at registration time but limits may change. Implement as soft warnings.
  • readOnly casing alignment. #56152 tracks this. Not blocking since both forms are stripped by the compiler.

This ticket was mentioned in ​Slack in #core-ai by gziolo. ​View the logs.


7 months ago

#3 @gziolo
6 months ago

  • Milestone Awaiting Review → Future Release

#4 @gziolo
6 months ago

  • Keywords 2nd-opinion added

#5 @gziolo
6 months ago

#65035 fixes the Abilities REST API (/wp/v2/abilities) leaking WordPress-internal schema keywords (sanitize_callback, validate_callback, arg_options) into input_schema and output_schema responses. These keywords aren't valid JSON Schema and break client-side validators — notably Gutenberg's AI tools integration.

This is a targeted fix for item 5 in this ticket's description (WordPress-only keywords) at the REST output boundary. The broader schema compiler proposed here would address the remaining structural incompatibilities (items 1–4 and 6) at the AI-facing boundary.

#6 @desrosj
6 months ago

  • Component AI → Abilities API

Moving tickets related to the Abilities API to a new sub-component.

#7 @desrosj
5 months ago

  • Keywords abilities removed

Removing abilities and abilities-api custom keywords. This is now indicated by the Abilities API component.

#8 @desrosj
5 months ago

  • Keywords ai-client, 2nd-opinion → ai-client 2nd-opinion
  • Summary Abilities API: Add schema compiler for AI tool calling compatibility → Add schema compiler for AI tool calling compatibility

This ticket was mentioned in ​PR #11988 on ​WordPress/wordpress-develop by ​@gziolo.


5 months ago
#9

  • Keywords has-patch has-unit-tests added

## What?

This hardens the Ability REST API schema preparation path by centralizing response schema transformations in one recursive helper.

The REST list controller now prepares both input and output schemas through prepare_schema_for_response(), which:

  • strips WordPress-internal schema keywords before exposing ability schemas through REST responses;
  • normalizes empty object defaults from array() to stdClass so they serialize as {};
  • applies those transformations recursively to nested sub-schemas.

## Why?

Ability schemas are consumed by REST clients and AI-facing tooling. The implementation already stripped internal schema keywords recursively, while empty object default normalization was only applied at the top level before that recursion. Combining these transformations into one helper gives core a clearer extraction point for future schema transformation work and ensures nested object defaults are represented consistently.

This follows the direction discussed in Core Trac ticket #64955 and builds on the REST cleanup from #65035.

## Testing

  • php -l src/wp-includes/rest-api/endpoints/class-wp-rest-abilities-v1-list-controller.php
  • php -l tests/phpunit/tests/rest-api/wpRestAbilitiesV1ListController.php
  • git diff --check -- src/wp-includes/rest-api/endpoints/class-wp-rest-abilities-v1-list-controller.php tests/phpunit/tests/rest-api/wpRestAbilitiesV1ListController.php
  • npm run test:php -- --filter Tests_REST_API_WpRestAbilitiesV1ListController

#10 @gziolo
5 months ago

  • Keywords ai-client 2nd-opinion removed
  • Milestone Future Release → 7.1

#11 @gziolo
5 months ago

In 62427:

Abilities API: Harden ability schema preparation for REST responses

Merge normalize_schema_empty_object_defaults() and strip_internal_schema_keywords() into a single recursive prepare_schema_for_response() helper on WP_REST_Abilities_V1_List_Controller. Empty object defaults now normalize to stdClass at every depth — not just the top level — so nested {} defaults serialize consistently alongside the existing internal-keyword stripping.

Follow-up to [62221], [61244].

Props gziolo, westonruter.
See #64955.

​@gziolo commented on ​PR #11988:


5 months ago
#12

A commit was made that fixes the Trac ticket referenced in the description of this pull request.

SVN changeset: 62427
GitHub commit: ​https://github.com/WordPress/wordpress-develop/commit/b226691d4b1d92126aa8c03b897e07ec18feef9a

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

This ticket was mentioned in ​PR #11997 on ​WordPress/wordpress-develop by ​@gziolo.


5 months ago
#13

## What?

This hardens the Ability REST API schema response preparation by switching from an internal-keyword deny-list to a schema-keyword allow-list.

Ability input and output schemas exposed through REST responses now keep only:

  • keywords returned by rest_get_allowed_schema_keywords();
  • a small local set of additional schema traversal keywords currently used by ability schemas.

This continues to preserve nested schema structures while stripping WordPress-internal keys such as sanitize_callback, validate_callback, and arg_options, along with other unsupported schema keywords such as examples.

## Why?

The previous implementation stripped only known internal keywords. An allow-list is stricter and better aligned with the REST API schema surface: unknown or unsupported schema keywords are not exposed to REST clients by default.

The local additional keyword list keeps backwards compatibility for schema traversal locations that are not currently returned by rest_get_allowed_schema_keywords(), while still making the response-preparation boundary more explicit.

## Testing

  • php -l src/wp-includes/rest-api/endpoints/class-wp-rest-abilities-v1-list-controller.php
  • php -l tests/phpunit/tests/rest-api/wpRestAbilitiesV1ListController.php
  • git diff --check -- src/wp-includes/rest-api/endpoints/class-wp-rest-abilities-v1-list-controller.php tests/phpunit/tests/rest-api/wpRestAbilitiesV1ListController.php
  • npm run typecheck:php -- src/wp-includes/rest-api/endpoints/class-wp-rest-abilities-v1-list-controller.php --no-progress
  • npm run test:php -- --filter Tests_REST_API_WpRestAbilitiesV1ListController

This ticket was mentioned in ​PR #12012 on ​WordPress/wordpress-develop by ​@gziolo.


4 months ago
#14

## Summary

Adds focused test coverage documenting how a top-level required keyword is treated during WP_Ability::validate_input(), and removes inert top-level required flags from existing test schemas.

## Background

WordPress validates object values against either a draft-04 required array of property names, or a draft-03 per-property required boolean (see rest_validate_value_from_schema()). A required flag placed at the root of a schema is never consulted by the validator, so it has no effect on validation. A number of test schemas in this file set required => true at the top level of simple-type schemas, which is inert and misleading.

## Changes

  • New data-provider-driven test_validate_input_top_level_required, covering both valid and invalid input for each variant:
    • top-level required: true — inert (only type is enforced)
    • top-level required: false — inert; does not permit null
    • required omitted — baseline; identical behavior
    • draft-04 required array on an object type — honored; enforces property presence
  • Removes the inert top-level required => true flags from simple-type schemas. The meaningful per-property required flags in the object data set are preserved.

This is a test-only change: it validates existing WP_Ability behavior and adds no production code.

🤖 Generated with ​Claude Code

This ticket was mentioned in ​PR #12013 on ​WordPress/wordpress-develop by ​@gziolo.


4 months ago
#15

## Summary

Normalizes the required keyword in an ability's input and output schemas to its JSON Schema draft-04 form when they are exposed through the REST API (wp-abilities/v1).

## Background

WordPress validates objects against either a draft-04 required array of property names on the object schema, or a draft-03 per-property required boolean (see rest_validate_value_from_schema()). Both work internally, but only the array form is valid JSON Schema draft-04. Ability schemas registered with the per-property boolean form (as shown in the wp_register_ability() examples) were therefore exposed to clients in a shape that does not conform to the draft-04 contract the endpoint advertises via $schema.

## Changes

In WP_REST_Abilities_V1_List_Controller::prepare_schema_for_response():

  • Collect per-property required: true booleans and emit them as a required: [ ... ] array of property names on the parent object schema — applied recursively, so nested objects each get their own array.
  • Strip required: false and any boolean required that has no draft-04 equivalent (for example on a scalar root schema).
  • Preserve and merge with an existing draft-04 required array, de-duplicating names.

The transformation only rewrites the REST response copy; server-side validation continues to use the stored schema via validate_input(), so behavior is unchanged. Adds test coverage for the conversion: top-level, nested objects, required: false, mixed draft-04 array + draft-03 booleans, and the scalar-root case.

🤖 Generated with ​Claude Code

#16 @gziolo
4 months ago

In 62440:

Abilities API: Add coverage for ignored schema validate_callback/sanitize_callback

Add characterization tests documenting that an ability ignores two REST-style schema keywords:

  • validate_callback is never invoked. The REST request layer calls it, but rest_validate_value_from_schema() does not.
  • There is no sanitization pass, so sanitize_callback never runs and input reaches the execute callback uncoerced.

Custom validation belongs in the wp_ability_validate_input / wp_ability_validate_output filters. Test-only change.

arg_options is intentionally not covered here: it is a registration-time helper for rest_get_endpoint_args_for_schema() with no runtime meaning for abilities.

Follow-up [61032], #64098.
See #64955.

#17 @gziolo
4 months ago

In 62441:

Abilities API: Add coverage for top-level required in input validation

Add a data-provider-driven test documenting that a required flag at the root of a schema is never consulted by WP_Ability::validate_input():

  • top-level required: true/false is inert; only type is enforced, and false does not permit null.
  • a draft-04 required array on an object type is honored and enforces property presence.

Also remove the inert top-level required => true flags from simple-type test schemas, keeping the meaningful per-property flags. Test-only change.

Follow-up [61032], #64098.
See #64955.

​@gziolo commented on ​PR #12012:


4 months ago
#18

A commit was made that references the Trac ticket included in the description of this pull request.

SVN changeset: 62441
GitHub commit: ​https://github.com/WordPress/wordpress-develop/commit/9a4f91d6b22eb32798bb0998b1b15bef7231b299

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

#19 @gziolo
4 months ago

In 62447:

REST API: Allow-list ability schema response keywords

Switch the Abilities REST API schema response preparation from an internal-keyword deny-list to an allow-list, so ability input and output schemas expose only keywords from rest_get_allowed_schema_keywords()\ plus a small set of additional draft-04 keywords. Unknown or WordPress-internal keywords (e.g. sanitize_callback, example, context, readonly) are no longer exposed to REST clients by default.

Props gziolo, jorgefilipecosta.
See #64955.

​@gziolo commented on ​PR #11997:


4 months ago
#20

A commit was made that references the Trac ticket included in the description of this pull request.

SVN changeset: 62447
GitHub commit: ​https://github.com/WordPress/wordpress-develop/commit/da7588be089fbaeb05ecb4d79dca3cf11815698d

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

#21 @gziolo
4 months ago

In 62449:

Abilities API: Normalize required schema shape for REST responses

Ability schemas are a public contract that REST clients, including the @wordpress/abilities JavaScript client, validate against as standard JSON Schema. The required keyword must therefore use the draft-04 array-of-property-names form, not the draft-03 per-property boolean that rest_validate_value_from_schema() also accepts on the server.

In WP_REST_Abilities_V1_List_Controller::prepare_schema_for_response(), collect per-property required: true booleans into a parent required array, recursively and inside array items. Strip required: false and boolean required values with no draft-04 equivalent, and honor rest_validate_object_value_from_schema() precedence where an existing draft-04 array wins. Only the REST response copy is rewritten; stored schemas and server-side validation are unchanged.

Props gziolo, westonruter, jorgefilipecosta.
See #64955.

​@gziolo commented on ​PR #12013:


4 months ago
#22

A commit was made that references the Trac ticket included in the description of this pull request.

SVN changeset: 62449
GitHub commit: ​https://github.com/WordPress/wordpress-develop/commit/63bf1d9049b62ff6cbaeb430130abf9f2e61e994

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

#23 @gziolo
4 months ago

The preparatory work is complete. Ability input and output schemas sent through the REST API are now aligned with how they're validated on the client in the @wordpress/abilities package: the required keyword is normalized to the standard draft-04 array form, and only the properties recognized by the REST API's own validation are returned from the server (WordPress-internal keys and unsupported keywords are stripped). This keeps the published schema consistent with what the server enforces and ensures it also passes the stricter validation applied on the client. Behavior is now covered by an additional set of unit tests on both the server and the client.

With this in place, we're in a good position to address the feedback raised by @jorgefilipecosta in ​https://github.com/WordPress/wordpress-develop/pull/12013#discussion_r3335149882 and start generalizing the schema handling so it's applied during ability registration as outlined in the ticket. Handling it universally means it would work seamlessly with the WP AI Client and the MCP adapter.

This ticket was mentioned in ​PR #12234 on ​WordPress/wordpress-develop by ​@gziolo.


4 months ago
#24

## What

Add a shared helper, wp_get_json_schema_allowed_keywords(), so Core has one place to decide which JSON Schema keywords may stay in schema output.

It supports two profiles:

  • rest-api — the existing REST API keyword set. This is the default and is used for REST route output.
  • draft-04 — a larger JSON Schema draft-04 set, used when publishing schemas to clients such as the Abilities API. It keeps documentation and passthrough keywords like $ref, definitions, allOf, not, dependencies, and additionalItems.

## Why

This centralizes schema keyword allow-listing in a general-purpose place and removes the Abilities-specific special casing from the REST controller. It is a preparatory step for the AI schema compiler work.

The change keeps a clear line between two ideas:

  • Preserved in schema output — keywords allowed to stay when a schema is serialized to JSON.
  • Validated by WordPress — keywords the server actually enforces.

REST route schema output now flows through the helper, so it can be adjusted with the new wp_json_schema_allowed_keywords filter. Validation internals still call rest_get_allowed_schema_keywords() directly, so REST validation behavior does not change.

## Testing

New unit tests cover the default profile and the unknown-value fallback, the draft-04 keyword set, and that the filter receives the schema profile. A REST server test confirms the filter reaches route schema output. The existing Abilities schema test now sources its allowed keywords from the shared helper.

npm run test:php -- --filter='test_wp_get_json_schema_allowed_keywords|test_get_data_for_route_includes_filtered_json_schema_keywords|test_core_abilities_schemas_use_only_valid_keywords'

🤖 Generated with ​Claude Code

#25 @gziolo
4 months ago

  • Summary Add schema compiler for AI tool calling compatibility → Improve JSON Schema compatibility when handling abilities

Update: After working through the schema flow in more detail, I think we can narrow the Core work here.

A large part of the original mismatch is now addressed by making REST/Abilities schema keyword handling more consistent with the draft-04-style schemas used by the client. With that in place, server and client can operate on similar canonical input/output schemas, and we likely do not need a dedicated wp_compile_ability_schema_for_ai() function in Core.

The remaining compatibility issues appear to be provider-specific. Different AI providers accept different JSON Schema subsets for tool calling and structured outputs, so a single WP Core compiler would either be too strict or would need provider-specific branching.

The proposed pivot is to handle this in the PHP AI Client/provider layer instead: add a default pass-through schema preparation method that is always used when serializing tool input schemas and output schemas for an LLM request. Individual providers could override that method when they need custom handling for their API.

So this ticket can shift from “add a Core ability schema compiler” to “improve schema compatibility for AI tool calling,” with the next step being an upstream PHP AI Client change for provider-specific schema adaptation.

#26 @gziolo
4 months ago

Update: I opened a focused upstream issue in the PHP AI Client to cover the provider-specific part of this problem:

​https://github.com/WordPress/php-ai-client/issues/256

After further review, the relevant compatibility surface for Abilities API tool calling is the input schema passed as FunctionDeclaration::parameters. The model only uses that schema to generate tool-call arguments. Ability output schemas are handled later outside the model.

The proposed PHP AI Client change would add a default no-op method that providers can override before serializing function declarations for tool-calling requests. This lets individual providers adapt tool input schemas for their own API requirements while keeping the canonical WordPress ability schema unchanged.


Follow-up intent: use the same prepared ability input schema for AI tool calling that REST exposes publicly.

using_abilities() currently passes WP_Ability::get_input_schema() directly into FunctionDeclaration::parameters. A follow-up PR should extract the REST abilities schema preparation logic into a shared helper and use it for both REST responses and AI tool declarations.

Last edited 4 months ago by gziolo (previous) (diff)

#27 @gziolo
4 months ago

In 62549:

REST API: Add a shared helper for JSON Schema allowed keywords.

Introduce wp_get_json_schema_allowed_keywords() as the single place to decide which JSON Schema keywords may stay in schema output. It supports two profiles: rest-api (the default, used for REST route output) and draft-04 (a wider draft-04 set used when publishing schemas to clients such as the Abilities API).

Route schema output now flows through the helper and the new wp_json_schema_allowed_keywords filter, replacing the Abilities-specific special casing in the REST list controller. Validation internals still call rest_get_allowed_schema_keywords() directly, so REST validation behavior does not change.

Props gziolo, apermo.
See #64955.

​@gziolo commented on ​PR #12234:


4 months ago
#28

A commit was made that lists the Trac ticket referenced in the description of this pull request.

SVN changeset: 62548
GitHub commit: ​https://github.com/WordPress/wordpress-develop/commit/859d7fea0e316b1bfc6bdfe8efa88e4e9402666b

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

This ticket was mentioned in ​PR #12283 on ​WordPress/wordpress-develop by ​@gziolo.


4 months ago
#29

## What

  • Introduces wp_prepare_json_schema_for_client() for preparing JSON Schemas that are exposed to clients.
  • Reuses the helper for Abilities REST responses and for WP AI Client ability input schemas used as function declaration parameters.
  • Moves detailed schema preparation coverage into the shared JSON Schema tests while leaving REST and AI Client tests focused on integration wiring.

## Why
Ability schemas can include WordPress-internal schema conveniences that are accepted by server-side REST validation but are not portable JSON Schema draft-04 forms. Preparing them through a shared helper keeps REST responses, frontend consumers, and AI tool declarations aligned.

## Testing

  • vendor/bin/phpcs --standard=phpcs.xml.dist src/wp-includes/json-schema.php src/wp-includes/rest-api/endpoints/class-wp-rest-abilities-v1-list-controller.php src/wp-includes/ai-client/class-wp-ai-client-prompt-builder.php tests/phpunit/tests/json-schema.php tests/phpunit/tests/ai-client/wpAiClientPromptBuilder.php tests/phpunit/tests/rest-api/wpRestAbilitiesV1ListController.php
  • php -l src/wp-includes/json-schema.php
  • git diff --check
  • npm run test:php -- --filter 'Tests_JSON_Schema|Tests_AI_Client_PromptBuilder|Tests_REST_API_WpRestAbilitiesV1ListController'

#30 @gziolo
4 months ago

@jason_the_adams, I opened ​https://github.com/WordPress/wordpress-develop/pull/12283. This ensures that the input schema passed to the WP AI Client is cleaned up of WordPress-specific artifacts, thereby increasing the chances of it being accepted by AI providers during the processing of tool calls. I would greatly appreciate your confirmation that the implementation is functioning correctly.

#31 @gziolo
3 months ago

  • Resolution → fixed
  • Status assigned → closed

In 62591:

Abilities API: Reuse JSON Schema client preparation

Introduce wp_prepare_json_schema_for_client() to prepare JSON Schemas that are exposed to clients, and reuse it for Abilities REST responses and for the AI Client ability input schemas that become function declaration parameters.

Ability schemas can include WordPress-internal schema conveniences that server-side REST validation accepts but that are not in portable JSON Schema draft-04 forms. Routing them through one shared helper keeps REST responses, frontend consumers, and and AI tool declarations aligned.

Move the detailed schemae shared JSON Schema tests, and keep the REST and AI Client tests focused on integration wiring.

Fixes #64955.

#32 @gziolo
3 months ago

  • Keywords has-dev-note added

Summary of what landed

The path here evolved during development, so a quick recap of the final shape against the original proposal.

The original plan was a dedicated wp_compile_ability_schema_for_ai() in Core producing provider-strict schemas. Working through the schema flow, we narrowed that down. Core now centralizes the work in two general-purpose helpers rather than an AI-specific compiler:

  • wp_get_json_schema_allowed_keywords() (62549) — single source of truth for which JSON Schema keywords may stay in schema output. Two profiles: rest-api (default, REST route output) and draft-04 (wider set used when publishing schemas to clients such as the Abilities API). Filterable via wp_json_schema_allowed_keywords, which receives the profile.
  • wp_prepare_json_schema_for_client() (62591) — prepares schemas exposed to clients, reused for Abilities REST responses and for the AI Client input schemas that become FunctionDeclaration::parameters. Strips WordPress-internal artifacts (sanitize_callback, arg_options, context, readonly, etc.) and normalizes required from the draft-03 per-property boolean to the draft-04 array form (62449).

The canonical ability schema is unchanged and server-side validation behavior is untouched. Earlier REST cleanup landed in 62427 and 62447.

What this does and does not do

The schema reaching the AI Client is now clean, portable draft-04. It is not provider-strict. We deliberately kept keywords like oneOf, minimum, pattern, and format in the draft-04 profile, since they are valid JSON Schema and useful to non-AI consumers. The remaining incompatibilities are provider-specific, which is why they belong upstream and not in Core: a single Core compiler would either be too strict for general clients or would need per-provider branching that does not belong in WordPress.

Follow-up

Provider-specific adaptation is tracked in ​WordPress/php-ai-client#256: each provider should be able to override the input schema for its own API requirements, which differ between providers and change over time.

#33 @milana_cap
2 months ago

Hi @gziolo, do we consider this last comment the Dev Note? Just checking to know what goes into the Field Guide. Thanks

#35 @gziolo
2 months ago

@milana_cap, great job with the dev note. I have reviewed it, and it perfectly explains the changes applied. Thank you!

#36 @justlevine
5 weeks ago

Hey @gziolo @jason_the_adams, can I get some clarification as to why y'all locked on "draft-04"? Is this a php-ai-client restriction? My understanding is most AI tools (and modern tools in general) rely on at least Draft 2019-09, or even draft-08-patch-1 which to my understanding was explicitly created to handle (and is used by) OpenAI-compliant tooling.

I'm also curious as to why the new json-schema.php functions depends on rest_*() functions, instead of rest_*() being rewritten to consume our new json_schema functions? I'm assuming the goal is still to decouple REST from JSON, at least in the long term.

(I'm exploring in the context of https://core.trac.wordpress.org/ticket/64209 , as noted there and the matching PR I believe deprecating args is more important and a conceptual prerequisite to deprecating entire abilities, and want to understand the barriers around using a spec that supports the deprecated keyword, before proposing any novel suggestions.)

This ticket was mentioned in ​Slack in #core-ai by justlevine. ​View the logs.


5 weeks ago

#38 @gziolo
5 weeks ago

@justlevine Draft 4 comes from WordPress’s existing schema conventions and the ajv-draft-04 validator used by @wordpress/abilities. It isn’t a php-ai-client restriction. This work aligned client-facing schemas with that existing contract while keeping server-side validation unchanged.

As explained in this comment, we moved away from the original Core compiler proposal. AI providers document their own subsets and restrictions, so choosing a newer draft alone wouldn’t ensure compatibility. For example, ​OpenAI excludes allOf, while ​Anthropic supports it with limitations. Provider-specific adaptation is tracked in ​WordPress/php-ai-client#256.

The rest_*() dependency reflects the incremental scope: the new helper reuses the existing REST keyword list, while REST schema output already consumes the new helper. Extracting the validation implementation into general-purpose JSON Schema functions would be a separate refactor.

This ticket didn’t settle a broader draft upgrade or decoupling effort.

This ticket was mentioned in ​PR #14068 on ​WordPress/wordpress-develop by ​@gziolo.


3 days ago
#39

What?

Adds a test for how wp_prepare_json_schema_for_client() handles the default of a schema. It uses a data provider with three cases:

  • An empty default on an object schema is encoded as {}.
  • A non-empty default on an object schema stays the same.
  • An empty default on an array schema stays [].

Why?

In PHP, an empty array is encoded as [] in JSON. For an object schema, clients expect {}. The helper already fixes this, but only a nested property was tested before, not the root schema.

Abilities rely on this. They can use 'default' => array() in an object input schema, and both the REST API and the AI client still send {}. This came up in ​https://github.com/WordPress/wordpress-develop/pull/12141.

Testing

npm run test:php -- --filter test_wp_prepare_json_schema_for_client_encodes_default

Use of AI Tools

The test and this description were drafted with Claude Code and reviewed by me.

🤖 Generated with ​Claude Code

Note: See TracTickets for help on using tickets.