Add additional array validation keywords to rest_validate_value_from_schema

[TimothyBlynJacobs]

JSON Schema describes minItems, maxItems and uniqueItems keywords to assist in validating array types: ​https://json-schema.org/draft/2019-09/json-schema-validation.html#rfc.section.6.4.1 I think this would be helpful to add support for.

[sorenbronsted]

I will work on this

[sorenbronsted]

In connection with working with these validations, the method rest_validate_value_from_schema is getting rather long. I would like to propose that the validation is refactored into smaller functions, all called from rest_validate_value_from_schema. Any objections?

[TimothyBlynJacobs]

rest_validate_value_from_schema is definitely getting rather long, but let's tackle that in a separate ticket.

Added min,max and unique to items

Trac ticket: https://core.trac.wordpress.org/ticket/48821

We should also do research about what the most correct way to check uniqueness would be. SORT_REGULAR does an == comparison which is of particular importance for [arrays and objects](​https://www.php.net/manual/en/language.operators.array.php).

For instance with the given schema and values, should those be considered unique or not?

`json
{

"type": "array",
"uniqueItems": true,
"items": {

"type": "array",
"items": {

"type": "string"

}

}

}
`

`json
[

[

"a",
"b",
"c"

],
[

"b",
"a",
"c"

]

]
`

Or what about

`json
{

"type": "array",
"uniqueItems": true,
"items": {

"type": [

"number",
"integer"

]

}

}
`

`json
[

1.0,
1

]
`

We should look at the spec, tests and other validators.

I have looked into justinrainbow approach and he uses var_exports and thereby converts everything to strings, but in my opinion he fails on numbers because [1,1.0] is unique according to justinrainbow but I think not, and the spec for uniqueItems is rather vague on what is unique means.
I will use justinrainbow approach but use the flag SORT_NUMERIC when the array type is declared numeric.
Thank you for your patience and feedback on my contributions :-)

Could you also take a look at ​https://github.com/ajv-validator/ajv/?

Thank you for your patience and feedback on my contributions :-)

No problem at all, thanks for taking on these tickets!

Trac ticket: https://core.trac.wordpress.org/ticket/48821

[TimothyBlynJacobs]

In 47923:

REST API: Support the (min|max)Items JSON Schema keywords.

A future commit will add support for the uniqueItems keyword.

Props sorenbronsted.
See #48821.

[TimothyBlynJacobs]

@sorenbronsted are you still taking a look at supporting uniqueItems? I'd like to have a patch for this by next week so we can give it some soak time before 5.5 Beta 1.

[sorenbronsted]

@TimothyBlynJacobs yes and I will take a refresh on this.

Could you also take a look at ​https://github.com/ajv-validator/ajv/?

Thank you for your patience and feedback on my contributions :-)

No problem at all, thanks for taking on these tickets!

The testcases I have written are taken are from [official testsuite](​https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/master/tests/draft2019-09/uniqueItems.json), and other implementations uses the same testsuite, so this implementation can the same as others including ajv.

Does the official test suite have tests covering the array/object behavior though?

If not, what I think we want is for arrays order should matter. [ 'a', 'b' ] is distinct from [ 'b', 'a' ]. That means that with an uniqueItems keyword the following would pass validation.

`json
[

[ "a", "b" ],
[ "b", "a" ]

]
`

`json
{

"type": "array",
"uniqueItems": true,
"items": {

"type": "array",
"items": {

"type": "string"

}

}

}
`

For objects, the order _should not_ matter. So the following example should fail validation.

`json
[

{

"a": "1",
"b": "2"

},
{

"b": "2",
"a": "1"

}

]
`

`json
{

"type": "array",
"uniqueItems": true,
"items": {

"type": "object",
"properties": {

"a": {

"type": "string",

},
"b": {

"type": "string"

}

}

}

}
`

This would match the idea that JSON arrays are "ordered" but JSON objects are "unordered".

AFAICT that is what ajv does

• ​https://runkit.com/timothybjacobs/5ef2cd72842f5d001b7f78a5
• ​https://runkit.com/timothybjacobs/5ef2cd9ccd1fe30013c27062

Does the official test suite have tests covering the array/object behavior though?

Yes it does. The above link is the official test suite for unique items.

For objects, the order _should not_ matter. So the following example should fail validation.

`json
[

{

"a": "1",
"b": "2"

},
{

"b": "2",
"a": "1"

}

]
`

`json
{

"type": "array",
"uniqueItems": true,
"items": {

"type": "object",
"properties": {

"a": {

"type": "string",

},
"b": {

"type": "string"

}

}

}

}
`

This would match the idea that JSON arrays are "ordered" but JSON objects are "unordered".

My implementation fails on the object case, so I will fix that. Actually I will implement the official test suite for unique items.

@TimothyBlynJacobs I have added the official json schema test suite for uniqueitems

Trac ticket: https://core.trac.wordpress.org/ticket/48821

Thanks @sorenbronsted!

Support for min and max items was added in https://core.trac.wordpress.org/changeset/47923.

Let's also skip additionalItems and positional items support please.

Let's also skip additionalItems and positional items support please.

Can I ask why you want to skip theses things, when I have put the time and effort into making all the tests work?

I think its a complicated feature to implement and is fairly unintuitive for users to use. I think an object with named properties is almost always a better DUX.

In the latest spec its also been removed from items and moved to a clearer tupleItems.

Something like this:

`php
/

• Checks if an array is made up of unique items. *
• @since 5.5.0 *
• @param array $array The array to check.
• @return bool True if the array contains unique items, false otherwise. */

function rest_validate_array_contains_unique_items( $array ) {

$seen = array();

foreach ( $array as $item ) {

$stabilized = rest_stabilize_value( $item );
$key = maybe_serialize( $stabilized );

if ( ! isset( $seen[ $key ] ) ) {

$seen[ $key ] = 1;

continue;

}

return false;

}

return true;

}

/

• Stabilizes a value following JSON Schema semantics. *
• For lists, order is preserved. For objects, properties are reordered alphabetically. *
• @since 5.5.0 *
• @param mixed $value The value to stabilize. Must already be sanitized. Objects should have been converted to arrays.
• @return mixed The stabilized value. */

function rest_stabilize_value( $value ) {

if ( is_scalar( $value )

is_null( $value ) ) {

return $value;

}

if ( is_object( $value ) ) {

_doing_it_wrong( FUNCTION, ( 'Cannot stabilize objects. Convert the object to an array first.' ), '5.5.0' );

return $value;

}

ksort( $value );

foreach ( $value as $k => $v ) {

$value[ $k ] = rest_stabilize_value( $v );

}

return $value;

}
`

A possible patch:

`patch
diff --git a/src/wp-includes/rest-api.php b/src/wp-includes/rest-api.php
index b2ea6e9654..8154d03af3 100644
--- a/src/wp-includes/rest-api.php
+++ b/src/wp-includes/rest-api.php
@@ -54,7 +54,7 @@ function register_rest_route( $namespace, $route, $args = array(), $override = f

_doing_it_wrong(

'register_rest_route',
sprintf(

• /* translators: %s: rest_api_init */

+ /* translators: %s: rest_api_init */

( 'REST API routes must be registered on the %s action.' ),
'<code>rest_api_init</code>'

),

@@ -1438,6 +1438,63 @@ function rest_handle_multi_type_schema( $value, $args, $param = ) {

return $best_type;

}

+/
+ * Checks if an array is made up of unique items.
+ *
+ * @since 5.5.0
+ *
+ * @param array $array The array to check.
+ * @return bool True if the array contains unique items, false otherwise.
+ */
+function rest_validate_array_contains_unique_items( $array ) {
+ $seen = array();
+
+ foreach ( $array as $item ) {
+ $stabilized = rest_stabilize_value( $item );
+ $key = serialize( $stabilized );
+
+ if ( ! isset( $seen[ $key ] ) ) {
+ $seen[ $key ] = 1;
+
+ continue;
+ }
+
+ return false;
+ }
+
+ return true;
+}
+
+/
+ * Stabilizes a value following JSON Schema semantics.
+ *
+ * For lists, order is preserved. For objects, properties are reordered alphabetically.
+ *
+ * @since 5.5.0
+ *
+ * @param mixed $value The value to stabilize. Must already be sanitized. Objects should have been converted to arrays.
+ * @return mixed The stabilized value.
+ */
+function rest_stabilize_value( $value ) {

+ if ( is_scalar( $value )

is_null( $value ) ) {

+ return $value;
+ }
+
+ if ( is_object( $value ) ) {
+ _doing_it_wrong( FUNCTION, ( 'Cannot stabilize objects. Convert the object to an array first.' ), '5.5.0' );
+
+ return $value;
+ }
+
+ ksort( $value );
+
+ foreach ( $value as $k => $v ) {
+ $value[ $k ] = rest_stabilize_value( $v );
+ }
+
+ return $value;
+}
+

/

• Validate a value based on a schema. *

@@ -1492,10 +1549,12 @@ function rest_validate_value_from_schema( $value, $args, $param = ) {

$value = rest_sanitize_array( $value );

• foreach ( $value as $index => $v ) {
• $is_valid = rest_validate_value_from_schema( $v, $argsitems?, $param . '. $index .?' );
• if ( is_wp_error( $is_valid ) ) {
• return $is_valid;

+ if ( isset( $argsitems? ) ) {
+ foreach ( $value as $index => $v ) {
+ $is_valid = rest_validate_value_from_schema( $v, $argsitems?, $param . '. $index .?' );
+ if ( is_wp_error( $is_valid ) ) {
+ return $is_valid;
+ }

}

}

@@ -1509,22 +1568,9 @@ function rest_validate_value_from_schema( $value, $args, $param = ) {

return new WP_Error( 'rest_invalid_param', sprintf( ( '%1$s must contain at most %2$s items.' ), $param, number_format_i18n( $argsmaxItems? ) ) );

}

• if ( isset( $argsuniqueItems? ) && $argsuniqueItems? ) {
• foreach ( $value as $index => &$object ) {
• if ( is_array( $object ) ) {
• ksort( $object );
• }
• }
• $unique = array_map(
• function ( $e ) {
• return var_export( $e, true );
• },
• $value
• );
• if ( count( $value ) !== count( array_unique( $unique ) ) ) {
• /* translators: 1: Parameter */
• return new WP_Error( 'rest_invalid_param', sprintf( ( '%1$s has duplicate items.' ), $param ) );
• }

+ if ( ! empty( $argsuniqueItems? ) && ! rest_validate_array_contains_unique_items( $value ) ) {
+ /* translators: 1: Parameter */
+ return new WP_Error( 'rest_invalid_param', sprintf( ( '%1$s has duplicate items.' ), $param ) );

}

}

@@ -1615,7 +1661,7 @@ function rest_validate_value_from_schema( $value, $args, $param = ) {

return new WP_Error(

'rest_invalid_param',
sprintf(

• /* translators: 1: Parameter, 2: Number of characters. */

+ /* translators: 1: Parameter, 2: Number of characters. */

_n( '%1$s must be at least %2$s character long.', '%1$s must be at least %2$s characters long.', $argsminLength? ),
$param,
number_format_i18n( $argsminLength? )

@@ -1627,7 +1673,7 @@ function rest_validate_value_from_schema( $value, $args, $param = ) {

return new WP_Error(

'rest_invalid_param',
sprintf(

• /* translators: 1: Parameter, 2: Number of characters. */

+ /* translators: 1: Parameter, 2: Number of characters. */

_n( '%1$s must be at most %2$s character long.', '%1$s must be at most %2$s characters long.', $argsmaxLength? ),
$param,
number_format_i18n( $argsmaxLength? )

@@ -1647,7 +1693,7 @@ function rest_validate_value_from_schema( $value, $args, $param = ) {

The "format" keyword should only be applied to strings. However, for backward compatibility,
we allow the "format" keyword if the type keyword was not specified, or was set to an invalid value.
if ( isset( $argsformat? )

&& ( ! isset( $argstype? ) + && ( ! isset( $argstype? )

'string' === $argstype?	! in_array( $argstype?, $allowed_types, true ) )
'string' === $argstype?	! in_array( $argstype?, $allowed_types, true ) )

) {

switch ( $argsformat? ) {

case 'hex-color':

@@ -1815,7 +1861,7 @@ function rest_sanitize_value_from_schema( $value, $args, $param = ) {

This behavior matches rest_validate_value_from_schema().
if ( isset( $argsformat? )

&& ( ! isset( $argstype? ) + && ( ! isset( $argstype? )

'string' === $argstype?	! in_array( $argstype?, $allowed_types, true ) )
'string' === $argstype?	! in_array( $argstype?, $allowed_types, true ) )

) {

switch ( $argsformat? ) {

case 'hex-color':

diff --git a/tests/phpunit/tests/rest-api/rest-schema-validation.php b/tests/phpunit/tests/rest-api/rest-schema-validation.php
index fae968b40e..37c03be6d1 100644
--- a/tests/phpunit/tests/rest-api/rest-schema-validation.php
+++ b/tests/phpunit/tests/rest-api/rest-schema-validation.php
@@ -906,11 +906,9 @@ class WP_Test_REST_Schema_Validation extends WP_UnitTestCase {

$this->assertWPError( rest_validate_value_from_schema( 'foobar', $schema ) );

}

• /
• * Data providor for uniqueitems tests
• * @return Generator
• */
• public function data_uniqueitems() {

+ public function data_unique_items() {
+ $all_types = array( 'object', 'array', 'null', 'number', 'integer', 'boolean', 'string' );
+

the following test suites is not supported at the moment
$skip = array(

'uniqueItems with an array of items',

@@ -919,6 +917,9 @@ class WP_Test_REST_Schema_Validation extends WP_UnitTestCase {

'uniqueItems=false with an array of items and additionalItems=false',

);
$suites = json_decode( file_get_contents( DIR . '/json_schema_test_suite/uniqueitems.json' ), true );

+
+ $tests = array();
+

foreach ( $suites as $suite ) {

if ( in_array( $suitedescription?, $skip, true ) ) {

continue;

@@ -929,26 +930,36 @@ class WP_Test_REST_Schema_Validation extends WP_UnitTestCase {

}
items is required for our implementation
if ( ! isset( $suiteschema?items? ) ) {

• $suiteschema?items? = array( 'type' => );

+ $suiteschema?items? = array(
+ 'type' => $all_types,
+ 'items' => array(
+ 'type' => $all_types,
+ ),
+ );

}
foreach ( $suitetests? as $test ) {

• yield array( $test, $suite );

+ $tests[] = array( $test, $suite );

}

}

+
+ return $tests;

}

/

• @ticket 48821 *

• * @dataProvider data_uniqueitems

+ * @dataProvider data_unique_items

*/

• public function test_uniqueitems( $test, $suite ) {

+ public function test_unique_items( $test, $suite ) {

$test_description = $suitedescription? . ': ' . $testdescription?;
$message = $test_description . ': ' . var_export( $testdata?, true );

+
+ $valid = rest_validate_value_from_schema( $testdata?, $suiteschema? );
+

if ( $testvalid? ) {

• $this->assertTrue( rest_validate_value_from_schema( $testdata?, $suiteschema? ), $message );

+ $this->assertTrue( $valid, $message );

} else {

• $this->assertWPError( rest_validate_value_from_schema( $testdata?, $suiteschema? ), $message );

+ $this->assertWPError( $valid, $message );

}

}

}

`

[TimothyBlynJacobs]

In 48357:

REST API: Add support for the uniqueItems keyword.

Props sorenbronsted.
Fixes #48821.

Merged.