Add Customizer state persistence in changesets (formerly “transactions”)

[westonruter]

The existence of modified settings in the Customizer is restricted to a browser window. When a user changes a control in the Customizer and a setting is thus modified, the changed setting is sent to the Customizer preview either by JavaScript (postMessage), or an Ajax POST request is made to the URL being previewed with the customized data sent along, and then this incoming $_POST data is added to filters so that the changes are reflected when WordPress builds the page to display in the preview iframe.

A downside to this current approach is that if the user navigates away from the Customizer, they lose their settings. To get around this, we added an AYS dialog in #25439, but this still doesn't account for browser crashes or system failures.

Another downside is that whenever the preview needs to load a new URL it has to re-send all the modified settings so that the Customizer preview will have them available to add to the filters, since the Customized data is not persisted in WordPress in any way. There's also a performance hit to continually send all data with each request, which was partially improved with #28580.
I
So I propose that we introduce persisted Customizer settings, in other words Customizer transactions changesets.

When opening the Customizer for the first time, a changeset UUID can be generated. Whenever a setting changes, an Ajax request sends the updated setting to WordPress to be persisted in a customize_changeset post which has a post_name corresponding to that UUID (or a post is created dynamically if not existing already). Any changes made in the Customizer then get amended to the same customize_changeset post, which has a key/value JSON blob as its post_content.

Instead of POSTing all of the customized settings to the preview, we then only have to reference the changeset UUID when loading URLs into the Customizer preview. Indeed, the patch I've worked on does this in a way that resolves #30028 (Load Customizer preview iframe with natural URL) and #23225 by injecting the changeset UUID as a query parameter for the URL being previewed and then amended to any site URL generated in the preview, so that navigating around the site in the preview (even following standard links with GET requests) will ensure that the changeset will continue to be referenced and loaded. In this way, when a changeset is updated the Customizer preview only has to do location.reload() instead of the current approach of doing an Ajax POST request followed by document.write() in the iframe window. (Nevertheless, the existing “seamless refresh” logic can remaij in which a loading PreviewIframe instance can replace the previously loaded one after it has finished loaded.)

As noted in #20714, my patch also injects the changeset UUID in form submissions (GET and POST), as well as in jQuery Ajax requests. This allows you to preview setting changes for full web applications in the Customizer.

Other side-benefits that Customizer changesets will bring us:

• Settings can be drafted and then returned to later.
• Settings can be collaborated on by multiple users (though not concurrently, without some Heartbeat system in place)
• The capability to publish a customize_changeset post can be limited by role, allowing 'Customizer contributors' to submit settings as pending review.
• Settings can be scheduled for going live at a later date by saving the changeset post simply with post_status=future (see #28721)
• With each save in the Customizer resulting in a new changeset post being created, then there is Customizer revision history (see #31088, #31089)
• Accessing the Customizer preview for a changeset needs no special capabilities since the changeset is updated by an authorized user via single Ajax request. This means that Customizer previews (frontend URLs with the changeset UUID amended) can be shared for anonymous users to review.
• Customizer Theme Switch (#31303) could preview another theme and refresh the Customizer without losing settings, and thus no AYS dialog would be needed. (See #36485.)

Something else that motivated my investigation into Customizer changesets is thinking about how the Customizer will relate to the JSON REST API. How can the REST API be improved with the Customizer? If the REST API provides a changesets endpoint for doing CRUD operations on Customizer settings, and if the REST API also has global recognition for a customize_changeset_uuid query parameter in all requests, then it becomes possible for the Customizer to be used to preview changes in applications that merely interact with the JSON REST API, as long as they include the changeset UUID in the requests.

There's a lot of exciting possibilities introduced with Customizer changesets.

Initial alpha Core patch for Customizer changesets can be seen at: ​https://github.com/xwp/wordpress-develop/pull/61

See Make Core blog post: ​https://make.wordpress.org/core/2015/01/26/customizer-transactions-proposal/

Related:

• #30028: Load Customizer preview iframe with natural URL
• #30936: Dynamically create WP_Customize_Settings for settings created on JS client
• #27355: Customizer: Add framework for partial preview refreshes
• #20714: Theme customizer: Impossible to preview a search results page
• #23225: Customizer is Incompatible with jQuery UI Tabs.
• #28721: Scheduled changes for the customizer
• #31088/#31089: Customizer revisions
• #31517: Customizer: show a notice after attempting to navigate to external links in live previews
• #34893: Improve Customizer setting validation model
• #34142: Links with preventDefault() don't have action prevented in Customizer
• #28227: Customizer: Error message instead of blank screen
• #31641: Theme Preview using "Customize.php" error
• #22037: Customizer: Live preview fetches page but does not display
• #36485: Lost pending customizer settings after theme change

[westonruter]

Putting this in 4.2 milestone because I do have a patch and it's my primary focus for this cycle, and will improve a lot of issues and make possible many enhancements.

[DrewAPicture]

Punting to a future release per @westonruter.

[westonruter]

See also #34893 for an improved setting validation model to enable transactional/atomic saving of settings, where if one setting is invalid, all of the settings are blocked from being saved until all are valid. In other words, the transaction should be blocked from commit until all are valid.

[NateWr]

It sounds like you're already aware of the issues using the Customizer with the REST API. I'd just add one more use-case.

If a theme declares REST API endpoints (for example, to drive custom controls), these endpoints are unavailable when switching to the theme in the customizer.

If possible, it would be great if a request with a customize_transaction_uuid could automatically load the appropriate theme before the rest_api_init call.

[westonruter]

@NateWr yes, any requests made in the context of a theme switch preview would need to include the theme as a parameter.

I talked with @rmccue about the PATCH method and that it isn't supported currently. This would really be needed if we were to use the REST API to amend changes (an update to a single setting) to a transaction post. Using a PUT with sparse fields (e.g. just including the one changed setting) would be ambiguous since a transaction would have any number of settings: it wouldn't be known if the intention was to override the resource to be that single setting or if the one supplied setting should just override the one field if already present in the transaction.

[westonruter]

New ticket for implementing support for PATCH requests in the REST API: #35822

[westonruter]

For a prototype of transactions, see work by @valendesigns on ​Customize Snapshots, which is basically “transactions lite”, implementing some of the key features that transactions will facilitate. The transactions proposal includes lower-level architectural changes to the Customizer, so it's not really appropriate as a feature plugin in itself. Nevertheless, as I say that, for 4.6 it would probably be good to create a new feature plugin with a WP_Customize_Manager fork that is loaded instead of the one currently in Core via a temp hook. Otherwise, the Customize Snapshots plugin gives a great perspective into an end-user feature for transactions.

[celloexpressions]

This ticket is high priority for the customize component, and will need to get in in the earlier part of the cycle so that all of the related tickets can be addressed before beta as well after the initial commit (which will fix many of them).

[westonruter]

I'll note that I updated the PR for trunk a week+ ago: ​https://github.com/xwp/wordpress-develop/pull/61

I need to set aside a few days (hopefully next week) to really dig back into it and wrap my mind around the many issues involved here.

[westonruter]

This isn't going to be early due to the level of effort remaining.

[westonruter]

In 37476:

Customize: Add setting validation model and control notifications to augment setting sanitization.

When a setting is invalid, not only will it be blocked from being saved but all other settings will be blocked as well. This ensures that Customizer saves aren't partial but are more transactional. User will be displayed the error in a notification so that they can fix and re-attempt saving.

PHP changes:

• Introduces WP_Customize_Setting::validate(), WP_Customize_Setting::$validate_callback, and the customize_validate_{$setting_id} filter.
• Introduces WP_Customize_Manager::validate_setting_values() to do validation (and sanitization) for the setting values supplied, returning a list of WP_Error instances for invalid settings.
• Attempting to save settings that are invalid will result in the save being blocked entirely, with the errors being sent in the customize_save_response. Modifies WP_Customize_Manager::save() to check all settings for validity issues prior to calling their save methods.
• Introduces WP_Customize_Setting::json() for parity with the other Customizer classes. This includes exporting of the type.
• Modifies WP_Customize_Manager::post_value() to apply validate after sanitize, and if validation fails, to return the $default.
• Introduces customize_save_validation_before action which fires right before the validation checks are made prior to saving.

JS changes:

• Introduces wp.customize.Notification in JS which to represent WP_Error instances returned from the server when setting validation fails.
• Introduces wp.customize.Setting.prototype.notifications.
• Introduces wp.customize.Control.prototype.notifications, which are synced with a control's settings' notifications.
• Introduces wp.customize.Control.prototype.renderNotifications() to re-render a control's notifications in its notification area. This is called automatically when the notifications collection changes.
• Introduces wp.customize.settingConstructor, allowing custom setting types to be used in the same way that custom controls, panels, and sections can be made.
• Injects a notification area into existing controls which is populated in response to the control's notifications collection changing. A custom control can customize the placement of the notification area by overriding the new getNotificationsContainerElement method.
• When a save fails due to setting invalidity, the invalidity errors will be added to the settings to then populate in the controls' notification areas, and the first such invalid control will be focused.

Props westonruter, celloexpressions, mrahmadawais.
See #35210.
See #30937.
Fixes #34893.

[westonruter]

Punting due to transactions being too large to finish patch and test in time for 4.6.

[MikeSchinkel]

Looking at this, I see it tagged as [Javascript] but if it is persistent then I assume it is also related to the back end.

I have a concern but the amount of documentation around this is too great for me to review at this moment. My concern is in making sure that if you add some type of "transactions" to WordPress that they not be coupled to Customizer but that they be a separate feature that could be accessed separately from the Customizer.

Can you speak to that?

[Fab1en]

@MikeSchinkel: the basic idea is to save ongoing settings changes in a persistent manner, so that it can be worked on later, shared with other and be resilient to browser crashes. At the end, this "transaction" can be applied live, so that settings are all modified at once. It is not related to traditional "database transactions" as you may think.

So, it does not only involves JavaScript, but it is closely tied to the Customizer.

[MikeSchinkel]

Replying to Fab1en:

So, it does not only involves JavaScript, but it is closely tied to the Customizer.

But must it be coupled to the Customizer? What if I want to invoke the same transaction concept from a collection of WP CLI scripts? Or from any number of other form and field plugins, such as Pods?

Coupling is generally consider ​Bad Practice(tm); do we really want to take that approach in WordPress core?

[Fab1en]

Replying to MikeSchinkel:

But must it be coupled to the Customizer?

Good question ! @westonruter, what's your opinion ? Would it be possible to access and modify Customizer transactions from outside of the Customizer ? @rmccue : what about adding an endpoint for that in the REST API ?

[westonruter]

@MikeSchinkel @Fab1en right, it's not really a JavaScript focus. I've removed it.

Transactions here make use of the architecture in Customizer's settings (WP_Customize_Setting) to model data in WordPress and provide the interface abstraction for getting the value(), doing logic to sanitize()/validate() a value, applying filters to preview changes, and lastly the logic for persisting the update into the database.

The ​Customize Snapshots plugin implements most of the key concepts on top of the existing Customizer in core. It introduces a customize_snapshot post type which has its post_content consisting of a JSON blob containing the current Customizer state (the dirty settings). Creating these snapshot post types does not depend on the Customizer UI at all. The can be created via WP-CLI or via the REST API. The Snapshots plugin has ​initial read support for listing out snapshots and inspecting their contents. The next step for REST API integration would be allowing ​updates to these snapshots.

Snapshots/transactions can be used completely independently of the Customizer UI. In fact, the Snapshots plugin includes a link in the Customizer UI to load up the frontend of the site with the snapshot UUID added for previewing outside of the Customizer, even if not logged in. If you have a snapshot UUID, you can add it to any request to WP, including REST API calls, to get a response back with the Customizer state associated with that UUID applied.

The REST API interface for manipulating snapshots/transactions would allow for completely alternative UIs for the Customizer. For example, a mobile app could start a new editing session which then writes changes into a snapshot/transaction. When in this session, the user could go into a preview mode which could then supply the snapshot UUID in all of its REST API requests, allowing the user to see how the app would behave once the changes in the snapshot/transaction/changeset are “committed”.

[westonruter]

Putting back on radar.

[westonruter]

I have a patch for testing! There are still several things I want to polish, but please give this PR a try: ​https://github.com/xwp/wordpress-develop/pull/161

Remember, the patch can be applied via grunt patch:https://github.com/xwp/wordpress-develop/pull/161

For collaboration on the patch, please do not amend the patch and upload new diffs to this ticket. Open PRs to the trac-30937 branch on GitHub or ask me and I'll set you up with direct push access.

Note that I've renamed “transactions” to “changesets” which I think makes more sense.

Here's a companion plugin which enables revisions for Changesets and provides access to the custom post type's admin screens: ​https://gist.github.com/westonruter/b2c9edb9a2ee83236dd9b2b4f177ae76

[celloexpressions]

There is a clear usability improvement from the start here, where navigating in the customizer preview feels much more natural, like you're navigating the frontend directly.

A few bugs I noticed initially in testing:

• Persistent preview refreshes when adding a widget.
• Trying to navigate to an external link causes the preview to refresh persistently (in one test) or navigates to the external site momentarily before redirecting back, or another time it just reloaded the current page.
• Changing a menu location (via the select input) cause the customizer to close at one point, prompting the AYS, which I declined. On the plus side, the settings were still there when I navigated back, but they had not been published.
• I got stuck in another set of persistent preview refreshes when removing menu items.
• Across a theme switch, the settings from the previous theme seem to be applied to the new theme (header image, menus, etc.). A theme switch should start a fresh changeset unless the theme in question had previous unpublished changes that should be restored, with the previous changeset being saved as a draft that could be restored later. There would need to be a distinction between options and theme_mods here too.

[westonruter]

@celloexpressions thanks for reviewing. I'm not able, however, to reproduce these issues you reported:

• Persistent preview refreshes when adding a widget.
• Changing a menu location (via the select input) cause the customizer to close at one point, prompting the AYS, which I declined. On the plus side, the settings were still there when I navigated back, but they had not been published.
• I got stuck in another set of persistent preview refreshes when removing menu items.
• Across a theme switch, the settings from the previous theme seem to be applied to the new theme (header image, menus, etc.). A theme switch should start a fresh changeset unless the theme in question had previous unpublished changes that should be restored, with the previous changeset being saved as a draft that could be restored later. There would need to be a distinction between options and theme_mods here too.

I think we'll need to do a screenshare so I can see what is going on.

I did however fix this issue:

• Trying to navigate to an external link causes the preview to refresh persistently (in one test) or navigates to the external site momentarily before redirecting back, or another time it just reloaded the current page.

Now when you hover over a link (or form) in the preview that is not previewable, the link will get a cursor: not-allowed and event.preventDefault(). Also, in regards to accessibility, the wp.a11y.speak() will be invoked to inform the user why clicking the link or submitting the form is not having an effect. This will finally mean that the edit post links, admin links, and external links will provide some hints to the user as to why clicking on them does nothing.

Here's a full list of the new changes pushed up to the feature branch:

• Add Previewer.ready method to replace anonymous closures
• Use explicit previewer variable instead of this or self
• Use links for URL parsing instead of regular expressions
• Make sure links in preview use HTTPS if parent frame uses HTTPS
• Normalize nav_menu_item URL scheme when parent frame is HTTPS to prevent selective refresh upon initial page load
• Ensure frontend can only be loaded into iframe for customizer preview
• Show expected error message or re-auth when accessing customize.php as unauth'ed user
• Add is_cross_domain and get_allowed_urls methods; export both into preview as well as pane
• Disallow users from clicking non-previewable links or submitting non-previewable forms
• Fix existing phpunit tests

See ​https://github.com/xwp/wordpress-develop/pull/161

[westonruter]

New changes: ​https://github.com/xwp/wordpress-develop/pull/161/files/67fc279..c3c6b82

• a1676d9 Fix jshint and jscs issues
• 93ca8f2 Add wp_generate_uuid4() with test
• 84d6e8d Ensure that UUID post_name customize_changeset is not dropped for pending status save for contributor users
• 6e5e61b Fix qunit tests by adding changeset fixture data
• c4b1abb Remove unused export of currentUserCapabilities
• 0e8bacf Create wp.customize.state before loading the preview
• 2276a56 Wait to set preview iframe src until processing finishes
• b8b48ed Clean up pending changeset update code
• c3c6b82 Restore use of url message for iframe navigation

[westonruter]

Key addition to the latest patch: now when you make a change to a theme_mod in theme A (like change background color to red) and then switch to another theme B and activate it instead, the next time you open theme A in the customizer for live preview the changes you made previously will be applied as your customizer state (the color will be background color will be red again). This is implementing ​feedback from @jorbin:

With this feature I would strongly encourage thinking in terms of user flow.

For example, "As a user, when I am live previewing and making changes to the twentyseventeen theme and then switch to live previewing the twentyten theme, I expect the customizations I made to twentyseventeen to be there the next time I live preview it"

Here are the full set of changes since last comment: ​https://github.com/xwp/wordpress-develop/pull/161/files/c3c6b82..8090d71

The full PR: ​https://github.com/xwp/wordpress-develop/pull/161

Commit log:

• ​fe506ff Fix test_previewing_with_switch_to_blog
• ​dc44bf5 Remove deprecated remove_preview_signature
• ​e2d39d4 De-duplicate parseQueryString and add utils to customize-base
• ​9bcb253 Remove references to previewer signature
• ​19b2224 Fix merging of changes on top of existing changes
• ​231a49e Store theme mods with active theme namespace; skip reading changeset value if theme namespace does not match current theme
• ​7fe74f4 Remove erroneous empty_customize_changeset_data error (causes publish failures)
• ​48ef530 Fix saving theme_mod settings and prepare for stashing non-activated themes' theme mods
• ​36987cb Register customize_changeset post type with can_export as false
• ​a3cc2e7 Add missing hook doc for customize_register
• ​9d5de88 Account for JSON_UNESCAPED_SLASHES not being defined
• ​d3da320 Include customize_changeset in list of exported post types if can_export even though _builtin
• ​131145f Create administrator user in wpSetUpBeforeClass and re-use in tests
• ​3918062 Harden changeset update requests to prevent race conditions from overlapping requests; re-send pending changes in next request upon failure
• ​16da045 Remove theme_mod from setting ID prefix since redundant
• ​55fa378 Prevent prematurely refreshing preview if existing changeset requests complete and there are new ones queued
• ​9749ef1 Introduce saving state and let UI listen for it and block saves if currently saving
• ​a90309c Fix typo where active theme settings were added to inactive theme mod settings
• ​0a37637 Ensure that old_sidebars_widgets_data setting is populated for previewed theme

(Log generator: git log c3c6b82..8090d71 ^origin/master --oneline --no-merges --reverse | sed 's#^\([^ ]*\)#* [https://github.com/xwp/wordpress-develop/pull/161/commits/\1 \1]#')

[jorbin]

Patch uploaded based on the PR for people that prefer testing that way. grunt patch:https://github.com/xwp/wordpress-develop/pull/161 can be used as well.

[westonruter]

Latest refinements: ​https://github.com/xwp/wordpress-develop/pull/161/files/8090d71..b582a43

• ​c3254d4 Prevent intercepting link clicks and form submissions if not in preview iframe; also skip widgets and nav menu integrations if not in iframe
• ​1edf58e Abort ajax requests using POST method, and prevent default forms using POST method
• ​185a354 Revert aborting POST ajax requests since breaks selective refresh
• ​f5d81e4 Add excludeCustomized option to wp.customize.previewer.query()
• ​9de3a51 Refine the how query vars are assembled for requests
• ​ec779ad Ignore preparing internal jump links. Fixes #34142
• ​b7018bb Fix populating post values when publishing a changeset post
• ​d2526fa Use setting._dirty flag to find settings to include in changeset; clear _dirty upon succssful save
• ​710deac Fix publishing changeset asynchronously in WP Cron
• ​b582a43 Also remove AYS dialog when clicking on Live Preview button in theme modal

[westonruter]

For how changesets facilitate the scheduling of changes, see comment on #28721:

The infrastructure for this has been implemented in the patch for #30937. […] There is no UI as part of the patch, so to test scheduling a change to the site title in 5 minutes:

1. Change the blogname in the customizer to “Scheduled Title” and note that that a changeset_uuid query param is added to the URL.
2. Open the console and (assuming your browser timezone is the same as the blog's timezone_string):

   inOneMinute = (new Date( new Date().valueOf() + 5 * 60 * 1000 )); 
   changesetDate = inOneMinute.getFullYear() + '-' + ( '00' + ( inOneMinute.getMonth() + 1 ) ).substr( -2 ) + '-' + ( '00' + inOneMinute.getDate() ).substr( -2 ) + ' ' + ( '00' + inOneMinute.getHours() ).substr( -2 ) + ':' + ( '00' + inOneMinute.getMinutes() ).substr( -2 ) + ':' + ( '00' + inOneMinute.getSeconds() ).substr( -2 );
   wp.customize.previewer.save( { 
       status: 'future', 
       date: changesetDate
    } )
   

Update: As of WordPress 4.9 the save call should be changed to the following so that the UI reflects the new state:

wp.customize.state( 'selectedChangesetStatus' ).set( 'future' );
wp.customize.state( 'selectedChangesetDate' ).set( changesetDate );
wp.customize.previewer.save(); // Reads from state for default values.

3. Assuming that WP Cron is running properly your environment, you should see the new site title “Scheduled Title” at the designated time.

Alternatively, here is how you can schedule a change using WP-CLI without going into the customizer at all:

wp post create \
    --post_type=customize_changeset \
    --post_name=$( uuidgen ) \
    --post_status=future \
    --post_date="$( wp eval "echo get_date_from_gmt( gmdate( 'Y-m-d H:i:s', time() + 5 * 60 ) );" )" \
    --post_content='{"blogname":{"value":"Scheduled Title"}}'

If WP Cron isn't firing, you can always wp cron event run publish_future_post

[westonruter]

Latest updates: ​https://github.com/xwp/wordpress-develop/pull/161/files/b582a43..6bf949b

• ​0c1b241 Fix phpdoc in \WP_Customize_Manager::validate_setting_values()
• ​645d4df Ensure changeset context (UUID, post ID, data) are properly set during publishing
• ​b4d5fd4 Add comment explaining why JSON_UNESCAPED_SLASHES is used
• ​0a5c943 Ensure core settings are registered before do_action(customize_register)
• ​cf9e605 Use refresh transport instead of postMessage if previewer no longer alive; remove keep-alive failure from causing navigation to previous URL and instead set new previewerAlive state; move interval magic numbers to settings.timeouts
• ​6bf949b Preserve history state when calling replaceState to update customize_changeset_uuid param; improve logic for detecting URL changes by comparing query params objects

[westonruter]

See new Make Core posts:

• ​Customize Changesets (formerly Transactions) Merge Proposal
• ​Customize Changesets Technical Design Decisions

[westonruter]

Latest updates: ​https://github.com/xwp/wordpress-develop/pull/161/files/6bf949b..c09a0eb

• ​44ab308 Ensure that customize state params persist through wp_redirect()
• ​1764f1b Re-use parseQueryString as opposed to manually manipulating
• ​170bd5c Restore 'refresh' behavior when previewUrl changes to prevent iframe from adding to browser history
• ​3639a8d Refresh the preview when clicking a link with a url matching the current url
• ​f706c5d Fix typo on timeouts property for updateChangeset, fixing failure to debounce updates
• ​da3a524 Fix updating state params to iframe src to replace instead of amend
• ​9bed2d4 Re-use processing state instead of introducing addPendingChangesetUpdateRequest
• ​ea00d19 Add jsdoc tags and fix typo in comment
• ​4fa326d Re-combine JS var declaration with assignment
• ​2351352 Clear cached changeset_data after a changeset post update
• ​5197ca3 Fix failure to remove changeset_uuid query param upon publishing
• ​ca3cb6c Rename customize_changeset_save filter to customize_changeset_save_data
• ​87ed513 Fix undefined index notice when restoring setting capability for setting added during customize_save_after
• ​58b31cf Keep revision numbers for changes to settings to keep track of which dirty settings need to be sent in a changeset update
• ​eef0b65 Eliminate exclude_stashed_theme_mods option, always including unstashed theme mods as basis for theme switch
• ​c8b3520 Only save settings when a changeset post transitions from non-publish to publish
• ​a58a822 Fix bug with passing status when saving changeset

[westonruter]

On Friday I realized a serious performance problem with how changesets were implemented. Which is now obvious to me now, the approach of trying to update the customize_changeset post every time (debounced 250ms) that a setting is updated will hammer the MySQL database with writes. DB writes are very slow compared with reads. What's even worse is that the HTTP request being made to update the changeset was blocking both the full refresh request and the selective refresh request, resulting in a customizer preview that was at least twice as slow.

The solution to the performance problem is to return to using a POST request for previewing changes when the changeset is not yet updated in the DB. Instead of doing an Ajax POST request and then using document.write() into an iframe, the patch instead creates a temporary form and sets the target window for the form to be iframe. This ensures that the document in the iframe will run in the context of the expected URL, and it will also allow us to amend the customized state with any unsaved setting values that have yet to be persisted into the changeset. So also with this change, the changeset is now auto-saved at the AUTOSAVE_INTERVAL and also whenever a blur event is triggered on the customizer window.

Full writeup on the problem and solution can be seen here: ​https://github.com/xwp/wordpress-develop/issues/170

These changes and others made over the weekend: ​https://github.com/xwp/wordpress-develop/pull/161/files/c09a0eb..0753db3

• ​27ae83c Eliminate updating changeset with every change in favor of autosave polling
• ​b4ff52c Remove marking of changeset-saved settings as clean, missed in 58b31cf
• ​1c82e40 Only increment setting revision for added setting if it is initially dirty
• ​0be8eca Introduce wp.customize.dirtyValues method; replace excludeCustomized with excludeCustomizedSaved option for previewer.query()
• ​5f5056d Short-circuit changeset update request if there are no pending changes
• ​2fc8ec8 Populate iframe window via form submitting post request with unsaved values instead of just src to URL with changeset UUID
• ​2cb33f3 Clarify that PreviewFrame.uuid() does not return an actual UUID
• ​7b94257 Remove obsolete reference to Previewer.request
• ​8f0bede Only update lastSavedRevision when changeset update successful
• ​469ffed Vastly simplify requestChangesetUpdate now that debouncing and request locking removed
• ​7f97497 Restore sending settingValidities and _dirty in full refresh
• ​838e415 Request changeset update when customizer window hidden or before unloaded
• ​13b89d4 Remove provision for plugins including customize_changeset on export screen for now
• ​86eec4b Fix _wp_scripts_maybe_doing_it_wrong in WP_Customize_Manager::wp_die()
• ​1794d2f Use GET and POST query vars instead of REQUEST when bootstrapping customizer
• ​5d5a529 Rewrite Ajax GET requests to be POST requests with customized state data included
• ​21f6bf3 Use AUTOSAVE_INTERVAL as changesetAutoSave
• ​ae5a980 Add data-src attribute to iframe to facilitate debugging
• ​70c52ce Add data-src attribute to iframe to facilitate debugging
• ​8b3c3df Ensure that changeset is updated prior to theme preview switch; eliminate replaceState requirement and AYS dialog
• ​f53f8f7 Use window blur instead of page visibility hidden to trigger save
• ​4df7928 Prevent background changeset update if another is currently in progress
• ​a6cd0f2 Remove unnecessary stateQueryParams setting
• ​63e16e2 Clarify when a customize_save request is transactional and allows revisions
• ​7e5fa46 Remove todos
• ​9273c08 Add object caching for changeset UUID to post ID lookup
• ​ea7d3e9 Load preview via iframe[src] if no unsaved dirty changes; otherwise, use form post request into window
• ​f222717 Skip incrementing processing state if requestChangesetUpdate will short-circuit
• ​f265835 Prevent date from being cleared in wp_update_post call
• ​cf91851 Allow date formats accepted by strtotime in customize_save requests
• ​9d615b8 Add link to start new customizer session when wp_die'd due to already-published changeset
• ​9ab8c26 Ensure save & publish button is disabled immediately once there are unsaved changes
• ​3cc1ea8 Break up methods to improve testability
• ​0753db3 Ensure changeset_status is returned as publish when sending future with non-future date

[westonruter]

Nearing completion on unit tests, for PHP at least.

Latest changes: ​https://github.com/xwp/wordpress-develop/pull/161/files/0753db3..001f3d3

• ​8059ce6 Add round of tests and test stubs
• ​bf84a1d Ensure wp_die() method is testable; send re-auth code if messenger channel is present
• ​63f34da Add tests for find_changeset_post_id and changeset_post_id methods
• ​b3ec194 Add tests for changeset_data and customize_preview_init
• ​e51a5ee Add test for add_state_query_params
• ​8068d46 Add tests for is_cross_domain and get_allowed_urls
• ​f12487d Fix IE9 failure to receive messages from preview due to host mismatch
• ​322358d Fix PHP 5.2 compat by checking that json_last_error() exists
• ​2b1b7a7 Add test for new args for validate_setting_values()
• ​aab7580 Test WP_Customize_Manager::unsanitized_post_values() with unstashed theme mods and changeset data
• ​fdfa20a Add tests for save_changeset_post and fix issues uncovered
• ​ffcfe06 Ensure changeset_uuid param is added as soon as a change is made
• ​6bfa681 Fix passing back changeset_status in customize_save response
• ​d2ccbd7 Use bool cast instead of boolval function for PHP compat
• ​001f3d3 Fix obtaining the self URL for subdirectory installs

[westonruter]

In 38810:

Customize: Implement customized state persistence with changesets.

Includes infrastructure developed in the Customize Snapshots feature plugin.

See ​https://make.wordpress.org/core/2016/10/12/customize-changesets-technical-design-decisions/

Props westonruter, valendesigns, utkarshpatel, stubgo, lgedeon, ocean90, ryankienstra, mihai2u, dlh, aaroncampbell, jonathanbardo, jorbin.
See #28721.
See #31089.
Fixes #30937.
Fixes #31517.
Fixes #30028.
Fixes #23225.
Fixes #34142.
Fixes #36485.

[westonruter]

In 38811:

Customize: Split out link click.preview and form submit.preview event handlers from anonymous functions into named methods on wp.customize.Preview for extensibility.

See #30937.

[westonruter]

In 38830:

Customize: Fix unit tests when twentyfifteen is WP_DEFAULT_THEME instead of twentysixteen.

Use an inactive core theme for previewing theme switch instead of assuming twentysixteen is installed and active and twentyfifteen is not.

See #30937.

[nikeo]

Replying to westonruter:
Hi @westonruter Thanks for this update and explanations. I have a quick question about the comment 75 if you don't mind.

So now the setting changes and the changeset silent updates are 2 different processes living their own lives.

When the preview frame gets refreshed with api.previewer.refresh(), the dirty settings are always posted (with a temporary <form>), even the ones that have been saved in the changeset. Is that correct ?

I'm currently developing a feature relying server side on the incoming $_POST['customized']
. I need to know if there's a plan to totally remove this source of customized data in the future, or if it will always be available and accessible server side. With the current implementation, I plan to do it with

$customized_posted_value = $setting -> unsanitized_post_values( 
    array( 
      'exclude_changeset' => true ,
      'exclude_post_data' => false
    )
  );

Can you also confirm this point ?

Thanks !

[westonruter]

Replying to nikeo:

So now the setting changes and the changeset silent updates are 2 different processes living their own lives.
When the preview frame gets refreshed with api.previewer.refresh(), the dirty settings are always posted (with a temporary <form>), even the ones that have been saved in the changeset. Is that correct ?

No, that's not correct. Actually, when you first load up the customizer and haven't made any changes, there will not be a form created at all, and the iframe will get a src that points to the previewed URL with the customize_changeset_uuid query param added. This is because all of the changes have been written into the changeset at that point. However, once you do make a change, then that change will not have been written into the changeset yet and a form will be created which will POST the customized value into the preview.

However, after the AUTOSAVE_INTERVAL of 60 seconds, or when focus is taken off of the customizer or it is unloaded, any such modifications get written to the changeset post. When there are no changes that are not yet written into the changeset, then it uses a regular iframe with a src. The POST data is only used for changes not yet written into the changeset. I ended up taking this approach because DB writes are very slow and so there needed to be a buffer between making changes to settings and the changes being written into the DB.

I'm currently developing a feature relying server side on the incoming $_POST['customized']
. I need to know if there's a plan to totally remove this source of customized data in the future, or if it will always be available and accessible server side. With the current implementation, I plan to do it with

$customized_posted_value = $setting -> unsanitized_post_values( 
    array( 
      'exclude_changeset' => true ,
      'exclude_post_data' => false
    )
  );

That won't work because once the changes have been written into the changeset, then the POST input will be empty. When all pending changes are written into the changeset, then the values are only sourced from the changeset and not from the POST input.

What feature are you developing that requires inspection of $_POST['customized']? Why can't you use $wp_customize->unsanitized_post_values() alone? It will contain the changeset values with any $_POST['customized'] values merged on top of it. If you need to override the values you can use $wp_customize->set_post_value().

Feel free to ping me on #core-customize in Slack to chat about this further as it's probably a better place to discuss your use case.

[nikeo]

Replying to westonruter:

OK thanks got it. The solution in my case will be to override the api.previewer.query method only in a refresh case, to always post the full set of customized values (and not only the ones not yet saved in the DB changeset)
The new utility api.dirtyValues(), with { unsaved : false } as param, will be my friend.

Feel free to ping me on #core-customize in Slack to chat about this further as it's probably a better place to discuss your use case.

I will for my next message about this.

[westonruter]

In 38979:

Customize: Add all dynamic settings for nav menus and nav menu items before previewing to ensure new entries (placeholders) will be recognized.

See #30937.

[westonruter]

In 39060:

Customize: Ensure state query params persist in preview through calls to history.pushState() & history.replaceState().

Allow history to be manipulated before DOM ready by sourcing state params from the current URL instead of from the wp.customize.settings object, since they will be the same anyway. This fixes a JS error since wp.customize.settings is not defined before DOM ready.

Amends [38810].
See #30937.
Fixes #38592.

[westonruter]

In 39180:

Customize: Prevent post_content and post_name from being modified when trashing customize_changeset posts.

See #30937.
Fixes #38719.

[westonruter]

In 39181:

Customize: Store modifying user ID with setting change written into changeset and restore current user when setting is being saved.

Restoring the current user context when saving a setting ensures filters apply as expected, such as Kses. When a user is not associated with a given setting change, continue to override capability to be exist when saving. Skip overwriting setting values in a changeset that have not changed, facilitating concurrent editing and amending a changeset by a user with fewer privileges.

See #30937.
Fixes #38705.

[westonruter]

In 39320:

Customize: Ensure that WP_Customize_Manager::save_changeset_post() returns setting_validities even for supplied values that are unchanged from values in changeset.

Check setting existence and authorization via WP_Customize_Manager::validate_setting_values() even for null values to account for custom params being added to settings, preventing failures from being silently ignored.

See #38705, #30937.
Fixes #38865.

[westonruter]

In 39332:

Customize: Remove iframe-specific behaviors from customize preview when previewing on frontend and not contained inside iframe.

• Strip out customize_messenger_channel from preview window URL when not contained in iframe.
• Allow interacting with unpreviewable links and forms when previewing customized state on frontend.

See #30937.
Fixes #38867.

[westonruter]

In 39409:

Customize: Reject a changeset update when a non-future date is provided and also ensure that a published changeset always gets set to the current date/time.

• Also moves checks from customize_save Ajax handler to the underlying WP_Customize_Manager::save_changeset_post() call which plugins may invoke directly.
• Ensures that customize_save_response filter is always passed an array, with error code available as code.

Props utkarshpatel, westonruter, sayedwp.
See #30937.
Fixes #38943.

[westonruter]

In 39410:

Customize: Reject a changeset update when a non-future date is provided and also ensure that a published changeset always gets set to the current date/time.

• Also moves checks from customize_save Ajax handler to the underlying WP_Customize_Manager::save_changeset_post() call which plugins may invoke directly.
• Ensures that customize_save_response filter is always passed an array, with error code available as code.

Props utkarshpatel, westonruter, sayedwp.
Merges [39409] into 4.7 branch.
See #30937.
Fixes #38943 for 4.7.

[westonruter]

In 39504:

Customize: Ensure changeset_uuid query param is removed from the customize.php window's location once a changeset has been published (committed) with starter content.

Props westonruter, dlh for testing.
See #30937.
Fixes #39081.

[westonruter]

In 39558:

Customize: Fix inability to delete nav menus by preventing preview filters from being added during customize_save admin ajax request.

Also prevent setting nav_menu_locations[...] values to NaN which gets sent as null.

Amends [38810].
See #30937.
Fixes #39103.

[dd32]

In 39570:

Customize: Fix inability to delete nav menus by preventing preview filters from being added during customize_save admin ajax request.

Also prevent setting nav_menu_locations[...] values to NaN which gets sent as null.

Props westonruter.
See #30937, [38810].
Merges [39558] to the 4.7 branch.
Fixes #39103.

[westonruter]

In 40041:

Customize: Extend auto-draft life of a customize_changeset post whenever modified.

Keep bumping the date for the auto-draft to preserve it from garbage-collection via wp_delete_auto_drafts() after 7 days.

See #30937.
Fixes #39713.

[dd32]

In 40099:

Customize: Extend auto-draft life of a customize_changeset post whenever modified.

Keep bumping the date for the auto-draft to preserve it from garbage-collection via wp_delete_auto_drafts() after 7 days.

Props westonruter.
Merges [40041] to the 4.7 branch.
See #30937.
Fixes #39713.

[westonruter]

In 41012:

Customize: Prevent edge case fatal error when attempting to save changes to a changeset that had previously been corrupted.

Check return value of WP_Customize_Manager::get_changeset_post_data() and return if error instead of assuming it is an array.

Amends [38810].
See #30937.
Fixes #41252.

[westonruter]

In 41626:

Customize: Introduce drafting and scheduling for Customizer changesets.

• Incorporates code from the Customize Snapshots and Customize Posts feature plugins.
• Adds a new Publish Settings section for managing the changeset status, scheduled date, and frontend preview link.
• Updates Publish button to reflect the status selected in the Publish Settings (including Save Draft and Schedule).
• Deactivates the Themes section when a non-publish status selected, and deactivates the Publish Settings section when previewing a theme switch.
• Introduces an outer section type (wp.customize.OuterSection in JS) for the Publish Settings section to use and for available widgets and available nav menu panels to use in the future. These sections can be expanded while other sections are expanded.
• Introduces WP_Customize_Date_Time_Control in PHP and wp.customize.DateTimeControl in JS for managing a date/time value.
• Keeps track of scheduled time and proactively publish from the client when the time arrives, as opposed to waiting for WP Cron.
• Auto-publishes a scheduled changeset when attempting to access one that missed its schedule.
• Starts a new changeset if attempting to save a changeset that was previously publish.
• Adds force arg to requestChangesetUpdate() to force an update request even when there are no pending changes.
• Adds utils methods for getCurrentTimestamp and getRemainingTime.
• Adds new state values for selectedChangesetStatus, changesetDate, selectedChangesetDate.
• Fixes logic for when to short-circuit check to close Customizer when there are unsaved changes.
• Adds getter methods for autosaved and branching parameters, with the latter applying the customize_changeset_branching filter.
• Call to establish_loaded_changeset on the fly when changeset_uuid() is called if no changeset UUID was specififed.
• De-duplicates logic for dismissing auto-draft changesets.
• Includes unit tests.

Builds on [41597].
Props sayedwp, westonruter, melchoyce, JoshuaWold, folletto, stubgo, karmatosed, dlh, paaljoachim, afercia, johnregan3, utkarshpatel, valendesigns.
See #30937.
Fixes #39896, #28721, #39275.

[westonruter]

In 42543:

Customize: Let default status for Customizer be draft if user does not have capability to publish.

Amends [41626].
Props sayedwp, westonruter.
See #30937.
Fixes #42686.

[westonruter]

In 42544:

Customize: Let default status for Customizer be draft if user does not have capability to publish.

Amends [41626].
Props sayedwp, westonruter.
See #30937.
Fixes #42686 for 4.9 branch.