Update CodeMirror to latest v5 version

[TobiasBg]

From what I can see in the JS file, we are using CodeMirror version 5.29.1 which is over 2 years old, according to ​https://codemirror.net/doc/releases.html

A lot has happened since then, with the current release being 5.49.2.

We should update and set a good example. Getting this in early for 5.4 will allow for testing.

[desrosj]

Thanks for this one, @TobiasBg!

Preferably, #41870 would be fixed before/at the same time as updating CodeMirror so that updating in the future is much easier.

Though that is not a steadfast blocker, if you are interested in this, I recommend looking at how to help out there first.

[davidbaumwald]

This ticket still needs patch and soak time. With 5.4 Beta 1 approaching, this is being moved to Future Release. If any maintainer or committer feels this can be resolved in time or wishes to assume ownership during a specific cycle, feel free to update the milestone accordingly.

[desrosj]

Just noting, the ​current version is now 5.65.9.

[jorbin]

When this is worked on, testing to see if #42822 is still valid should be included.

[westonruter]

Latest is now ​5.65.18. This was released Sep 20, 2024.

Nevertheless, CodeMirror v5 is now legacy. What would be preferable (but much riskier and more work) is updating to v6.

[westonruter]

CodeMirror was introduced in #41897. The version used in the wp-codemirror script is 5.29.1-alpha-ee20357 which remains unchanged since it was introduced 9 years ago in [41376] (to fix #12423/#39892).

So yes, it is very overdue for updating.

Trac ticket: https://core.trac.wordpress.org/ticket/48456

For Gemini CLI, I provided the following spec/plan:

From commit dbace684e214c434ede007bc69d964155d18fcf6 you can see that CodeMirror was originally introduced. In a91d3980dfee019e09b91919dac290716082203e it was integrated into the Customizer. Note also in 5bb7fc10c5185f26eb52cfa2c39bcbb80d65173a that JSHint was removed due to license compatibility. This was many years ago. The version of CodeMirror installed then was 5.29.1-alpha-ee20357. However, the latest version of CodeMirror v5 is 5.65.18 as seen at <​https://github.com/codemirror/codemirror5/releases>. Granted, there is now a complete rewrite in CodeMirror v6, but before we upgrade to v6 we should first upgrade to the latest v5, as requested in <https://core.trac.wordpress.org/ticket/48456>. Part of the reason why we didn't upgrade previously is that there was no script to facilitate upgrading.

I want you to come up with a plan for how we might upgrade to the latest version of CodeMirror v5, taking into account all of the release notes for all versions of CodeMirror since 5.29.1, or rather, since 5.29.0 <​https://github.com/codemirror/codemirror5/releases/tag/5.29.0>. I don't actually see that 5.29.1 was ever released since the actual release that included commit <​https://github.com/codemirror/codemirror5/commit/ee20357> was 5.30.0. In any case, I need a plan to upgrade CodeMirror which is currently located in @src/js/_enqueues/vendor/codemirror/ (where initially it was located at @src/wp-includes/js/codemirror but with ef37f002ee3189cbe647a853d287eb4e5e1b0fba there was a build step introduced which caused files to be moved around).

So I need a plan for:

1. A new Grunt task like codemirror:update which will handle pulling down the latest version of CodeMirror v5 and to update the versions for wp-codemirror script/style in @src/wp-includes/script-loader.php as well as the other script handles in that directory, specifically the add-ons for jshint, jsonlint, htmlhint, csslint. The jshint add-on was replaced in 5bb7fc10c5185f26eb52cfa2c39bcbb80d65173a with esprima, so that is out of scope for updating.
2. Updating dependencies of the wp-codemirror script to account for any back-compat breakages. Note that the code-editor script/style is the main dependency. Take care to note how the code-editor dependency is used, namely in the wp-theme-plugin-editor script, the wp_enqueue_code_editor() function where the script gets enqueued. The wp_enqueue_code_editor() function is used in @src/wp-includes/widgets/class-wp-widget-custom-html.php and src/wp-includes/customize/class-wp-customize-code-editor-control.php as well as in @src/wp-admin/theme-editor.php and @src/wp-admin/plugin-editor.php.

You will not make any changes to the codebase for now. Only come up with a plan. Remember that backwards compatibility is paramount in WordPress, so take care to consider how existing ecosystem code may be extending CodeMirror in 5.29.1-alpha-ee20357 which has been present in core for 9 years.

I also provided ​https://github.com/WordPress/better-code-editing as context for where the CodeMirror integration originally came from.

I then iterated on the plan with Gemini for awhile, and came up with the following:

# Plan for Upgrading CodeMirror to v5.65.18

This plan upgrades CodeMirror to version 5.65.18 by transitioning from a manually bundled artifact to a build-generated asset sourced from npm. It mirrors the integration strategy of the better-code-editing plugin (the feature's origin) while adapting to the modern wordpress-develop build system (Webpack) and updating dependencies to their latest compatible versions.

## 1. Dependency Management [COMPLETED]
Add codemirror and required linters to package.json dependencies. We are updating these to the latest versions compatible with CodeMirror 5, rather than using the outdated versions from the original plugin.

• Action: Update package.json dependencies with:
  • "codemirror": "5.65.18"
  • "csslint": "1.0.5"
  • "htmlhint": "1.1.4"
  • "jsonlint": "1.6.3"
  • "esprima": "4.0.1"
• Note: jshint is excluded because Core uses a custom fakejshint.js wrapper (powered by esprima) instead of the standard library.

## 2. Source File Cleanup [COMPLETED]
Remove the pre-bundled CodeMirror files and standalone linters from the source tree, as they will be replaced by build artifacts.

• Action: Delete from src/js/_enqueues/vendor/codemirror/:
  • codemirror.min.js
  • codemirror.min.css
  • csslint.js
  • esprima.js
  • htmlhint.js
  • jsonlint.js
• Keep: fakejshint.js and htmlhint-kses.js (Core-specific wrappers).

## 3. Build Tooling Implementation [COMPLETED]

### 3.1. Webpack Entry Point (tools/vendors/codemirror-entry.js)
Create an entry point that replicates the original codemirror.manifest.js. This file aggregates the core library, modes, keymaps, and addons into a single bundle and exposes the global object.

• Content:

  // Define CodeMirror globally before other imports to ensure they attach to it.
  var CodeMirror = require( 'codemirror/lib/codemirror' );
  
  // Keymaps
  require( 'codemirror/keymap/emacs' );
  require( 'codemirror/keymap/sublime' );
  require( 'codemirror/keymap/vim' );
  
  // Addons (Hinting)
  require( 'codemirror/addon/hint/show-hint' );
  require( 'codemirror/addon/hint/anyword-hint' );
  require( 'codemirror/addon/hint/css-hint' );
  require( 'codemirror/addon/hint/html-hint' );
  require( 'codemirror/addon/hint/javascript-hint' );
  require( 'codemirror/addon/hint/sql-hint' );
  require( 'codemirror/addon/hint/xml-hint' );
  
  // Addons (Linting)
  require( 'codemirror/addon/lint/lint' );
  require( 'codemirror/addon/lint/css-lint' );
  require( 'codemirror/addon/lint/html-lint' );
  require( 'codemirror/addon/lint/javascript-lint' );
  require( 'codemirror/addon/lint/json-lint' );
  
  // Addons (Other)
  require( 'codemirror/addon/comment/comment' );
  require( 'codemirror/addon/comment/continuecomment' );
  require( 'codemirror/addon/fold/xml-fold' );
  require( 'codemirror/addon/mode/overlay' );
  require( 'codemirror/addon/edit/closebrackets' );
  require( 'codemirror/addon/edit/closetag' );
  require( 'codemirror/addon/edit/continuelist' );
  require( 'codemirror/addon/edit/matchbrackets' );
  require( 'codemirror/addon/edit/matchtags' );
  require( 'codemirror/addon/edit/trailingspace' );
  require( 'codemirror/addon/dialog/dialog' );
  require( 'codemirror/addon/display/autorefresh' );
  require( 'codemirror/addon/display/fullscreen' );
  require( 'codemirror/addon/display/panel' );
  require( 'codemirror/addon/display/placeholder' );
  require( 'codemirror/addon/display/rulers' );
  require( 'codemirror/addon/fold/brace-fold' );
  require( 'codemirror/addon/fold/comment-fold' );
  require( 'codemirror/addon/fold/foldcode' );
  require( 'codemirror/addon/fold/foldgutter' );
  require( 'codemirror/addon/fold/indent-fold' );
  require( 'codemirror/addon/fold/markdown-fold' );
  require( 'codemirror/addon/merge/merge' );
  require( 'codemirror/addon/mode/loadmode' );
  require( 'codemirror/addon/mode/multiplex' );
  require( 'codemirror/addon/mode/simple' );
  require( 'codemirror/addon/runmode/runmode' );
  require( 'codemirror/addon/runmode/colorize' );
  require( 'codemirror/addon/runmode/runmode-standalone' );
  require( 'codemirror/addon/scroll/annotatescrollbar' );
  require( 'codemirror/addon/scroll/scrollpastend' );
  require( 'codemirror/addon/scroll/simplescrollbars' );
  require( 'codemirror/addon/search/search' );
  require( 'codemirror/addon/search/jump-to-line' );
  require( 'codemirror/addon/search/match-highlighter' );
  require( 'codemirror/addon/search/matchesonscrollbar' );
  require( 'codemirror/addon/search/searchcursor' );
  require( 'codemirror/addon/tern/tern' );
  require( 'codemirror/addon/tern/worker' );
  require( 'codemirror/addon/wrap/hardwrap' );
  require( 'codemirror/addon/selection/active-line' );
  require( 'codemirror/addon/selection/mark-selection' );
  require( 'codemirror/addon/selection/selection-pointer' );
  
  // Modes
  require( 'codemirror/mode/meta' );
  require( 'codemirror/mode/clike/clike' );
  require( 'codemirror/mode/css/css' );
  require( 'codemirror/mode/diff/diff' );
  require( 'codemirror/mode/htmlmixed/htmlmixed' );
  require( 'codemirror/mode/http/http' );
  require( 'codemirror/mode/javascript/javascript' );
  require( 'codemirror/mode/jsx/jsx' );
  require( 'codemirror/mode/markdown/markdown' );
  require( 'codemirror/mode/gfm/gfm' );
  require( 'codemirror/mode/nginx/nginx' );
  require( 'codemirror/mode/php/php' );
  require( 'codemirror/mode/sass/sass' );
  require( 'codemirror/mode/shell/shell' );
  require( 'codemirror/mode/sql/sql' );
  require( 'codemirror/mode/xml/xml' );
  require( 'codemirror/mode/yaml/yaml' );
  
  // Global Exposure
  if ( ! window.wp ) {
      window.wp = {};
  }
  window.wp.CodeMirror = CodeMirror;
  

### 3.2. Webpack Configuration [COMPLETED]
Create tools/webpack/codemirror.config.js to handle the JS bundling.

• Input: tools/vendors/codemirror-entry.js
• Output: src/wp-includes/js/codemirror/codemirror.min.js

### 3.3. Grunt Configuration (Gruntfile.js) [COMPLETED]
Update Gruntfile.js to manage the full build process for CodeMirror assets.

1. Webpack Task: Added a codemirror target pointing to the new config.
2. CSS Concatenation: Added a concat:codemirror target to bundle styles into codemirror.css:
   • node_modules/codemirror/lib/codemirror.css
   • node_modules/codemirror/addon/hint/show-hint.css
   • node_modules/codemirror/addon/lint/lint.css
   • node_modules/codemirror/addon/dialog/dialog.css
   • node_modules/codemirror/addon/display/fullscreen.css
   • node_modules/codemirror/addon/fold/foldgutter.css
   • node_modules/codemirror/addon/merge/merge.css
   • node_modules/codemirror/addon/scroll/simplescrollbars.css
   • node_modules/codemirror/addon/search/matchesonscrollbar.css
   • node_modules/codemirror/addon/tern/tern.css
3. CSS Minification: Added a cssmin:codemirror target.
4. Copy Task: Added a copy:codemirror target to copy linters and preserve custom wrappers.
5. Build Integration: Added build:codemirror to the main build task.

IMPORTANT: Build artifacts generated in src/wp-includes/js/codemirror/ are ignored by Git (via the /src/wp-includes/js rule in .gitignore). These files must NOT be staged or committed.

## 4. Script Loader Update (src/wp-includes/script-loader.php) [COMPLETED]
Update the registered versions for the following handles to match the new packages:

• wp-codemirror (v5.65.18)
• csslint (v1.0.5)
• htmlhint (v1.1.4)
• jsonlint (v1.6.3)
• esprima (v4.0.1)

## 5. Verification [COMPLETED]

1. Build: Run grunt build:codemirror --dev and verify existence of:
   • src/wp-includes/js/codemirror/codemirror.min.js
   • src/wp-includes/js/codemirror/codemirror.min.css
   • Linter files in the same directory.
2. Runtime: Check window.CodeMirror.version and window.wp.CodeMirror in the browser console.
3. Functionality:
   • Test Theme Editor (syntax highlighting, line numbers).
   • Test Custom CSS (linting warnings).
   • Test Custom HTML Widget.

I then told Gemini to execute the plan, creating commits for each logical step.

# Initial Gemini Review

The changes look solid and represent a significant improvement in how CodeMirror is managed in WordPress Core, moving from committed artifacts to a proper build process backed by npm dependencies.

I have a few items for you to address:

1. License Header in JS Bundle: The previous codemirror.min.js included a comment block with license information. Your new Webpack config (tools/webpack/codemirror.config.js) uses TerserPlugin with extractComments: false and comments: false, which strips all comments.
   • Action: Please configure TerserPlugin to preserve @license or @preserve comments, or use webpack.BannerPlugin to prepend the CodeMirror license information to the generated codemirror.min.js.

Addressed in 2fb47e41d624d90749a7fbe8dfb46027db671115, 3fb228150ce80d7439e6497eb96aa51741df8d72, 79295b3b6000c31bc4b1ea85459bcb6471c034ab

2. CSS Source Mapping/Banner: In Gruntfile.js, you added a process function to concat:codemirror:

   process: function( src, filepath ) {
       return '/* Source: ' + filepath.replace( 'node_modules/', '' ) + '*/\n' + src;
   }
   

This adds comments to the concatenated CSS file (codemirror.css), but the cssmin:codemirror task will likely strip these comments when generating codemirror.min.css.

• Action: Verify if these source comments are intended to remain in the minified file. If not, this is fine for the unminified version, but ensure the license is preserved here as well if required.

Yes. These comments are useful for when the unminified file is erved.

3. Completeness of Addons: The list of requires in tools/vendors/codemirror-entry.js is extensive.
   • Observation: Please double-check that this list matches the functionality provided by the previous 5.29.1 bundle. Specifically, ensure no used modes or addons (like match-highlighter or specific keymaps) were accidentally omitted if they were previously present.

4. fakejshint.js and htmlhint-kses.js Location:
   • You are copying these from SOURCE_DIR + 'js/_enqueues/vendor/codemirror/' to WORKING_DIR + 'wp-includes/js/codemirror/'.
   • Observation: Since these are Core-specific wrappers and not external vendor code (strictly speaking), ensure they are tracked correctly in git. The delete command removed the *other* files in that directory, leaving these two. This seems correct, but just confirming their source location is intended to stay in vendor.

5. Script Loader Versions:
   • In src/wp-includes/script-loader.php, you updated the versions.
   • Nit: jshint is still listed as 2.9.5. Since it relies on esprima (which you updated to 4.0.1) and fakejshint.js, verify if fakejshint.js needs any updates to work with the newer esprima version.

Overall, excellent work modernizing this dependency chain. Fix the license header issue, and it should be ready.

[westonruter]

I'm really happy with how quickly this upgrade came together, thanks to the assists from Gemini and Copilot. This is ready for review. I've done initial smoke tests to confirm the syntax highlighting and linting continues to work as expected in the current core surfaces: theme/plugin file editors, Additional CSS in Customizer, and Custom HTML widget (legacy).

I tested this PR in Playground, by installing my plugin (TablePress), which embeds CodeMirror on its "Plugin Options" screen, for adding CSS code.
Everything worked fine with the new version, no issues :-)

[westonruter]

Once this is fixed I'm keen to explore the v6 upgrade, which will be much more liable to cause backwards compatibility breakages for extensions. But hopefully we can find shims to help bridge the gap.

[westonruter]

In addition to the PR attached to this ticket, I've opened a sub-PR to replace Esprima with Espree in the JavaScript linter: ​https://github.com/westonruter/wordpress-develop/pull/4 This is important because Esprima does not support ES6, so any scripts using const, let and the like currently show up as syntax errors.

This deprecates the use of fakejshint.js while at the same time registering espree as a script module. This then gets imported dynamically by a new javascript-lint.js which replaces that module in the CodeMirror build. Note that since classic scripts cannot explicitly depend on script modules (#61500), this is currently implemented as a bit of a hack. A new wp-codemirror script module is registered with an empty string src. This module has espree added as a dynamic dependency. The empty string has the effect of preventing the wp-codemirror script module from being printed, while at the same time any of its dependencies still get added to the importmap.

I expect to file a new ticket for this Esprima to Espree upgrade, once CodeMirror v5 is upgraded.

[westonruter]

In 61539:

External Libraries: Upgrade CodeMirror to latest v5 in addition to updating CSSLint, Esprima, HTMLHint, and JSONLint.

This installs npm packages for codemirror, csslint, esprima, htmlhint, and jsonlint to replace the libraries which had been copied into SVN. A new grunt build:codemirror task is responsible for building CodeMirror as part of the build process. This finally revisits the original CodeMirror integration which was originally developed in the ​Better Code Editing feature plugin in 2017.

Package	Old Version	New Version
codemirror	5.29.1-alpha-ee20357	5.65.20
esprima	4.0.0	4.0.1
jsonlint	1.6.2	1.6.3
htmlhint	0.9.14-xwp	1.8.0

Follow-up to [41376].

Props westonruter, jonsurrell, tobiasbg, desrosj, adamsilverstein, WraithKenny, rafa8626, netweb.
See #12423.
Fixes #48456, #41870.

[westonruter]

In 61611:

Code Editor: Switch from Esprima to Espree for JavaScript linting in CodeMirror.

Esprima is no longer maintained, and it does not support the latest JavaScript features in ES11, as Espree does.

• New Linter Integration: Introduces src/js/_enqueues/vendor/codemirror/javascript-lint.js using espree for parsing and error reporting, replacing the dependency on jshint and esprima scripts.
• Script Modules: Registers espree as a script module and leverages the module_dependencies argument in wp_register_script() to ensure espree is available as a dynamic import.
• Editor Settings: Updates wp_get_code_editor_settings() to use ES11 (ECMAScript 2020) defaults and synchronizes JSHint settings from .jshintrc for compatibility.
• Editable Extensions: Adds .mjs to the list of editable file extensions for plugins and themes.
• Deprecations: Marks esprima and jshint script handles as deprecated.
• Build Tools: Updates Webpack configuration to bundle espree as a module and use the new local javascript-lint.js.

Developed in ​https://github.com/WordPress/wordpress-develop/pull/10806

Follow-up to [61587], [61544], [61539], [42547].

Props westonruter, jonsurrell.
See #64562, #61500, #48456, #42850.
Fixes #64558.

This is a follow-up to:

• ​https://github.com/WordPress/wordpress-develop/pull/10806
• ​https://github.com/WordPress/wordpress-develop/pull/10778

To address:

• [x] Fix TypeScript issues with code-editor.js.
• [x] Address ​https://github.com/WordPress/wordpress-develop/pull/10806#discussion_r2787837736. See dd7579f62c2733dc32b9b23e4d56ac462a0a83f2.

In subsequent PR:

• [ ] Remove jQuery dependency

Trac ticket: https://core.trac.wordpress.org/ticket/48456

## Gemini Summary

This branch introduces comprehensive TypeScript static analysis for JavaScript files using JSDoc, modernizes several Code Editor components, and resolves logical and build-process issues.

#### 1. TypeScript Static Analysis & Type Safety

• Infrastructure: Introduced tsconfig.json configured for JS checking (checkJs: true) and strict mode (strict: true).
• Global Definitions: Created typings/globals.d.ts to define global interfaces for wp, jQuery, _, Backbone, and HTMLHint.
• Enhanced Type Definitions:
  • Defined precise typedefs in code-editor.js (CodeEditorInstance, LintingController, CombinedLintOptions, etc.) to replace generic Object and Function types.
  • Integrated CodeMirror addon types (lint, show-hint) using triple-slash directives.
  • Successfully eliminated almost all generic any usage in favor of specific intersection types and unions.
• Strict Checking Enablement: Resolved numerous strict-mode issues, including null safety for optional callbacks and properties.

#### 2. Code Modernization & Refactoring

• code-editor.js:
  • Refactored to use const and let variable declarations.
  • Replaced deprecated APIs: event.keyCode with event.key and substr() with slice().
  • Switched to native DOM APIs where appropriate (Element.classList, Node.contains).
  • Standardized multi-line object literals with trailing commas.
• htmlhint-kses.js:
  • Refactored to modern ES syntax (const/let, arrow functions, template literals, and for...of loops).
  • Applied Prettier formatting for consistency.

#### 3. Build Process & Reorganization

• File Relocation: Moved WordPress-maintained CodeMirror extensions (fakejshint.js, htmlhint-kses.js, javascript-lint.js) from vendor/ to lib/ to distinguish them from third-party vendor code.
• Redundancy Cleanup: Deleted the redundant esprima.js source file; the build now correctly sources the latest version from the npm package.
• Linter Optimization:
  • Fixed a race condition/double-linting bug during initialization by constructing complete options before editor creation.
  • Removed obsolete nested options.options for linting, following modern CodeMirror standards.
• ESLint Alignment: Added a local .eslintrc.json in lib/codemirror/ to correctly signal ES module parsing for javascript-lint.js while maintaining compatibility for other files.

#### 4. Dependencies

• Added @types/jquery, @types/codemirror, @types/underscore, @types/espree, and @types/htmlhint as devDependencies to support the static analysis effort.

All changes have been verified via tsc and jshint.

## Fixed PhpStorm Inspections

## Use of AI Tools

I used Gemini CLI for granular commits, each of which I reviewed.