WordPress.org

Make WordPress Core

Opened 13 months ago

Last modified 6 weeks ago

#43731 new task (blessed)

Use Webpack + NPM scripts to build all the things

Reported by: omarreiss Owned by:
Milestone: Future Release Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords:
Focuses: Cc:

Description

With the introduction of Webpack in WordPress core, we could consider using it to perform build tasks currently performed with Grunt.

The main advantage of having Webpack build everything is that this would reduce the different tools developers would have to learn in the standard development process for WordPress core. NPM scripts would also make the available tools more discoverable and understandable through package.json.

Having a unified approach towards building / bundling also helps in other area's, such as general asset management and devtooling. It could for example be great if we could start using Webpack dev server at some point to serve assets in development and introduce hot module replacement.

Webpack would be totally suitable for this goal, as can also be read from its documentation:

Webpack is a module bundler. Its main purpose is to bundle JavaScript files for usage in a browser, yet it is also capable of transforming, bundling, or packaging just about any resource or asset.

Webpack provides a unified approach towards building. For example: It would be quite easy to configure Webpack to perform the following tasks:

The above tasks are just some of the more prominent examples. But anything is possible really. NPM scripts can be used to invoke the separate tasks like linting, building etc.

Attachments (4)

jshint.patch (7.9 KB) - added by youknowriad 13 months ago.
JSHint to NPM script
jshint.2.patch (8.1 KB) - added by youknowriad 13 months ago.
I was missing some files in the original patch. That npm script line is way too big, I was thinking of creating a separate bin/jshint script, what do you think?
wordpress_webpack.patch (342.0 KB) - added by herregroen 10 months ago.
wordpress_webpack.2.patch (309.7 KB) - added by herregroen 10 months ago.

Download all attachments as: .zip

Change History (24)

This ticket was mentioned in Slack in #core-js by adamsilverstein. View the logs.


13 months ago

#2 @netweb
13 months ago

  • Type changed from enhancement to task (blessed)

#3 @dingo_bastard
13 months ago

I had some issues when using copy-webpack-plugin to bundle my plugin, and I used https://github.com/gregnb/filemanager-webpack-plugin. Worked better than copy-webpack-plugin.

@youknowriad
13 months ago

JSHint to NPM script

#4 @youknowriad
13 months ago

So I started looking at this today, and the Gruntfile.js looks way too complex to move to WebPack on one step. I propose we do it iteratively: move task by task to npm scripts while keeping Grunt as the task manager, and at some point, it will become obvious that we can just remove it.

I know we're also planning to move to ESlint instead of JSHint and this would also be easier if the JSHint setup was a separate npm script. So in this first patch I'm proposing to move the JSHint script to an npm script, still call it in the Gruntfile to keep the toolchain as is (precommit).

I don't have a strong confidence in Trac and the workflow around the patches and stuff. Let me know if I'm doing something wrong.

Last edited 13 months ago by youknowriad (previous) (diff)

@youknowriad
13 months ago

I was missing some files in the original patch. That npm script line is way too big, I was thinking of creating a separate bin/jshint script, what do you think?

This ticket was mentioned in Slack in #core-js by omarreiss. View the logs.


12 months ago

This ticket was mentioned in Slack in #core by jeremyfelt. View the logs.


11 months ago

This ticket was mentioned in Slack in #core-js by davis. View the logs.


11 months ago

#8 @herregroen
10 months ago

During the course of WordCamp Europe I've prepared a patch that will build all source javascript using WebPack.

This allows us to have a single pipeline through which all JS passes. All grunt tasks that were previously involved in building Javascript have been replaced with equivalent WebPack plugins so that everything is handled in one place.

The main advantage of doing so is significantly improved watch times for JS, most files are now under 100ms after these changes on my machine and even the biggest files only take around 350ms.

Overall build times are mostly the same as vendor and node_module dependencies are currently handled in the exact same manner ( copy, minify and validate ). However this does open the path to start importing dependencies such as jQuery through require and building them with the CommonsChunkPlugin, doing so should significantly speed up the overall build time.

Another advantage is that this open up the path to start splitting up some of the extremely large JS files that exist in WordPress, such as the Customiser, into a much more modular approach while maintaining backwards compatibility in our output files. This should make it trivial to set up the Customiser in a similar way as the Media Library.

This will also make it much easier to start extracting useful parts of WordPress JS to the Packages repo and importing these as a dependency into WordPress as well as easily allowing access to the same dependencies for example Gutenberg is using without having to duplicate code or jump through any hoops.

WebPack does add it's own bootstrap code to each file it's build which slightly increases file sizes, while with most files this is an extremely minimal increase we do have some JS files that are only several lines and in these cases it's a relatively large increase. I think the ideal solution here would be to phase out these files and instead use the options WebPack provides to import these where necessary, but if this is problematic on the short term it's possible to adapt the production build to instead of fully building these files simply copy and minify them to ensure there's no difference in the final build.

The current patch successfully builds all JS with no files missing and all tests passing.

#9 @jorbin
10 months ago

@herregroen An initial quick scan through the patch shows a lot of docs changes, including many being deleted and doc blocks being condensed.

#10 @herregroen
10 months ago

@jorbin My bad, I'd based my changes on Omar's working copy of the @output tags which were compressed.

I've uploaded a new patch that's based on the latest trunk.

It is still based on Omar's patch in #44371 as that's required for this to work. His changes there are included in my patch to allow everything to actually work and be testable. Most of the comment changes are from that patch and have simply been moved from the var declaration to the actual assignment.

The only comment change should be removing the @output tag from the embed.js file as this can't be build by webpack due to the issue linked in the comment. I've expanded that comment to explain this.

Last edited 10 months ago by herregroen (previous) (diff)

This ticket was mentioned in Slack in #core-js by adamsilverstein. View the logs.


10 months ago

This ticket was mentioned in Slack in #core-js by aduth. View the logs.


10 months ago

This ticket was mentioned in Slack in #core-js by herregroen. View the logs.


9 months ago

This ticket was mentioned in Slack in #core by omarreiss. View the logs.


9 months ago

#15 @mnelson4
8 months ago

I believe using npm and webpack may address https://core.trac.wordpress.org/ticket/44271.

FYI we use npm for our plugin's build process. One significant advantage is that we can use the command webpack --watch --watch-poll (see https://github.com/eventespresso/event-espresso-core/blob/master/docs/AA--Javascript-in-EE/build-process.md#watch) which uses poling instead of waiting for the OS to indicate that a file was changed. This is helpful because normal watching doesn't work when used from inside a VM (like VVV) and your host machine is on Windows.

#16 @omarreiss
8 months ago

In 43577:

General: Explicitly assigns all JS globals to the window.

Many variables in the JavaScript were defined in the global scope without being explicitly assigned to the window. When built with Webpack, the code gets encapsulated in anonymous functions and those implicit globals get assigned to the wrong scope. This patch prevents that from happening.

Fixes #44371. See #43731.

This ticket was mentioned in Slack in #core-js by adamsilverstein. View the logs.


8 months ago

This ticket was mentioned in Slack in #core-js by aduth. View the logs.


8 months ago

#19 @pento
3 months ago

  • Milestone changed from Awaiting Review to Future Release
  • Version trunk deleted

This ticket was mentioned in Slack in #core-js by netweb. View the logs.


6 weeks ago

Note: See TracTickets for help on using tickets.