Build can fail when `src` folder contains folders with many files

[adamsilverstein]

For local development, I've typically symlinked plugins I am working on into the src/wp-content/plugins folder. These plugins sometimes include a large number of files (typically in a node_modules folder) that causes the copy part of the core build process to hang or fail. Since r43309 which requires a build step for core JavaScript and working out of the build folder, I am having trouble getting the process to complete. I've seen several mentions of this issue in core slack. It is possible this is also an issue for the themes folder for some developers.

In addition, since the build process wipes out the build folder with each build, I can't put my plugins folder in build instead of src.

One possible solution: exclude the plugins folder when wiping build and skip copying any non-core plugins during the copy phase. If we get more explicit about what we delete and copy, developers can have more flexibility in their local setups.

[adamsilverstein]

in 44256.diff​ :

• limit copy to exclude plugins and wildcard prefixes
• exclude plugins folder when cleaning, add a clean:plugins task that does clear plugins

Q: should we add similar explicit paths for theme copies/removal?

[adamsilverstein]

cc: @johnbillion with this patch, you should be able to leave plugins in the build folder - does this fix the issues you were experiencing? What do you think of the approach? any other ideas for how to resolve this?

[pento]

I like this approach, thanks @adamsilverstein!

I think we should be similarly explicit with themes, too.

[netweb]

Replying to adamsilverstein:

It is possible this is also an issue for the themes folder for some developers.

Replying to adamsilverstein:

Q: should we add similar explicit paths for theme copies/removal?

Yes please, themes using SASS, PostCSS etc for theme dev along with Grunt or Gulp would be quite common practice

I like the VCS change to only copy the root .git/.svn folders

Should the clean task also clean the upgrade and uploads folders?

[adamsilverstein]

Should the clean task also clean the upgrade and uploads folders?

Yes, good point, I will add that.

I like the VCS change to only copy the root .git/.svn folders

Yea, the original line:

'!**/.{svn,git}/**'

attempts to exclude any .svn or .git files from copying. The problem here is the prefix which causes the copy process to attempt to traverse every possible prefix path (eg every folder) which leads to the hung process.

In my testing at least on my local system these folders are ignored by default (in which case we could remove the line). It is possible that on other systems/setups that is not the case, I'm going to see if I can figure out where/why this line was added.

[adamsilverstein]

in 44256.2.diff​:

• clean upgrade and uploads folders
• explicit theme includes in build copy step
• even more explicit paths

[netweb]

• Why only copy the twenty 12/13/14 themes? What about twenty 15/16/17 themes?

• I see what you did there with Hello Dolly, nice try, please fix the typo hello.php, not helo.php 🤣

[adamsilverstein]

Why only copy the twenty 12/13/14 themes? What about twenty 15/16/17 themes?

ooops, I didn't realize I had omitted those, will add.

I see what you did there with Hello Dolly, nice try, please fix the typo hello.php, not helo.php 🤣

Doh! that would have been a sneaky trick :)

[adamsilverstein]

in 44256.3.diff​ :

• create the build and clean lists from the same array so same paths are always cleaned and copied. the copy files list has some additional exclusions added with concat
• correct hello typo
• include all themes

[johnbillion]

Could the list of themes be changed to wp-content/themes/twenty* ?

[pento]

In 44256.5.diff​:

• Replace the explicit list of themes with wp-content/themes/twenty*: we already use this shorthand in a few other places in Gruntfile.js.
• Added the /** suffix to the themes and Akismet, so their files are copied.

This is working pretty well for me, let's land it and see what happens.

[pento]

In 43329:

Build Tools: grunt build should only copy Core files.

Historically, grunt build has copied all files from the src directory to the build directory. This is usually fine, but can be super slow when there are lots of custom plugins or themes in the src directory.

To rectify this, we now only copy Core plugins and themes to build.

Props adamsilverstein, pento, johnbillion.
Fixes #44256.