WordPress.org

Make WordPress Core

Opened 20 months ago

Last modified 5 months ago

#38603 assigned defect (bug)

Explore adding Yarn and a `yarn.lock` file for NPM module management

Reported by: netweb Owned by: netweb
Milestone: 5.0 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: needs-patch
Focuses: Cc:

Description

YARN: Yarn: A new package manager for JavaScript:

  • Fast: Yarn caches every package it downloads so it never needs to download the same package again. It also parallelizes operations to maximize resource utilization so install times are faster than ever.
  • Reliable: Using a detailed, concise lockfile format and a deterministic algorithm for installs, Yarn is able to guarantee that an install that worked on one system will work exactly the same way on any other system.
  • Secure: Yarn uses checksums to verify the integrity of every installed package before its code is executed.

Yarn is similar to a npm-shrinkwrap.json file though with vast performance increases and improved cross platform compatibility/reliably.

Here's some quick stats:

  • time npm install (First run, existing npm cache)
    • 176.87 seconds
  • time npm install (First run, no existing npm cache)
    • 237.97 seconds
  • time yarn (Initial Yarn run)
    • 21.40 seconds
  • time yarn (Subsequent install after creating yarn.lock and rm -rf node_modules/ && time yarn)
    • 18.82 seconds

Change History (12)

This ticket was mentioned in Slack in #core by netweb. View the logs.


20 months ago

#2 @netweb
19 months ago

Issue of note: https://github.com/yarnpkg/yarn/issues/681 Pinning dependencies behaves differently than npm.

The above issue is not currently of concern, though if we were to have to "pin" an NPM module to a specific semantic version it would become a concern.

#4 @netweb
19 months ago

  • Milestone changed from Awaiting Review to Future Release

Yarn currently is very new and very shiny...

Enough complications arouse in #38657 with 4.7's npm-shrinkwrap.json and as Yarn doesn't currently behave nicely with NPM's shrinkwrap process let's wait a while until it's a little less new and shiny...

#5 @swissspidy
15 months ago

#40421 was marked as a duplicate.

#6 @jnylen0
15 months ago

I should've searched and found this ticket before opening #40421, but before we make a decision here I'd like to see us take one of two steps, with the goal of preserving a clean source tree for people (like me) who are already using yarn:

  • Commit yarn.lock to the source tree. This does not impose a requirement that people start to use yarn, rather it merely provides additional benefits if you do (deterministic installs including all versions of sub-dependencies).
  • Ignore yarn.lock in git and svn. This also does not create any kind of expectation that people must use yarn instead of npm install, though there are a lot of benefits. This will allow people to continue using yarn if they wish, with a clean source tree.

#7 @netweb
15 months ago

I've not revisited the issues previously raised for a while, I'll take a look again in the coming days.

FYI: The "build" server, i.e. build.svn.wordpress.org is not setup for Yarn so switching to Yarn requires some coordination with meta and systems to get it implemented when we can switch to Yarn.

I'm hoping the previous concerns have been addressed as Yarn is soooo much faster 🤞

#8 @jnylen0
15 months ago

@netweb as a first step that doesn't impose a commitment or requirement to use yarn, what do you think about committing yarn.lock soon? In addition to allowing those of us already using yarn to have a clean source tree, this will allow us to get more testing of potential incompatibilities.

#9 @netweb
14 months ago

Replying to netweb:

Issue of note: https://github.com/yarnpkg/yarn/issues/681 Pinning dependencies behaves differently than npm.

The above issue is not currently of concern, though if we were to have to "pin" an NPM module to a specific semantic version it would become a concern.

This appears to been resolved overnight https://github.com/yarnpkg/yarn/issues/681#issuecomment-298036459 :+1:


Replying to jnylen0:

@netweb as a first step that doesn't impose a commitment or requirement to use yarn, what do you think about committing yarn.lock soon? In addition to allowing those of us already using yarn to have a clean source tree, this will allow us to get more testing of potential incompatibilities.

As a workaround running yarn install --no-lockfile will use Yarn to install the npm modules without generating a yarn.lock file, it's not quite as fast as installing with a yarn.lock file being generated, but still faster than npm install

#10 @netweb
14 months ago

A couple of things, NPM 5 is due out tomorrow, it purportedly has a massive performance increase comparable to Yarn, https://github.com/npm/npm/pull/16244

Yarn also includes a yet to be undocumented --frozen-lockfile option, this sounds as if (further research required) it might be a good way in keeping package.json, yarn.lock and npm-shrinkwrap.json in sync with each other.

Npm vs Yarn Performance stats: https://github.com/npm/npm/pull/16244#issuecomment-297190219

It'll be interesting in the coming days to run some performance tests here for our use case

#11 @netweb
13 months ago

Related: #40938 Add npm package-lock.json for npm v5.x

#12 @netweb
5 months ago

  • Milestone changed from Future Release to 5.0
  • Owner set to netweb
  • Status changed from new to assigned

Once a few dependencies have been updated following WordPress' "build" server and this repo bumping Node.js to 8.9.3 for WordPress 5.0 let's commit both a yarn.lock and package-lock.json to the repo, users can choose their tool of choice.

Using the synp package should allow us to keep both lock files in sync programmatically.

Note: See TracTickets for help on using tickets.