WordPress.org

Make WordPress Core

Opened 2 years ago

Closed 2 months ago

Last modified 8 weeks ago

#38603 closed defect (bug) (maybelater)

Explore adding Yarn and a `yarn.lock` file for NPM module management

Reported by: netweb Owned by: netweb
Milestone: Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: needs-patch
Focuses: Cc:

Description

YARN: Yarn: A new package manager for JavaScript:

  • Fast: Yarn caches every package it downloads so it never needs to download the same package again. It also parallelizes operations to maximize resource utilization so install times are faster than ever.
  • Reliable: Using a detailed, concise lockfile format and a deterministic algorithm for installs, Yarn is able to guarantee that an install that worked on one system will work exactly the same way on any other system.
  • Secure: Yarn uses checksums to verify the integrity of every installed package before its code is executed.

Yarn is similar to a npm-shrinkwrap.json file though with vast performance increases and improved cross platform compatibility/reliably.

Here's some quick stats:

  • time npm install (First run, existing npm cache)
    • 176.87 seconds
  • time npm install (First run, no existing npm cache)
    • 237.97 seconds
  • time yarn (Initial Yarn run)
    • 21.40 seconds
  • time yarn (Subsequent install after creating yarn.lock and rm -rf node_modules/ && time yarn)
    • 18.82 seconds

Change History (15)

This ticket was mentioned in Slack in #core by netweb. View the logs.


2 years ago

#2 @netweb
2 years ago

Issue of note: https://github.com/yarnpkg/yarn/issues/681 Pinning dependencies behaves differently than npm.

The above issue is not currently of concern, though if we were to have to "pin" an NPM module to a specific semantic version it would become a concern.

#4 @netweb
2 years ago

  • Milestone changed from Awaiting Review to Future Release

Yarn currently is very new and very shiny...

Enough complications arouse in #38657 with 4.7's npm-shrinkwrap.json and as Yarn doesn't currently behave nicely with NPM's shrinkwrap process let's wait a while until it's a little less new and shiny...

#5 @swissspidy
2 years ago

#40421 was marked as a duplicate.

#6 @jnylen0
2 years ago

I should've searched and found this ticket before opening #40421, but before we make a decision here I'd like to see us take one of two steps, with the goal of preserving a clean source tree for people (like me) who are already using yarn:

  • Commit yarn.lock to the source tree. This does not impose a requirement that people start to use yarn, rather it merely provides additional benefits if you do (deterministic installs including all versions of sub-dependencies).
  • Ignore yarn.lock in git and svn. This also does not create any kind of expectation that people must use yarn instead of npm install, though there are a lot of benefits. This will allow people to continue using yarn if they wish, with a clean source tree.

#7 @netweb
2 years ago

I've not revisited the issues previously raised for a while, I'll take a look again in the coming days.

FYI: The "build" server, i.e. build.svn.wordpress.org is not setup for Yarn so switching to Yarn requires some coordination with meta and systems to get it implemented when we can switch to Yarn.

I'm hoping the previous concerns have been addressed as Yarn is soooo much faster 🤞

#8 @jnylen0
2 years ago

@netweb as a first step that doesn't impose a commitment or requirement to use yarn, what do you think about committing yarn.lock soon? In addition to allowing those of us already using yarn to have a clean source tree, this will allow us to get more testing of potential incompatibilities.

#9 @netweb
23 months ago

Replying to netweb:

Issue of note: https://github.com/yarnpkg/yarn/issues/681 Pinning dependencies behaves differently than npm.

The above issue is not currently of concern, though if we were to have to "pin" an NPM module to a specific semantic version it would become a concern.

This appears to been resolved overnight https://github.com/yarnpkg/yarn/issues/681#issuecomment-298036459 :+1:


Replying to jnylen0:

@netweb as a first step that doesn't impose a commitment or requirement to use yarn, what do you think about committing yarn.lock soon? In addition to allowing those of us already using yarn to have a clean source tree, this will allow us to get more testing of potential incompatibilities.

As a workaround running yarn install --no-lockfile will use Yarn to install the npm modules without generating a yarn.lock file, it's not quite as fast as installing with a yarn.lock file being generated, but still faster than npm install

#10 @netweb
23 months ago

A couple of things, NPM 5 is due out tomorrow, it purportedly has a massive performance increase comparable to Yarn, https://github.com/npm/npm/pull/16244

Yarn also includes a yet to be undocumented --frozen-lockfile option, this sounds as if (further research required) it might be a good way in keeping package.json, yarn.lock and npm-shrinkwrap.json in sync with each other.

Npm vs Yarn Performance stats: https://github.com/npm/npm/pull/16244#issuecomment-297190219

It'll be interesting in the coming days to run some performance tests here for our use case

#11 @netweb
22 months ago

Related: #40938 Add npm package-lock.json for npm v5.x

#12 @netweb
14 months ago

  • Milestone changed from Future Release to 5.0
  • Owner set to netweb
  • Status changed from new to assigned

Once a few dependencies have been updated following WordPress' "build" server and this repo bumping Node.js to 8.9.3 for WordPress 5.0 let's commit both a yarn.lock and package-lock.json to the repo, users can choose their tool of choice.

Using the synp package should allow us to keep both lock files in sync programmatically.

#13 @jorbin
5 months ago

  • Milestone changed from 5.0 to 5.1

#14 @pento
2 months ago

  • Milestone 5.1 deleted
  • Resolution set to maybelater
  • Status changed from assigned to closed

Yarn is a little faster than NPM on my computer, running against trunk, but not the wild difference it was two years ago.

So, in the spirit of decisions, not options, I'm fine with sticking with NPM for now.

This ticket was mentioned in Slack in #core-js by pento. View the logs.


8 weeks ago

Note: See TracTickets for help on using tickets.