npm install does not work on wordpress-trunk on new Mac M1's

[justinahinon]

I've been trying to run a development environment to contribute to WordPress on my new MacBook M1 for some time now, but I go through a lot of mistakes every time.

The first one is that some libraries used by WordPress Core are not yet supported on the ARM architectures of the new Macs (I guess).

For example, running npm install from a folder with WordPress Trunk returns this error:

Error
    at /Users/segbedji/wp/wordpress-develop/node_modules/grunt-contrib-qunit/node_modules/puppeteer/lib/BrowserFetcher.js:112:19
    at FSReqCallback.oncomplete (fs.js:183:21)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! puppeteer@4.0.1 install: `node install.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the puppeteer@4.0.1 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

After some research, the puppeteer package has several incompatibilities with the Macs M1, and many issues have already been reported on their repo: ​https://github.com/puppeteer/puppeteer/issues/6622, ​https://github.com/puppeteer/puppeteer/issues/6634.

Some solutions have been proposed on the issues. Like this one (​https://github.com/puppeteer/puppeteer/issues/6622#issuecomment-759840596) that doesn't work on my end.

But it looks like there is an official fix here ​https://github.com/puppeteer/puppeteer/commit/9a8479a52a7d8b51690b0732b2a10816cd1b8aef. So I guess upgrading the package version on our end could fix the issue.

[justinahinon]

I may have found a workaround for this.

So, it seems to me that the versions of Node from 14 onwards are "universal". So you need to install a lower version of Node (branch 12) from a terminal emulated with Rosetta 2. Preferably install this version with NVM.

Then from this terminal, run npm install, and it should work correctly.

[desrosj]

Hey @justinahinon,

Do you recall which version of NodeJS you were using when running npm install originally? Core runs on 14.x, so that should be the version you use. As of a few weeks ago, all devDependencies are updated to the latest versions (except Webpack related ones, which are at the highest version that supports the version of Webpack trunk uses).

If you're still having issues and are able to narrow down which packages are problematic, we can look at updating them!

[justinahinon]

Replying to desrosj:

Do you recall which version of NodeJS you were using when running npm install originally? Core runs on 14.x, so that should be the version you use. As of a few weeks ago, all devDependencies are updated to the latest versions (except Webpack related ones, which are at the highest version that supports the version of Webpack trunk uses).

I was using Node LTS 14.15

[afragen]

@justinahinon node v15.x compiles and installs under arm64 (Apple Silicon). Here's what I did.

nvm is installed via plugin zsh-nvm for oh-my-zsh!

You will need Xcode Command Line Tools installed.
xcode-select --install, may need sudo

1. Open a terminal session.
2. Run nvm install 15 and allow it time to compile/build.
3. Run nvm use 15
4. Run nvm alias default 15, so every shell session uses this version of node by default

After the above, you can open a new terminal session where wordpress-develop is installed.

1. export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true , this avoids a current error. ​https://github.com/puppeteer/puppeteer/issues/6622
2. npm install, you can also start over by rm -rf node_modules && npm install
3. npm run build

If you're looking to run the npm run env:xxxx instructions you will need to install version of Docker for Apple Silicon, ​https://docs.docker.com/docker-for-mac/apple-m1/. Note that this Docker app does not automatically update you must update yourself.

If you want to run the above Docker version you will need to modify the .env to use mariadb as the database using latest version.

[SergeyBiryukov]

Related: #52690

[talldanwp]

I've also encountered this after getting a new laptop, but a curious thing is that there's no such problem in the Gutenberg project.

For some background, Gutenberg only uses the puppeteer-core package. That package doesn't install Chromium when running npm install, instead chromium is installed when running the test command:
​https://github.com/WordPress/gutenberg/blob/trunk/packages/scripts/scripts/test-e2e.js#L31-L33

This is distinct from the puppeteer package, which installs chromium whenever running npm install (which can be a frustrating waste of time).

My first thought was that perhaps core should also use puppeteer-core, but then I was surprised to find it already does. It uses the same tooling as Gutenberg.

Looking into the dependencies a bit further, it looks like the issue is that the grunt-contrib-qunit package has puppeteer as a dependency, and an old version at that. This is what's causing the issue. Versions of puppeteer are locked to specific chromium versions, so it's likely that the version of chromium that project depends on doesn't support the M1.

IMO, any fix should be made upstream to grunt-contrib-qunit, or perhaps WordPress should look into an alternative to that project that doesn't cause chromium to be installed when running npm install.

As a workaround you can set the PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true environment variable, which allows npm install to finish, but running tests won't work. Technically you should be able to install chromium independently (brew install chromium) and also set PUPPETEER_EXECUTABLE_PATH, but I haven't been able to get that working.

edit: actually, I do know the reason for the last thing I mentioned. e2e tests are using puppeteer-core and that doesn't support the environment variables!

[antonvlasenko]

Solution that worked for me when I got my M1 Mac a couple of months ago:

1. brew install chromium
2. Go to ~/.zshrc file and add the following lines:

    bash
   export PUPPETEER_EXECUTABLE_PATH=`which chromium`
   export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1
   export PATH="/opt/homebrew/opt/node@14/bin:$PATH"
   

/opt/homebrew/opt/node@14/binpath must match the actual node version installed on your Mac).

3. Log out of your user account and log in again.
4. Try to install and/or build Gutenberg.

[afragen]

Also for Apple Silicon you might need to add the following to your docker-compose.yml under the mysql section so that the correct image of mysql will be used.

mysql:
  platform: linux/amd64

[gziolo]

The issue with npm install got fixed with [53069]. There is the remaining issue with running npm run env because of some issues with MySQL.

[gziolo]

For the issue with MySQL, I followed the following article:
​https://www.jesusamieiro.com/installing-wordpress-develop-on-m1-chip/

I executed docker pull --platform linux/x86_64 mysql:5.7 which is probably similar to what @afragen shared but it doesn't require changes to the docker-compose.yml file.

The other way to fix it is to change the database to MariaDB v10 as suggested by @youknowriad.

[afragen]

@gziolo you are correct. I have actually tried all 3 methods. They all work.

I don't know if it's possible for the docker-compose.yml` to do a check for a the processor and add the correct platform automatically?

Trac ticket: https://core.trac.wordpress.org/ticket/52356

[johnbillion]

I've opened a PR at ​https://github.com/WordPress/wordpress-develop/pull/2650 which uses the ​https://hub.docker.com/r/amd64/mysql/ and ​https://hub.docker.com/r/amd64/mariadb/ images on arm64 machines instead of using the --platform property. Tested and working for me on an M1 MacBook Pro, would appreciate testing from others.

I've tested this PR and it works on my M1 Mac. Thanks @johnbillion

[afragen]

I have tested @johnbillion’s PR and it works great.

[aristath]

I had the same issue a while back before seeing this ticket, and switching to MariaDB worked perfectly fine. The changes I made are ​https://github.com/WordPress/wordpress-develop/compare/trunk...aristath:try/env/mariadb?expand=1 but using the patch from above works just as well.

[johnbillion]

Yeah switching to MariaDB works too but I'd like to leave MySQL in place so the choice remains, and so that testing can be performed on both.

@justinahinon @talldanwp @antonvlasenko @gziolo Could you give the PR a try on your M1s please?

[desrosj]

#54814 was marked as a duplicate.

[costdev]

This ticket was discussed in the bug scrub. Adding needs-testing so that this ticket can move forward.

[johnbillion]

If we could get a few more thumbs on on this change we can pull it into 6.0. Any testers both with M1s and without would be appreciated.

[aristath]

Tested once again and confirmed 👍

npm run env:start works great.

When I run npm run env:install I see the following error:

Status: Downloaded newer image for wordpressdevelop/cli:latest
Creating wordpress-develop-svn_cli_run ... done
ERROR 2002 (HY000): Can't connect to MySQL server on 'mysql' (115)
ERROR: 1
child_process.js:866
    throw err;
    ^

Error: Command failed: docker-compose run --rm cli config create --dbname=wordpress_develop --dbuser=root --dbpass=password --dbhost=mysql --path=/var/www/src --force
    at checkExecSyncError (child_process.js:790:11)
    at execSync (child_process.js:863:15)
    at wp_cli (/Users/gziolo/Projects/wordpress-develop-svn/tools/local-env/scripts/install.js:52:2)
    at Object.<anonymous> (/Users/gziolo/Projects/wordpress-develop-svn/tools/local-env/scripts/install.js:14:1)

Am I missing anything?

@gziolo Thanks for testing, that sounds like you're encountering https://core.trac.wordpress.org/ticket/50342 which is a bug unrelated to this change. If you run npm run env:install again it should work.

[SergeyBiryukov]

Moving to 6.0 per comment:22, as testing seems to confirm that the PR works as expected.

@gziolo Thanks for testing, that sounds like you're encountering https://core.trac.wordpress.org/ticket/50342 which is a bug unrelated to this change. If you run npm run env:install again it should work.

I can confirm it resolves the issue. All good in that case. I'm super excited about this patch, thank you so much 🎉

[johnbillion]

In 53358:

Build/Test Tools: Allow the local development environment to run on Apple M1 machines without requiring local configuration changes.

The amd64/mysql and amd64/mariadb official images from Docker are compatible with x86 running on the ARM64v8 architecture of M1 machines.

Props afragen, justinahinon, desrosj, talldanwp, antonvlasenko, gziolo, aristath, johnbillion

Fixes #52356

[afragen]

@johnbillion does this need to be reopened and backported to 6.0?

[johnbillion]

You're too quick for me :-)

[johnbillion]

In 53359:

Build/Test Tools: Allow the local development environment to run on Apple M1 machines without requiring local configuration changes.

The amd64/mysql and amd64/mariadb official images from Docker are compatible with x86 running on the ARM64v8 architecture of M1 machines.

Merges [53358] to the 6.0 branch.

Fixes #52356

Committed with https://core.trac.wordpress.org/changeset/53358 🎉

[SergeyBiryukov]

Follow-up: #55914

Usage of the amd64/mysql and amd64/mariadb official images from Docker was introduced in https://core.trac.wordpress.org/ticket/52356 as they are compatible with an x64 container running on a host machine using ARM64 architecture (ie. Apple M1 machines).

However they're also compatible with an x64 host machine which should mean they can be used by default instead of only when the host uses ARM64. This allows some image juggling to be removed.

## Testing

• Ensure Docker Desktop is running if necessary
• While still on your current branch (eg. trunk) run npm run env:stop
• Checkout the branch for this PR
• Run npm run env:start
• Run some tests that use the database container, eg. npm run test:php

## Todo:

• [ ] Test on x64 Intel Mac
• [x] Test on ARM64 M1 Mac
• [x] Test on x64 Linux (GitHub Actions)
• [ ] Test on x64 Windows

FWIW, _without_ this patch, I cannot run the dev environment locally on my M1 Pro. npm run env:start fails with

no matching manifest for linux/arm64/v8 in the manifest list entries

when attempting to install the mysql image. (Related ​Slack convo.)

If you're on Apple Silicon, you can try to reproduce this issue on trunk by running npm run env:reset followed by npm run env:start.

I've had the same exact issue as @ockham and it was fixed by cherry-picking ​34d9b9c (this PR) on top of trunk.

Tested on x64 Intel Mac (stats below):

{{{sh
$ node -v
14.17.0

$ docker -v
Docker Version 20.10.17

}}}

Docker Desktop v 4.12.0
Compose v 2.20.2

• npm run env:reset Just in case
• gh pr checkout 2700
• npm install
• npm run build:dev
• npm run env:start
• npm run env:install
• npm run test:php

Tests pass 👌

OK, but incomplete, skipped, or risky tests!
Tests: 13829, Assertions: 45228, Skipped: 59, Risky: 30.

• Tested in Windows WSL (ubuntu base): Success
• Tested in Windows 11 native: Success
• Tested in Linux native, no docker desktop (Pop! OS 22.04): Success

Thanks @gmovr @withinboredom 👍 👍

https://core.trac.wordpress.org/changeset/54096