Make WordPress Core

Opened 8 months ago

Closed 4 months ago

Last modified 4 months ago

#59930 closed defect (bug) (fixed)

Problems pulling relevant MySQL Docker container

Reported by: desrosj's profile desrosj Owned by: desrosj's profile desrosj
Milestone: 6.5 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch needs-testing fixed-major
Focuses: Cc:

Description

Recently, I started receiving this error on my Apple Silicone machine.

no matching manifest for linux/arm64/v8 in the manifest list entries
node:child_process:935
    throw err;
    ^

Error: Command failed: docker-compose up -d wordpress-develop
    at checkExecSyncError (node:child_process:861:11)
    at execSync (node:child_process:932:15)
    at Object.<anonymous> (Sites/wordpress-svn/trunk/tools/local-env/scripts/start.js:22:1)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:22:47 {
  status: 18,
  signal: null,
  output: [ null, null, null ],
  pid: 17288,
  stdout: null,
  stderr: null
}

I'm not sure what happened to cause these errors to start, but any pull or start commands where the environment had not been set up previously fail.

Attachments (2)

59930.diff (401 bytes) - added by desrosj 8 months ago.
bump-mysql-to-8-2.diff (1.1 KB) - added by Bernhard Reiter 8 months ago.
Use MySQL 8.2, don't specify platform

Download all attachments as: .zip

Change History (32)

@desrosj
8 months ago

#1 @johnbillion
8 months ago

I'm on a MacBook Pro M1 and my environment also won't start up now since clearing my local images and volumes. The above patch works but only if I uncheck the "Use Rosetta for x86/amd64 emulation on Apple Silicon" checkbox in the settings for Docker Desktop.

#2 @desrosj
8 months ago

59930.diff fixes the problem for me. For more context, I'm running the following:

  • Docker Desktop: 4.25.0 (126437)
  • Compose: v2.23.0-desktop.1
  • Engine: 24.0.6
  • Credential Helper: v0.7.0
  • Kubernetes: v1.28.2

This is on macOS Ventura 13.6.x with an Apple M2 Max.

#4 @afragen
8 months ago

I’ll have to check if the new Docker Desktop stopped my setup from working. I have some php config settings for xDebug and a docker-compose.override.yml file.

My setup is documented here, https://gist.github.com/afragen/a10d0475b71bfb79745df52ed3300924

I’ll have to retest with current Docker Desktop.

#5 @Bernhard Reiter
8 months ago

I just ran into this on my MacBook Pro (M1 Pro, i.e. Apple Silicone, with macOS Sonoma 14.1.1), after an npm run env:reset.

FWIW, I'm not using Docker Desktop but colima (installed from brew; I also ran brew update && brew upgrade).

The patch fixes the issue for me; I had to run another npm run env:reset though (as it would otherwise still fail on one of the Docker images).


FWIW, I'm still getting a warning, but it doesn't seem to be critical:

 ! php The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

@Bernhard Reiter
8 months ago

Use MySQL 8.2, don't specify platform

#6 @Bernhard Reiter
8 months ago

BTW it seems like the reason that we need the amd64 image for MySQL is that there is no `arm64` image for MySQL v5.7 (which is our default).

Should we consider changing that default version to 8.x (for which there are arm64 images)? The patch I just uploaded works for me. (Don't forget to run npm run env:reset before trying it out!)

Last edited 8 months ago by Bernhard Reiter (previous) (diff)

#7 follow-up: @desrosj
8 months ago

@bernhard-reiter does 59930.diff also work for you?

I would be in favor of bumping the default version of MySQL to 8.0, which is the latest LTS version. 8.1 and 8.2 support is more complicated. Even though they are production-grade releases, they are "innovation releases" that are only supported for roughly 1 quarter (~4 months). It's still not clear how or if we should support them (see #59779 for more details there).

bump-mysql-to-8-2.diff seems good at the surface, but doesn't solve the problem. When someone with arm64 changes the version of MySQL being used to 5.7 without knowledge of this issue or a way around it, then their environment would show the problem and back to square one. It would be great if there were conditional statements supported within Docker configuration files, but they only support variable substitution. So in order to dynamically configure the right platform (if running arm64 & DB type is MySQL & version <= 5.7, use linux/amd64 instead).

#8 in reply to: ↑ 7 @Bernhard Reiter
8 months ago

Replying to desrosj:

@bernhard-reiter does 59930.diff also work for you?

It does, yes!

I would be in favor of bumping the default version of MySQL to 8.0, which is the latest LTS version. 8.1 and 8.2 support is more complicated. Even though they are production-grade releases, they are "innovation releases" that are only supported for roughly 1 quarter (~4 months). It's still not clear how or if we should support them (see #59779 for more details there).

Right, makes sense. I wasn't aware of their LTS/"innovation" release split, but LTS makes more sense, of course.

bump-mysql-to-8-2.diff seems good at the surface, but doesn't solve the problem. When someone with arm64 changes the version of MySQL being used to 5.7 without knowledge of this issue or a way around it, then their environment would show the problem and back to square one. It would be great if there were conditional statements supported within Docker configuration files, but they only support variable substitution. So in order to dynamically configure the right platform (if running arm64 & DB type is MySQL & version <= 5.7, use linux/amd64 instead).

Heh, I went through a similar thought process (also tried looking for a conditional syntax in the Docker compose file).

Ultimately, I thought it might be permissible to set a default like this, even if that means that users will have to exert more caution when trying to use a different version (for which no arm64 image exists, in this case); the error message might provide enough of a pointer in the right direction. But yeah, not sure it's worth confusing some users, unless we see e.g. huge performance benefits from using arm64 images on Apple Silicone.

Anyway, I'm fine with proceeding with your patch for now -- bumping the MySQL version is something we can consider doing later.

This ticket was mentioned in Slack in #core by jorbin. View the logs.


8 months ago

#10 @afragen
7 months ago

I've gotten around to testing with my setup, ie using a docker-compose.override.yml containing the following.

services:
  mysql:
    image: ${LOCAL_DB_TYPE-mysql}:8.0
    platform: linux/arm64
  php:
    platform: linux/amd64
    environment:
      - LOCAL_PHP_XDEBUG=true

    volumes:
      - ../custom-php-config.ini:/usr/local/etc/php/conf.d/php-config.ini
      - ./:/var/www

I still have some issues with getting xDebug to stop at breakpoints, but on an Apple Silicon Mac it loads and works with the following commands.

npm install && npm run build:dev && npm run env:start && npm run env:install

Last edited 7 months ago by afragen (previous) (diff)

#11 @johnbillion
7 months ago

  • Keywords commit added

There's #50342 for that MySQL container issue.

I think this change of image should be made, along with an update to the readme to make it clear that Rosetta needs to be disabled in the Docker settings.

#13 @desrosj
7 months ago

  • Keywords commit removed

I've been thinking about this the last few weeks, and the more I mull it over the more I lean towards using a solution similar to bump-mysql-to-8-2.diff instead of 59930.diff.

By adding platform: linux/amd64 to the compose file, we're applying a bandaid for old versions of MySQL that potentially has a negative impact for contributors using more modern versions (8.0+), or skipping MySQL entirely and using MariaDB. Instead of doing this, I think we should increase the default version used from 5.7 to 8.0. Versions 8.1 and 8.2 are "innovation releases", and a larger conversation needs to be had around how we support these versions. See #59779 for more info.

Updating the default version is a change that should have been done by now and is overdue, in my opinion.

For the second part (solving the problem when contributors choose to run older MySQL versions locally), we could take one of two approaches:

  • Add documentation to the README file that notes contributors using Apple Silicone machines will need to have a docker-compose.override.yml when using MySQL 5.7 or lower. An example file can also be included as a snippet.
  • Add platform: ${LOCAL_ARCH_TYPE} to the docker-compose.yml file and dynamically set the value of this environment variable based on the actual architecture in use and the values provided for LOCAL_DB_TYPE and LOCAL_DB_VERSION. Something like when LOCAL_DB_TYPE == 'mysql' && LOCAL_DB_VERSION <= 5.7 && architecture == 'arm64', use linux/amd64 instead.

We don't currently run automated testing on non-Linux images in GitHub Action, so the second option would only be required when contributors are running the local environment on ARM.

#15 @desrosj
7 months ago

  • Keywords needs-testing added

Another aspect of this issue is that 5.7 will still be the default in older branches. So the platform: solution is likely needed for WP branches <- 6.3. I think the 8.0 change can be reasonably backported to the 6.4 branch because it only affects the local Docker environment and does not affect any preferred defaults in the software itself.

I created 2 PRs that propose the following:

  • Bump the default in trunk to 8.0, adding notes in the README and docker-compose.yml file about changing MySQL version on ARM.
  • Add platform: linux/amd64 change to branches 6.3 and earlier.

In my testing, I've also found that the state of the Rosetta setting when using MySQL 8.0+ does not prevent the environment from starting and running. Would love confirmation of this.

#16 @Bernhard Reiter
6 months ago

Thanks a lot for filing those PRs, @desrosj!

I just checked https://hub.docker.com/r/amd64/mysql (which is what we're referring to from the `.env` file), and I noticed that MySQL 5.7 isn't even listed there anymore.

I dug a bit deeper and found that it was removed recently; the commit is titled "Remove 5.7 entirely (EOL)".

So if that means that MySQL 5.7 isn't even available for amd64 (i.e. x86-64) anymore, should we go ahead with this PR (and possibly drop the other one)?

#17 @desrosj
6 months ago

I saw that today as well. It looks like they removed it as an available container from their documentation, and from the repository that is responsible for building and pushing out new containers, but the old containers that were previously built and tagged for 5.7.x are still available: https://hub.docker.com/_/mysql/tags?page=1&name=5.7.

It's unlikely that they delete those manually. For example, the 5.5 and 5.6 containers are also still listed there. Just not mentioned in their docs.

#18 @desrosj
6 months ago

Also, MySQL 8.3 was released yesterday, though the Docker containers for that version are not yet available. Once they're published, I think the bump-mysql-to-8-2.diff should be updated to use 8.3 (provided tests pass and there are no compatibility issues).

@Bernhard Reiter commented on PR #5834:


6 months ago
#19

Hmm, weird (and somewhat worrying) that two of the build jobs are failing on MacOS:

https://i0.wp.com/github.com/WordPress/wordpress-develop/assets/96308/fb50a919-dd48-4c9c-8bf7-d24fb947491d

It's "Gutenberg running from src" and "Gutenberg running from build"; the Core counterparts seem to be passing. Wondering if we also need to change some environment setting in GB? In case the build downloads some platform-specific binary dependencies maybe? 🤔

#20 @huzaifaalmesbah
6 months ago

I face this issue on my MacBook Pro M1. after apply this patch working.

This ticket was mentioned in Slack in #core by ocean90. View the logs.


6 months ago

#23 @desrosj
5 months ago

  • Owner set to desrosj
  • Resolution set to fixed
  • Status changed from new to closed

In 57568:

Build/Test Tools: Fix bug pulling local environment containers on Apple silicone.

The MySQL Docker containers for versions 5.7 and below do not support recent Apple silicone chips. Previously this was fixed by including amd64/ as a prefix to the image name in the docker-compose.yml file (see [54096]). However, this stopped working after recent updates to Docker Desktop.

This changeset removes the amd64/ prefix for the image used as the database container and raises the default version of MySQL in the local development environment to the current LTS version (8.0). Because this version is still maintained, there are arm64 containers available to use.

This also documents a new workaround for contributors looking to run the local Docker environment using MySQL 5.7 or earlier, which entails creating a small docker-compose.override.yml.

Props bernhard-reiter, johnbillion, afragen, huzaifaalmesbah.
Fixes #59930.

#25 @desrosj
5 months ago

  • Keywords fixed-major added
  • Resolution fixed deleted
  • Status changed from closed to reopened

Reopening to backport [57568] to 6.4 after a bit of soak time, and the secondary PR to 6.3 and back.

#26 @johnbillion
5 months ago

In 57678:

Docs: Correct the URLs that document the available images for MySQL and MariaDB for the local development environment.

See #59930

#27 @swissspidy
5 months ago

#60381 was marked as a duplicate.

#28 @swissspidy
4 months ago

@desrosj Was this enough soak time before backporting?

#29 @desrosj
4 months ago

  • Resolution set to fixed
  • Status changed from reopened to closed

In 57862:

Build/Test Tools: Fix bug pulling local environment containers on Apple silicone.

The MySQL Docker containers for versions 5.7 and below do not support recent Apple silicone chips. Previously this was fixed by including amd64/ as a prefix to the image name in the docker-compose.yml file (see [54096]). However, this stopped working after recent updates to Docker Desktop.

This changeset removes the amd64/ prefix for the image used as the database container and raises the default version of MySQL in the local development environment to the current LTS version (8.0). Because this version is still maintained, there are arm64 containers available to use.

This also documents a new workaround for contributors looking to run the local Docker environment using MySQL 5.7 or earlier, which entails creating a small docker-compose.override.yml.

Merges [57568] to the 6.4 branch.

Props bernhard-reiter, johnbillion, afragen, huzaifaalmesbah.
Fixes #59930.

#30 @desrosj
4 months ago

In 57863:

Docs: Correct the URLs that document the available images for MySQL and MariaDB for the local development environment.

Merges [57678] to the 6.4 branch.

See #59930.

#31 @desrosj
4 months ago

Thanks for the reminder @swissspidy. I've gone and backported the changes to 6.4, and I've created #60822 for backporting further.

Note: See TracTickets for help on using tickets.