Problems pulling relevant MySQL Docker container

[desrosj]

Recently, I started receiving this error on my Apple Silicone machine.

no matching manifest for linux/arm64/v8 in the manifest list entries
node:child_process:935
    throw err;
    ^

Error: Command failed: docker-compose up -d wordpress-develop
    at checkExecSyncError (node:child_process:861:11)
    at execSync (node:child_process:932:15)
    at Object.<anonymous> (Sites/wordpress-svn/trunk/tools/local-env/scripts/start.js:22:1)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:22:47 {
  status: 18,
  signal: null,
  output: [ null, null, null ],
  pid: 17288,
  stdout: null,
  stderr: null
}

I'm not sure what happened to cause these errors to start, but any pull or start commands where the environment had not been set up previously fail.

[johnbillion]

I'm on a MacBook Pro M1 and my environment also won't start up now since clearing my local images and volumes. The above patch works but only if I uncheck the "Use Rosetta for x86/amd64 emulation on Apple Silicon" checkbox in the settings for Docker Desktop.

[desrosj]

59930.diff​ fixes the problem for me. For more context, I'm running the following:

• Docker Desktop: 4.25.0 (126437)
• Compose: v2.23.0-desktop.1
• Engine: 24.0.6
• Credential Helper: v0.7.0
• Kubernetes: v1.28.2

This is on macOS Ventura 13.6.x with an Apple M2 Max.

[desrosj]

I think this is likely due to the recent ​Docker Desktop 4.25 version that was released on October 30th.

[afragen]

I’ll have to check if the new Docker Desktop stopped my setup from working. I have some php config settings for xDebug and a docker-compose.override.yml file.

My setup is documented here, ​https://gist.github.com/afragen/a10d0475b71bfb79745df52ed3300924

I’ll have to retest with current Docker Desktop.

[Bernhard Reiter]

I just ran into this on my MacBook Pro (M1 Pro, i.e. Apple Silicone, with macOS Sonoma 14.1.1), after an npm run env:reset.

FWIW, I'm not using Docker Desktop but ​colima (installed from brew; I also ran brew update && brew upgrade).

The patch fixes the issue for me; I had to run another npm run env:reset though (as it would otherwise still fail on one of the Docker images).

---

FWIW, I'm still getting a warning, but it doesn't seem to be critical:

 ! php The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

[Bernhard Reiter]

Use MySQL 8.2, don't specify platform

[Bernhard Reiter]

BTW it seems like the reason that we need the amd64 image for MySQL is that ​there is no `arm64` image for MySQL v5.7 (which is our default).

Should we consider changing that default version to 8.x (for which there are arm64 images)? The patch I just uploaded works for me. (Don't forget to run npm run env:reset before trying it out!)

[desrosj]

@bernhard-reiter does 59930.diff​ also work for you?

I would be in favor of bumping the default version of MySQL to 8.0, which is the latest LTS version. 8.1 and 8.2 support is more complicated. Even though they are production-grade releases, they are "innovation releases" that are only supported for roughly 1 quarter (~4 months). It's still not clear how or if we should support them (see #59779 for more details there).

bump-mysql-to-8-2.diff​ seems good at the surface, but doesn't solve the problem. When someone with arm64 changes the version of MySQL being used to 5.7 without knowledge of this issue or a way around it, then their environment would show the problem and back to square one. It would be great if there were conditional statements supported within Docker configuration files, but they only support ​variable substitution. So in order to dynamically configure the right platform (if running arm64 & DB type is MySQL & version <= 5.7, use linux/amd64 instead).

[Bernhard Reiter]

Replying to desrosj:

@bernhard-reiter does 59930.diff​ also work for you?

It does, yes!

I would be in favor of bumping the default version of MySQL to 8.0, which is the latest LTS version. 8.1 and 8.2 support is more complicated. Even though they are production-grade releases, they are "innovation releases" that are only supported for roughly 1 quarter (~4 months). It's still not clear how or if we should support them (see #59779 for more details there).

Right, makes sense. I wasn't aware of their LTS/"innovation" release split, but LTS makes more sense, of course.

bump-mysql-to-8-2.diff​ seems good at the surface, but doesn't solve the problem. When someone with arm64 changes the version of MySQL being used to 5.7 without knowledge of this issue or a way around it, then their environment would show the problem and back to square one. It would be great if there were conditional statements supported within Docker configuration files, but they only support ​variable substitution. So in order to dynamically configure the right platform (if running arm64 & DB type is MySQL & version <= 5.7, use linux/amd64 instead).

Heh, I went through a similar thought process (also tried looking for a conditional syntax in the Docker compose file).

Ultimately, I thought it might be permissible to set a default like this, even if that means that users will have to exert more caution when trying to use a different version (for which no arm64 image exists, in this case); the error message might provide enough of a pointer in the right direction. But yeah, not sure it's worth confusing some users, unless we see e.g. huge performance benefits from using arm64 images on Apple Silicone.

Anyway, I'm fine with proceeding with your patch for now -- bumping the MySQL version is something we can consider doing later.

[afragen]

I've gotten around to testing with my setup, ie using a docker-compose.override.yml containing the following.

services:
  mysql:
    image: ${LOCAL_DB_TYPE-mysql}:8.0
    platform: linux/arm64
  php:
    platform: linux/amd64
    environment:
      - LOCAL_PHP_XDEBUG=true

    volumes:
      - ../custom-php-config.ini:/usr/local/etc/php/conf.d/php-config.ini
      - ./:/var/www

I still have some issues with getting xDebug to stop at breakpoints, but on an Apple Silicon Mac it loads and works with the following commands.

npm install && npm run build:dev && npm run env:start && npm run env:install

[johnbillion]

There's #50342 for that MySQL container issue.

I think this change of image should be made, along with an update to the readme to make it clear that Rosetta needs to be disabled in the Docker settings.

Trac ticket: https://core.trac.wordpress.org/ticket/59930

[desrosj]

I've been thinking about this the last few weeks, and the more I mull it over the more I lean towards using a solution similar to bump-mysql-to-8-2.diff​ instead of 59930.diff​.

By adding platform: linux/amd64 to the compose file, we're applying a bandaid for old versions of MySQL that potentially has a negative impact for contributors using more modern versions (8.0+), or skipping MySQL entirely and using MariaDB. Instead of doing this, I think we should increase the default version used from 5.7 to 8.0. Versions 8.1 and 8.2 are "innovation releases", and a larger conversation needs to be had around how we support these versions. See #59779 for more info.

Updating the default version is a change that should have been done by now and is overdue, in my opinion.

For the second part (solving the problem when contributors choose to run older MySQL versions locally), we could take one of two approaches:

• Add documentation to the README file that notes contributors using Apple Silicone machines will need to have a docker-compose.override.yml when using MySQL 5.7 or lower. An example file can also be included as a snippet.
• Add platform: ${LOCAL_ARCH_TYPE} to the docker-compose.yml file and dynamically set the value of this environment variable based on the actual architecture in use and the values provided for LOCAL_DB_TYPE and LOCAL_DB_VERSION. Something like when LOCAL_DB_TYPE == 'mysql' && LOCAL_DB_VERSION <= 5.7 && architecture == 'arm64', use linux/amd64 instead.

We don't currently run automated testing on non-Linux images in GitHub Action, so the second option would only be required when contributors are running the local environment on ARM.

Trac ticket: https://core.trac.wordpress.org/ticket/59930

[desrosj]

Another aspect of this issue is that 5.7 will still be the default in older branches. So the platform: solution is likely needed for WP branches <- 6.3. I think the 8.0 change can be reasonably backported to the 6.4 branch because it only affects the local Docker environment and does not affect any preferred defaults in the software itself.

I created 2 PRs that propose the following:

• Bump the default in trunk to 8.0, adding notes in the README and docker-compose.yml file about changing MySQL version on ARM.
• Add platform: linux/amd64 change to branches 6.3 and earlier.

In my testing, I've also found that the state of the Rosetta setting when using MySQL 8.0+ does not prevent the environment from starting and running. Would love confirmation of this.

[Bernhard Reiter]

Thanks a lot for filing those PRs, @desrosj!

I just checked ​https://hub.docker.com/r/amd64/mysql (which is what we're referring to from the `.env` file), and I noticed that MySQL 5.7 isn't even listed there anymore.

I dug a bit deeper and found that it was ​removed recently; the ​commit is titled "Remove 5.7 entirely (EOL)".

So if that means that MySQL 5.7 isn't even available for amd64 (i.e. x86-64) anymore, should we go ahead with ​this PR (and possibly drop ​the other one)?

[desrosj]

I saw that today as well. It looks like they removed it as an available container from their documentation, and from the repository that is responsible for building and pushing out new containers, but the old containers that were previously built and tagged for 5.7.x are still available: ​https://hub.docker.com/_/mysql/tags?page=1&name=5.7.

It's unlikely that they delete those manually. For example, the 5.5 and 5.6 containers are also still listed there. Just not mentioned in their docs.

[desrosj]

Also, MySQL 8.3 was released yesterday, though the Docker containers for that version are not yet available. Once they're published, I think the bump-mysql-to-8-2.diff​ should be updated to use 8.3 (provided tests pass and there are no compatibility issues).

Hmm, weird (and somewhat worrying) that ​two ​of the build jobs are failing on MacOS:

It's "Gutenberg running from src" and "Gutenberg running from build"; the Core counterparts seem to be passing. Wondering if we also need to change some environment setting in GB? In case the build downloads some platform-specific binary dependencies maybe? 🤔

[huzaifaalmesbah]

I face this issue on my MacBook Pro M1. after apply this patch working.

[desrosj]

In 57568:

Build/Test Tools: Fix bug pulling local environment containers on Apple silicone.

The MySQL Docker containers for versions 5.7 and below do not support recent Apple silicone chips. Previously this was fixed by including amd64/ as a prefix to the image name in the docker-compose.yml file (see [54096]). However, this stopped working after recent updates to Docker Desktop.

This changeset removes the amd64/ prefix for the image used as the database container and raises the default version of MySQL in the local development environment to the current LTS version (8.0). Because this version is still maintained, there are arm64 containers available to use.

This also documents a new workaround for contributors looking to run the local Docker environment using MySQL 5.7 or earlier, which entails creating a small docker-compose.override.yml.

Props bernhard-reiter, johnbillion, afragen, huzaifaalmesbah.
Fixes #59930.

Merged in https://core.trac.wordpress.org/changeset/57568.

[desrosj]

Reopening to backport [57568] to 6.4 after a bit of soak time, and the secondary PR to 6.3 and back.

[johnbillion]

In 57678:

Docs: Correct the URLs that document the available images for MySQL and MariaDB for the local development environment.

See #59930

[swissspidy]

#60381 was marked as a duplicate.

[swissspidy]

@desrosj Was this enough soak time before backporting?

[desrosj]

In 57862:

Build/Test Tools: Fix bug pulling local environment containers on Apple silicone.

The MySQL Docker containers for versions 5.7 and below do not support recent Apple silicone chips. Previously this was fixed by including amd64/ as a prefix to the image name in the docker-compose.yml file (see [54096]). However, this stopped working after recent updates to Docker Desktop.

This changeset removes the amd64/ prefix for the image used as the database container and raises the default version of MySQL in the local development environment to the current LTS version (8.0). Because this version is still maintained, there are arm64 containers available to use.

This also documents a new workaround for contributors looking to run the local Docker environment using MySQL 5.7 or earlier, which entails creating a small docker-compose.override.yml.

Merges [57568] to the 6.4 branch.

Props bernhard-reiter, johnbillion, afragen, huzaifaalmesbah.
Fixes #59930.

[desrosj]

In 57863:

Docs: Correct the URLs that document the available images for MySQL and MariaDB for the local development environment.

Merges [57678] to the 6.4 branch.

See #59930.

[desrosj]

Thanks for the reminder @swissspidy. I've gone and backported the changes to 6.4, and I've created #60822 for backporting further.

This was merged.