#57187 closed enhancement (fixed)
Add devcontainer + Codespaces support
Reported by: | helen | Owned by: | |
---|---|---|---|
Milestone: | 6.2 | Priority: | normal |
Severity: | normal | Version: | |
Component: | Build/Test Tools | Keywords: | |
Focuses: | Cc: |
Description (last modified by )
Finally opening up a Trac ticket after posting on Make/Core :)
I'd like to start adding "official" support for remote dev environments to contribute to core (including testing) - I believe this makes contributions more accessible to a wider audience, by making fully-featured dev environments available on more devices and getting people started much faster (i.e. you click a button on a repo instead of braving conference wifi to download Docker images or passing around USB sticks).
This ticket is more specifically for devcontainer + Codespaces support in core so you can start a Codespace from GitHub web - devcontainers are a set of config files that can be used with local Docker as well via VSCode (and otherwise). Codespaces is a GitHub-hosted solution to then run those devcontainers in the cloud (Azure) with the default ability to open VSCode for the Web connected to said container for a completely in-browser experience, meaning you can have a fully-featured code editing, building, and app running experience on a tablet.
Here's what I think is needed but need opinions on where we should have boundaries between published images, devcontainers, and setup scripts as I am not myself particularly knowledgeable about containers
- A baseline image that can be used for any number of dev needs - core, Gutenberg, other plugins and themes, testing, etc. This should probably include Composer, the appropriate Node+npm versions, and wp-cli. I have never yet gotten Xdebug to work in Codespaces. My current PR uses the Docker-published
wordpress
image, not the ones we use forwp-env
- seems to me it would be nice to have something that spins up faster but I don't know enough here. - A
devcontainer.json
that includes some VSCode extensions (or explicitly decide not to include any). - Documentation, esp in the Make/Core Handbook. :)
Full disclosure: I do currently work at GitHub; not in this area, but I do have access to helpful resources as needed. There are a number of services out there for remote dev with varying tiers of free usage, over time it would be great to support multiple options.
Change History (30)
This ticket was mentioned in PR #3670 on WordPress/wordpress-develop by helen.
2 years ago
#1
- Keywords has-patch added
This ticket was mentioned in Slack in #core by helen. View the logs.
2 years ago
This ticket was mentioned in PR #4031 on WordPress/wordpress-develop by samruddhikhandale.
22 months ago
#3
Hi 👋
I am Samruddhi, a Software Engineer for GitHub, working with the GitHub Codespaces team, and a maintainer for the dev containers org 👩💻
Thank you @helen for doing all the ground work for setting up a dev container (ref: https://github.com/WordPress/wordpress-develop/pull/3670). This PR is greatly inspired from the contents of #3670 with few tweaks.
Thanks,
Samruddhi
#5
follow-up:
↓ 7
@
22 months ago
- Description modified (diff)
- Milestone changed from Awaiting Review to 6.2
I'm pulling this into the 6.2 milestone to indicate my intention to commit this, though I'm going to consider it "beta support" for Codespaces and we can kick it to a future milestone for further work as makes sense. It has no impact on the core build as everything is outside of src
so I have no qualms about bringing it in during beta, though I will wait a day or so for any discussions folks might want to have about this.
I got some help from an engineer on Codespaces (samruddhikhandale) who's made this more robust as a first run and it would be great to have this in trunk
so that folks can try it out at the upcoming WordCamp Asia contributor day on February 17. If you haven't yet used Codespaces, this allows people to essentially "hit play" and a cloud container spins up with trunk running from src
installed at a portforwarded URL (private by default but can even be opened to the public), and VSCode connects to it from the browser or the desktop app. This does mean requiring connectivity to work, but contributors won't need to download images to their machines over venue wifi, and can even contribute from mobile devices and tablets with the in-browser VSCode option. 120 "core hours" are free for individual users, which really equates to 60 hours of usage at 2 cores (decent) or 30 hours at 4 cores (zippy). The underpinning of this is a devcontainer, which is an open spec that means you can also run the same environment via local Docker or any other tool that supports devcontainers. I just happen to be able to solicit direct help from my coworkers who are specifically working to enable OSS adoption of the tool and we have the GH mirror already :)
I would very much value feedback from a contributor day type of event so that we as a project can make a decision about whether this is worth getting further help on solidifying a container strategy, as the current iteration relies on the wordpress
image that is actually published by Docker, not us. That's the primary reason I would consider this "beta support". I've also not tested multisite or running any tests in the container.
This ticket was mentioned in Slack in #core by helen. View the logs.
22 months ago
#7
in reply to:
↑ 5
@
22 months ago
Replying to helen:
it would be great to have this in
trunk
so that folks can try it out at the upcoming WordCamp Asia contributor day
Indeed, this sounds like it would make it easier to get started on a contributor day :)
samruddhikhandale commented on PR #4031:
22 months ago
#8
Once a contributor closes their VS Code browser window, how long does the Codespace remain active for?
By default, codespaces time out after 30 minutes of inactivity. When a codespace times out it is stopped and will no longer incur charges for compute usage.
Doc reference - here
@peterwilsoncc Let me know if this info would be helpful in the doc, happy to add that.
@peterwilsoncc commented on PR #4031:
22 months ago
#9
@samruddhikhandale Given the default spend limit is set to zero, I think it's fine to leave it out for now. It's easy enough to add something later if it becomes an issue.
#10
@
22 months ago
- Keywords commit added
I've both tested and approved the PR; this looks good for commit in the coming days.
Testing process:
- built a codespace from the code dropdown on the PR
- checked npm and node versions in terminal: they match the expected versions from the
nvm
config file - successfully logged in to WP Admin and viewed the front end
I broke the final step initially due to impatience, a note to be patient has since been added to the readme file.
@manfcarlo commented on PR #4031:
22 months ago
#11
Maybe some people who are unfamiliar with codespaces would be interested to know that I have been using a codespace to develop a plugin for the plugin directory, and the youth of the product absolutely shows. I have been hit by two major outages in little over a month, the latest one being ongoing for more than 48 hours and preventing me from releasing a new version to the plugin directory. The outage was reported as resolved, but either you are not testing closely enough, or deliberately misreporting to hide the high duration of downtime. The support ticket I submitted for the previous outage took 7 days to be answered, so I'm expecting 7 full days of downtime and 7 days delay to a plugin release that should already be completed by now. With service like this, there is no chance I will upgrade to the paid tier anytime soon. Not voting down this patch at all, but I wouldn't be encouraging the events team to jump into codespaces with both feet just yet, or it could result in an attendee's whole contributor day being wasted.
I actually think WordPress Playground would be a better long-term solution for this purpose, but it too obviously has a long way to go.
22 months ago
#12
Hi @carlomanf - totally understood that you've experienced frustrations. We use Codespaces for our own dev environments at GitHub so any outages you experience we do as well. I cannot speak to this product area as far as work goes and don't think this is the greatest venue to go back and forth about it, in any case.
As said, this is a way to get a sense of how this particular tool and cloud dev environments as a general concept might benefit the core contribution case. devcontainers as an open spec should mean that this can lay the groundwork for generally being usable across different options based on what works best for the contributor, which should always be our goal - to make it easier for folks to contribute given their differing needs and constraints. A Wasm option would also be great! Let's just not let ourselves get stuck in attempting to be perfect from the start, that's already happened to this effort a couple times.
22 months ago
#14
Committed in https://core.trac.wordpress.org/changeset/55303
#15
@
22 months ago
- Keywords has-patch commit removed
Notes:
- I think we need to commit the correct permissions on
.devcontainer/install-tools.sh
and.devcontainer/setup.sh
rather than changing them at runtime, because otherwise they show as modified which interferes with doing version control tasks. - It took 11 minutes to get from clicking "create a new codespace" to the environment being fully ready with WordPress installed - this is manageable with set expectations as a trial run but we need to explore our options for bringing this down. It was 2.5 minutes to start the container and open VSCode, so I believe it's
npm install
that is taking the bulk of the time here. Under 3 minutes total seems ideal to me although it's just an arbitrary number. I'll ask for help in understanding our options - because contributors should be running the Codespace from a fork in order to create a branch and open a PR, I don't think prebuilds work for this case. - The setup script output is getting lost - at some point in iterations we were writing this to
setup.log
which is why it's in the.gitignore
but it's gotten dropped - should decide what to do about that (write it to a file, output it to the terminal, something else?). Output to the terminal is helpful in reassuring users things are still moving along.
#16
@
22 months ago
I can't get the devcontainer to start up in VS Code locally, I'm getting the following error.
Shall we use this ticket as a thread for now or shall we create separate tickets for issues now that it's been merged?
ERROR: Service 'wordpress' failed to build : Build failed [14996 ms] Error: Command failed: docker-compose --project-name wp_devcontainer -f /Users/john/sites/wp/.devcontainer/docker-compose.yml -f /Users/john/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data/docker-compose/docker-compose.devcontainer.build-1676035102524.yml build [14996 ms] at gF (/Users/john/.vscode/extensions/ms-vscode-remote.remote-containers-0.275.1/dist/spec-node/devContainersSpecCLI.js:1872:412) [14996 ms] at process.processTicksAndRejections (node:internal/process/task_queues:96:5) [14996 ms] at async rie (/Users/john/.vscode/extensions/ms-vscode-remote.remote-containers-0.275.1/dist/spec-node/devContainersSpecCLI.js:1872:2381) [14996 ms] at async tie (/Users/john/.vscode/extensions/ms-vscode-remote.remote-containers-0.275.1/dist/spec-node/devContainersSpecCLI.js:1854:2396) [14996 ms] at async Fie (/Users/john/.vscode/extensions/ms-vscode-remote.remote-containers-0.275.1/dist/spec-node/devContainersSpecCLI.js:1921:2266) [14996 ms] at async Vf (/Users/john/.vscode/extensions/ms-vscode-remote.remote-containers-0.275.1/dist/spec-node/devContainersSpecCLI.js:1921:3241) [14996 ms] at async eoe (/Users/john/.vscode/extensions/ms-vscode-remote.remote-containers-0.275.1/dist/spec-node/devContainersSpecCLI.js:2045:17324) [14996 ms] at async Qse (/Users/john/.vscode/extensions/ms-vscode-remote.remote-containers-0.275.1/dist/spec-node/devContainersSpecCLI.js:2045:17065) [14998 ms] Exit code 1 [14999 ms] Command failed: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/john/.vscode/extensions/ms-vscode-remote.remote-containers-0.275.1/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/john/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-14b29ec9-458f-4953-9888-1ba9c74df4941676035088525 --workspace-folder /Users/john/sites/wp --workspace-mount-consistency cached --id-label devcontainer.local_folder=/Users/john/sites/wp --id-label devcontainer.config_file=/Users/john/sites/wp/.devcontainer/devcontainer.json --log-level debug --log-format json --config /Users/john/sites/wp/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true [14999 ms] Exit code 1
Environment
- VS Code 1.75.1
- macOS 12.3
- Docker Desktop 4.7.1
#17
@
22 months ago
@johnbillion I'm fine with continuing on this ticket for the moment, I do think making sure it at least works consistently locally is important to keeping this in core. Are you getting any other output? It is coming up for me although it looks like we need to do some conditional setting of the workspace folder, that's where it fails and doesn't fully finish setting up WordPress for me right now.
#18
@
22 months ago
Here's the full terminal log: https://gist.github.com/johnbillion/1a51bad2eb523e491cc08582513bd749
There's an error on line 142 in addition to the one I reported above:
=> ERROR [dev_containers_target_stage 3/6] RUN cd /tmp/build-features/co 0.4s ------ > [dev_containers_target_stage 3/6] RUN cd /tmp/build-features/common-utils_1 && chmod +x ./devcontainer-features-install.sh && ./devcontainer-features-install.sh: #11 0.129 =========================================================================== #11 0.129 Feature : Common Utilities #11 0.129 Description : Installs a set of common command line utilities, Oh My Zsh!, and sets up a non-root user. #11 0.129 Id : ghcr.io/devcontainers/features/common-utils #11 0.129 Version : 2.0.8 #11 0.129 Documentation : https://github.com/devcontainers/features/tree/main/src/common-utils #11 0.129 Options : #11 0.129 INSTALLZSH="true" #11 0.129 CONFIGUREZSHASDEFAULTSHELL="false" #11 0.129 INSTALLOHMYZSH="true" #11 0.129 UPGRADEPACKAGES="true" #11 0.129 USERNAME="wordpress" #11 0.129 USERUID="automatic" #11 0.129 USERGID="automatic" #11 0.129 NONFREEPACKAGES="false" #11 0.129 =========================================================================== #11 0.173 Packages to verify are installed: apt-utils openssh-client gnupg2 dirmngr iproute2 procps lsof htop net-tools psmisc curl tree wget rsync ca-certificates unzip bzip2 zip nano vim-tiny less jq lsb-release apt-transport-https dialog libc6 libgcc1 libkrb5-3 libgssapi-krb5-2 libicu[0-9][0-9] liblttng-ust[0-9] libstdc++6 zlib1g locales sudo ncdu man-db strace manpages manpages-dev init-system-helpers libssl1.1 git #11 0.244 Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB] #11 0.305 Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB] #11 0.315 Err:1 http://deb.debian.org/debian bullseye InRelease #11 0.315 At least one invalid signature was encountered. #11 0.334 Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB] #11 0.338 Err:2 http://deb.debian.org/debian-security bullseye-security InRelease #11 0.338 At least one invalid signature was encountered. #11 0.360 Err:3 http://deb.debian.org/debian bullseye-updates InRelease #11 0.360 At least one invalid signature was encountered. #11 0.361 Reading package lists... #11 0.366 W: GPG error: http://deb.debian.org/debian bullseye InRelease: At least one invalid signature was encountered. #11 0.366 E: The repository 'http://deb.debian.org/debian bullseye InRelease' is not signed. #11 0.366 W: GPG error: http://deb.debian.org/debian-security bullseye-security InRelease: At least one invalid signature was encountered. #11 0.366 E: The repository 'http://deb.debian.org/debian-security bullseye-security InRelease' is not signed. #11 0.366 W: GPG error: http://deb.debian.org/debian bullseye-updates InRelease: At least one invalid signature was encountered. #11 0.366 E: The repository 'http://deb.debian.org/debian bullseye-updates InRelease' is not signed. #11 0.367 ERROR: Feature "Common Utilities" (ghcr.io/devcontainers/features/common-utils) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/common-utils for help troubleshooting this error.
#19
@
22 months ago
@johnbillion Seems like this might be a cache issue (what isn't...), could you try running docker system prune
and rebuild the dev container to see if that fixes things? Not ideal but if that's the issue then we can explore a better experience from there.
We've got some other fixes incoming for running on local Docker - don't know if I would consider it a great experience but at least it'll get a contributor up and running if they bravely want to try this instead of wp-env
right now.
This ticket was mentioned in Slack in #core by sergey. View the logs.
22 months ago
#22
@
22 months ago
On the items I noted in 15:
- The correct permissions made it from the svn propset to Git and then to GitHub correctly! Hooray!
- @craiglpeters confirmed that we cannot use prebuilds effectively because of forking, this is probably worthy of chasing down via another ticket if we keep the devcontainer (i.e. actually close this ticket).
- Turns out this goes into the creation log that Codespaces helpfully prompts you to check out if you want more information, I think that's fine then. Just worth noting in docs.
Going to leave this open over the weekend for WordCamp Asia feedback, and if nothing blocking comes out of there I'll close this as fixed and we can move into details to get it to something that is "fully supported" rather than this beta-esque state.
#23
@
22 months ago
Did this get tested much at WordCamp Asia contributor day?
The devcontainer is working fine for me now, I had to clear out some Docker networking config that was also affecting containers in other projects.
Brain dump items that we can either address here or in follow-up tickets:
- README.md needs to document the three environment choices now available to devs (Codespaces, local devcontainer, local environment).
- The devcontainer doesn't use the same environment for WordPress that you get when you follow the "Local development" instructions in the readme. We're going to end up with two confusingly separate but similar environments. Could and should the same environment be used to run WordPress within the devcontainer?
setup.sh
assumes the workspace directory name is/workspaces/wordpress-develop
but the local directory name may differ. The${localWorkspaceFolderBasename}
variable needs to be passed in to that script fromdevcontainer.json
.
This ticket was mentioned in Slack in #core by costdev. View the logs.
22 months ago
This ticket was mentioned in Slack in #core by mukeshpanchal27. View the logs.
22 months ago
#26
@
22 months ago
This ticket was discussed during the bug scrub. @helen @johnbillion Is there any further work likely to be done on this before Thursday 9th March (6.2 RC1)? Or is this ready to close and the items in comment:23 addressed in follow-up tickets?
Additional props: @mukesh27
This ticket was mentioned in Slack in #core by costdev. View the logs.
22 months ago
#28
@
22 months ago
- Resolution set to fixed
- Status changed from new to closed
This ticket was discussed during the bug scrub. As we're about to enter RC, closing this ticket out for 6.2 and iteration can happen in follow-up tickets. 🙂
Pulling over a PR I was originally noodling on here: https://github.com/helen/wordpress-develop/pull/53
This works - you can spin up a new Codespace on this branch and after waiting patiently for the setup script to run (it's too quiet, you have to watch
setup.log
to check on progress), you should be able to go to the preview URL and log in and everything. A 2-core works fine, 4-core is quite speedy but of course uses double the core hours.However, I don't think this is ready to go into core :) I don't think it needs to be perfect but it feels a little weird to use the Docker-published
wordpress
image, and there are some details in here that don't feel right either.Here are the lingering todo items I currently have:
www-data
user?/wp-admin
kicks me tohttps://localhost/wp-admin
- no idea why, going towp-login.php
works fine, maybe something in the container somewhereTrac ticket: https://core.trac.wordpress.org/ticket/57187#ticket