Make WordPress Core

Opened 8 months ago

Closed 7 months ago

#64587 closed defect (bug) (fixed)

AI Guidelines should be referenced in the pull request template

Reported by: westonruter Owned by: westonruter
Priority: normal Milestone: 7.0
Component: AI Version:
Severity: normal Keywords: has-patch changes-requested
Cc: Focuses:

Description

New ​WordPress AI Guidelines have been published. See ​announcement. These should be referenced in the ​pull request template since a lot of new PRs are being opened with the assistance of AI tooling.

For reference, the WordPress/performance repo's ​pull request template currently includes this AI guidance.

Change History (17)

This ticket was mentioned in ​PR #10850 on ​WordPress/wordpress-develop by ​@westonruter.


8 months ago
#1

  • Keywords has-patch added

Trac ticket: https://core.trac.wordpress.org/ticket/64587

This incorporates aspects of the ​PR template from the WordPress/performance repo, namely the new “Use of AI Tools” section.

​@justlevine commented on ​PR #10850:


8 months ago
#2

I'm not familiar with the conversation and Chesterton fences that went into WordPress/performance, but as I outline in ​https://github.com/WordPress/ai/issues/77, I'd a recommend a checklist (or commented list if we care about GitHub UI semantics) versus a "free text" section.

IMO much better DX choice than expecting contributors - even if/after reading the guidelines- to intuitively know and remember what they need to proactively disclose / how granularly to go, etc. Also has the added benefit of guiding users to potential constructive uses of AI without being overbearing.

​@westonruter commented on ​PR #10850:


8 months ago
#3

@justlevine I think that's a good suggestion, but I'm wary of adding too much to the template since it is currently pretty streamlined. Maybe start with simple freeform and links to the (in progress) guidelines and then iterate once the guidelines are more mature? Maybe the guidelines could have such a list of bullet points incorporated into it for copying into the PR template. It could even have an example PR template as part of the guidelines. My primary concern is to get something out the door sooner than later to help address the influx of PRs, many of which appear to be authored with in AI.

​@justlevine commented on ​PR #10850:


8 months ago
#4

@westonruter yup agree 100%, especially the part that something here is better than nothing.

Maybe the guidelines could have such a list of bullet points incorporated into it for copying into the PR template. It could even have an example PR template as part of the guidelines.

My hope is to create less cognitive load, so while I love the idea of providing some examples _of_ what disclosure looks like, I think a copy->paste flow from an external ref document isn't worth the friction.

Still, even if it's just 1 extra sentence in the , I do think we should provide at least _some_ same-place signposting. Personally, I feel it's more important to have this while waiting for the guidelines have time percolate, since the whole point is we're asking folks to do something _now_ that's new to contributors, guideline authors, and really the entire ecosystem, before there's anywhere else to provide relatively clear direction.

(Related: Some fun reading at where folks are discussing the issues and potential tools GitHub could offer maintainers to triage slop ​https://github.com/orgs/community/discussions/185387)

​@westonruter commented on ​PR #10850:


8 months ago
#5

Still, even if it's just 1 extra sentence in the , I do think we should provide at least _some_ same-place signposting.

Would you add a suggestion for what you have in mind?

#6 @westonruter
8 months ago

Related: #63901 (Add AGENTS.md for the project)

Having this context will be very helpful for Copilot (and other agents) to perform accurate suggestions and reviews.

#7 @westonruter
8 months ago

In 61593:

General: Add AI Guidelines to the pull request template.

Developed in ​https://github.com/WordPress/wordpress-develop/pull/10850

Props westonruter, justlevine, mukesh27, jeffpaul.
See #64587.

​@westonruter commented on ​PR #10850:


8 months ago
#8

I've committed the initial version of this in r61593.

#9 @westonruter
8 months ago

Even though I've committed the initial version of this in [61593], I'm leaving this open for additional refinement.

#10 @desrosj
8 months ago

  • Keywords changes-requested added

Thanks for adding this, all!

I have one request for a refinement. Could we add an example that shows the information contributors are expected to include and how to ideally format it? And possibly a template that can just be filled in below that.

There is a ​great example in the guidelines. But even when someone is very familiar with the guidelines and expectations, it can be (easy to forget|hard to remember) everything that should be included (as a Core Committer, I drive a good amount of traffic to the ​Commit Messages best practices page).

This would also help to make disclosures more consistent and easier to spot or scan while reviewing PRs.

#11 @desrosj
7 months ago

I noticed just now that the Gutenberg repository's PR template needs to be updated to include this as well.

#13 @gziolo
7 months ago

  • Milestone 7.0 → Future Release

This ticket isn't strictly tied to WP 7.0.

This ticket was mentioned in ​PR #11287 on ​WordPress/wordpress-develop by ​@desrosj.


7 months ago
#14

This adds an example disclosure statement for the Use of AI Tools section in the pull request template.

By including a template, contributors are encouraged to leave a detailed disclosure and reminded of what details are important.

Trac ticket: Core-64587.

## Use of AI Tools

AI assistance: None

#15 @desrosj
7 months ago

  • Milestone Future Release → 7.0

@gziolo since there were commits made during the 7.0 cycle, it's best to leave this ticket in the milestone for tracking purposes. If needed, closing the ticket out and creating a new one to continue is usually best.

Since this only has a small follow up change left, I went ahead and created a PR for that.

​@desrosj commented on ​PR #11287:


7 months ago
#16

The only issue I see moving forward is that when agents create a template, they aren't aware they should follow it, since they use the API directly. So something to think about seperately. There is no AGENTS.md file, so it's unclear to me how to help contribute folks on that front.

@gziolo That's fair. But I think that this is specifically aimed at human contributors. Any further configuration or template required for the robots can be added and discussed separately.

#17 @desrosj
7 months ago

  • Resolution → fixed
  • Status assigned → closed

In 62049:

General: Include example for disclosing AI usage.

Follow up to [61593].

Props jeffpaul, westonruter, gziolo, mukesh27.
Fixes #64587.

Note: See TracTickets for help on using tickets.