Make WordPress Core

Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#42573 closed defect (bug) (fixed)

Templates not working properly

Reported by: precies's profile precies Owned by: dd32's profile dd32
Milestone: 4.9.1 Priority: high
Severity: normal Version: 4.9
Component: Themes Keywords:
Focuses: Cc:

Description

Hello,

We updated our WordPress to version 4.9 and found problems with our custom templates files in our child-theme.

We always create our own template files for custom website for our customers.

The problem:
Since the update we only see one custom template file. If we add another one it wont show up under Templates when we edit a page. (See attached screenshot).

https://imgur.com/cCBo7KY

When we did a downgrade to WordPress 4.8.3 we were able to select all of our templates again.

Can we fix this by ourselves or is it a bug that needed to be fixed by WordPress?

Kind regards,

Joep van Dongen

Attachments (6)

only-one-template-4.9.png (11.1 KB) - added by precies 6 years ago.
Only one custom template file te select
42573.patch (999 bytes) - added by ibenic 6 years ago.
Disabling the cache if WP_DEBUG is true
42573.2.patch (2.3 KB) - added by mariovalney 6 years ago.
42573.3.patch (2.2 KB) - added by mariovalney 6 years ago.
Allow developers to flush files cache
42573.4.patch (1.2 KB) - added by mariovalney 6 years ago.
Removing cache
42573.5.patch (1.4 KB) - added by schlessera 6 years ago.
Only use caching for a depth > 1, and reduce expiry to 5 minutes (updated)

Download all attachments as: .zip

Change History (121)

@precies
6 years ago

Only one custom template file te select

#1 @Clorith
6 years ago

Hiya, and welcome to Trac!

In r41806 we added caching to the list of files fetched from themes, this means any changes to the filesystem won't instantly show up, it can take as long as an hour for them to show up.

It was originally intended for the code editor screen, but I'm not entirely sure it's bad that it also affected the selector for page templates, if a theme has a lot of files that part of the backend could be slowed down by the need to iterate over files often.

#2 follow-up: @ibenic
6 years ago

Maybe we could skip that caching if WP_DEBUG is true? Could be useful while developing a new theme.

#3 follow-ups: @ocean90
6 years ago

Not sure if the caching was a good idea there. But bumping the version of the theme allows you to refresh the cache.

@ibenic
6 years ago

Disabling the cache if WP_DEBUG is true

#4 @precies
6 years ago

Hello guys,

It was caching of the files. After an hour they showed up. I think it is a good idea to disable cache while wp_debug is true. Because while developing a page is a bit anoying to wait for an hour;)

And thanks for the fast response:)

#5 @robscott
6 years ago

Will be a puzzling one the first time you see it! Maybe worth adding (or documenting) how to force clear the caches?

#6 @SergeyBiryukov
6 years ago

  • Milestone changed from Awaiting Review to 4.9.1

#7 @SergeyBiryukov
6 years ago

#42567 was marked as a duplicate.

#8 in reply to: ↑ 3 @MarcusAnthony
6 years ago

Replying to ocean90:

Not sure if the caching was a good idea there. But bumping the version of the theme allows you to refresh the cache.

Thank you - this simple solution worked!

#9 in reply to: ↑ 3 @dghizoni
6 years ago

Replying to ocean90:

Not sure if the caching was a good idea there. But bumping the version of the theme allows you to refresh the cache.

Thanks! Worked for me too.

@mariovalney
6 years ago

@mariovalney
6 years ago

Allow developers to flush files cache

#10 follow-up: @mariovalney
6 years ago

Hey people, thanks for this ticket (saved some hours of work).
I agree ignore cache while WP_DEBUG is true is a good way to allow developers see the changes fast.

About 42573.3.patch, I suggest we can allow developers to flush that cache someway.

In 42573.2, I added the check before WP_DEBUG to avoid warnings because I did not remember it's always defined (ignore it, please - my bad).

#11 in reply to: ↑ 10 @MarcusAnthony
6 years ago

Replying to mariovalney:

Hey people, thanks for this ticket (saved some hours of work).
I agree ignore cache while WP_DEBUG is true is a good way to allow developers see the changes fast.

About 42573.3.patch, I suggest we can allow developers to flush that cache someway.

In 42573.2, I added the check before WP_DEBUG to avoid warnings because I did not remember it's always defined (ignore it, please - my bad).

Allowing us developers to flush cache would be a huge asset.

#12 @leobaiano
6 years ago

#42591 was marked as a duplicate.

#13 @Vishnja1
6 years ago

I'm also not sure caching was a good idea there, at least this can be explicit option (in wp-config for example). Otherwise it would be a surpise for any WP developer how to edit/add theme template files in the nearest future). We should be bumping the version of the theme!

#14 follow-ups: @robscott
6 years ago

So the solution to this issue is to version bump the theme.

If you're adding a whole new page template, I would respectfully suggest that the theme version must be bumped. However, in development, this would be confusing the first time it were encountered.

Most systems with caching should have a "flush all the caches" button somewhere. I like the "if debug is on don't cache anything" option too.

Last edited 6 years ago by robscott (previous) (diff)

#15 in reply to: ↑ 14 @robscott
6 years ago

urgh replied to myself instead of editing. Lunch time.

Last edited 6 years ago by robscott (previous) (diff)

#16 @swissspidy
6 years ago

#42601 was marked as a duplicate.

#17 in reply to: ↑ 14 ; follow-up: @mariovalney
6 years ago

  • Keywords has-patch added

Replying to robscott:

So the solution to this issue is to version bump the theme.

If you're adding a whole new page template, I would respectfully suggest that the theme version must be bumped. However, in development, this would be confusing the first time it were encountered.

Most systems with caching should have a "flush all the caches" button somewhere.

A "flush all caches" could be a good feature but for now these patches seems good.

For a "flush all" we should think about wp_object_cache and the cases we do it with transient (like this case - are there another cases?) and maybe create ways to cache plugins be flushed in same button (so we are not just adding one more button to users).

I like the "if debug is on don't cache anything" option too.

I really agree. It should be a good practice to cache plugins too.

Last edited 6 years ago by mariovalney (previous) (diff)

#18 in reply to: ↑ 17 @robscott
6 years ago

Replying to mariovalney:

For a "flush all" we should think about wp_object_cache and the cases we do it with transient (like this case - are there another cases?) and maybe create ways to cache plugins be flushed in same button (so we are not just adding one more button to users).

Yes its a wider thing really. Would be most excellent if there was one WP trigger to flush caches; transients which (may) no longer apply; and "everything" that could be a one process cache clear-ify. Hmm, wonder if this idea exists anywhere!?

#19 follow-up: @metropolis_john
6 years ago

There definitely needs to be a way to disable this completely as it makes theme development a chore. Having to bump version number, or the suggestion of having to clear the cache manually, every time a new template is added is not a solution.

#20 @Clorith
6 years ago

#42611 was marked as a duplicate.

#21 @andrewinsideout
6 years ago

Please add the option to disable this. I just wasted a bunch of time trying to figure out what I was doing wrong. Thanks to the people who suggested changing the version number... but what a pain while developing.

#22 @ibenic
6 years ago

The WP_DEBUG should be enough if you ask me for local development. If someone uploads a new template through FTP, let's say, their live/staging site with a disabled WP_DEBUG, the page templates won't be refreshed.

Maybe we could also add a button that could delete the cache on request. This button could be under Appearance > Themes on the active theme block. Something like this: https://imgur.com/Xwh3Qyo ? But that could be a little too intrusive to regular users? Not sure.

WP_DEBUG + version change might be just enough for developers.

#23 in reply to: ↑ 19 @ryan360
6 years ago

Replying to metropolis_john:

There definitely needs to be a way to disable this completely as it makes theme development a chore. Having to bump version number, or the suggestion of having to clear the cache manually, every time a new template is added is not a solution.

Agreed. I can't believe this was overlooked. I just wasted a bunch of time because of this!

#24 follow-up: @Clorith
6 years ago

  • Keywords needs-patch added; has-patch removed

We'll definitely not add any buttons to the UI that will rarely, if ever, see use by actual users.

If we do decide to keep the transient in place, I do like the WP_DEBUG check, with it in use there's no need for the extra functions to clear things, as the version bump invalidation should take care of things for end users, but the patch needs refined a little.

I looked at 42573.3.patch and it just checks if WP_DEBUG is true or false, it also needs a check if it's defined altogether.

On a side note, I agree with adding the transient here, now that we recursively scan through the whole theme directory the operation can become heavy when we take into account 3rd party premium themes as well which are known for bundling plugins and other large code bases making them substantially more time consuming than they used to be.

#25 in reply to: ↑ 24 ; follow-up: @SergeyBiryukov
6 years ago

Replying to Clorith:

If we do decide to keep the transient in place, I do like the WP_DEBUG check, with it in use there's no need for the extra functions to clear things, as the version bump invalidation should take care of things for end users, but the patch needs refined a little.

I don't think the WP_DEBUG check is enough here. Plenty of users that create custom templates never heard of WP_DEBUG, and bumping the theme version is not an obvious step either.

Could we disable the caching for files in the theme's root directory (which seems to be where most users upload custom templates), and only use it for subdirectories?

#26 in reply to: ↑ 2 @webdados
6 years ago

Replying to ibenic:

Maybe we could skip that caching if WP_DEBUG is true? Could be useful while developing a new theme.

That's exactly my opinion.

(I've just lost 30 minutes debugging WP core to find this issue while developing a new theme)

#27 @theMikeD
6 years ago

Another vote for a way to disable this.

#28 in reply to: ↑ 25 ; follow-up: @theMikeD
6 years ago

Replying to SergeyBiryukov:

Replying to Clorith:

If we do decide to keep the transient in place, I do like the WP_DEBUG check, with it in use there's no need for the extra functions to clear things, as the version bump invalidation should take care of things for end users, but the patch needs refined a little.

I don't think the WP_DEBUG check is enough here. Plenty of users that create custom templates never heard of WP_DEBUG, and bumping the theme version is not an obvious step either.

Could we disable the caching for files in the theme's root directory (which seems to be where most users upload custom templates), and only use it for subdirectories?

This feel like a half measure and would not help with themes that put template in subfolders. WP_DEBUG along with documenting this somewhere would be a good first step. An option somewhere in the UI to flush the template cache would also help non-developers. But WP_DEBUG at a minimum.

#29 follow-up: @mariovalney
6 years ago

As performance must be a theme developer concern as well, maybe we can disable this as default and add a support to developers who creates themes with a lot of files activate it.

We can do that cache just in parent theme too.

This way we are covering the 3rd party premium themes and not being a problem to users who creates files into your (child) themes.

Last edited 6 years ago by mariovalney (previous) (diff)

#30 in reply to: ↑ 29 ; follow-ups: @ibenic
6 years ago

Replying to mariovalney:

We can do that cache just in parent theme too.

This way we are covering the 3rd party premium themes and not being a problem to users who creates files into your (child) themes.

Not sure about that. I have seen a lot of child themes having much more templates than the parent themes. Some of the parent themes are just used as frameworks such as the Genesis Framework.

I would consider maybe adding a simple (?) tooltip next to Page Template (?) which on hover would say that the templates are cached and if new templates were added, they should bump the theme/child theme version? It seems logical to bump the version since new templates were added.

#31 in reply to: ↑ 30 ; follow-ups: @webdados
6 years ago

Replying to ibenic:

It seems logical to bump the version since new templates were added.

Unless you're on active development and then bumping the version number is just a hassle

#32 in reply to: ↑ 31 @theMikeD
6 years ago

Replying to webdados:

Replying to ibenic:

It seems logical to bump the version since new templates were added.

Unless you're on active development and then bumping the version number is just a hassle

Exactly this.

#33 in reply to: ↑ 31 @ibenic
6 years ago

Replying to webdados:

Replying to ibenic:

It seems logical to bump the version since new templates were added.

Unless you're on active development and then bumping the version number is just a hassle

Of course! Bumping the version should help those who are changing a theme on a production site where they won't define the WP_DEBUG as true.

In my opinion, active development should be done on your local machine so turning WP_DEBUG to TRUE should not be an issue (if the patch is applied).

Last edited 6 years ago by ibenic (previous) (diff)

#34 follow-ups: @fatmedia
6 years ago

What exactly is the benefit of this? If a theme has so many templates that it would benefit from caching the admin meta box I think there's probably a bigger problem than some cache is going to solve...

Reverting this would be a win for sane theme authors. Why improve things for authors who choose to do ridiculous things?

#35 @mindctrl
6 years ago

Could we disable the caching for files in the theme's root directory (which seems to be where most users upload custom templates), and only use it for subdirectories?

I don't think that would adequately address the issue. I've seen a number of themes that store templates outside the root directory, such as wp-content/themes/theme_name/templates/.

#36 in reply to: ↑ 34 @ryanduff
6 years ago

Replying to fatmedia:

What exactly is the benefit of this? If a theme has so many templates that it would benefit from caching the admin meta box I think there's probably a bigger problem than some cache is going to solve...

Reverting this would be a win for sane theme authors. Why improve things for authors who choose to do ridiculous things?

Agreed. I'd yet to see a theme that has anywhere near 100 levels deep of a folder structure. If that's the case, the theme is the problem.

Again we've introduced unintended consequences to core by fixing a problem that never needed to be solved in the first place.

+1 to revert any form of caching around this and maybe revisit the changes in r41806 to see what problem they're solving and if this is even appropriate.

Nobody should be forced to use WP_DEBUG, nor should another useless button to purge this unnecessary transient be added somewhere in wp-admin.

As a developer, if I deploy a file update to a client site (live or staging) I should be able to use that template right away. Not be forced to wait up to an hour just for the cache to expire to select it. Think of the poor experience that a user is going to experience if they update a file in a theme and can't select it? That goes against the goals of the project plain and simple.

#37 follow-up: @robscott
6 years ago

@ryanduff if it were very simple to flush the caches, would you still have an issue with this being present? I like caching things that do not (normally) change, particularly if it can speed up admin pageload in poor and shared hosting... otherwise "WordPress is slow" will be the user takeaway.

Surely here we're talking about a surprise to a developer in their workflow versus slow(er) admin performance for *all* users of WordPress.

Assuming developers get a solution as a result of this conversation, that would be a win win? Without reverting anything?

#38 @jzankowski
6 years ago

Over 2 hours wasted because of this. Please fix this in a way that at least makes it clear what's happening and what option to flip to disable the caching behavior.

#39 in reply to: ↑ 37 ; follow-up: @ryanduff
6 years ago

Replying to robscott:

@ryanduff if it were very simple to flush the caches, would you still have an issue with this being present? I like caching things that do not (normally) change, particularly if it can speed up admin pageload in poor and shared hosting... otherwise "WordPress is slow" will be the user takeaway.

Surely here we're talking about a surprise to a developer in their workflow versus slow(er) admin performance for *all* users of WordPress.

Assuming developers get a solution as a result of this conversation, that would be a win win? Without reverting anything?

No. Again, r41806 got "clever" by introducing a max depth of 100 levels to the file list function. This is not needed nor is caching around it. Find me a bug where someone complained the page editor screen was too slow because their theme was bloated with files. What does this fix? Nada.

If you have slow admin performance it's because (a) Core changed to scan 100 levels deep in your theme folder and (b) you're using a theme so bloated it's now affected by this change and you should probably find a new theme.

This is not just a developer issue. It's only a matter of time before end users run into this issue as well (if they haven't already reported it in the forums).

The bigger issue here is template files not being available because the list is cached, not that the editor now supports editing theme files 100 levels deep.

#40 in reply to: ↑ 39 @metropolis_john
6 years ago

Replying to ryanduff:

This is not just a developer issue. It's only a matter of time before end users run into this issue as well (if they haven't already reported it in the forums).

Yes, one thing I noticed is the page template list being cached when trying to assign a custom field group to a specific page template in the Advanced Custom Fields plugin. End users could definitely run into this without any idea of how to fix it.

#41 in reply to: ↑ 28 @SergeyBiryukov
6 years ago

Replying to theMikeD:

Replying to SergeyBiryukov:

I don't think the WP_DEBUG check is enough here. Plenty of users that create custom templates never heard of WP_DEBUG, and bumping the theme version is not an obvious step either.

Could we disable the caching for files in the theme's root directory (which seems to be where most users upload custom templates), and only use it for subdirectories?

This feel like a half measure and would not help with themes that put template in subfolders. WP_DEBUG along with documenting this somewhere would be a good first step. An option somewhere in the UI to flush the template cache would also help non-developers. But WP_DEBUG at a minimum.

I meant disabling caching in the theme's root directory in addition to the WP_DEBUG check, not instead of it. A UI option seems like a non-starter though, as it contradicts the WordPress' philosophy of "decisions, not options".

That said, I agree with @ryanduff in comment:36, caching should not be necessary here at all.

#42 @johnbillion
6 years ago

  • Priority changed from normal to high

Agreed also, this caching should be removed.

#43 @pkoceniak
6 years ago

This should 100% be disabled by default.. with a setting somewhere that allows you to enable it.
I don't understand the logic behind this feature at all.. if your theme is built properly, than there should never be an issue here.
Just wasted over an hour tearing my hair out trying to figure out why my templates weren't showing.

This ticket was mentioned in Slack in #core-customize by clorith. View the logs.


6 years ago

#46 in reply to: ↑ 30 @mariovalney
6 years ago

Replying to ibenic:

Not sure about that. I have seen a lot of child themes having much more templates than the parent themes. Some of the parent themes are just used as frameworks such as the Genesis Framework.

Yeah... You are right.

I agree with everyone: remove this cache (or "disable as default and add a support to developers who creates themes with a lot of files activate it").

Last edited 6 years ago by mariovalney (previous) (diff)

@mariovalney
6 years ago

Removing cache

#47 @swb1192
6 years ago

I'm seeing an issue with a client site where child theme templates are being picked up, but not the parent theme page templates as of 4.9. Is that a separate issue?

#48 @robscott
6 years ago

@ryanduff am not even playing devil's advocate on this... I did not like the original implementation nor did I see it as necessary. BUT there was presumably some thinking behind it which may not need to be thrown out?

I am working on a solution to postmeta slowness for which the proposed fix was also caching. Also in that case, I did not think caching would be a good solution.

Caching is not a band aid to fix inefficient code.

That said, we should not just scrap something because it raises tickets - if it is the right thing to do. In this case, what was the original ill being fixed?

#49 follow-ups: @westonruter
6 years ago

Here is a workaround plugin which will fix the issue until 4.9.1 comes out: https://gist.github.com/westonruter/6c2ca0e5a4da233bf4bd88a1871dd950

#50 in reply to: ↑ description @dbhiems
6 years ago

Thank you @westonruter for making the fix into a plugin. It is working for me.

I think this is a bigger problem than just fixing bugs in a new release, because the 4.9 wordpress update essentially disabled a core wordpress feature with no explanation or way to fix it for the average user. It honestly makes me wonder how something like this was able to make it into the release in the first place. Like what is the decision making process to allow a change like that without appropriate review of the consequences for users. Any user like myself would have no idea that now, after creating a new page template, we will not see it appear for up to an hour, so for all intents and purposes custom page templates is now non functional. I spent a lot of time trying to figure out why my page templates didn't show up anymore, when I was doing exactly the same thing a few days ago with no problems. Literally copying the code of an existing page template (which showed up in the template list dropdown) and saving to a new renamed template would not show up in the list. It was a completely bizarre and frustrating situation. I really hope this gets looked at as a systematic issue beyond just fixing the bug because it I feel it undermines the trust in the excellence in wordpress with users and the expectation that new releases will improve things for the user.

#51 @majick
6 years ago

I see the advantages of applying file list caching here, but breaking things for end users and developers through the unintended consequences is something that can be avoided without much fuss... My suggestions:

  1. Identify what events could clear the file list cache for a theme when in the admin side? Maybe something simple like clearing the theme-specific file list in the function get_page_templates so at least it doesn't affect the standard UI selection etc.? eg.
    function get_page_template( $post = null, $post_type = 'page' ) {
        $theme = wp_get_theme();
        $theme->clear_cache( 'post_templates' );
        delete_transient( sanitize_key( 'files_' . $theme->cache_hash . '-' . $theme->get( 'Version' ) ) ); 
        return array_flip( $theme->get_page_templates( $post, $post_type ) );
    }
    

I'm not sure why this transient key structure? Seems to have come from the plugin file list side? But it's a bit of a mismatch to the rest of the theme class transient keys, otherwise it could be more simply cleared using WP Theme clear_cache method - which in any case is the other place where the theme file list could (and probably should) be cleared.

  1. Add a filter to disable file caching for a particular theme, this way theme developers could set the filter in an mu-plugin or something locally and not have to worry about the theme file list cache while they are developing. eg. in WP Theme method get_files instead of $all_files = get_transient( $transient_key ); just:
$all_files = apply_filters( 'theme_file_list_cache', get_transient( $transient_key ), $this );

That could be used like this for example:

add_filter('theme_file_list_cache', function($filelist, $theme) {
    if ($theme->get_stylesheet() == 'my-theme') {return false;}
    return $filelist;
}), 10, 2 );

Also as a sidenote going from just a few subdirectory levels to 100 levels seems like going from one extreme to the other... Sure it's well beyond time that it increased, but it seems it's this extreme swing that then required the caching for performance, which then caused this problem. 10 would have been a safe maximum depth, with a filter for anyone really wanting more if needed in a particular plugin / theme.

On a related note, only using a depth of 1 for searching page templates in the WP Theme method get_post_templates method while the levels are now cached 100 deep seems a bit bizarre. Doesn't it precludes template directory structures as simple as /content/post/ ? Any reason not to simply bump this value up to 2 or 3?

#52 in reply to: ↑ 49 ; follow-up: @ijignesh
6 years ago

I was having same issue today in latest wordpress 4.9.

I just created new folder named page-templates and copied my custom template file in that folder and voila! my template name shows up on edit page in admin area.

Can anyone confirm is this working for them as well?

Thanks,

Replying to westonruter:

Here is a workaround plugin which will fix the issue until 4.9.1 comes out: https://gist.github.com/westonruter/6c2ca0e5a4da233bf4bd88a1871dd950

#53 in reply to: ↑ 52 @webdados
6 years ago

Replying to ijignesh:

I just created new folder named page-templates and copied my custom template file in that folder and voila! my template name shows up on edit page in admin area.

Can anyone confirm is this working for them as well?

Not for me

#54 @becky.absolute
6 years ago

Just a thought that I had ... what if an ENVIRONMENT variable was introduced (in some form). Then the code to cache the file listing could do a check of this ENVIRONMENT variable and only cache the listing if the value was set to "production" (or something along those lines).

I personally don't set WP_DEBUG to true while I'm developing my themes and plugins, unless I'm troubleshooting an issue. But it would be easy for me to set up an ENVIRONMENT variable in the wp-config.php file when I'm setting up my local development environment or a staging environment.

I've seen thing kind of variable in CodeIgniter and in Laravel. And I can imagine that it would be useful in other situations moving forward than just this one.

#55 in reply to: ↑ 34 @gaasbeekmw
6 years ago

Replying to fatmedia:

What exactly is the benefit of this? If a theme has so many templates that it would benefit from caching the admin meta box I think there's probably a bigger problem than some cache is going to solve...

Reverting this would be a win for sane theme authors. Why improve things for authors who choose to do ridiculous things?

Exactly, it also moves away from what Wordpress was all about, a default which you extend by using plugins. If someone requires this then there are plenty of options to add functionality via plugins. No need to have something like this in the core.

#56 follow-up: @maximejobin
6 years ago

As a business owner of a maintenance service where we do our best to keep things as stable as possible for our clients, two things are really surprising to me here:

  1. How could a feature like that be proposed, accepted and introduced into core without anyone noticing it? I did not see any information about this in the release notes or in the documentation. Did I miss anything? I truly hope it was documented somewhere and I missed it.
  1. Some people propose a patch, to add a button in the UI, to consider looking for WP_DEBUG, ... And then suddenly decisions are made to remove this functionality based on no real explanation. I totally agree with @ryanduff about the fact that it "seems" to be solving a non-issue. Is it really ? As if the most used web software in the world could accept code in its core without anybody noticing it and remove it in a heartbeat.

Can anybody from the core team step up and let us know:

  • Why was this feature introduced ? What problem does it solve? If any...
  • What is the status on their thoughts now that this seems to cause more trouble than good?
  • What will be done in the future to make sure we don't break things with functionalities like this ?

I just want to be clear, I'm not trying to point a finger at anyone. I'm looking for a long-term solution because it seems unacceptable to me that a software like WordPress, used by hundreds of millions of websites, have issues like this.

#57 in reply to: ↑ 56 @westonruter
6 years ago

Replying to maximejobin:

  • Why was this feature introduced ? What problem does it solve? If any...

See above. It's related to changes in the file editor in #6531 via r41806. In previous version of WordPress, files would only be listed 2-levels deep. In 4.9, the entire directory tree for a theme is now listed regardless of depth. Recursively gathering a list of files can be expensive for large themes, so that is why caching was enabled. An unintended side effect of the caching is that the same directory listing function get_files is used both for the theme editor and for gathering page templates.

One change could be to only use caching when calling get_files with a $depth greater than 1. Note that get_post_templates will explicitly call get_files with a $depth of 1 already. I believe the only place where infinite depth of -1 is used is for the theme editor, and in that low-traffic'ed place, caching may be overkill anyway. So as long as we can confirm that infinite depth calls to get_files is limited to file editor, I would +1 removal of the caching.

#58 @stinkyweezle
6 years ago

Clearing all transients will apparently flush the filename cache if that helps anyone.

#59 in reply to: ↑ 3 @efpacc
6 years ago

Replying to ocean90:

Not sure if the caching was a good idea there. But bumping the version of the theme allows you to refresh the cache.

Thanks, this does the work until we get an update, also there is a plugin to add a clear cache button i found on this ticket https://wordpress.org/support/topic/updated-to-4-9-wont-detect-page-template/ from a user named connor, direct link to the plugin here https://github.com/connorlacombe/WP-Clear-File-Cache , hope we get a permanent solution soon.

#60 @hereswhatidid
6 years ago

+1 for removal or at least better error handling when the cached files are no longer found to explain what happened. Perhaps that would be a good place to put a "Clear File Cache" link if no where else.

#61 @efpacc
6 years ago

+1 for removal or perhaps a message on the theme editor saying the files there are being cached and a link or button to clear as desired or even better deactivate it while on development to have more control of it.

Last edited 6 years ago by efpacc (previous) (diff)

#62 in reply to: ↑ 49 @danni240
6 years ago

  • Resolution set to worksforme
  • Status changed from new to closed

Replying to westonruter:

Here is a workaround plugin which will fix the issue until 4.9.1 comes out: https://gist.github.com/westonruter/6c2ca0e5a4da233bf4bd88a1871dd950

Thank you so much for this. Worked perfectly!

#63 @mindctrl
6 years ago

  • Resolution worksforme deleted
  • Status changed from closed to reopened

#64 @Howdy_McGee
6 years ago

I wouldn't mind seeing the cache being used in template functions such as locate_template().

Initially, I was all in favor of just using WP_DEBUG as the enable/disable flag for caching theme files until I realize that on live sites, I still keep WP_DEBUG and WP_DEBUG_LOG enabled to take care of any issues that crop up. So I would need to make a choice, what's better for this live site, caching the theme files or being able to log any potential issues that may crop-up upon plugin / theme updating?

I like the functionality but I don't like ( along with many others ) it interfering with my development speed. I'm in favor of an additional wp-config flag to enable / disable this type of caching functionality; that's my vote.

#65 @teresa17
6 years ago

  • Version changed from 4.9 to trunk

I have been watching this ticket #42573 because customizer fails to load since the WP 4.9 upgrade on a one page site using [Avata Pro] template I have been trying to complete an update for http://evolutioncovers.com but have NO access to edit widgets or cover page sections etc since 4.9 was installed via WP update. Jumping in to inform that I tried two different [clear cache patches] without resolve in my case. Appreciate all of your work on this because - PHP (I know not} Will continue to watch and hope for a resolution.

#66 follow-up: @Clorith
6 years ago

  • Version changed from trunk to 4.9

#67 in reply to: ↑ 66 @teresa17
6 years ago

Replying to Clorith:
I have been watching this ticket #42573 because customizer fails to load since the WP 4.9 upgrade on a one page site using [Avata Pro] template I have been trying to complete an update for http://evolutioncovers.com but have NO access to edit widgets or cover page sections etc since 4.9 was installed via WP update. Jumping in to inform that I tried two different [clear cache patches] without resolve in my case. Appreciate all of your work on this because - PHP (I know not} Will continue to watch and hope for a resolution.

DISREGARD any changes made by my reply re{: Version changed from 4.9 to trunk} did not see until after submitting

#68 @Levdbas
6 years ago

If the caching is only effectively used in the file editor, I vote for removal.

Another option to opt-in to caching for larger themes with loads of folders. If a developer makes such a big theme it should be his or her consideration to implement this kind of caching. This way it ain't an inconvenience for small themes.

#69 in reply to: ↑ 49 @Olly - OWMC
6 years ago

Replying to westonruter:

Here is a workaround plugin which will fix the issue until 4.9.1 comes out: https://gist.github.com/westonruter/6c2ca0e5a4da233bf4bd88a1871dd950

Nice one! Works perfectly.

+1 for removing caching. There has to be a better way for improving admin page loading times. The consequences of going this route have probably cost the planet 10s of thousands of hours. @ryanduff 's comments are salient.

#70 @dd32
6 years ago

Here's what I'm thinking for 4.9.1:

  • Page Templates: Remove the caching by default, keep it looking 1 folder deep only
  • File Editor: Allow opt-in caching, used mostly by the file editor
    • Reduce the cache time to a slightly more reasonable 5-15 minutes (instead of an hour), which seems plenty for a low-traffic area to me.

Additionally these thoughts came to mind, based on the ticket feedback here:

  • Consider disabling the cache completely when WP_DEBUG is set - this seems like a bad idea. It'd effectively cause the cached template issue to not exist when a developer started debugging.
  • Consider only caching the data when fetching it takes longer than a threshold - Does it take 5seconds? Cache it. 50ms? Just query it every time - This could have unintended consequences, an issue which only happens sometimes, gone as soon as they start trying to fix it.

Thinking out loud here, Filesystem caching has always been a minefield, the FS includes a cache already, unfortunately the act of accessing those directories can be slow even if the FS is caching information.

The idea of only having the cache kick in if the speed of looking up this information was slow is an interesting one to me, it would effectively mean that there is no cache, but the moment that the hosts disk is overloaded or you have a 100-folder-deep theme then it starts to cache it for a short amount of time - hopefully long enough to let you do whatever you're doing without further delays.
In this scenario the page templates would still be cached, but only if it was slow in the first place.

Thoughts?

#71 @don_alejandro
6 years ago

An issue I'm not seeing explicitly mentioned is that the 4.9 update effectively frelled live sites that are not under development and have not made recent template changes. Also, if the template cache is supposed to update every hour or so, I’ve not seen any evidence of that.

I updated my sites several days ago and am now receiving emails about pages that were fine before updating to 4.9 – WordPress still has not gotten around to noticing the collection of page templates included in my parent theme. For example, the 4.9 update failed to notice any of my parent theme page templates except for the 'default' template. Pages using the default template or a child theme template were fine, but any pages using one of the optional parent theme templates were reset to the 'default' template. I couldn’t even fix the pages until after finding one of the custom plugins especially developed to clear this new cache.

In addition to clearing the cache, I first had to copy all of the parent theme page templates into my child theme folder in order to get WordPress 4.9 to notice optional page templates included in the parent theme. Arrrghhh...

I'll be spending the rest of the day copying template files, clearing the template cache on ten different websites and looking at pages to see which ones I need to manually reset to use the proper template.

Last edited 6 years ago by don_alejandro (previous) (diff)

#72 follow-up: @tdmalone
6 years ago

Until this is fixed - it’s probably worth noting that instead of adding a temporary plugin or bumping your style.css version (which you might not want to do until releasing), if you’ve got wp-cli installed you could run wp transient delete —all.

#73 in reply to: ↑ 72 @westonruter
6 years ago

Replying to tdmalone:

Until this is fixed - it’s probably worth noting that instead of adding a temporary plugin or bumping your style.css version (which you might not want to do until releasing), if you’ve got wp-cli installed you could run wp transient delete --all.

Running wp transient delete --all does not work when you have a persistent object cache enabled, like Memcached or Redis. In that case, you'll need to run wp cache flush. See https://github.com/wp-cli/cache-command/blob/9c3d686/src/Transient_Command.php#L247

This ticket was mentioned in Slack in #hosting-community by clorith. View the logs.


6 years ago

#75 @samoya22
6 years ago

Please make this caching stop. I spent the better part of an hour asking myself if I'd finally pushed myself over the edge. In the middle of a development sprint. I guess I can't imagine a world in which cached template lists are useful, at all.

This ticket was mentioned in Slack in #core-committers by johnbillion. View the logs.


6 years ago

This ticket was mentioned in Slack in #core by johnbillion. View the logs.


6 years ago

#78 @Otto42
6 years ago

What is the problem that this caching solves, exactly? File IO? Is it a major performance drag? I think that this seems a lot like "early optimization".

I vote to remove the transient unless there is a really surprisingly good reason for it which I am not aware of.

#79 @schlessera
6 years ago

@Otto42 For the file lists in the editors, as we don't control the structure that will get parsed and the depth that will need to be traversed, this can potentially take minutes to complete. The caching is a kind of safeguard to not enable people to bring the entire server down because they totally trash IO bandwidth.

I've added a new patch 42573.5.patch that disables the caching for theme files if the requested depth is 1 or less. The template files are all being scanned with a requested depth of 1 for the PHP files and 0 for the CSS file. So, this should effectively disable caching in regards to the templating completely, while still keeping it as a safeguard for the code editor.

Also, I reduced the expiration of the cache from 1 hour to 5 minutes. This will still throttle I/O usage, while reducing the negative side effects of caching.

@schlessera
6 years ago

Only use caching for a depth > 1, and reduce expiry to 5 minutes (updated)

#80 @Otto42
6 years ago

@schlessera I get that file IO is a thing, but the previous editor didn't have this cache, and I don't recall any reports of people having issues with it trashing their IO. Yes, you could conceivably make a plugin nest so deeply that a simple scandir takes forever and breaks the site, but realistically, PHP only has 30 seconds to run and it will get killed off before that happens. Also, we'd remove the plugin from the directory for being stupid like that.

So, again, what *actual* problem does this solve? Not theoretical, but is there any real instance of a plugin that takes more than a few seconds to do a scandir of? Jetpack is the biggest most complex plugin I'm aware of, and it works just fine.

So, yeah, I'm still voting to just remove the transient entirely. It's creating problems, not solving them. Your patch for 5 minutes is great and all, but it's probably better to not have a cache than to cache preemptively for no real reason.

#81 @schlessera
6 years ago

@Otto42 Yes, the previous editor didn't have that problem, but that editor also limited the folder recursion to two levels deep. Therefore, it didn't run the risk of getting caught in endless node_modules hierarchies or similar.

With the new editor, the recursion limit was lifted (technically, it still stops at 100 levels deep), so the risk is much bigger that something bad will happen.

This ticket was mentioned in Slack in #core by nerrad. View the logs.


6 years ago

#83 @schlessera
6 years ago

Oh, what I should mention as well: This is not about plugins, they are not part of this ticket. This is about the get_files() function that is being used for themes.

A lot of themes nowadays contain custom JavaScript code, that relies on dependencies pulled in via npm/yarn. Given that the code editor is meant to be used for live coding of themes on the server, this will in turn mean that the node_modules folder will also need to reside on that server, and it can typically contain tens of thousands of files.

#84 @mark-k
6 years ago

@schlessera I think that what you basically say is that with themes that follow modern development practice (and actually plugins do that as well, so not sure why the separation at all), the whole editor thing is "false advertisement", as you can not actually edit a JS or CSS file and see results as either the theme has a build process that needs to run (JS) or you better do the change at SASS/LESS files to have them properly apply everywhere.

I agree with otto42, the impact of the caching should first be measured against a proper setup with a complex theme on a typical hosting server, not a local dev machine, as it has too many processes running to be able to deduct anything from testing on it. Then you can compare performance of cached version to non cached, and different caching intervals. My gut feeling, is that since we are talking here about human scale response times, 5 minutes will be too short and it will feel like there is no caching at all.

IMO for 4.9.1 caching should just be scrapped. This is the path of least resistance and least risk. Then if some complex themes create an actual problems to users you will be able to have a judgment call on whether it is a theme problem that is not worthy of being handled in core.

In any case, for people that have the editor disabled, caching should never be triggered, or better, have the caching limited to the context of the theme/plugin editor.

#85 @abramarket
6 years ago

Ug another vote to disable this - why on earth would caching this list be useful? I am developing a theme that will need to add several templates - this literally makes it a nightmare to do one of the most basic things in WordPress :(

This ticket was mentioned in Slack in #core by rabmalin. View the logs.


6 years ago

#87 @BackuPs
6 years ago

I have been banging my head on this for hours. I deleted a file in my template as it was no longer needed because of changes in woocommerce and i got a warning message that the file was missing

file_get_contents(/var/docs/website.tst/public/wp-content/themes/mytheme/single-product/product-image.php): failed to open stream:

Thanks for this ! The fix on github works fine. https://gist.github.com/westonruter/6c2ca0e5a4da233bf4bd88a1871dd950

Is this gonna be fixed in 4.9.1 or do we need to keep on adding that function?

#88 @mofeenster
6 years ago

Oh wow this really made my day more fun, I managed to find this thread after comparing 2 different installs of WP and playing "spot the difference", then working out that google will possibly assist in finding the answer.

You know when you have a limited amount of time to get a job done and then you can't because you end up de-bugging something like this...

Last edited 6 years ago by mofeenster (previous) (diff)

#89 @dd32
6 years ago

  • Owner set to dd32
  • Resolution set to fixed
  • Status changed from reopened to closed

In 42242:

Theme/Plugin Editor: Remove the caching added in [41806] as it causes more problems than it fixes.

While caching here seemed like a good idea in theory, in practice the cache would be often stale causing development issues.
We exclude common folders (such as node_modules) from the scanning to avoid directories which are not useful to the end-user, so as long as those exclusion lists are held up this shouldn't cause too much of a degredation in the future.
We may consider adding caching here again in the future if it's determined that it is really needed.

Props precies, ibenic, mariovalney, schlessera, and all the others who commented on the ticket(s).
This partually reverts [41806].
See #6531.
Fixes #42573.

#90 @dd32
6 years ago

In 42243:

Theme/Plugin Editor: Remove the caching added in [41806] as it causes more problems than it fixes.

While caching here seemed like a good idea in theory, in practice the cache would be often stale causing development issues.
We exclude common folders (such as node_modules) from the scanning to avoid directories which are not useful to the end-user, so as long as those exclusion lists are held up this shouldn't cause too much of a degredation in the future.
We may consider adding caching here again in the future if it's determined that it is really needed.

Props precies, ibenic, mariovalney, schlessera, and all the others who commented on the ticket(s).
This partually reverts [41806].
Merges [42242] to the 4.9 branch.
See #6531.
Fixes #42573 for 4.9.

#91 @dd32
6 years ago

#42706 was marked as a duplicate.

#92 follow-up: @BackuPs
6 years ago

Is this gonna be fixed in the beta wp 5.0 and above aswell? I see the same issue in those builds.

#93 @swissspidy
6 years ago

#42709 was marked as a duplicate.

#94 in reply to: ↑ 92 @dd32
6 years ago

  • Keywords needs-patch removed

Replying to BackuPs:

Is this gonna be fixed in the beta wp 5.0 and above aswell? I see the same issue in those builds.

Yes, it was committed against trunk (5.0 alpha) however the nightly builds only refresh every 24hours and I believe I committed a few hours after the previous build.

#95 @SergeyBiryukov
6 years ago

#42728 was marked as a duplicate.

#96 in reply to: ↑ 3 @kaleelahamed
6 years ago

Replying to ocean90:

Not sure if the caching was a good idea there. But bumping the version of the theme allows you to refresh the cache.

Thank you so much. This saved me a lot of time :)

#97 @maricate
6 years ago

  • Focuses template added
  • Keywords reporter-feedback ui-feedback added
  • Severity changed from normal to critical

None of these fixes worked here.
I'm updated to the 4.9.1 and still not working/showing up.
Any suggestions?
I really need to create new page templates and use it but it continues not working.

#98 follow-up: @mariovalney
6 years ago

  • Focuses template removed
  • Keywords reporter-feedback ui-feedback removed
  • Severity changed from critical to normal

Hi @maricate, how are you?

The 4.9.1 version fixed it (I confirmed it in my fresh install righ now).

Can you open a topic in support forum please?

#99 in reply to: ↑ 98 @maricate
6 years ago

Replying to mariovalney:

Hi @maricate, how are you?

The 4.9.1 version fixed it (I confirmed it in my fresh install righ now).

Can you open a topic in support forum please?

Very weird...
I disabled the CDN option and all the other cache options.
And the problem persists. Maybe a problem with the host?
I will contact the host.

#100 follow-up: @simonp303
6 years ago

I am still not seeing templates:

I updated from 4.8.2 to 4.9.1

I ran wp cache flush and got Success: The cache was flushed.

I have also enabled WP_DEBUG

Still not seeing any templates

Also...

Ran wp transient delete --all and updated theme version number

still no joy

Last edited 6 years ago by simonp303 (previous) (diff)

#101 in reply to: ↑ 100 ; follow-up: @mariovalney
6 years ago

Hi @simonp303, how are you?
Like I said to @maricate, the 4.9.1 version fixed it.

Can you open a topic in support forum please?

#102 in reply to: ↑ 101 ; follow-up: @simonp303
6 years ago

Replying to mariovalney:

Hi @simonp303, how are you?
Like I said to @maricate, the 4.9.1 version fixed it.

Can you open a topic in support forum please?

But it clearly hasn't fixed it... :) so this bug should be re-opened

#103 in reply to: ↑ 102 ; follow-up: @mariovalney
6 years ago

Replying to simonp303:

But it clearly hasn't fixed it... :) so this bug should be re-opened

It's working here. And as I said, I confirmed it in one fresh install after 4.9.1.
So... I guess it could be a problem in your instalation/theme and shouldn't be reopen.

#104 in reply to: ↑ 103 ; follow-ups: @javorszky
6 years ago

Hi @mariovalney how are you?

Replying to mariovalney:

Replying to simonp303:

But it clearly hasn't fixed it... :) so this bug should be re-opened

It's working here. And as I said, I confirmed it in one fresh install after 4.9.1.
So... I guess it could be a problem in your instalation/theme and shouldn't be reopen.

I'm glad it works for you. It's great, super even!

However that doesn't mean that @simonp303 doesn't have a problem with this functionality. In my view the best qualities of open source is, well, open source, and collaboration, and WordPress is especially famous for an inclusive, welcoming, and collaborative tone, so all problems, regardless of the reporter's level of technical expertise or language capabilities (English as a second language, etc) can get help and are not discouraged from future participation by people who shoot them down. Hope that's fair :).

Going back to the issue. Downgrading back to 4.8.3 makes the template appear. Using 4.9.1 doesn't. There is a genuine problem here. It warrants further examination.

#105 in reply to: ↑ 104 @mariovalney
6 years ago

@javorszky I'm fine and hope you are too.

I'm not saying @simonp303 doesn't have a problem. I believe he has a problem. That's why I recommended to create a topic in support forum.

I'm sorry if I didn't sound collaborative indicating the issue don't happens in a fresh install and linking the support channel to try to solve his problem.

Last edited 6 years ago by mariovalney (previous) (diff)

#106 in reply to: ↑ 104 ; follow-up: @don_alejandro
6 years ago

Replying to javorszky:

I wonder if you are using a child theme? When this bug first appeared, only my child-theme templates and the parent theme's default page template appeared as options in an edit window. I had to copy all my parent theme page templates to my child theme folder and then clear the cache (I used the 'WP Clear File Cache' plugin by Seismic).

If WordPress is still not registering parent-theme templates the problem may not be totally fixed...

#107 in reply to: ↑ 106 ; follow-up: @simonp303
6 years ago

Hi @don_alejandro :)

Replying to don_alejandro:

Replying to javorszky:

I wonder if you are using a child theme? When this bug first appeared, only my child-theme templates and the parent theme's default page template appeared as options in an edit window. I had to copy all my parent theme page templates to my child theme folder and then clear the cache (I used the 'WP Clear File Cache' plugin by Seismic).

If WordPress is still not registering parent-theme templates the problem may not be totally fixed...

It is a parent theme - so no child theme. I have cleared all caches and transients that I was able to with WP-CLI. It is a very very basic theme as well. Nothing unusual about it and no unusual plugins (only Yoast and ACF).

The issue is that I would not be here had I not tried many many many things :) Hence why I feel this is still an issue. The whole point of WordPress is to develop it and make your own themes. Its no good saying it works in a different setup. That just sounds like a developer response like "well it works fine for me" etc :p

#108 in reply to: ↑ 107 @theMikeD
6 years ago

The issue is that I would not be here had I not tried many many many things :) Hence why I feel this is still an issue. The whole point of WordPress is to develop it and make your own themes. Its no good saying it works in a different setup. That just sounds like a developer response like "well it works fine for me" etc :p

That's true...but so is, "It's not working for me," without providing any information on how to duplicate the problem. Nobody is going to be able to help unless the issue can be duplicated, and so far there are no steps, links or downloads on how to do so.

When you do decide to provide these steps, please be as detailed as possible, as if you are writing for someone without experience with WP (within reason of course!). I have found more than once that the act of writing the duplication steps in this way reveals the true source of the issue. And make sure you're using no plugins and a core theme to start :)

HTH

...Mike

#109 follow-up: @shishx86
6 years ago

I have the same problem with child theme.

All plugins disabled.
Parent Theme - Twenty Twelve
WP version 4.9.1 (Clear Core, I tested)
Runung on MAMP 4.2.1
Templates placed in page-templates/
One of Template header (I tried name it on English):

/**
 * Template Name: Карта
 *
 * Description: Twenty Twelve loves the no-sidebar look as much as
 * you do. Use this page template to remove the sidebar from any page.
 *
 * Tip: to remove the sidebar from all posts and pages simply remove
 * any active widgets from the Main Sidebar area, and the sidebar will
 * disappear everywhere.
 *
 * @package WordPress
 * @subpackage Twenty_Twelve
 * @since Twenty Twelve 1.0
 */

Helps only rollback to 4.8.4

Last edited 6 years ago by shishx86 (previous) (diff)

#110 in reply to: ↑ 109 ; follow-up: @mariovalney
6 years ago

Hi @shishx86, how are you?

I tried to reproduce the problem but was not able.
The only moment I had a template problem was when I tried to give permission 444 to page-templates/ folder.

Can you make sure the folder/file is readable by WordPress?

I already tried to save the file with many encodings to make sure is not that kind of problem. And I was able to reproduce the problem with UTF-16.

Which encoding are you using to your files?

But in both cases the problem happens in 4.8.4 too... :(

#111 in reply to: ↑ 110 @shishx86
6 years ago

Replying to mariovalney:

Hi @shishx86, how are you?

I tried to reproduce the problem but was not able.
The only moment I had a template problem was when I tried to give permission 444 to page-templates/ folder.

Can you make sure the folder/file is readable by WordPress?

I already tried to save the file with many encodings to make sure is not that kind of problem. And I was able to reproduce the problem with UTF-16.

Which encoding are you using to your files?

But in both cases the problem happens in 4.8.4 too... :(

@mariovalney, I can give you login/password and you can see all yourself. If OK, I'll find you on facebook and give you temp credentials.

#112 @dd32
6 years ago

#42962 was marked as a duplicate.

#113 follow-up: @dd32
6 years ago

As it turns out, there still exists some caching ability of page template names in WordPress, which has been there for over 6 years - r20029
However, WordPress specifically does not enable the caching of page templates, a plugin would need to specifically enable that.
If you're still experiencing issues here, I'd suggest looking at whether you have any caching plugins installed, or a wp-content/object-cache.php file in place.

#114 in reply to: ↑ 113 @shishx86
6 years ago

Replying to dd32:

As it turns out, there still exists some caching ability of page template names in WordPress, which has been there for over 6 years - r20029
However, WordPress specifically does not enable the caching of page templates, a plugin would need to specifically enable that.
If you're still experiencing issues here, I'd suggest looking at whether you have any caching plugins installed, or a `wp-content/object-cache.php file in place.

I tried to disable all plugins and not found any objects-cache.
This issue reproduced only on one quick-built site for friend...
If you interested in I can give you temp access (FTP+admin panel) and you can see it your self.
Contact me on facebook

This ticket was mentioned in Slack in #accessibility by postphotos. View the logs.


6 years ago

Note: See TracTickets for help on using tickets.