WordPress.org

Make WordPress Core

Opened 4 weeks ago

Closed 2 weeks ago

Last modified 11 days ago

#42573 closed defect (bug) (fixed)

Templates not working properly

Reported by: precies Owned by: dd32
Milestone: 4.9.1 Priority: high
Severity: normal Version: 4.9
Component: Themes Keywords:
Focuses: Cc:

Description

Hello,

We updated our WordPress to version 4.9 and found problems with our custom templates files in our child-theme.

We always create our own template files for custom website for our customers.

The problem:
Since the update we only see one custom template file. If we add another one it wont show up under Templates when we edit a page. (See attached screenshot).

https://imgur.com/cCBo7KY

When we did a downgrade to WordPress 4.8.3 we were able to select all of our templates again.

Can we fix this by ourselves or is it a bug that needed to be fixed by WordPress?

Kind regards,

Joep van Dongen

Attachments (6)

only-one-template-4.9.png (11.1 KB) - added by precies 4 weeks ago.
Only one custom template file te select
42573.patch (999 bytes) - added by ibenic 4 weeks ago.
Disabling the cache if WP_DEBUG is true
42573.2.patch (2.3 KB) - added by mariovalney 4 weeks ago.
42573.3.patch (2.2 KB) - added by mariovalney 4 weeks ago.
Allow developers to flush files cache
42573.4.patch (1.2 KB) - added by mariovalney 3 weeks ago.
Removing cache
42573.5.patch (1.4 KB) - added by schlessera 3 weeks ago.
Only use caching for a depth > 1, and reduce expiry to 5 minutes (updated)

Download all attachments as: .zip

Change History (105)

@precies
4 weeks ago

Only one custom template file te select

#1 @Clorith
4 weeks ago

Hiya, and welcome to Trac!

In r41806 we added caching to the list of files fetched from themes, this means any changes to the filesystem won't instantly show up, it can take as long as an hour for them to show up.

It was originally intended for the code editor screen, but I'm not entirely sure it's bad that it also affected the selector for page templates, if a theme has a lot of files that part of the backend could be slowed down by the need to iterate over files often.

#2 follow-up: @ibenic
4 weeks ago

Maybe we could skip that caching if WP_DEBUG is true? Could be useful while developing a new theme.

#3 follow-ups: @ocean90
4 weeks ago

Not sure if the caching was a good idea there. But bumping the version of the theme allows you to refresh the cache.

@ibenic
4 weeks ago

Disabling the cache if WP_DEBUG is true

#4 @precies
4 weeks ago

Hello guys,

It was caching of the files. After an hour they showed up. I think it is a good idea to disable cache while wp_debug is true. Because while developing a page is a bit anoying to wait for an hour;)

And thanks for the fast response:)

#5 @robscott
4 weeks ago

Will be a puzzling one the first time you see it! Maybe worth adding (or documenting) how to force clear the caches?

#6 @SergeyBiryukov
4 weeks ago

  • Milestone changed from Awaiting Review to 4.9.1

#7 @SergeyBiryukov
4 weeks ago

#42567 was marked as a duplicate.

#8 in reply to: ↑ 3 @MarcusAnthony
4 weeks ago

Replying to ocean90:

Not sure if the caching was a good idea there. But bumping the version of the theme allows you to refresh the cache.

Thank you - this simple solution worked!

#9 in reply to: ↑ 3 @dghizoni
4 weeks ago

Replying to ocean90:

Not sure if the caching was a good idea there. But bumping the version of the theme allows you to refresh the cache.

Thanks! Worked for me too.

@mariovalney
4 weeks ago

@mariovalney
4 weeks ago

Allow developers to flush files cache

#10 follow-up: @mariovalney
4 weeks ago

Hey people, thanks for this ticket (saved some hours of work).
I agree ignore cache while WP_DEBUG is true is a good way to allow developers see the changes fast.

About 42573.3.patch, I suggest we can allow developers to flush that cache someway.

In 42573.2, I added the check before WP_DEBUG to avoid warnings because I did not remember it's always defined (ignore it, please - my bad).

#11 in reply to: ↑ 10 @MarcusAnthony
4 weeks ago

Replying to mariovalney:

Hey people, thanks for this ticket (saved some hours of work).
I agree ignore cache while WP_DEBUG is true is a good way to allow developers see the changes fast.

About 42573.3.patch, I suggest we can allow developers to flush that cache someway.

In 42573.2, I added the check before WP_DEBUG to avoid warnings because I did not remember it's always defined (ignore it, please - my bad).

Allowing us developers to flush cache would be a huge asset.

#12 @leobaiano
4 weeks ago

#42591 was marked as a duplicate.

#13 @Vishnja1
4 weeks ago

I'm also not sure caching was a good idea there, at least this can be explicit option (in wp-config for example). Otherwise it would be a surpise for any WP developer how to edit/add theme template files in the nearest future). We should be bumping the version of the theme!

#14 follow-ups: @robscott
4 weeks ago

So the solution to this issue is to version bump the theme.

If you're adding a whole new page template, I would respectfully suggest that the theme version must be bumped. However, in development, this would be confusing the first time it were encountered.

Most systems with caching should have a "flush all the caches" button somewhere. I like the "if debug is on don't cache anything" option too.

Last edited 4 weeks ago by robscott (previous) (diff)

#15 in reply to: ↑ 14 @robscott
4 weeks ago

urgh replied to myself instead of editing. Lunch time.

Last edited 4 weeks ago by robscott (previous) (diff)

#16 @swissspidy
4 weeks ago

#42601 was marked as a duplicate.

#17 in reply to: ↑ 14 ; follow-up: @mariovalney
4 weeks ago

  • Keywords has-patch added

Replying to robscott:

So the solution to this issue is to version bump the theme.

If you're adding a whole new page template, I would respectfully suggest that the theme version must be bumped. However, in development, this would be confusing the first time it were encountered.

Most systems with caching should have a "flush all the caches" button somewhere.

A "flush all caches" could be a good feature but for now these patches seems good.

For a "flush all" we should think about wp_object_cache and the cases we do it with transient (like this case - are there another cases?) and maybe create ways to cache plugins be flushed in same button (so we are not just adding one more button to users).

I like the "if debug is on don't cache anything" option too.

I really agree. It should be a good practice to cache plugins too.

Last edited 4 weeks ago by mariovalney (previous) (diff)

#18 in reply to: ↑ 17 @robscott
4 weeks ago

Replying to mariovalney:

For a "flush all" we should think about wp_object_cache and the cases we do it with transient (like this case - are there another cases?) and maybe create ways to cache plugins be flushed in same button (so we are not just adding one more button to users).

Yes its a wider thing really. Would be most excellent if there was one WP trigger to flush caches; transients which (may) no longer apply; and "everything" that could be a one process cache clear-ify. Hmm, wonder if this idea exists anywhere!?

#19 follow-up: @metropolis_john
4 weeks ago

There definitely needs to be a way to disable this completely as it makes theme development a chore. Having to bump version number, or the suggestion of having to clear the cache manually, every time a new template is added is not a solution.

#20 @Clorith
4 weeks ago

#42611 was marked as a duplicate.

#21 @andrewinsideout
4 weeks ago

Please add the option to disable this. I just wasted a bunch of time trying to figure out what I was doing wrong. Thanks to the people who suggested changing the version number... but what a pain while developing.

#22 @ibenic
4 weeks ago

The WP_DEBUG should be enough if you ask me for local development. If someone uploads a new template through FTP, let's say, their live/staging site with a disabled WP_DEBUG, the page templates won't be refreshed.

Maybe we could also add a button that could delete the cache on request. This button could be under Appearance > Themes on the active theme block. Something like this: https://imgur.com/Xwh3Qyo ? But that could be a little too intrusive to regular users? Not sure.

WP_DEBUG + version change might be just enough for developers.

#23 in reply to: ↑ 19 @ryan360
4 weeks ago

Replying to metropolis_john:

There definitely needs to be a way to disable this completely as it makes theme development a chore. Having to bump version number, or the suggestion of having to clear the cache manually, every time a new template is added is not a solution.

Agreed. I can't believe this was overlooked. I just wasted a bunch of time because of this!

#24 follow-up: @Clorith
3 weeks ago

  • Keywords needs-patch added; has-patch removed

We'll definitely not add any buttons to the UI that will rarely, if ever, see use by actual users.

If we do decide to keep the transient in place, I do like the WP_DEBUG check, with it in use there's no need for the extra functions to clear things, as the version bump invalidation should take care of things for end users, but the patch needs refined a little.

I looked at 42573.3.patch and it just checks if WP_DEBUG is true or false, it also needs a check if it's defined altogether.

On a side note, I agree with adding the transient here, now that we recursively scan through the whole theme directory the operation can become heavy when we take into account 3rd party premium themes as well which are known for bundling plugins and other large code bases making them substantially more time consuming than they used to be.

#25 in reply to: ↑ 24 ; follow-up: @SergeyBiryukov
3 weeks ago

Replying to Clorith:

If we do decide to keep the transient in place, I do like the WP_DEBUG check, with it in use there's no need for the extra functions to clear things, as the version bump invalidation should take care of things for end users, but the patch needs refined a little.

I don't think the WP_DEBUG check is enough here. Plenty of users that create custom templates never heard of WP_DEBUG, and bumping the theme version is not an obvious step either.

Could we disable the caching for files in the theme's root directory (which seems to be where most users upload custom templates), and only use it for subdirectories?

#26 in reply to: ↑ 2 @webdados
3 weeks ago

Replying to ibenic:

Maybe we could skip that caching if WP_DEBUG is true? Could be useful while developing a new theme.

That's exactly my opinion.

(I've just lost 30 minutes debugging WP core to find this issue while developing a new theme)

#27 @theMikeD
3 weeks ago

Another vote for a way to disable this.

#28 in reply to: ↑ 25 ; follow-up: @theMikeD
3 weeks ago

Replying to SergeyBiryukov:

Replying to Clorith:

If we do decide to keep the transient in place, I do like the WP_DEBUG check, with it in use there's no need for the extra functions to clear things, as the version bump invalidation should take care of things for end users, but the patch needs refined a little.

I don't think the WP_DEBUG check is enough here. Plenty of users that create custom templates never heard of WP_DEBUG, and bumping the theme version is not an obvious step either.

Could we disable the caching for files in the theme's root directory (which seems to be where most users upload custom templates), and only use it for subdirectories?

This feel like a half measure and would not help with themes that put template in subfolders. WP_DEBUG along with documenting this somewhere would be a good first step. An option somewhere in the UI to flush the template cache would also help non-developers. But WP_DEBUG at a minimum.

#29 follow-up: @mariovalney
3 weeks ago

As performance must be a theme developer concern as well, maybe we can disable this as default and add a support to developers who creates themes with a lot of files activate it.

We can do that cache just in parent theme too.

This way we are covering the 3rd party premium themes and not being a problem to users who creates files into your (child) themes.

Last edited 3 weeks ago by mariovalney (previous) (diff)

#30 in reply to: ↑ 29 ; follow-ups: @ibenic
3 weeks ago

Replying to mariovalney:

We can do that cache just in parent theme too.

This way we are covering the 3rd party premium themes and not being a problem to users who creates files into your (child) themes.

Not sure about that. I have seen a lot of child themes having much more templates than the parent themes. Some of the parent themes are just used as frameworks such as the Genesis Framework.

I would consider maybe adding a simple (?) tooltip next to Page Template (?) which on hover would say that the templates are cached and if new templates were added, they should bump the theme/child theme version? It seems logical to bump the version since new templates were added.

#31 in reply to: ↑ 30 ; follow-ups: @webdados
3 weeks ago

Replying to ibenic:

It seems logical to bump the version since new templates were added.

Unless you're on active development and then bumping the version number is just a hassle

#32 in reply to: ↑ 31 @theMikeD
3 weeks ago

Replying to webdados:

Replying to ibenic:

It seems logical to bump the version since new templates were added.

Unless you're on active development and then bumping the version number is just a hassle

Exactly this.

#33 in reply to: ↑ 31 @ibenic
3 weeks ago

Replying to webdados:

Replying to ibenic:

It seems logical to bump the version since new templates were added.

Unless you're on active development and then bumping the version number is just a hassle

Of course! Bumping the version should help those who are changing a theme on a production site where they won't define the WP_DEBUG as true.

In my opinion, active development should be done on your local machine so turning WP_DEBUG to TRUE should not be an issue (if the patch is applied).

Last edited 3 weeks ago by ibenic (previous) (diff)

#34 follow-ups: @fatmedia
3 weeks ago

What exactly is the benefit of this? If a theme has so many templates that it would benefit from caching the admin meta box I think there's probably a bigger problem than some cache is going to solve...

Reverting this would be a win for sane theme authors. Why improve things for authors who choose to do ridiculous things?

#35 @mindctrl
3 weeks ago

Could we disable the caching for files in the theme's root directory (which seems to be where most users upload custom templates), and only use it for subdirectories?

I don't think that would adequately address the issue. I've seen a number of themes that store templates outside the root directory, such as wp-content/themes/theme_name/templates/.

#36 in reply to: ↑ 34 @ryanduff
3 weeks ago

Replying to fatmedia:

What exactly is the benefit of this? If a theme has so many templates that it would benefit from caching the admin meta box I think there's probably a bigger problem than some cache is going to solve...

Reverting this would be a win for sane theme authors. Why improve things for authors who choose to do ridiculous things?

Agreed. I'd yet to see a theme that has anywhere near 100 levels deep of a folder structure. If that's the case, the theme is the problem.

Again we've introduced unintended consequences to core by fixing a problem that never needed to be solved in the first place.

+1 to revert any form of caching around this and maybe revisit the changes in r41806 to see what problem they're solving and if this is even appropriate.

Nobody should be forced to use WP_DEBUG, nor should another useless button to purge this unnecessary transient be added somewhere in wp-admin.

As a developer, if I deploy a file update to a client site (live or staging) I should be able to use that template right away. Not be forced to wait up to an hour just for the cache to expire to select it. Think of the poor experience that a user is going to experience if they update a file in a theme and can't select it? That goes against the goals of the project plain and simple.

#37 follow-up: @robscott
3 weeks ago

@ryanduff if it were very simple to flush the caches, would you still have an issue with this being present? I like caching things that do not (normally) change, particularly if it can speed up admin pageload in poor and shared hosting... otherwise "WordPress is slow" will be the user takeaway.

Surely here we're talking about a surprise to a developer in their workflow versus slow(er) admin performance for *all* users of WordPress.

Assuming developers get a solution as a result of this conversation, that would be a win win? Without reverting anything?

#38 @jzankowski
3 weeks ago

Over 2 hours wasted because of this. Please fix this in a way that at least makes it clear what's happening and what option to flip to disable the caching behavior.

#39 in reply to: ↑ 37 ; follow-up: @ryanduff
3 weeks ago

Replying to robscott:

@ryanduff if it were very simple to flush the caches, would you still have an issue with this being present? I like caching things that do not (normally) change, particularly if it can speed up admin pageload in poor and shared hosting... otherwise "WordPress is slow" will be the user takeaway.

Surely here we're talking about a surprise to a developer in their workflow versus slow(er) admin performance for *all* users of WordPress.

Assuming developers get a solution as a result of this conversation, that would be a win win? Without reverting anything?

No. Again, r41806 got "clever" by introducing a max depth of 100 levels to the file list function. This is not needed nor is caching around it. Find me a bug where someone complained the page editor screen was too slow because their theme was bloated with files. What does this fix? Nada.

If you have slow admin performance it's because (a) Core changed to scan 100 levels deep in your theme folder and (b) you're using a theme so bloated it's now affected by this change and you should probably find a new theme.

This is not just a developer issue. It's only a matter of time before end users run into this issue as well (if they haven't already reported it in the forums).

The bigger issue here is template files not being available because the list is cached, not that the editor now supports editing theme files 100 levels deep.

#40 in reply to: ↑ 39 @metropolis_john
3 weeks ago

Replying to ryanduff:

This is not just a developer issue. It's only a matter of time before end users run into this issue as well (if they haven't already reported it in the forums).

Yes, one thing I noticed is the page template list being cached when trying to assign a custom field group to a specific page template in the Advanced Custom Fields plugin. End users could definitely run into this without any idea of how to fix it.

#41 in reply to: ↑ 28 @SergeyBiryukov
3 weeks ago

Replying to theMikeD:

Replying to SergeyBiryukov:

I don't think the WP_DEBUG check is enough here. Plenty of users that create custom templates never heard of WP_DEBUG, and bumping the theme version is not an obvious step either.

Could we disable the caching for files in the theme's root directory (which seems to be where most users upload custom templates), and only use it for subdirectories?

This feel like a half measure and would not help with themes that put template in subfolders. WP_DEBUG along with documenting this somewhere would be a good first step. An option somewhere in the UI to flush the template cache would also help non-developers. But WP_DEBUG at a minimum.

I meant disabling caching in the theme's root directory in addition to the WP_DEBUG check, not instead of it. A UI option seems like a non-starter though, as it contradicts the WordPress' philosophy of "decisions, not options".

That said, I agree with @ryanduff in comment:36, caching should not be necessary here at all.

#42 @johnbillion
3 weeks ago

  • Priority changed from normal to high

Agreed also, this caching should be removed.

#43 @pkoceniak
3 weeks ago

This should 100% be disabled by default.. with a setting somewhere that allows you to enable it.
I don't understand the logic behind this feature at all.. if your theme is built properly, than there should never be an issue here.
Just wasted over an hour tearing my hair out trying to figure out why my templates weren't showing.

This ticket was mentioned in Slack in #core-customize by clorith. View the logs.


3 weeks ago

#46 in reply to: ↑ 30 @mariovalney
3 weeks ago

Replying to ibenic:

Not sure about that. I have seen a lot of child themes having much more templates than the parent themes. Some of the parent themes are just used as frameworks such as the Genesis Framework.

Yeah... You are right.

I agree with everyone: remove this cache (or "disable as default and add a support to developers who creates themes with a lot of files activate it").

Last edited 3 weeks ago by mariovalney (previous) (diff)

@mariovalney
3 weeks ago

Removing cache

#47 @swb1192
3 weeks ago

I'm seeing an issue with a client site where child theme templates are being picked up, but not the parent theme page templates as of 4.9. Is that a separate issue?

#48 @robscott
3 weeks ago

@ryanduff am not even playing devil's advocate on this... I did not like the original implementation nor did I see it as necessary. BUT there was presumably some thinking behind it which may not need to be thrown out?

I am working on a solution to postmeta slowness for which the proposed fix was also caching. Also in that case, I did not think caching would be a good solution.

Caching is not a band aid to fix inefficient code.

That said, we should not just scrap something because it raises tickets - if it is the right thing to do. In this case, what was the original ill being fixed?

#49 follow-ups: @westonruter
3 weeks ago

Here is a workaround plugin which will fix the issue until 4.9.1 comes out: https://gist.github.com/westonruter/6c2ca0e5a4da233bf4bd88a1871dd950

#50 in reply to: ↑ description @dbhiems
3 weeks ago

Thank you @westonruter for making the fix into a plugin. It is working for me.

I think this is a bigger problem than just fixing bugs in a new release, because the 4.9 wordpress update essentially disabled a core wordpress feature with no explanation or way to fix it for the average user. It honestly makes me wonder how something like this was able to make it into the release in the first place. Like what is the decision making process to allow a change like that without appropriate review of the consequences for users. Any user like myself would have no idea that now, after creating a new page template, we will not see it appear for up to an hour, so for all intents and purposes custom page templates is now non functional. I spent a lot of time trying to figure out why my page templates didn't show up anymore, when I was doing exactly the same thing a few days ago with no problems. Literally copying the code of an existing page template (which showed up in the template list dropdown) and saving to a new renamed template would not show up in the list. It was a completely bizarre and frustrating situation. I really hope this gets looked at as a systematic issue beyond just fixing the bug because it I feel it undermines the trust in the excellence in wordpress with users and the expectation that new releases will improve things for the user.

#51 @majick
3 weeks ago

I see the advantages of applying file list caching here, but breaking things for end users and developers through the unintended consequences is something that can be avoided without much fuss... My suggestions:

  1. Identify what events could clear the file list cache for a theme when in the admin side? Maybe something simple like clearing the theme-specific file list in the function get_page_templates so at least it doesn't affect the standard UI selection etc.? eg.
    function get_page_template( $post = null, $post_type = 'page' ) {
        $theme = wp_get_theme();
        $theme->clear_cache( 'post_templates' );
        delete_transient( sanitize_key( 'files_' . $theme->cache_hash . '-' . $theme->get( 'Version' ) ) ); 
        return array_flip( $theme->get_page_templates( $post, $post_type ) );
    }
    

I'm not sure why this transient key structure? Seems to have come from the plugin file list side? But it's a bit of a mismatch to the rest of the theme class transient keys, otherwise it could be more simply cleared using WP Theme clear_cache method - which in any case is the other place where the theme file list could (and probably should) be cleared.

  1. Add a filter to disable file caching for a particular theme, this way theme developers could set the filter in an mu-plugin or something locally and not have to worry about the theme file list cache while they are developing. eg. in WP Theme method get_files instead of $all_files = get_transient( $transient_key ); just:
$all_files = apply_filters( 'theme_file_list_cache', get_transient( $transient_key ), $this );

That could be used like this for example:

add_filter('theme_file_list_cache', function($filelist, $theme) {
    if ($theme->get_stylesheet() == 'my-theme') {return false;}
    return $filelist;
}), 10, 2 );

Also as a sidenote going from just a few subdirectory levels to 100 levels seems like going from one extreme to the other... Sure it's well beyond time that it increased, but it seems it's this extreme swing that then required the caching for performance, which then caused this problem. 10 would have been a safe maximum depth, with a filter for anyone really wanting more if needed in a particular plugin / theme.

On a related note, only using a depth of 1 for searching page templates in the WP Theme method get_post_templates method while the levels are now cached 100 deep seems a bit bizarre. Doesn't it precludes template directory structures as simple as /content/post/ ? Any reason not to simply bump this value up to 2 or 3?

#52 in reply to: ↑ 49 ; follow-up: @ijignesh
3 weeks ago

I was having same issue today in latest wordpress 4.9.

I just created new folder named page-templates and copied my custom template file in that folder and voila! my template name shows up on edit page in admin area.

Can anyone confirm is this working for them as well?

Thanks,

Replying to westonruter:

Here is a workaround plugin which will fix the issue until 4.9.1 comes out: https://gist.github.com/westonruter/6c2ca0e5a4da233bf4bd88a1871dd950

#53 in reply to: ↑ 52 @webdados
3 weeks ago

Replying to ijignesh:

I just created new folder named page-templates and copied my custom template file in that folder and voila! my template name shows up on edit page in admin area.

Can anyone confirm is this working for them as well?

Not for me

#54 @becky.absolute
3 weeks ago

Just a thought that I had ... what if an ENVIRONMENT variable was introduced (in some form). Then the code to cache the file listing could do a check of this ENVIRONMENT variable and only cache the listing if the value was set to "production" (or something along those lines).

I personally don't set WP_DEBUG to true while I'm developing my themes and plugins, unless I'm troubleshooting an issue. But it would be easy for me to set up an ENVIRONMENT variable in the wp-config.php file when I'm setting up my local development environment or a staging environment.

I've seen thing kind of variable in CodeIgniter and in Laravel. And I can imagine that it would be useful in other situations moving forward than just this one.

#55 in reply to: ↑ 34 @gaasbeekmw
3 weeks ago

Replying to fatmedia:

What exactly is the benefit of this? If a theme has so many templates that it would benefit from caching the admin meta box I think there's probably a bigger problem than some cache is going to solve...

Reverting this would be a win for sane theme authors. Why improve things for authors who choose to do ridiculous things?

Exactly, it also moves away from what Wordpress was all about, a default which you extend by using plugins. If someone requires this then there are plenty of options to add functionality via plugins. No need to have something like this in the core.

#56 follow-up: @maximejobin
3 weeks ago

As a business owner of a maintenance service where we do our best to keep things as stable as possible for our clients, two things are really surprising to me here:

  1. How could a feature like that be proposed, accepted and introduced into core without anyone noticing it? I did not see any information about this in the release notes or in the documentation. Did I miss anything? I truly hope it was documented somewhere and I missed it.
  1. Some people propose a patch, to add a button in the UI, to consider looking for WP_DEBUG, ... And then suddenly decisions are made to remove this functionality based on no real explanation. I totally agree with @ryanduff about the fact that it "seems" to be solving a non-issue. Is it really ? As if the most used web software in the world could accept code in its core without anybody noticing it and remove it in a heartbeat.

Can anybody from the core team step up and let us know:

  • Why was this feature introduced ? What problem does it solve? If any...
  • What is the status on their thoughts now that this seems to cause more trouble than good?
  • What will be done in the future to make sure we don't break things with functionalities like this ?

I just want to be clear, I'm not trying to point a finger at anyone. I'm looking for a long-term solution because it seems unacceptable to me that a software like WordPress, used by hundreds of millions of websites, have issues like this.

#57 in reply to: ↑ 56 @westonruter
3 weeks ago

Replying to maximejobin:

  • Why was this feature introduced ? What problem does it solve? If any...

See above. It's related to changes in the file editor in #6531 via r41806. In previous version of WordPress, files would only be listed 2-levels deep. In 4.9, the entire directory tree for a theme is now listed regardless of depth. Recursively gathering a list of files can be expensive for large themes, so that is why caching was enabled. An unintended side effect of the caching is that the same directory listing function get_files is used both for the theme editor and for gathering page templates.

One change could be to only use caching when calling get_files with a $depth greater than 1. Note that get_post_templates will explicitly call get_files with a $depth of 1 already. I believe the only place where infinite depth of -1 is used is for the theme editor, and in that low-traffic'ed place, caching may be overkill anyway. So as long as we can confirm that infinite depth calls to get_files is limited to file editor, I would +1 removal of the caching.

#58 @stinkyweezle
3 weeks ago

Clearing all transients will apparently flush the filename cache if that helps anyone.

#59 in reply to: ↑ 3 @efpacc
3 weeks ago

Replying to ocean90:

Not sure if the caching was a good idea there. But bumping the version of the theme allows you to refresh the cache.

Thanks, this does the work until we get an update, also there is a plugin to add a clear cache button i found on this ticket https://wordpress.org/support/topic/updated-to-4-9-wont-detect-page-template/ from a user named connor, direct link to the plugin here https://github.com/connorlacombe/WP-Clear-File-Cache , hope we get a permanent solution soon.

#60 @hereswhatidid
3 weeks ago

+1 for removal or at least better error handling when the cached files are no longer found to explain what happened. Perhaps that would be a good place to put a "Clear File Cache" link if no where else.

#61 @efpacc
3 weeks ago

+1 for removal or perhaps a message on the theme editor saying the files there are being cached and a link or button to clear as desired or even better deactivate it while on development to have more control of it.

Last edited 3 weeks ago by efpacc (previous) (diff)

#62 in reply to: ↑ 49 @danni240
3 weeks ago

  • Resolution set to worksforme
  • Status changed from new to closed

Replying to westonruter:

Here is a workaround plugin which will fix the issue until 4.9.1 comes out: https://gist.github.com/westonruter/6c2ca0e5a4da233bf4bd88a1871dd950

Thank you so much for this. Worked perfectly!

#63 @mindctrl
3 weeks ago

  • Resolution worksforme deleted
  • Status changed from closed to reopened

#64 @Howdy_McGee
3 weeks ago

I wouldn't mind seeing the cache being used in template functions such as locate_template().

Initially, I was all in favor of just using WP_DEBUG as the enable/disable flag for caching theme files until I realize that on live sites, I still keep WP_DEBUG and WP_DEBUG_LOG enabled to take care of any issues that crop up. So I would need to make a choice, what's better for this live site, caching the theme files or being able to log any potential issues that may crop-up upon plugin / theme updating?

I like the functionality but I don't like ( along with many others ) it interfering with my development speed. I'm in favor of an additional wp-config flag to enable / disable this type of caching functionality; that's my vote.

#65 @teresa17
3 weeks ago

  • Version changed from 4.9 to trunk

I have been watching this ticket #42573 because customizer fails to load since the WP 4.9 upgrade on a one page site using [Avata Pro] template I have been trying to complete an update for http://evolutioncovers.com but have NO access to edit widgets or cover page sections etc since 4.9 was installed via WP update. Jumping in to inform that I tried two different [clear cache patches] without resolve in my case. Appreciate all of your work on this because - PHP (I know not} Will continue to watch and hope for a resolution.

#66 follow-up: @Clorith
3 weeks ago

  • Version changed from trunk to 4.9

#67 in reply to: ↑ 66 @teresa17
3 weeks ago

Replying to Clorith:
I have been watching this ticket #42573 because customizer fails to load since the WP 4.9 upgrade on a one page site using [Avata Pro] template I have been trying to complete an update for http://evolutioncovers.com but have NO access to edit widgets or cover page sections etc since 4.9 was installed via WP update. Jumping in to inform that I tried two different [clear cache patches] without resolve in my case. Appreciate all of your work on this because - PHP (I know not} Will continue to watch and hope for a resolution.

DISREGARD any changes made by my reply re{: Version changed from 4.9 to trunk} did not see until after submitting

#68 @Levdbas
3 weeks ago

If the caching is only effectively used in the file editor, I vote for removal.

Another option to opt-in to caching for larger themes with loads of folders. If a developer makes such a big theme it should be his or her consideration to implement this kind of caching. This way it ain't an inconvenience for small themes.

#69 in reply to: ↑ 49 @Olly - OWMC
3 weeks ago

Replying to westonruter:

Here is a workaround plugin which will fix the issue until 4.9.1 comes out: https://gist.github.com/westonruter/6c2ca0e5a4da233bf4bd88a1871dd950

Nice one! Works perfectly.

+1 for removing caching. There has to be a better way for improving admin page loading times. The consequences of going this route have probably cost the planet 10s of thousands of hours. @ryanduff 's comments are salient.

#70 @dd32
3 weeks ago

Here's what I'm thinking for 4.9.1:

  • Page Templates: Remove the caching by default, keep it looking 1 folder deep only
  • File Editor: Allow opt-in caching, used mostly by the file editor
    • Reduce the cache time to a slightly more reasonable 5-15 minutes (instead of an hour), which seems plenty for a low-traffic area to me.

Additionally these thoughts came to mind, based on the ticket feedback here:

  • Consider disabling the cache completely when WP_DEBUG is set - this seems like a bad idea. It'd effectively cause the cached template issue to not exist when a developer started debugging.
  • Consider only caching the data when fetching it takes longer than a threshold - Does it take 5seconds? Cache it. 50ms? Just query it every time - This could have unintended consequences, an issue which only happens sometimes, gone as soon as they start trying to fix it.

Thinking out loud here, Filesystem caching has always been a minefield, the FS includes a cache already, unfortunately the act of accessing those directories can be slow even if the FS is caching information.

The idea of only having the cache kick in if the speed of looking up this information was slow is an interesting one to me, it would effectively mean that there is no cache, but the moment that the hosts disk is overloaded or you have a 100-folder-deep theme then it starts to cache it for a short amount of time - hopefully long enough to let you do whatever you're doing without further delays.
In this scenario the page templates would still be cached, but only if it was slow in the first place.

Thoughts?

#71 @don_alejandro
3 weeks ago

An issue I'm not seeing explicitly mentioned is that the 4.9 update effectively frelled live sites that are not under development and have not made recent template changes. Also, if the template cache is supposed to update every hour or so, I’ve not seen any evidence of that.

I updated my sites several days ago and am now receiving emails about pages that were fine before updating to 4.9 – WordPress still has not gotten around to noticing the collection of page templates included in my parent theme. For example, the 4.9 update failed to notice any of my parent theme page templates except for the 'default' template. Pages using the default template or a child theme template were fine, but any pages using one of the optional parent theme templates were reset to the 'default' template. I couldn’t even fix the pages until after finding one of the custom plugins especially developed to clear this new cache.

In addition to clearing the cache, I first had to copy all of the parent theme page templates into my child theme folder in order to get WordPress 4.9 to notice optional page templates included in the parent theme. Arrrghhh...

I'll be spending the rest of the day copying template files, clearing the template cache on ten different websites and looking at pages to see which ones I need to manually reset to use the proper template.

Last edited 3 weeks ago by don_alejandro (previous) (diff)

#72 follow-up: @tdmalone
3 weeks ago

Until this is fixed - it’s probably worth noting that instead of adding a temporary plugin or bumping your style.css version (which you might not want to do until releasing), if you’ve got wp-cli installed you could run wp transient delete —all.

#73 in reply to: ↑ 72 @westonruter
3 weeks ago

Replying to tdmalone:

Until this is fixed - it’s probably worth noting that instead of adding a temporary plugin or bumping your style.css version (which you might not want to do until releasing), if you’ve got wp-cli installed you could run wp transient delete --all.

Running wp transient delete --all does not work when you have a persistent object cache enabled, like Memcached or Redis. In that case, you'll need to run wp cache flush. See https://github.com/wp-cli/cache-command/blob/9c3d686/src/Transient_Command.php#L247

This ticket was mentioned in Slack in #hosting-community by clorith. View the logs.


3 weeks ago

#75 @samoya22
3 weeks ago

Please make this caching stop. I spent the better part of an hour asking myself if I'd finally pushed myself over the edge. In the middle of a development sprint. I guess I can't imagine a world in which cached template lists are useful, at all.

This ticket was mentioned in Slack in #core-committers by johnbillion. View the logs.


3 weeks ago

This ticket was mentioned in Slack in #core by johnbillion. View the logs.


3 weeks ago

#78 @Otto42
3 weeks ago

What is the problem that this caching solves, exactly? File IO? Is it a major performance drag? I think that this seems a lot like "early optimization".

I vote to remove the transient unless there is a really surprisingly good reason for it which I am not aware of.

#79 @schlessera
3 weeks ago

@Otto42 For the file lists in the editors, as we don't control the structure that will get parsed and the depth that will need to be traversed, this can potentially take minutes to complete. The caching is a kind of safeguard to not enable people to bring the entire server down because they totally trash IO bandwidth.

I've added a new patch 42573.5.patch that disables the caching for theme files if the requested depth is 1 or less. The template files are all being scanned with a requested depth of 1 for the PHP files and 0 for the CSS file. So, this should effectively disable caching in regards to the templating completely, while still keeping it as a safeguard for the code editor.

Also, I reduced the expiration of the cache from 1 hour to 5 minutes. This will still throttle I/O usage, while reducing the negative side effects of caching.

@schlessera
3 weeks ago

Only use caching for a depth > 1, and reduce expiry to 5 minutes (updated)

#80 @Otto42
3 weeks ago

@schlessera I get that file IO is a thing, but the previous editor didn't have this cache, and I don't recall any reports of people having issues with it trashing their IO. Yes, you could conceivably make a plugin nest so deeply that a simple scandir takes forever and breaks the site, but realistically, PHP only has 30 seconds to run and it will get killed off before that happens. Also, we'd remove the plugin from the directory for being stupid like that.

So, again, what *actual* problem does this solve? Not theoretical, but is there any real instance of a plugin that takes more than a few seconds to do a scandir of? Jetpack is the biggest most complex plugin I'm aware of, and it works just fine.

So, yeah, I'm still voting to just remove the transient entirely. It's creating problems, not solving them. Your patch for 5 minutes is great and all, but it's probably better to not have a cache than to cache preemptively for no real reason.

#81 @schlessera
3 weeks ago

@Otto42 Yes, the previous editor didn't have that problem, but that editor also limited the folder recursion to two levels deep. Therefore, it didn't run the risk of getting caught in endless node_modules hierarchies or similar.

With the new editor, the recursion limit was lifted (technically, it still stops at 100 levels deep), so the risk is much bigger that something bad will happen.

This ticket was mentioned in Slack in #core by nerrad. View the logs.


3 weeks ago

#83 @schlessera
3 weeks ago

Oh, what I should mention as well: This is not about plugins, they are not part of this ticket. This is about the get_files() function that is being used for themes.

A lot of themes nowadays contain custom JavaScript code, that relies on dependencies pulled in via npm/yarn. Given that the code editor is meant to be used for live coding of themes on the server, this will in turn mean that the node_modules folder will also need to reside on that server, and it can typically contain tens of thousands of files.

#84 @mark-k
3 weeks ago

@schlessera I think that what you basically say is that with themes that follow modern development practice (and actually plugins do that as well, so not sure why the separation at all), the whole editor thing is "false advertisement", as you can not actually edit a JS or CSS file and see results as either the theme has a build process that needs to run (JS) or you better do the change at SASS/LESS files to have them properly apply everywhere.

I agree with otto42, the impact of the caching should first be measured against a proper setup with a complex theme on a typical hosting server, not a local dev machine, as it has too many processes running to be able to deduct anything from testing on it. Then you can compare performance of cached version to non cached, and different caching intervals. My gut feeling, is that since we are talking here about human scale response times, 5 minutes will be too short and it will feel like there is no caching at all.

IMO for 4.9.1 caching should just be scrapped. This is the path of least resistance and least risk. Then if some complex themes create an actual problems to users you will be able to have a judgment call on whether it is a theme problem that is not worthy of being handled in core.

In any case, for people that have the editor disabled, caching should never be triggered, or better, have the caching limited to the context of the theme/plugin editor.

#85 @abramarket
3 weeks ago

Ug another vote to disable this - why on earth would caching this list be useful? I am developing a theme that will need to add several templates - this literally makes it a nightmare to do one of the most basic things in WordPress :(

This ticket was mentioned in Slack in #core by rabmalin. View the logs.


3 weeks ago

#87 @BackuPs
2 weeks ago

I have been banging my head on this for hours. I deleted a file in my template as it was no longer needed because of changes in woocommerce and i got a warning message that the file was missing

file_get_contents(/var/docs/website.tst/public/wp-content/themes/mytheme/single-product/product-image.php): failed to open stream:

Thanks for this ! The fix on github works fine. https://gist.github.com/westonruter/6c2ca0e5a4da233bf4bd88a1871dd950

Is this gonna be fixed in 4.9.1 or do we need to keep on adding that function?

#88 @mofeenster
2 weeks ago

Oh wow this really made my day more fun, I managed to find this thread after comparing 2 different installs of WP and playing "spot the difference", then working out that google will possibly assist in finding the answer.

You know when you have a limited amount of time to get a job done and then you can't because you end up de-bugging something like this...

Last edited 2 weeks ago by mofeenster (previous) (diff)

#89 @dd32
2 weeks ago

  • Owner set to dd32
  • Resolution set to fixed
  • Status changed from reopened to closed

In 42242:

Theme/Plugin Editor: Remove the caching added in [41806] as it causes more problems than it fixes.

While caching here seemed like a good idea in theory, in practice the cache would be often stale causing development issues.
We exclude common folders (such as node_modules) from the scanning to avoid directories which are not useful to the end-user, so as long as those exclusion lists are held up this shouldn't cause too much of a degredation in the future.
We may consider adding caching here again in the future if it's determined that it is really needed.

Props precies, ibenic, mariovalney, schlessera, and all the others who commented on the ticket(s).
This partually reverts [41806].
See #6531.
Fixes #42573.

#90 @dd32
2 weeks ago

In 42243:

Theme/Plugin Editor: Remove the caching added in [41806] as it causes more problems than it fixes.

While caching here seemed like a good idea in theory, in practice the cache would be often stale causing development issues.
We exclude common folders (such as node_modules) from the scanning to avoid directories which are not useful to the end-user, so as long as those exclusion lists are held up this shouldn't cause too much of a degredation in the future.
We may consider adding caching here again in the future if it's determined that it is really needed.

Props precies, ibenic, mariovalney, schlessera, and all the others who commented on the ticket(s).
This partually reverts [41806].
Merges [42242] to the 4.9 branch.
See #6531.
Fixes #42573 for 4.9.

#91 @dd32
2 weeks ago

#42706 was marked as a duplicate.

#92 follow-up: @BackuPs
2 weeks ago

Is this gonna be fixed in the beta wp 5.0 and above aswell? I see the same issue in those builds.

#93 @swissspidy
2 weeks ago

#42709 was marked as a duplicate.

#94 in reply to: ↑ 92 @dd32
2 weeks ago

  • Keywords needs-patch removed

Replying to BackuPs:

Is this gonna be fixed in the beta wp 5.0 and above aswell? I see the same issue in those builds.

Yes, it was committed against trunk (5.0 alpha) however the nightly builds only refresh every 24hours and I believe I committed a few hours after the previous build.

#95 @SergeyBiryukov
2 weeks ago

#42728 was marked as a duplicate.

#96 in reply to: ↑ 3 @kaleelahamed
2 weeks ago

Replying to ocean90:

Not sure if the caching was a good idea there. But bumping the version of the theme allows you to refresh the cache.

Thank you so much. This saved me a lot of time :)

#97 @maricate
11 days ago

  • Focuses template added
  • Keywords reporter-feedback ui-feedback added
  • Severity changed from normal to critical

None of these fixes worked here.
I'm updated to the 4.9.1 and still not working/showing up.
Any suggestions?
I really need to create new page templates and use it but it continues not working.

#98 follow-up: @mariovalney
11 days ago

  • Focuses template removed
  • Keywords reporter-feedback ui-feedback removed
  • Severity changed from critical to normal

Hi @maricate, how are you?

The 4.9.1 version fixed it (I confirmed it in my fresh install righ now).

Can you open a topic in support forum please?

#99 in reply to: ↑ 98 @maricate
11 days ago

Replying to mariovalney:

Hi @maricate, how are you?

The 4.9.1 version fixed it (I confirmed it in my fresh install righ now).

Can you open a topic in support forum please?

Very weird...
I disabled the CDN option and all the other cache options.
And the problem persists. Maybe a problem with the host?
I will contact the host.

Note: See TracTickets for help on using tickets.