Shortcodes are not working in page templates in the new "Beta" editor

[nextend_ramona]

The shortcode block does not transform the shortcodes in the page template of the new "beta" editor (located at Appearance > Editor (beta)) of the full site editing when the shortcode is added to the "content" area, not into the "Header" or "Footer". (It works fine from these two.)

For testing I created a new plugin which only contains the following code:

<?php
add_shortcode('note', 'tt_test_note');
function tt_test_note($atts, $content = null) {
    ob_start();
    ?>
    <div class="note" style="margin: 30px 0; padding: 27px 40px; background: #fcf3dc; border-radius: 5px;">
        <p><?php echo $content; ?></p>
    </div>
    <?php
    return ob_get_clean();
}

Result

New “beta” editor: ​https://imgur.com/o4bPPa6
Page content: ​https://imgur.com/9j3gusE
Frontend: ​https://imgur.com/576sFam

Environment

1. WordPress 5.9-RC1-52464
2. Active theme: Twenty Twenty-Two Version: 1.0
3. Active plugins: only the shortcode plugin that has the code above and nothing else
4. PHP: 7.2.34-28+ubuntu18.04.1+deb.sury.org+1

What’s the issue?

When using the shortcode [note]test[/note] in Gutenberg’s own shortcode module works in the actual page, but not in the new "beta" full site editor's "content" area. (I'm sorry, I'm not sure what that is called exactly.) The shortcode works fine from the "Header" and "Footer" area, just not between them.

Steps to reproduce

1. Create a new plugin and add the code mentioned above
2. Go to Appearance > Editor
3. Click on the WordPress icon at the left site
4. Select Templates > Page
5. Add a new shortcode block above Post title (​https://imgur.com/5lFPgtE)
6. Write this code there: [note]test[/note]
7. Save the template
8. Create a new page
9. Add a new shortcode block and write the same shortcode there (​https://imgur.com/5lFPgtE)
10. Save the page and check it

You’ll see the [note]test[/note] code is displayed above the post title, but the yellow block shows correctly up below.
​https://imgur.com/orEyhvI

I've also opened a post in the alpha-beta forums:
​https://wordpress.org/support/topic/shortcodes-are-not-working-in-page-templates/

[nextend_ramona]

The shortcode is rendered in header and page content ("header" and "in page content" yellow box) but it doesn't render in the "content" area of the full side editor

[sebastienserre]

Hello,

I confirm I've reproduced the Ramona behaviour by adding the [products] WooCommerce Shortcode

[audrasjb]

Hi,

We're now tracking this bug in the following Gutenberg issue: ​https://github.com/WordPress/gutenberg/issues/37890

I'm closing the Trac ticket as reported-upstream.

[garrett-eclipse]

@audrasjb with this being fixed upstream in ​Gutenberg#37545 is there any plans to pull it into core for the 5.9 release?

Testing RC2 the issue is present on plugins using shortcodes in their block rendering as well as any shortcodes placed into the WP Shortcode block. Installing the latest Gutenberg resolves the issue so the fix upstream seems to work nicely and is just missing from core.

[hellofromTonya]

Following up:

​The fix from Gutenberg was backported to Core's trunk with [52595] and 5.9-branch with [52596].

[manfcarlo]

This should be re-opened because the fix was found to cause unwanted effects and is expected to be removed in 5.9.1, ​see discussion. A new fix could potentially be included into 6.0 and targeted directly at core.

[manfcarlo]

I think it might be helpful to collate the history of the shortcode question, which I took the time to research:

• ​February 2020: template parts were changed from using the_content filter to explicitly applying do_shortcode
• ​February 2020: a pull request was opened to remove do_shortcode from template parts, but it was never merged
• ​July 2020: a discussion was opened about whether to support shortcodes in block templates (the heading mentions parts, but I think the author was referring to templates)
• ​December 2021: shortcodes were added to block templates in the Gutenberg plugin
• ​January 2022: shortcodes were "mistakenly" (?) added to block templates in core during the RC (!) stage of 5.9

I also noticed that the ​recent roadmap for 6.0 by @matveb mentioned the exploration of "inline tokens." Some of the discussions linked above (such as ​this comment) did cite the usefulness of shortcodes as "inline tokens" as a reason to support them within blocks, although others such as @azaozz disagreed.

Personally, I am neutral to whether shortcodes are supported or not, and will gladly defer to those with a greater involvement in the project. However, the shortcode question has suffered from scattered discussions and a lack of leadership, meaning that code changes have been made without the full context of arguments that were previously and validly put forward, and that's what I'm not so neutral about.

[zeshanb]

Replying to manfcarlo:

This should be re-opened because the fix was found to cause unwanted effects and is expected to be removed in 5.9.1, ​see discussion. A new fix could potentially be included into 6.0 and targeted directly at core.

The issue is still there for 6.1. The content of the shortcode displays right below the body tag instead of the area where the shortcode tag was used in gutenburg editor.

This bug is for shortcodes using dynamic content where either a plugin does some calculations and returns the results on the template of a wordpress site. Static content shortcodes work fine.

[zeshanb]

Shortcode with dynamic content works in version 5.7

[zeshanb]

In version 6.1 content is displayed right below the body tag

[zeshanb]

Shortcode dynamic content processed by php and inserted right below body tag