Option to load comments asynchronously, for speed

[markjaquith]

This comes as a suggestion from Google, who wants to help publishing platforms deliver content faster.

The idea is to have an option to not deliver comments in the page's initial HTML request, but to load them with an asynchronous JavaScript call. This would cut down on page generation time, reduce page size, and make page content load and show up faster.

This also allows caching plugins to avoid flushing the cache for that page when a new comment is approved. They'd only have to monitor for post content changes. This means more cache hits, which translates into a faster average page load.

This solution would need to handle comment permalink requests, and scroll to that comment once the comments were loaded. We'd also have to delay comment-related JS (like threading support) until the comments were loaded.

Is this core? Can we do it without requiring explicit theme support?

[filosofo]

Replying to markjaquith:

The idea is to have an option to not deliver comments in the page's initial HTML request, but to load them with an asynchronous JavaScript call. This would cut down on page generation time, reduce page size, and make page content load and show up faster.

Possibly. It also means:

• Two requests where there were one. (See #14374 ?)
• Problems with whatever browsers aren't supporting that particular JS
• Extra bandwidth to request the JS that wasn't needed before.
• Comments that aren't indexed in search results.

This also allows caching plugins to avoid flushing the cache for that page when a new comment is approved. They'd only have to monitor for post content changes. This means more cache hits, which translates into a faster average page load.

This could be addressed directly by the caching plugins. And you still have to flush the comments cache. And because the comments are now separated from the rest of the page, there are more server requests and more cache-processing overhead.

This solution would need to handle comment permalink requests, and scroll to that comment once the comments were loaded. We'd also have to delay comment-related JS (like threading support) until the comments were loaded.

All of which suggests lots of extra JS, or JS libraries, etc., that are best left to the discretion of the theme author.

Is this core? Can we do it without requiring explicit theme support?

It sounds like something we should make easier for people to implement with plugins and themes, if they can't already:

• Perhaps a front-end JSON response API for better JS queries of stuff like comments.
• Improving comments query performance
• Encouraging theme authors to standardize common page element IDs and / or theme action hooks, so plugins can more easily let JS replace elements.
• Setting comment paging to a lower number of comments per page by default.

[filosofo]

s/were one/was one/

[scribu]

Comments that aren't indexed in search results.

That's a deal breaker right there.

I think everyone prefers to have indexed comments rather than to have posts that load a little faster.

Suggest closing as wontfix.

[ericmann]

Suggest closing as wontfix.

I think one statement about a possible consequence of this system doesn't automatically mean we should close this as wontfix. I, for one, would like to see this expanded upon. A front-end JSON API would make this easy to implement for theme and plug-in authors without requiring it on every site.

We should be thinking of ways to improve WordPress that don't break existing sites and too many people have systems that might not be compatible with a catch-all refactoring of the comment system anyway.

Yes, some people prefer indexed comments over post load time. But some people prefer optimized post load time over indexed comments. Why not build the system so it can handle both paradigms?

[wjm]

Replying to ericmann:

Yes, some people prefer indexed comments over post load time. But some people prefer optimized post load time over indexed comments. Why not build the system so it can handle both paradigms?

I totally second you on this.
This feature could be enabled under the Settings>Discussion

Also, there could be a hybrid version that loads in plain html the first page of comments and from the second on they are retrieved via asynchronus javascript, but the second page is still links for search engines and people with no-javascript support (That would be the best of both worlds --no additional requests, and everything indexed)

[hanifb]

Replying to scribu:

I think everyone prefers to have indexed comments rather than to have posts that load a little faster.

Why not disable the async when is_robot() == true ?

[nacin]

is_robots() is for dynamically serving a robots.txt file, not for HTTP requests by robots.

[hanifb]

Replying to nacin:

is_robots() is for dynamically serving a robots.txt file, not for HTTP requests by robots.

I see, but maybe an is_crawler() function that recognizes the biggest crawlers could do it and some options for custom UA´s that isnt included.

[hotforwords]

This is very important and needs to be addressed as highly trafficked websites like Mashable use Disqus because Disqus is able to show the first 10 comments with a link to Show All Comments PLUS Disqus is able to index the comments properly as they state here: ​http://wiki.disqus.net/FAQ#Willsearchenginesstillindexmycomments.3F

As they state:

---

Disqus provides plugins for self-hosted blogs, running software such as WordPress or MovableType, that uses our API to interface with Disqus. Search engines will index this content perfectly.

---

So they offer some kind of solution for Self Hosted Blogs that allows for the proper indexing of the comments, they offer another solution for non-self-hosted blogs which puts the comments on the Discus site, but we are not interested in that solution.

Disqus charges $500/mo to integrate comments with your existing comments and existing users, so I cannot justify using them currently, plus is seems a shame to move all commenting outside of Wordpress!

If someone could look at how Disqus is addressing the indexing issue that might be the solution we need.

[filosofo]

From your link:

Disqus uses JavaScript and helps search engines index the comments by displaying the discussion threads on your Disqus community page.

So the comments are being indexed, but it's on Disqus' site. There's no special guidance that can be applied to XHR-retrieved comments on one's own site.

---

As someone who browses with JS (and cookies) turned off by default for security reasons, I think this fad of making comments JS-only creates a bad user experience. Sites that use Disqus and the like are saying at least one of the following to someone like me:

• We don't really care if you can read or contribute to our content.
• We want to force you to expose your client to a third-party site that runs a client-side script and requires a cookie.

To disable JS, I use the NoScript Firefox add-on, which according to the Mozilla site has been downloaded over 72 million times, so I don't think I'm alone. Also, with the expanding number of web clients that have limited JS abilities, including many screen readers and phones, the number of people who can't participate in those comments is growing.

I love JavaScript; I'm a JavaScript developer. However, as with everything it has its place, and it should be used to enhance not diminish the experience of users.

So let's provide the ammunition people need to do this if they choose, perhaps with an XML or JSON comments query API, but let's not pull the trigger by default.

[ryan]

​http://code.google.com/web/ajaxcrawling/docs/getting-started.html

[scribu]

Nice. I'm all for it, then.

If we add the <meta> tag to single.php (only when there are comments), we won't have to change any other front-end markup.

Of course, that's just the first step.

[filosofo]

Ryan and I chatted about this in IRC the other day, and I think we both came to the conclusion that this is something that is best left to plugins for now. That's because it's something that is not necessarily in the interests of the vast majority of WP users.

However, we should make it easier for plugins to implement such a system without forcing the user to make theme edits.

I would like to do the following:

• Add filter(s) to wp_list_comments to make this doable by a plugin
• Write a proof-of-concept plugin that uses that filter to provide this functionality for those who want it.

[jane]

Closing this, given 16 months with no activity and an opinion from Ryan that it's plugin territory. If re-opening, please include a patch.