Improve Javascript Word Counting for TinyMCE

[westi]

This is a follow on from #8759 to further improve the code.

While we improved it in 3.4 we can do a better job to more fully support the customisation for all different locales.

For know issues see the comments on the previous ticket.

[chriscct7]

Now that we have utf8mb4 done, has anyone tested to see if the word counter works for languages like Chinese? There is/was a plugin for this: ​https://wordpress.org/plugins/chinese-word-count/

[chriscct7]

Sample 你吃饭了吗 returns a count of 0 words.

[chriscct7]

Actually this plugin seems to do a good job at fixing this: ​https://wordpress.org/plugins/word-count-with-double-byte-character/

All that plugin does is deregister the version of word counter in core and enqueue a slighly modified version. We could patch ours with their changes

[paulwilde]

Related #30966.

[chriscct7]

If #30966 doesn't ship with 4.3, this should be so we can at least provide accurate word counts for multibyte character words in 4.3 now that we support them fully.

[chriscct7]

Marking for 4.3 inclusion, if #30966 makes the deadline for 4.3 we should use that, else use this and we'll get #30966 in for 4.4

[chriscct7]

Missed some formatting

[chriscct7]

Turns out #30966 fixes a slightly different issue. Related but not duplicates :)

[obenland]

@azaozz, could you review and commit/punt when you get a chance?

[iseulde]

Let's get #30966 in first. :)

[obenland]

Replying to iseulde:

Let's get #30966 in first. :)

Is that going to happen before beta?

[iseulde]

Let's try to get it in today.

[iseulde]

@chriscct7 We'll also need some basic unit tests for this.

[chriscct7]

Replying to iseulde:

@chriscct7 We'll also need some basic unit tests for this.

If someone can write them, I can look over them if needed. I don't think I'll have the time to come up with a robust set of tests before my (college) classes for the day start in 10 minutes.

[azaozz]

Looking at 20738.2.patch, it has three loops, one of them nested and is testing each individual character, twice. This seems far too slow in longer posts. It would count any non-ASCII char, including emoji, etc.

Also, seems the character count works differently for Japanese and Chinese, one counts spaces, the other doesn't according to the comments and patches on the original ticket: https://core.trac.wordpress.org/ticket/8759#comment:15.

In that terms maybe it will be better to concentrate on #30966 and close this one as "close enough" duplicate.

[obenland]

Replying to azaozz:

In that terms maybe it will be better to concentrate on #30966 and close this one as "close enough" duplicate.

Let's do that.