Category view incorrectly counting custom posts

[fireproofsocks]

If you define a custom post-type something like this:

Array
(
    [taxonomies] => Array
        (
            [0] => category
            [1] => post_tag
        )
    
    [label] => Book
    [rewrite_slug] => 
    [query_var] => 
    [public] => 
    [show_ui] => 1
    [show_in_nav_menus] => 1
    [publicly_queryable] => 
    [exclude_from_search] => 1
)

Even if the post-type is NOT public or NOT publicly_queriable (see above), the counts of the posts in each category still shows, totally ignoring the settings of the post-type.

Expected Output

I would expect the count to be related to the results displayed. If there are no visible results for that category (i.e. the user can't see them due to public settings), I would expect the count to go to zero.

Actual Output

The count of posts in the given category seems to have nothing to do with the visible results.

This is related to this bug: #18950

Related: #14084

The more I think about it, I think that this should be tied to the exclude_from_search parameter, since category views are in essence a type of search, whereas publicly_queryable would control whether any aspect of the post is visible anywhere on the front-end.

Related: #14076, #20708