Fixing extreme admin page renders due to broken systemd-resolved making WordPress appear slow

[dfavor]

Feature Request

I've marked this as a feature request, because this is not a WordPress issue + any WordPress site running on a systemd based OS can be effected.

This makes WordPress appear to run slow (which is a total myth), so likely this is good information for somewhere in the Codex.

My request is for Codex to reflect unexplained slowness in WordPress admin pages may be due to systemd-resolved + if this code is running, replace it with a working DNS resolver + retest.

Related tickets...

https://core.trac.wordpress.org/ticket/40266 - Previous WordPress ticket.

​https://github.com/rmccue/Requests/issues/272 - Upstream GitHub ticket.

This problem occurs when two conditions exist, which are very common.

1) OS Distro uses the systemd abomination (my opinion).

2) Users logged in as admin visit any page.

This seems to occur as follows...

1) systemd-resolved caching is completely broken + every DNS lookup generates a recursive lookup, rather than pulling DNS records from previously cached lookups.

This is extremely easy to verify.

Just issue host google.com repeatedly + you'll see multi-second lags, before IP is returned. These lags occur every time the host command is issued for any DNS lookup.

So caching never seems to work... ever...

And executing this - while : ; do host google.com ; done - will sometimes recreate the problem.

Since this only seems to occur sometimes, seems like some DNS lookup returns data which corrupts systemd-resolved memory, which allows the process to continue running + never work again.

2) systemd-resolved hangs for 5 seconds + times out + returns an error, which might be the reason the Requests library has hostname/ip missing in some exceptions. This was the original ticket issue.

This is the behavior, once #1 seeming memory corruption occurs.

3) systemd-resolved, once hung for a specific hostname, seems hung forever. In other words, once a time out occurs for any hostname lookup, future lookups fail with a time out also.

You can see this behavior using the Query Monitor plugin, tracking HTTP requests. Once a request goes red/fail, it never recovers.

The Simple Fix

Nuke systemd-resolved + replace with a working DNS system.

I host client sites in LXC containers, so container startup includes...

1) If missing, install dnsmasq-base (Debian/Ubuntu).

2) If systemd-resolved exists, nuke it - disable it in systemd + remove all related packages.

For Ubuntu, the commands I use are...

systemctl stop systemd-resolved
systemctl disable systemd-resolved
apt-get -yqq purge libnss-resolve

The disable (counter intuitive) is required, because systemd Ubuntu packaging is broken where systemd tries to start systemd-resolved, even after it's purged from system, so doing the disable speeds up boots (where systemd hangs trying to start systemd-resolved).

3) Start dnsmasq using a highly optimized caching config, which avoids all disk i/o.

Admin pages taking 10-30 seconds now render first time in <10 seconds + subsequent times <1sec, as expected.

[swissspidy]

Hey there,

Please note that there's no need to create a ticket for this. The ​WordPress Codex, like Wikipedia, is a public wiki that can be edited by anyone. If you happen to have some information you think is valuable for others, just log in with your WordPress.org username and password and edit/add the page in question. It can be useful to post this stuff in a blog post first and then link to that blog post.