| 1 | (11:48:55) photomatt: okay, going to flood to catch ryan up :) |
|---|
| 2 | (11:49:03) photomatt: did you guys see the mysql guy in #wordpress last night? |
|---|
| 3 | (11:49:03) photomatt: [11:32AM] photomatt: he suggested some really interesting improvements |
|---|
| 4 | (11:49:03) photomatt: [11:32AM] photomatt: in my tests it took the main post query from 0.12 to 0.05 |
|---|
| 5 | (11:49:07) photomatt: [11:32AM] photomatt: on average |
|---|
| 6 | (11:49:09) photomatt: [11:32AM] photomatt: http://p.defau.lt/?tiJutSuV86iBJfUPGq_h0g |
|---|
| 7 | (11:49:12) photomatt: [11:33AM] photomatt: http://p.defau.lt/?_hp_YagOMRNnFgeXdhpbpw |
|---|
| 8 | (11:49:14) photomatt: [11:33AM] photomatt: (the second is 4.1 only though) |
|---|
| 9 | (11:49:33) photomatt: rboren: if we could eliminate the DISTINCT and GROUP BY from post queries that don't need it |
|---|
| 10 | (11:49:43) photomatt: and add this suggested index |
|---|
| 11 | (11:49:53) photomatt: (he also suggested making post_type a shorter varchar, maybe 10-20) |
|---|
| 12 | (11:50:10) photomatt: well, try out the queries, it helps a lot |
|---|
| 13 | (11:52:51) domas [n=midom@wikipedia/Midom] entered the room. |
|---|
| 14 | (11:53:00) domas: hello! can I lurk here? :) |
|---|
| 15 | (11:53:17) photomatt: totally |
|---|
| 16 | (11:53:43) photomatt: domas is the fellow who suggested the changes I pasted above |
|---|
| 17 | (11:53:59) domas: did you collect ones that I pasted afterwards in channel? |
|---|
| 18 | (11:54:16) photomatt: got two from p.defau.lt |
|---|
| 19 | (11:54:30) rboren: I'm making the following changes: |
|---|
| 20 | (11:54:31) rboren: http://p.defau.lt/?tiJutSuV86iBJfUPGq_h0g |
|---|
| 21 | (11:54:47) rboren: And shortening the varchars for post_type and post_status |
|---|
| 22 | (11:55:06) domas: category query is important too |
|---|
| 23 | (11:55:14) photomatt: domas: out of curiosity, why is the index in that order? |
|---|
| 24 | (11:55:24) photomatt: (post_status,post_type,post_date) |
|---|
| 25 | (11:55:31) domas: photomatt: it could be type,status,date |
|---|
| 26 | (11:55:39) domas: status,type - for WHERE, date for ORDER BY |
|---|
| 27 | (11:56:31) domas: maybe you need to add ID there |
|---|
| 28 | (11:56:44) domas: at the end |
|---|
| 29 | (11:56:56) domas: (so you could GROUP BY date DESC, ID DESC |
|---|
| 30 | (11:57:12) domas: of course, if post_date was unique, you could just GROUP BY date |
|---|
| 31 | (11:57:21) domas: but you'd have to code according to that |
|---|
| 32 | (11:57:23) photomatt: mysql can only use one key per query? |
|---|
| 33 | (11:57:38) photomatt: post_date might not be unique |
|---|
| 34 | (11:57:48) domas: then append ID at the end. |
|---|
| 35 | (11:58:12) domas: photomatt: not really. mysql in 5.0 can do index merge for multiple keys, but it won't compile a composite key out of separate indexes |
|---|
| 36 | (11:58:19) domas: (and that would be painful operation anyway) |
|---|
| 37 | (11:58:30) photomatt: k |
|---|
| 38 | (11:58:56) domas: composite keys are used everywhere |
|---|
| 39 | (12:00:40) domas: these two queries: |
|---|
| 40 | (12:00:40) domas: 66 Query SELECT * FROM wp_categories WHERE cat_ID > 0 HAVING category_count > 0 ORDER BY cat_name asc |
|---|
| 41 | (12:00:40) domas: 66 Query SELECT * FROM wp_categories WHERE cat_ID > 0 HAVING link_count > 0 ORDER BY cat_name ASC |
|---|
| 42 | (12:00:55) domas: could possibly be rewritten into link_count>0 or category_count>0 |
|---|
| 43 | (12:01:21) domas: this one: |
|---|
| 44 | (12:01:22) domas: 66 Query SELECT COUNT(DISTINCT ID) FROM wp_posts WHERE 1=1 AND (post_type = 'post' AND (post_status = 'publish')) |
|---|
| 45 | (12:01:30) domas: can be eliminated too, of course :) |
|---|
| 46 | (12:02:09) domas: I didn't look yet at the object cache code |
|---|
| 47 | (12:02:31) photomatt: for post queries we also sometimes have "(post_author = 1 AND post_status = 'private'" |
|---|
| 48 | (12:02:38) photomatt: to get private posts for the current logged in user |
|---|
| 49 | (12:03:02) photomatt: but that might be a rare enough case that it doesn't need to be optimized for |
|---|
| 50 | (12:03:15) domas: *nod* |
|---|
| 51 | (12:03:18) domas: I'm looking at anon hits |
|---|
| 52 | (12:04:21) photomatt: but it might be easier just to create a separate section in the admin for private posts and leave them out of public listings all-together |
|---|
| 53 | (12:04:27) photomatt: it's a weird feature anyway |
|---|
| 54 | (12:04:57) domas: 'cron' and 'rewrite_rules' should go into autoload options :) |
|---|
| 55 | (12:05:22) domas: as for categories, this seems fun: |
|---|
| 56 | (12:05:23) domas: 68 Query SELECT * FROM wp_categories |
|---|
| 57 | (12:05:23) domas: 68 Query SELECT cat_ID FROM wp_categories |
|---|
| 58 | (12:06:07) domas: btw, for categories, this query is executed twice too: |
|---|
| 59 | (12:06:08) domas: 68 Query SELECT COUNT(DISTINCT ID) FROM wp_posts LEFT JOIN wp_post2cat ON (wp_posts.ID = wp_post2cat.post_id) WHERE 1=1 AND (category_id = 4) AND (post_type = 'post' AND (post_status = 'publish')) |
|---|
| 60 | (12:06:53) domas: it's fast enough and using the (`post_status`,`post_type`,`post_date`,`ID`) index, but still :) |
|---|
| 61 | (12:08:37) MarkJaquith: photomatt: New wp.org/support/ logins don't seem to be syncing with Trac... you aware? |
|---|
| 62 | (12:08:47) photomatt: yes |
|---|
| 63 | (12:09:01) photomatt: haven't been since the server move, I'm going to look at it ina bit |
|---|
| 64 | (12:09:02) MarkJaquith: Cool. I figured it had something to do with the server move or something |
|---|
| 65 | (12:09:21) domas: for rss feeds: |
|---|
| 66 | (12:09:21) domas: 70 Query SELECT post_modified_gmt FROM wp_posts WHERE post_status = 'publish' ORDER BY post_modified_gmt DESC LIMIT 1 |
|---|
| 67 | (12:09:21) domas: 70 Query SELECT post_date_gmt FROM wp_posts WHERE post_status = 'publish' ORDER BY post_date_gmt DESC LIMIT 1 |
|---|
| 68 | (12:09:29) domas: both fields have no indexes. |
|---|
| 69 | (12:10:00) domas: so either these values have to be maintained somewhere else |
|---|
| 70 | (12:10:23) domas: on the second query you may just use post_date |
|---|
| 71 | (12:10:31) domas: instead of _gmt, as for ordering value should be the same |
|---|
| 72 | (12:11:03) MarkJaquith: there is an edge case where they aren't... not sure it matters |
|---|
| 73 | (12:11:22) domas: well, it's rss feed |
|---|
| 74 | (12:11:43) domas: on every hit to rss feed you order the column again and again. |
|---|
| 75 | (12:11:49) domas: it's optimized by mysql though |
|---|
| 76 | (12:11:57) domas: that is, only one row is used for a sort |
|---|
| 77 | (12:13:38) domas: woo, article view. |
|---|
| 78 | (12:14:18) domas: the index I created was used again for several queries |
|---|
| 79 | (12:15:28) MarkJaquith: what'd you index? |
|---|
| 80 | (12:15:29) domas: comments: |
|---|
| 81 | (12:15:51) domas: MarkJaquith: on posts: type,status,date,id |
|---|
| 82 | (12:16:10) domas: now comments ask for this index: postid,approved,date |
|---|
| 83 | (12:18:13) mdawaffe is now known as mdawaffe|lunch |
|---|
| 84 | (12:19:37) domas: with this index comments would not be sorted :) |
|---|
| 85 | (12:19:46) domas: filesorted, just an index range scan |
|---|
| 86 | (12:20:17) domas: it picks comments_aproved index |
|---|
| 87 | (12:20:28) domas: I provided a hint to ignore it |
|---|
| 88 | (12:21:19) domas: http://p.defau.lt/?DRnl1EN_6SB7UhBfgzLdrw |
|---|
| 89 | (12:21:56) domas: the /*!4000.. specifies that the ignore index clause should be used on 4.0 or newer |
|---|
| 90 | (12:22:04) domas: I don't remember if 3.23 actually supported IGNORE INDEX :) |
|---|
| 91 | (12:22:20) photomatt: wow, you weren't kidding about the mysqlisms ;) |
|---|
| 92 | (12:22:46) domas: well, /* is official SQL comment |
|---|
| 93 | (12:22:56) domas: so, on any other platform it would not be executed |
|---|
| 94 | (12:23:01) domas: oracle uses /*+blahblah |
|---|
| 95 | (12:23:48) photomatt: got it |
|---|
| 96 | (12:24:11) domas: anyway, on proper deployments you don't need this hint |
|---|
| 97 | (12:24:22) domas: it's just my test dataset of single article provides skewed view to optimizer |
|---|
| 98 | (12:24:37) domas: not much of optimizations can be done with single row, can it? :) |
|---|
| 99 | (12:25:10) MarkJaquith: heh... at least not any that'll add up to what you can do while optimizing one of the posts/category queries |
|---|
| 100 | (12:25:41) domas: hehe |
|---|
| 101 | (12:25:51) rboren: FYI: http://trac.wordpress.org/ticket/2604 |
|---|
| 102 | (12:26:01) domas: well, I did that analysis post on my real dataset |
|---|
| 103 | (12:26:08) domas: which at least has >20 posts and >20 comments, ha ha ha |
|---|
| 104 | (12:27:12) BigJibby [n=matt@mtl58-2-131-132.dialup.sprint-canada.net] entered the room. |
|---|
| 105 | (12:27:17) MarkJaquith: domas: running 1.5.2, right? |
|---|
| 106 | (12:27:49) domas: MarkJaquith: yessir |
|---|
| 107 | (12:28:05) MarkJaquith: (just reading your blog post now) |
|---|
| 108 | (12:28:39) domas: hehe, sorry for sarcasm, it wasn't directed towards any of you, just for general problem of not dealing with performance |
|---|
| 109 | (12:29:07) domas: I'm doing performance engineering for opensource project, so every time I have to LART people about various issues |
|---|
| 110 | (12:29:08) MarkJaquith: DISTINCT needs to go away for single-table post queries for sure |
|---|
| 111 | (12:29:22) domas: MarkJaquith: for multiple-table you'd be better using proper GROUP BY |
|---|
| 112 | (12:30:13) domas: http://bugs.mysql.com/bug.php?id=18182 <--- this one if fixed would improve your category query performance even more :) |
|---|
| 113 | (12:30:37) MarkJaquith: "I’d really like if it just used "publish" status for front page and would not care about anything else. " Good news... that's what's happening for 2.1 |
|---|
| 114 | (12:31:14) domas: hehe, I just looked at our bugs system |
|---|
| 115 | (12:31:23) domas: I've opened 6 bugs with words 'index' in description :) |
|---|
| 116 | (12:31:26) domas: *word |
|---|
| 117 | (12:31:58) MarkJaquith: The hidden benefit there is that you can now cache the queries, because you don't have that dynamic timestamp variable in each query |
|---|
| 118 | (12:32:13) ***domas enjoys the fading yellow |
|---|
| 119 | (12:33:24) domas: caching is nice for small sites enjoying lots of reads and no writes |
|---|
| 120 | (12:33:37) domas: none of sites I work with or clients of mine have such situations :) |
|---|
| 121 | (12:33:45) photomatt: that's the bulk of WP installations |
|---|
| 122 | (12:33:58) photomatt: if you think we have perf problems now, you should have seen 0.72 |
|---|
| 123 | (12:34:00) photomatt: :) |
|---|
| 124 | (12:34:31) rboren: Not many people were around to witness 0.72 perf problems. :-) |
|---|
| 125 | (12:34:43) photomatt: I remember doing a profile and finding 60% of our time being spent on convert_char or something like that |
|---|
| 126 | (12:34:44) domas: haha, I'm fascinated enough to remember mediawiki 1.3/1.4 ;-) |
|---|
| 127 | (12:35:03) photomatt: which was individually looping over EVERY SINGLE CHARACTER in every post |
|---|
| 128 | (12:35:04) domas: photomatt: well, an example of badly optimized situation was |
|---|
| 129 | (12:35:23) domas: when for interwiki prefixes, for every link there was a hit to our memcached cluster |
|---|
| 130 | (12:35:31) ***MarkJaquith is so glad he only got involved with WordPress right before 1.2 release |
|---|
| 131 | (12:35:34) domas: and it takes ~2ms each |
|---|
| 132 | (12:36:08) domas: so it used to spend 50ms or more on fetching absolutely static data |
|---|
| 133 | (12:36:27) domas: and the interesting part, why it was not noticed |
|---|
| 134 | (12:36:40) domas: it always was hidden beneath other tasks that appeared 'long' in profiling trees |
|---|
| 135 | (12:36:52) domas: only proper kcachegrind analysis allowed to track such issues |
|---|
| 136 | (12:37:12) domas: moral of the story: do not assume the task is slow until you know what is slow in it. |
|---|
| 137 | (12:37:50) ***ringmaster re-reads the story of MediaWiki innards and impales himself on a fork. |
|---|
| 138 | (12:37:50) photomatt: what's kcachegrind? |
|---|
| 139 | (12:38:17) domas: ringmaster: which story? |
|---|
| 140 | (12:38:26) ringmaster: The one you just told. :) |
|---|
| 141 | (12:38:51) domas: photomatt: http://dammit.lt/2006/01/18/mediawiki-graphic-profile/ |
|---|
| 142 | (12:39:03) domas: photomatt: one (two actually) pictures are better than thousand words. |
|---|
| 143 | (12:39:05) MarkJaquith: domas: in 2.x, WP now stores comment_count in the posts table, as you suggested (still reading!) |
|---|
| 144 | (12:39:16) photomatt: that's purty |
|---|
| 145 | (12:39:36) domas: photomatt: I was working on cpu profiling instead of real time profiling |
|---|
| 146 | (12:39:42) domas: but ditched that for a while |
|---|
| 147 | (12:39:45) domas: may resume the effort some day |
|---|
| 148 | (12:45:49) domas: hehe, wanna see a pic of wordpress kcachegrind graph? :) |
|---|
| 149 | (12:46:01) photomatt: sounds perfect, I'll definitely try to make it |
|---|
| 150 | (12:46:40) ***MarkJaquith puts his money down on smiley regexes |
|---|
| 151 | (12:46:44) MarkJaquith: (kidding) |
|---|
| 152 | (12:46:55) domas: haha, someone asked for that |
|---|
| 153 | (12:46:59) domas: in mediawiki |
|---|
| 154 | (12:47:41) photomatt: sure |
|---|
| 155 | (12:48:15) domas: I hate such features :) |
|---|
| 156 | (12:48:18) domas: pain to maintain |
|---|
| 157 | (12:48:22) domas: breaks other syntax |
|---|
| 158 | (12:48:25) domas: performance hog |
|---|
| 159 | (12:49:07) MarkJaquith: 1) wait 2) wait 3) wait 4) wait 5) wait 6) wait 7) almost there 8) SUNGLASSES! |
|---|
| 160 | (12:50:09) photomatt: ha! |
|---|
| 161 | (12:50:15) photomatt: LOL |
|---|
| 162 | (12:51:38) domas: ;-D |
|---|
| 163 | (12:52:49) MarkJaquith: error_bot: google wordpress smilies |
|---|
| 164 | (12:52:52) error_bot: MarkJaquith: Search took 0.17 seconds: WordPress Smilies [ Tempus Fugit | TxFx.net ]: <http://txfx.net/2004/08/18/wordpress-smilies/>; WordPress Smilies: <http://www.tamba2.org.uk/wordpress/smilies/>; Using Smilies « WordPress Codex: <http://codex.wordpress.org/Using_Smilies> |
|---|
| 165 | (12:53:16) MarkJaquith: (that first result would be mine, domas) |
|---|
| 166 | (12:55:49) photomatt: domas: would love to see kcachegrind output for WP |
|---|
| 167 | (12:56:43) domas: photomatt: hehe |
|---|
| 168 | (12:56:47) domas: ---> Fetching graphviz |
|---|
| 169 | (12:56:48) domas: ---> Attempting to fetch graphviz-2.6.tar.gz from http://www.graphviz.org/pub/graphviz/ARCHIVE/ |
|---|
| 170 | (12:56:52) domas: forgot to install some deps before |
|---|
| 171 | (12:56:55) domas: on this machine |
|---|
| 172 | (12:57:18) domas: well, I don't have enough data |
|---|
| 173 | (12:57:23) domas: so the results would be... skewed again :) |
|---|
| 174 | (12:57:33) domas: oh, downloaded |
|---|
| 175 | (12:57:52) domas: this LCD compiles things really fast :) |
|---|
| 176 | (13:00:11) photomatt: oh that's right you still need the dump file |
|---|
| 177 | (13:00:23) photomatt: rboren: do you still have that giant MT dump I sent you? I can't find it |
|---|
| 178 | (13:00:34) rboren: Lemme look... |
|---|
| 179 | (13:03:54) rboren: Found it. |
|---|
| 180 | (13:04:02) rboren: I'll upload it somewhere... |
|---|
| 181 | (13:05:31) BigJibby left the room (quit: Read error: 104 (Connection reset by peer)). |
|---|
| 182 | (13:05:41) domas: compress it too, thanks! |
|---|
| 183 | (13:11:30) domas: damn, it got mad with wp profile |
|---|
| 184 | (13:12:12) domas: there's some code that is executed directly via require_once() instead of calling as functions |
|---|
| 185 | (13:12:40) photomatt: ? |
|---|
| 186 | (13:15:09) domas: well, the most expensive functions are require_once |
|---|
| 187 | (13:15:12) domas: and require |
|---|
| 188 | (13:15:35) domas: instead of loading file with functions and calling them, you execute it directly |
|---|
| 189 | (13:15:49) domas: so there's no difference between actually loading file and executing code |
|---|
| 190 | (13:15:50) domas: in the profile |
|---|
| 191 | (13:16:02) masquerade [n=masquera@c-68-49-145-242.hsd1.de.comcast.net] entered the room. |
|---|
| 192 | (13:16:05) photomatt: huh |
|---|
| 193 | (13:16:25) photomatt: well because of the amount of code, we've always suspected that the biggest overhead was simply parsing it all |
|---|
| 194 | (13:16:31) domas: not really |
|---|
| 195 | (13:16:33) photomatt: which is why an opcode cache helps so much |
|---|
| 196 | (13:16:53) domas: but yeah, opcode helps |
|---|
| 197 | (13:17:56) domas: it spent 5% of real time on mysql ;-D |
|---|
| 198 | (13:18:44) domas: eh, I can't provide you with a nice graph, just several small ones probably |
|---|
| 199 | (13:19:57) photomatt: too much junk? |
|---|
| 200 | (13:20:36) domas: ah, or doesn't show too much |
|---|
| 201 | (13:20:40) domas: it's much better if interactive |
|---|
| 202 | (13:20:49) domas: for analysis of details |
|---|
| 203 | (13:21:23) domas: and maybe xdebug acting funny on this box :( |
|---|
| 204 | (13:21:31) domas: ok, I'll do one export |
|---|
| 205 | (13:23:27) domas: http://dammit.lt/wordpress-kcg.png |
|---|
| 206 | (13:24:08) domas: most of code is hidden in require or require_once calls |
|---|
| 207 | (13:24:10) domas: :( |
|---|
| 208 | (13:24:57) photomatt: weird |
|---|
| 209 | (13:25:05) photomatt: I'm not sure what we're doing that would make it appear so weird |
|---|
| 210 | (13:25:32) masquerade: Anyone have Zend Studio's Debug Server setup? Its profiling might help out in this case |
|---|
| 211 | (13:25:50) photomatt: lots of time in load_template, it looks like |
|---|
| 212 | (13:26:09) MarkJaquith: lot of file_exists calls eh? |
|---|
| 213 | (13:26:44) domas: you should really move away execution from script loading |
|---|
| 214 | (13:27:03) photomatt: I still don't understand exactly what you mean by that |
|---|
| 215 | (13:27:09) domas: require();require(); dothis(); dothat(); |
|---|
| 216 | (13:27:21) photomatt: oh I see |
|---|
| 217 | (13:27:29) photomatt: so the action is happening inside the requires |
|---|
| 218 | (13:27:34) ringmaster: As opposed to require(<? dothis());require(<? dothat()); |
|---|
| 219 | (13:28:03) domas: well, not really |
|---|
| 220 | (13:28:11) photomatt: I think he means that a file includes another file which has the executing code in it |
|---|
| 221 | (13:28:11) domas: the problem is require(code;code;no;functions) |
|---|
| 222 | (13:28:28) domas: e.g. wp-settings |
|---|
| 223 | (13:28:28) photomatt: so we don't execute anything inside of index.php, for example |
|---|
| 224 | (13:28:43) domas: has lots of code but 3 tiny functions |
|---|
| 225 | (13:29:19) photomatt: nothing executes from wp-includes, I think |
|---|
| 226 | (13:29:23) photomatt: but other files do |
|---|
| 227 | (13:29:47) photomatt: I take that back, nothing in functions.php, template* executes any code, just loads it |
|---|
| 228 | (13:30:13) masquerade: I fail to see how any of that is or could be a performance trap, though |
|---|
| 229 | (13:30:42) domas: not much of performance trap |
|---|
| 230 | (13:30:45) MarkJaquith: masquerade: it's just a performance mask... with this tool. |
|---|
| 231 | (13:30:49) domas: it just obstructs clear view of what is happening where |
|---|
| 232 | (13:31:20) domas: anyway, branches can still be analyzed |
|---|
| 233 | (13:31:21) masquerade: try APD |
|---|
| 234 | (13:31:33) domas: masquerade: it won't give you graphic profile :) |
|---|
| 235 | (13:31:46) domas: you should _try_ kcachegrind :) |
|---|
| 236 | (13:31:53) masquerade: I'd rather have the text than that mess any day, for PHP |
|---|
| 237 | (13:32:49) domas: ah: http://flake.defau.lt/pics/mediawiki.png |
|---|
| 238 | (13:32:54) domas: this is how it usually outputs stuff |
|---|
| 239 | (13:33:16) domas: (that picture details the situation I talked about here ;-) |
|---|
| 240 | (13:33:49) domas: masquerade: the nice thing about that is 'where is this method called, what is called by it, how much time was spent in multiple places, etc' |
|---|
| 241 | (13:35:34) domas: e.g. this is all database activity profiled: http://dammit.lt/wordpress-kcg-sql.png |
|---|
| 242 | (13:35:52) domas: (on an empty db though, I'll try importing a dump soon) |
|---|
| 243 | (13:36:50) MarkJaquith: photomatt: the load_template just does require_once... so that's misleading. |
|---|
| 244 | (13:37:44) masquerade: I'm spoiled by Zend Studio's profiler |
|---|
| 245 | (13:37:46) photomatt: yep |
|---|
| 246 | (13:37:57) photomatt: masquerade: andy has been using that |
|---|
| 247 | (13:38:19) masquerade: I'd pick it any day over callgrind/kcachegrind style dumps for PHP source |
|---|
| 248 | (13:38:42) masquerade: (Although I do understand the value of callgrind, its invaluable when working with C) |
|---|
| 249 | (13:47:34) domas: anyway, I'm not selling anything here, hehehe |
|---|
| 250 | (13:49:27) domas: ok, with more data |
|---|
| 251 | (13:49:47) domas: http://p.defau.lt/?GyzOyrdWQ1eScjMefMXOoQ |
|---|
| 252 | (13:49:55) domas: this may be probably loaded linked to posts read |
|---|
| 253 | (13:49:57) domas: joined |
|---|
| 254 | (13:51:59) MarkJaquith: Wait, it did that all in one load? |
|---|
| 255 | (13:52:19) domas: yes |
|---|
| 256 | (13:52:27) domas: for anon |
|---|
| 257 | (13:53:00) MarkJaquith: You have the object cache disabled, or is wp-content not server writable? |
|---|
| 258 | (13:53:07) masquerade: MarkJaquith, he's on 1.5 |
|---|
| 259 | (13:53:20) masquerade: (or according to the blog post, not 2.0 at least) |
|---|
| 260 | (13:53:25) domas: masquerade: 2.1-alpha |
|---|
| 261 | (13:53:25) MarkJaquith: masquerade: wp_usermeta |
|---|
| 262 | (13:53:38) masquerade: ahh, I walked in halfway through all this, excuse me |
|---|
| 263 | (13:53:39) domas: MarkJaquith: I guess it's object cache disabled |
|---|
| 264 | (13:54:00) domas: MarkJaquith: it's "default install" |
|---|
| 265 | (13:54:02) domas: of 2.1 alpha |
|---|
| 266 | (13:54:07) domas: and wp-content is writable |
|---|
| 267 | (13:54:10) domas: ok, I'll enable it |
|---|
| 268 | (13:54:43) MarkJaquith: domas: ah yeah, it is disabled by default in 2.1 |
|---|
| 269 | (13:55:11) MarkJaquith: so you have posts by each of these authors on the loaded page, right? |
|---|
| 270 | (13:55:20) domas: yes |
|---|
| 271 | (13:55:41) ***domas notices cache on filesystem sucks if NFS is used ;-) |
|---|
| 272 | (13:56:08) MarkJaquith: So it should be grabbing the user IDs from the posts query and doing one query with ID IN (1,2,3,4,5) eh? |
|---|
| 273 | (13:56:11) domas: but I guess people are already sane at these times |
|---|
| 274 | (13:56:24) domas: MarkJaquith: you can just join user directly to posts. |
|---|
| 275 | (13:56:36) domas: SELECT * from wp_posts join wp_users ON (.. |
|---|
| 276 | (13:56:41) domas: well, * is evil |
|---|
| 277 | (13:56:42) domas: but anyway |
|---|
| 278 | (13:57:24) MarkJaquith: hm, only problem with that is that multiple post queries would do possibly redundant user joins |
|---|
| 279 | (13:58:04) domas: that's why you'd avoid multiple post queries :) |
|---|
| 280 | (13:58:27) MarkJaquith: well yes, not in the default template |
|---|
| 281 | (13:59:26) domas: on the other hand |
|---|
| 282 | (13:59:28) domas: it's much cheaper |
|---|
| 283 | (13:59:45) domas: than having multiple queries, ipc, .. |
|---|
| 284 | (14:00:14) domas: I voted for GROUP_CONCAT() for categories too, but this bastard is in mysql 4.1 and later only |
|---|
| 285 | (14:00:14) masquerade: heh, just about every other theme and plugin on the planet uses query_posts() for a second post query, so I'm not really sure avoiding multiple post queries is an option |
|---|
| 286 | (14:00:30) domas: masquerade: but you should not care about join price in this case. |
|---|
| 287 | (14:00:34) domas: a join is done on an index |
|---|
| 288 | (14:00:36) domas: and on hot data |
|---|
| 289 | (14:00:39) domas: it's _cheap_ |
|---|
| 290 | (14:00:47) domas: whatever people tell to you otherwise, don't listen. |
|---|
| 291 | (14:01:05) MarkJaquith: heh, I expect you know what you're talking about here |
|---|
| 292 | (14:01:36) domas: well, when you see a join of 50 tables |
|---|
| 293 | (14:01:43) domas: then you think about better in-db optimization techniques |
|---|
| 294 | (14:01:47) domas: rather than splitting it |
|---|
| 295 | (14:01:51) domas: :) |
|---|
| 296 | (14:02:30) MarkJaquith: this would be a massive code change. |
|---|
| 297 | (14:02:30) photomatt: domas: hot data? |
|---|
| 298 | (14:02:43) domas: photomatt: it's in RAM |
|---|
| 299 | (14:02:58) domas: photomatt: you've read it miliseconds ago, it's in RAM! :) |
|---|
| 300 | (14:03:03) domas: ergh, nanoseconds even |
|---|
| 301 | (14:04:29) domas: heh, kubrick fetches are annoying |
|---|
| 302 | (14:05:14) domas: http://p.defau.lt/?6kSNMyxeSoWHy6ute45mow |
|---|
| 303 | (14:05:38) MarkJaquith: domas: I filed a bug for that |
|---|
| 304 | (14:05:42) photomatt: we don't cache "cache misses" |
|---|
| 305 | (14:05:50) photomatt: right now |
|---|
| 306 | (14:06:11) domas: well, my original issue about rss feed inside options table still matters :) |
|---|
| 307 | (14:06:21) domas: photomatt: it should be autoloaded |
|---|
| 308 | (14:06:22) MarkJaquith: yeah, why are those autoloaded? |
|---|
| 309 | (14:06:34) photomatt: the options don't exist |
|---|
| 310 | (14:06:35) MarkJaquith: domas: er, should be? |
|---|
| 311 | (14:07:00) domas: ah, right |
|---|
| 312 | (14:07:05) domas: get it :) |
|---|
| 313 | (14:07:10) photomatt: the RSS cache in the DB is a poor man's cache, writing to the filesystem is a real pain in the enviroments we run in |
|---|
| 314 | (14:07:24) domas: photomatt: use other DB for that |
|---|
| 315 | (14:07:28) domas: pew |
|---|
| 316 | (14:07:29) domas: table. |
|---|
| 317 | (14:07:30) photomatt: and loading the remote RSS all the time is not nice and slow |
|---|
| 318 | (14:07:44) photomatt: why? |
|---|
| 319 | (14:07:57) photomatt: more data in the options table doesn't slow it down, does it? |
|---|
| 320 | (14:08:31) domas: well, optimizer sees 70 rows |
|---|
| 321 | (14:08:40) domas: and notices that most of them are autoloaded |
|---|
| 322 | (14:08:47) domas: so it doesn't even pick autoload index |
|---|
| 323 | (14:09:02) domas: but there's _one_ row that weights 100x more than all other rows combined |
|---|
| 324 | (14:09:23) domas: of course, it's negligible data amount ;-) |
|---|
| 325 | (14:09:25) MarkJaquith: there will still be autoload = no rows even if RSS cache was moved |
|---|
| 326 | (14:09:37) domas: MarkJaquith: those would not weight 100k |
|---|
| 327 | (14:09:57) MarkJaquith: hrm, so it's the size of it that matters? |
|---|
| 328 | (14:09:58) domas: I'm a bit anal here about that, maybe I shouldn't stress it too much |
|---|
| 329 | (14:10:17) domas: I usually explain this situation as 'the elephant among the ants' |
|---|
| 330 | (14:10:50) domas: you just can't apply neither elephant rules to all environment, nor ant rules |
|---|
| 331 | (14:10:51) MarkJaquith: Heh. Well, I'll say this... SELECT * FROM wp_options; is hell from the command line because of those feeds ;-) |
|---|
| 332 | (14:11:02) domas: MarkJaquith: too. |
|---|
| 333 | (14:11:03) masquerade: just wait until you see the amount of data plugins could and do store ;-) |
|---|
| 334 | (14:11:13) domas: haha, lol |
|---|
| 335 | (14:11:14) domas: screw plugins |
|---|
| 336 | (14:11:47) mdawaffe|lunch is now known as mdawaffe |
|---|
| 337 | (14:11:57) domas: soo... |
|---|
| 338 | (14:11:57) domas: SELECT DISTINCT YEAR(post_date) AS `year`, MONTH(post_date) AS `month`, count(ID) as posts FROM wp_posts WHERE post_type = 'post' AND post_status = 'publish' GROUP BY YEAR(post_date), MONTH(post_date) ORDER BY post_date DESC |
|---|
| 339 | (14:11:59) domas: this one |
|---|
| 340 | (14:12:00) mdawaffe: MarkJaquith: did you get a chance to look at 2561? |
|---|
| 341 | (14:12:02) domas: is the one I hate most actually |
|---|
| 342 | (14:12:06) MarkJaquith: mdawaffe: not yet |
|---|
| 343 | (14:12:11) mdawaffe: no worries |
|---|
| 344 | (14:12:11) photomatt: haha me too |
|---|
| 345 | (14:12:45) domas: use per-month cache in a table. :) |
|---|
| 346 | (14:13:04) domas: publish an article - increase. delete an article - decrease. \o/ |
|---|
| 347 | (14:15:00) domas: oh wait, query cache :) |
|---|
| 348 | (14:16:00) MarkJaquith: Here's that non-existent option bug report: http://trac.wordpress.org/ticket/2268 |
|---|
| 349 | (14:16:18) domas: as the timestamp is not put into queries, it's much more efficient with query cache of course ;-) |
|---|
| 350 | (14:16:32) domas: hehe, wanna add one more mysqlism? :) |
|---|
| 351 | (14:17:01) ringmaster left the room (quit: "Find me at http://asymptomatic.net, where I remain ambivalent."). |
|---|
| 352 | (14:17:11) MarkJaquith: go for it |
|---|
| 353 | (14:17:16) domas: SELECT /*! 40000 SQL_CACHE */ ... |
|---|
| 354 | (14:17:17) domas: ;-) |
|---|
| 355 | (14:17:29) domas: for queries you really want to cache. |
|---|
| 356 | (14:18:04) domas: there's a setting for mysqld, which caches only queries with such tags. |
|---|
| 357 | (14:18:34) domas: so, it'd cache in such cases :) |
|---|
| 358 | (14:20:36) MarkJaquith: Would require people to have an appropriate setup though |
|---|
| 359 | (14:20:41) domas: yes |
|---|
| 360 | (14:21:03) domas: minor issue anyway :) |
|---|