WordPress.org

Make WordPress Core

Opened 4 years ago

Last modified 11 hours ago

#20263 reviewing defect (bug)

Backticks in dbDelta cause warning and actually causes a query to alter all columns and indexes to run even if none have changed

Reported by: Driskell Owned by: ocean90
Milestone: 4.6 Priority: normal
Severity: normal Version: 1.5
Component: Database Keywords: has-unit-tests has-patch
Focuses: Cc:

Description (last modified by ocean90)

Backticks in a CREATE TABLE in a dbDelta call are causing the below warning, and running a query to alter all columns and indexes with backticks even if they haven't changed:
Notice: Undefined offset: 1 in /home/backuptechnology/public_html/wp-admin/includes/upgrade.php on line 1544

Seems this started a long long time ago after this was fixed: #8014

Seems there is a match on line 1587 of WordPress 3.3.1 wp-admin/includes/upgrade.php:

               // For every field in the table
                foreach ($tablefields as $tablefield) {
                        // If the table field exists in the field array...
                        if (array_key_exists(strtolower($tablefield->Field), $cfields)) {
                                // Get the field type from the query
                                preg_match("|".$tablefield->Field." ([^ ]*( unsigned)?)|i", $cfields[strtolower($tablefield->Field)], $matches);

The preg_match bit is what fails to match because it is looking for "fieldname" instead of "fieldname".
So this is where the warning occurs, and why it assumes the type of the field is wrong.
This means the previous patch 3 years ago probably meant when you have backticks in your CREATE TABLE, when you run dbDelta it actually runs an ALTER TABLE for every single column in the table, and then an ALTER TABLE ADD INDEX / PRIMARY for every single KEY and PRIMARY (there's more than just the above code that needs adjusting), causing duplicate key errors, but ignored. Guess it kinda works though...

Also seems during the DESCRIBE it doesn't use backticks so if someone has a table with a reserved name (stupid I know) it would break dbDelta completely, but that's me digressing I just think getting rid of the warnings for CREATE TABLES with backticks will be good.

But explains why my plugin takes a few seconds to activate hahaha. :-)

I'll provide a patch at some point this week or next unless someone has already done so.

Attachments (6)

20263.patch (677 bytes) - added by kurtpayne 4 years ago.
Remove backticks when comparing fields and indexes
20263.diff (791 bytes) - added by davidmosterd 14 months ago.
20263.2.diff (1.6 KB) - added by ocean90 2 weeks ago.
20263.3.diff (1.5 KB) - added by ocean90 36 hours ago.
20263.4.diff (3.9 KB) - added by ocean90 33 hours ago.
20263.5.diff (8.8 KB) - added by ocean90 11 hours ago.

Download all attachments as: .zip

Change History (27)

#1 @johnbillion
4 years ago

  • Cc johnbillion added

@kurtpayne
4 years ago

Remove backticks when comparing fields and indexes

#2 @kurtpayne
4 years ago

  • Cc kpayne@… added
  • Keywords has-patch added

I can replicate the original problem. Here's the sample plugin I used to test this:

https://gist.github.com/2151282

Just activate the plugin and look for errors. Deactivate the plugin to reset.

#3 @nacin
4 years ago

  • Version changed from 3.4 to 1.5

Definitely not new, setting version to 1.5 when dbDelta came around.

Adding ` to the list of characters to trim() might be enough, yes?

#4 @kurtpayne
4 years ago

Adding ` to the trim list won't fix it because a backtick can occur in the middle of the string:

`user_ip` int(11) DEFAULT 0,

#5 @timfs
4 years ago

What's so complicated? Took me 10 seconds:

preg_match("|`?" . preg_quote($tablefield->Field) . "`? ([^ ]*( unsigned)?)|i", $cfields[strtolower($tablefield->Field)], $matches);

This also fixes problems with tablenames that contain "special" characters, in MySQL for example foo.bar.[b-a-z] is a perfectly valid tablename, besides using non-sanitized external input is always a very bad idea.

However, this Regex is pretty fragile anyways, it can't handle multiple spaces, it can't handle tabs, it can't handle spaces between the parentheses enclosing the column length, etc... someone should really rewrite that whole stuff, it might explode by a slight sneeze.

Version 0, edited 4 years ago by timfs (next)

#6 @dd32
18 months ago

  • Component changed from Upgrade/Install to Database

#7 @andddd
17 months ago

It's been couple of years. Can this be finally fixed because my plugins spits a ton of errors on each update. @timfs proposed a reasonable patch.

Last edited 17 months ago by andddd (previous) (diff)

@davidmosterd
14 months ago

#8 @davidmosterd
14 months ago

Little context: Here with a mentor (Mark Jaquith) on contributor day WordCamp London 2015 and just looking to get my first commit going, found this ticket :)

We did this assumption: When you supply a CREATE TABLE statement, dbDelta() should handle what MySQL gives you when you export the CREATE TABLE statement

I tried to simplify this case by first removing the first appearance of the column name from the query, and then ltrim() from the remainder any remaining spaces and an optional backtick (in case the column name was enclosed in backticks). The string should now start with the data type, maybe having " unsigned" behind it. The remaining preg_match() is quite simple now.

#9 @markjaquith
14 months ago

  • Keywords early added
  • Milestone changed from Awaiting Review to Future Release

#10 @dd32
11 months ago

#30655 was marked as a duplicate.

This ticket was mentioned in Slack in #glotpress by ocean90. View the logs.


2 months ago

#12 @ocean90
2 months ago

  • Keywords needs-unit-tests added

We noticed the same with GlotPress which has a colum "references", see https://github.com/GlotPress/GlotPress-WP/pull/343.

It seems to work with BackPress because BP_SQL_Schema_Parser::delta() supports backticks, see https://backpress.trac.wordpress.org/browser/trunk/includes/class.bp-sql-schema-parser.php?marks=118,130,144#L117.

#13 @jrfoell
6 weeks ago

Just joining the conversation here, hoping this gets resolved since it seems like an easy fix. Still an issue in WP 4.4 :(

@ocean90
2 weeks ago

#14 @ocean90
2 weeks ago

20263.2.diff adds a simple unit test but it probably needs a few more.

@davidmosterd: I noticed that the strtolower() for $tablefield->Field got removed in 20263.diff. Not sure if that was intended or not.

#15 @ocean90
2 days ago

  • Description modified (diff)
  • Keywords has-unit-tests added; needs-unit-tests removed
  • Milestone changed from Future Release to 4.6
  • Owner set to ocean90
  • Severity changed from minor to normal
  • Status changed from new to reviewing

#16 @pento
2 days ago

Rather than doing the ltrim() magic, I'd be inclined to add the `s to the regex. That's the same as how wpdb::get_table_from_query() behaves, as well as the BackPress example that @ocean90 linked to.

#17 @ocean90
36 hours ago

In 37538:

Database: Support backticks around field names when parsing a query for the field type.

Avoids an undefined index PHP warning by dbDelta().

Props davidmosterd, ocean90.
See #20263.

@ocean90
36 hours ago

@ocean90
33 hours ago

#19 @ocean90
33 hours ago

  • Keywords needs-patch added; has-patch early removed

Turns out the approach in 20263.patch doesn't work because it would break a query where backticks are required, like for reserved keywords. 20263.4.diff includes a test for that.

In 20263.4.diff I tried to escape every field in our custom tables which doesn't quite work because of the array_search() call.

I'll take another look tomorrow and will probably copy parts of BP_SQL_Schema_Parser:: get_index_definition() and https://backpress.trac.wordpress.org/browser/trunk/includes/class.bp-sql-schema-parser.php?marks=118-143#L117 to wpdb.

This ticket was mentioned in Slack in #glotpress by ocean90. View the logs.


13 hours ago

#21 @ocean90
12 hours ago

  • Keywords has-patch added; needs-patch removed

20263.5.diff normalizes the index definitions so they match with the result of SHOW INDEX FROM in https://core.trac.wordpress.org/browser/trunk/src/wp-admin/includes/upgrade.php?rev=37538&marks=2274-2337#L2274.

The regular expressions are copied from BackPress, slightly changed to have named capture groups.

This patch fixes also #34959, #34871, #34873 and #34869. #34874 would be possible too.

Last edited 11 hours ago by ocean90 (previous) (diff)

@ocean90
11 hours ago

Note: See TracTickets for help on using tickets.