Plugin data migration

[anonymized_14235950]

I'm not a dev, so this may not be possible like I envision, so forgive me if it is not.

I have used a bunch of plugins that make it hard, or even nearly impossible, to migrate data between them. Sure, if you use a calendar or other plugin that is centered around some kind of standard file export format like ICS then you can probably do it, but many aren't like that.

One example is Gravity Forms. There is only one plugin I could find that allows me to import data into GF and that costs $99. Insane. There are many other examples.

WordPress and its plugin, being open source, should (in my mind) make it easy to switch from one plugin to another without losing data.

Here is what I envision:

1. A user has BuddyPress installed and has a few hundred members who have all filled out custom profile fields that they created with BuddyPress.

2. A competitor to BuddyPress is released, which the user installs and activates.

3. The user selects the option under the plugin to import data into it.

4. WordPress asks the user which plugin they'd like to import the data from.

5. User selects BuddyPress.

6. WordPress scans the databases BuddyPress created and comes up with a list that the user can review to select which data goes where. This would be similar to when you are importing a CSV file into an email newsletter program, if you are familiar with that. The user would see a list of last names and select "Last Name" as the column header, a list of phone miners and select "Phone Number," etc.

7. WordPress would then migrate all that data into the new plugin and the user would be able to rest easy knowing that they can easily switch back or switch to another plugin instantly without losing his users' data.

I think this is a glaring hold in WordPress and is greatly needed for obvious reasons. Technology, especially open source software, changes so quickly that it is important for users to not have their data locked in to one plugin.

I believe WordPress could change that in a big way, and this could spur competition and quality among plugins as a result, along with offering an incredibly simple and easy solution to users everywhere.

[swissspidy]

Plugins store data in so many different ways. WordPress simply cannot know how each one works. In my opinion it's the job of the plugin to import data from another plugin correctly.

[anonymized_14235950]

Replying to swissspidy:

Plugins store data in so many different ways. WordPress simply cannot know how each one works. In my opinion it's the job of the plugin to import data from another plugin correctly.

Isn't most text-based data stored in the database? I don't necessarily mean ALL data, like photos and other media. I just mean text based info like in the example I gave above.

[knutsp]

Under the current paradigm for what plugins may do and how they do it, there is no way WordPress can know what data was stored by which plugin.

Even the plugin author may not know. All metadata, all options and all custom post types would have to be either prefixed with a plugin slug or an extra key field for this.

Would we want something like that? In my view, cleary no.

[anonymized_14235950]

Replying to knutsp:

Under the current paradigm for what plugins may do and how they do it, there is no way WordPress can know what data was stored by which plugin.

Even the plugin author may not know. All metadata, all options and all custom post types would have to be either prefixed with a plugin slug or an extra key field for this.

Would we want something like that? In my view, cleary no.

Why not? Just curious. I would think it would be very beneficial to have that. That way, not only could we migrate data between plugins more easily, but since WordPress would know what data came from where then it would be able to allow us to remove all the plugin's data upon uninstall rather than trusting the plugin to do it itself. After using a bunch of plugins for years on one site, it can get messy with the dead files being left around by old plugins you've deleted.

[knutsp]

Plugins are not self contained applications. They are pieces of code that becomes part of th WordPress core application once activated. Even if it may have benefits to know what data is "owned" by a plugin, there are data that should not be owned. Data saved should become part of a common database. This enables plugins to extend other plugins and reuse data directly.

Even now, plugins that want a complete separation may create their own DB tables, hopefully properly prefixed with a slug that clearly identifies the plugin hat created it.

And as others have said, it should be the responsibility for each plugin that wants to import data another plugin has saved, or whatever, to properly either reuse them as is, or import them and store them with new keys, when needed.

So if a plugin wants to "take over" for, say BuddyPress, it may just use the data as they are. They er not protected and accessible for any plugin. Not marked who it belongs to, and that's fine. This need for an import is really non-existent.

Your suggestion will just complicate things, in my view.

This suggestion implies a so great paradigm shift that I also doubt Trac is a suitable place for discussing what it would take. I suggest writing a blog post, and make suggestions on how to implement it in practice. "Hopeless" things may become less strange if it can be implemented elegantly, flexible, compatible and easy to understand. This is just a wish and a general idea. Others may think different, for all I know.

[boonebgorges]

Thanks for the ticket, freetheweb. I'm a member of the BuddyPress team in addition to the WordPress team, so I'm acutely aware of the importance of decent import/export tools for custom data.

That being said, there's little that WP can do all by itself. As noted in the thread above, plugins store data in many different ways: custom post types, custom metadata for posts and users, custom options, custom database tables. WordPress can't predict all of these, which means it'll be nearly impossible to build a general tool that is able to recognize and identify all types of data created by plugins.

And recognition/reading is the easy part. It's one thing for WordPress to look through the database, recognize non-native data, and show it to the user (this is the "export" part of the equation). But because each "import" plugin expects data to be in different formats, there's literally no way for WP to know where to put it.

The best that WP can do is to improve its import and export APIs in the following ways:

• Better support for arbitrary data types and data structures, including support for various open data standards.
• A way for plugins to tell WP that a given piece of data is of a given type. So: BuddyPress might decide to represent "friendship" data using FOAF. WP's API would have a way for BP to announce this, so that the data is properly formatted in the xml/json file that's generated as part of the export. Then another plugin could announce to WP that it can ingest FOAF data from such a file.
• UIs that support the workflows described above.

Some technical foundations for kind of workflow are proposed in #22435.

I'm going to close this ticket as maybelater on the basis of the above. If we get a more robust export/import API, we can then start talking about what it would mean to support standardized data types. Only then will WP be able to play middleman in the types of migrations you've proposed here.

[knutsp]

@boonebgorges: Good answer. I highly support the plans to standardize data types and perhaps the input control type for it. (But not some kind of ownership to plugins, except for custom DB tables.)

So this is the way to go to ensure the "commonness" and reusability of stored data among themes and plugins, as opposed to this tickets original suggestion.