Decouple the Menu System from Posts and Taxonomies

[MikeSchinkel]

I'm proposing this idea for consideration, and hope that it will be taken seriously and at least contemplated rather than have someone close it reflexively.

I'd like to propose we consider create menu classes and then decoupling the menu system from the post types and taxonomy systems and store menus in the options table as serialized objects/arrays. This could probably be done in a mostly backward compatible manner such that it could mirror the menu data into the existing structure, it would only fail if someone updates the menu information in the posts and taxonomy structures using direct SQL.

Having used the menu system in a variety of projects I've come to the conclusion that the added complexity of using posts and taxonomy for menus provides no benefit over having a much simpler serialized object hierarchy such that we have always abandoned the built-in system and rolled our own; if anyone knows of use-cases where the current system is preferrable, please advise.

The reason I posted this ticket today is because of @voilamedia's issue with edits to taxonomy terms breaking his or her menu. Here are the relevant comments from that ticket:

When you create a custom taxonomy, (post types most likely would do this too but I didn't want to test it) then add terms to the taxonomy as you would. Then add a term of that taxonomy as the name of custom menu that you set up with the new menu manager feature, than add menu items it modifies the term and the menu it modifies both the menu term and the custom taxonomy term.

I know this would be a big change from how the existing system stores menus but I think it could make working with menus significantly easier for themers and plugin developers and get them to use the menu system in their own works more frequently than they currently do. It would solve at least one user issue that won't be solved before significant surgery to much of the core and it's database tables which isn't going to happen quickly.

I hope you'll give this ticket some serious consideration as I think it could really improve the WordPress API.

[wonderboymusic]

Moving to serialized objects? No.

[MikeSchinkel]

Replying to wonderboymusic:

Moving to serialized objects? No.

16 minutes? Thanks for taking time to contemplate the idea and let other's weigh in on it.

[wonderboymusic]

You have 90 menus in the options table, all containing the same post in their hierarchy tree - that post gets deleted ... You would have to 1) load every menu and search its tree or 2) search LONGTEXT via SQL LIKE and then reconstruct each affected tree.

Also, there's no patch.

[MikeSchinkel]

Replying to wonderboymusic:

You have 90 menus in the options table, all containing the same post in their hierarchy tree - that post gets deleted ... You would have to 1) load every menu and search its tree or 2) search LONGTEXT via SQL LIKE and then reconstruct each affected tree.

Who said there is only one way to implement this? Your concerns show you are not thinking this through.

Also, there's no patch.

Why spend several days writing a patch when someone is going to to close the ticket in 16 minutes?!?!?

Just re-open the ticket, let other people weigh in and then decide. If it's ultimately a bad idea -- it might be -- but let the process work instead of just killing it w/o any other input.

[nacin]

While I probably would have let this play out for a bit before closing the ticket, I think it's also helpful to be honest when an idea is dead in the water from the start. Conversation can continue even when a ticket is closed. I strongly agree with not doing this. I have some good news, though.

Being involved in building menus in 3.0 probably subtracted a few years from my life. What we ended up with worked, but obviously has its issues. That said, I don't think the schema is the problem. I think we were just using custom post types and custom taxonomies for the first time and did it the "long way". It ended up being over-engineered in nearly every way, and it could benefit from a lot of TLC on the internals.

Anyway — the primary reason you've proposed for doing this is that custom taxonomies share term objects. This is a design decision dating to 2.3, but it isn't the way it has to be. In fact, it doesn't require significant surgery, and re-doing all of menus *certainly* would not happen any quicker than the outpatient surgery required. Re-doing menus is terrible waste of time — even a fraction of the time that would otherwise be spent on that could be used to improve menus in general, without butchering the schema.

The complete plan is outlined in ​http://make.wordpress.org/core/2013/07/28/potential-roadmap-for-taxonomy-meta-and-post-relationships/. Solving the user issue presented here simply requires completing the pre-requisite steps that are needed before actually merging the tables, doing term meta, and then considering post relationships. These steps would solve this for all custom taxonomies, not just menus, and would put us on a path to considering term meta and post relationships — which I imagine you would not like to slow down. :-)

Those tickets are (in order, as each blocks the next) — #17689, #22023, #5809, and #21950. I've proposed tackling these for 3.9 (hinted ​in this post), which will solve this problem.

So, in the end, this is a non-starter for two reasons:

• We're not going to re-architecting an entire schema to move to a serialized array, especially not to fix a deficiency in the custom taxonomy API.
• We are planning to fix the custom taxonomy deficiency very soon, which will render the problem here moot and make custom taxonomies a better building block for others.

So, yes, it's safe to close this — but it's also clear everyone wins.

[MikeSchinkel]

Replying to nacin:

Being involved in building menus in 3.0 probably subtracted a few years from my life.

I can believe it. :)

Anyway — the primary reason you've proposed for doing this is that custom taxonomies share term objects.

To be clear, that was not the primary reason. That issue was the one that got me to post this ticket and it provided a reason I thought core might get behind since it affects users. My primary reason is to make menus less fragile in plugins and themes, especially when one is trying to add custom functionality. and the secondary reason is to remove unintended consequences for when doing things that inadvertently affect menus, albeit a distant second.

So, in the end, this is a non-starter for two reasons:

Fine. I just wish time was allowed for others to voice their opinions and discuss options for potential implementation before deciding this, or deciding other things, are not a good idea. The "We know better so there's no need for discussion" approach can seriously demotivate people who want to contribute, ​people other than just me.

[F J Kaiser]

Replying to nacin:

That said, I don't think the schema is the problem.

I agree with you here. The idea of using Taxonomies and CPTs IMO is a good one. Provides enough flexibility and gives low end devs enough public API/Template Tags, etc./tools to interact with it.

Replying to MikeSchinkel:

My primary reason is to make menus less fragile in plugins and themes, especially when one is trying to add custom functionality. and the secondary reason is to remove unintended consequences for when doing things that inadvertently affect menus, albeit a distant second.

Switching to something else doesn't fix the underlying problem. CPTs and Taxonomies need some care: true. But we shouldn't just switch and start to build and maintain yet another system. IMHO everything in WP should rely on CPTs and Taxonomies - even admin pages and the admin menu. The more we build on CPTS & CTs, the better we can focus on it, the better WP gets as there're less things pulling off resources. And I'm hardly against serialized arrays and even more against serialized arrays in tables and completely against stuffing menu items in the objects table. I hope I don't need to, but I'm able to elaborate that in detail.

Replying to nacin:

I think we were just using custom post types and custom taxonomies for the first time and did it the "long way". It ended up being over-engineered in nearly every way, and it could benefit from a lot of TLC on the internals.

Very true, that. But to be honest: It doesn't only need "love on the internals". It should be rewritten from scratch. No one can fix that mess. Stating that as I thought I could fix it this weekend and provide a patch. (Yes, I'm aware of the times when it was written and the PHP version it was written in).

[nacin]

Replying to MikeSchinkel:

My primary reason is to make menus less fragile in plugins and themes, especially when one is trying to add custom functionality. and the secondary reason is to remove unintended consequences for when doing things that inadvertently affect menus, albeit a distant second.

Then it needs a better API, not a (worse) schema.

I just wish time was allowed for others to voice their opinions and discuss options for potential implementation before deciding this, or deciding other things, are not a good idea. The "We know better so there's no need for discussion" approach can seriously demotivate people who want to contribute

Sure. You catch more flies with honey, I get that. But let's face it, moving this to serialized options is a non-starter. It's never gonna happen. Keeping this ticket open is not the best use of anyone's time or focus. If you want to still comment, feel free. Comments remain open.

[MikeSchinkel]

But we shouldn't just switch and start to build and maintain yet another system. IMHO everything in WP should rely on CPTs and Taxonomies - even admin pages and the admin menu.

I used to think that, but after lots of attempts to use CPTs for many things in my own libraries I've move away from that in some cases. If you overload them too much you end up with a very fragile system where it's far too easy to break something when you are working on something else.

And I'm hardly against serialized arrays and even more against serialized arrays in tables and completely against stuffing menu items in the objects table.

I am usually against serialized arrays being stored in the database because most things really do need to be processable by SQL. But for menus I've yet to find myself in a position where I've needed to process menus using SQL (other than that's how WordPress currently works.) I'm not saying reasons do not exist but simply that I've not seen them and one of the reasons for this ticket is to hear from others if there are indeed reasons that serialization does not fit this use-case.

I hope I don't need to, but I'm able to elaborate that in detail.

Please do.

Very true, that. But to be honest: It doesn't only need "love on the internals". It should be rewritten from scratch. No one can fix that mess.

That I agree with.

[F J Kaiser]

Replying to MikeSchinkel:

If you overload them too much you end up with a very fragile system where it's far too easy to break something when you are working on something else.

Agree that it's (currently) not the most robust system and it surely has its edges and shortcomings. But - at least for me - that doesn't mean that we should start spreading forces by introducing new systems. Let's face it: CPTs and CTs are what drives WPs content engine and they aren't going away.

I am usually against serialized arrays being stored in the database because most things really do need to be processable by SQL. But for menus I've yet to find myself in a position where I've needed to process menus using SQL (other than that's how WordPress currently works.) I'm not saying reasons do not exist but simply that I've not seen them and one of the reasons for this ticket is to hear from others if there are indeed reasons that serialization does not fit this use-case.

I hope I don't need to, but I'm able to elaborate that in detail.

Please do.

The only valid use case for serialized content that I can see is a log. Write log once, read all entries at once, but have no searching, no updating. Pretty much like with a NoSQL DB. Just with a regular clean up task.

Anyway, here're some reasons why I'd never consider using serialized menu items in the options table:

(1) Structural changes are expensive. You'll have to read all the data, filter it, change and then add it back to the stack, then update every single element/entry. This would only be worth it, if every item is exactly the same as every other one. Else you'll have to craft a fresh API to deal with all the variations. And variations (CSS classes, XFN, etc.) can be found all over menu items.
(2) Serialized content can't be queried and filtered inside the DB layer. That means that every attempt to craft dynamic navigation based on arguments would be a pain. Imagine seasonally changing menu items in an online boutique/shop.
(3) Stuff like our "post type archive links"-plugin [A] would be much harder to accomplish as the menu items foundation wouldn't be that dynamic anymore: Check rewrites/routing on the fly, use the WP_Query API, etc.
(4) The options table is for options. Not for menu items. It's already tough enough to keep that one clean when test driving someones plugin or theme. If they shouldn't be a CPT or CT, then please a completely separate table. Then someone could at least add different menus for different themes and it would be much easier to keep them - think about multisites as well.
(5) I can't see a real reason to use the system if you aren't satisfied. There have been enough cases where I hard coded a menu into a theme. Just because it was faster, it was the only menu on this site and no one would ever change it. Rule of thumb: Only make dynamic what needs to be dynamic. And from what I could read, you don't really need dynamic.

[A] ​http://wordpress.org/plugins/post-type-archive-links/

[MikeSchinkel]

CPTs and CTs are what drives WPs content engine and they aren't going away.

I'm a big fan of ​CPTs and CTs, but I also prefer to use a screwdriver to drive screws instead of a hammer. Not every tool, no matter how good, it the right tool for every job.

The only valid use case for serialized content that I can see is a log. Write log once, read all entries at once, but have no searching, no updating. Pretty much like with a NoSQL DB. Just with a regular clean up task.

There are numerous other valid use-cases for serialized content besides a log. Here are two:

1. A cache for data; unserializing is the fastest PHP way to move from disk to a complex in-memory data structure when those data structures are always initialized the same for every page load.

2. Plugin or theme options, because ​Otto says so.

OTOH, I can't envision lots being all that useful as serialized arrays but I'm always willing to assume maybe I'm not seeing the full picture.

(1) Structural changes are expensive

Agreed in general, but not for this use-case. Menus model to an object and lists of other objects better than they model to database tables, and an object structure containing everything you'd want to know about a Menu takes a lot of code and SQL queries to load up, ever page load.

Menus are updated very infrequently, so another 100 milliseconds to update a menu is not going to be noticed by anyone who is not being pedantic. OTOH, loading menus from the current table structure takes a lot more time during page load and that affects users and google rankings; that is what should be optimized, not how long it takes to update something that is likely updated less than 0.01% of the time it is online and available for visitors to access pages.

(2) Serialized content can't be queried and filtered inside the DB layer.

Agreed in general, but not for the menu use-case. It's usually important not to serialize data stored into postmeta or usermeta because you frequently need to query on it, but I still cannot envision how that would ever need to relate to menus.

Also the WordPress API discourages direct SQL as it is only acceptable when the API doesn't support a use-case. So the coding model to craft dynamic navigation would be to use the API, not to run SQL, and if the API didn't exist it would just require extending it.

Maybe it's just not clear what I'm envisioning, but I'm struggling to envision anything that would be more difficult for this use-case compared to what we have now.

(3) Stuff like our "post type archive links"-plugin [A] would be much harder to accomplish as the menu items foundation wouldn't be that dynamic anymore: Check rewrites/routing on the fly, use the WP_Query API, etc.

I looked through the code for your plugin and I'm confident that I can say not only would your plugin be easier to write if core used the approach for menus I advocated in this ticket but your function ajax_add_post_type() would become much simpler with ~1/2 as much code, your function setup_archive_item() would work very similarly, and chances are you wouldn't even need the function for the 'wp_setup_nav_menu_item' hook.

(4) The options table is for options. Not for menu items.

Where is it written that there is a '"for" related to the options table and that that "for" does not include menu items? Seriously?

The options table is a table of name-value data pairs for global use across a site. Currently you can find options as well as serialized data for at least Widgets, User Roles, Cron tasks and Transients most of which are used similar to how I proposed serialized data be used for Menus. Menus are not out of line considering what else core stores here.

Also, the WordPress database provides a collection of data storage patterns, and we as developers should strive to use those patterns if at all possible before we go and roll our own tables. Options would work perfectly for Menus (at least I think it would.)

It's already tough enough to keep that one clean when test driving someones plugin or theme.

Funny, that's what I'm saying about the posts table. :)

If they shouldn't be a CPT or CT, then please a completely separate table.

Proposing a new tables is pretty much a non-starter for WordPress, more so than most anything else. Adding a new table is a HUGE decision, and one that won't be taken lightly by TPTB. To add a new Table WordPress.com would have to run a migration script over millions of sites, which is I understand one reason adding a table is such a big deal.

Then someone could at least add different menus for different themes and it would be much easier to keep them -

Why is another table any better?

think about multisites as well.

I'm pretty sure what I proposed could work equally well with multisite as it would for single site. Multisite has an options table per site; what am I missing that you see here?

(5) I can't see a real reason to use the system if you aren't satisfied.

But I can see a real reason. I'd like to be able to offer end-users the ability to update their own menus and be able to more easily implement changes without making the menu system less robust. If I hardcode them, then users require me to change them; that's what I'm trying to avoid.

[MikeSchinkel]

Replying to nacin:

let's face it, moving this to serialized options is a non-starter. It's never gonna happen. Keeping this ticket open is not the best use of anyone's time or focus. If you want to still comment, feel free. Comments remain open.

At least one person told me "Implement it yourself." I unfortunately don't have time to do that right now, but if I do attempt to implement this in the future and for my own (client) needs, will you be supportive of adding hooks that I would need to make it work with the current API and UI?

[F J Kaiser]

Replying to MikeSchinkel:

Not every tool, no matter how good, it the right tool for every job.

True. But when I look at the box of screw driver bits that I got in my (real life) toolbox for each kind of screw head, I'd wish that there would be less variation. This would allow me to own less tools and instead get better crafted ones. Guess we can end the comparisons here.

There are numerous other valid use-cases for serialized content besides a log. Here are two: A cache for data; (...) Plugin or theme options

How is cache or options different from what I described as log? (rhetorical question)

(1) Structural changes are expensive

Agreed in general, but not for this use-case. (...) Menus are updated very infrequently, so another 100 milliseconds to update a menu is not going to be noticed by anyone who is not being pedantic.

I'm talking about structural changes of the menu system itself. For e.g. properties of menu items are changed with an update of WP core.

Where is it written that there is a "for" related to the options table and that that "for" does not include menu items? Seriously?

Seriously. Else it'd be name options and other stuff, right? Yes, currently people are using it as if it would be name crap and more crap, but I wouldn't set goals that low.

Currently you can find options as well as serialized data for at least Widgets, User Roles, Cron tasks and Transients (...)

Ok... core partly doesn't use it that different than users do.

I looked through the code for your plugin (...)

I was afraid you would. Not the best example, but the one that came to my mind quickest. Still my point stays: Reduced abilities for dynamic querying.

Why is another table any better?

think about multisites as well.

Multisite has an options table per site; what am I missing that you see here?

Users are shared across multisites. That's what I'd propose (in this fictional scenario) for menus/items as well. Imagine multisite for different languages, all sharing the same menu with just some minor variations. Or magazine and shop, sharing the same main menu. Etc.

[MikeSchinkel]

How is cache or options different from what I described as log? (rhetorical question)

If in your mind they are not different then for the purposes of this discussion they are the same. I don't have a need to make the distinction if you don't.

I'm talking about structural changes of the menu system itself. For e.g. properties of menu items are changed with an update of WP core.

Other than the ability to write directly to the tables w/o using the WordPress API I'm pretty sure it could all be made compatible, both for SQL read and for any other interactions (except SQL update or insert.)

Seriously. Else it'd be name options and other stuff, right?

I don't think there is such a thing as a definition of "options" such that would disallow storing something that needs to be global.

We'll just have to agree to disagree on this one.

Yes, currently people are using it as if it would be name crap and more crap, but I wouldn't set goals that low.

Yes, the current ones are the core committers. :)

I was afraid you would.

Sorry, twas your example. :)

I think maybe your response to the idea was a gut-level "no" which if so means you making an assumption that this proposal (would have) caused problems but you can't put your finger on any specific example? It feels like you can only find principles which you think apply and I think don't apply? (I'm not criticizing; I've been in the situation where I felt a change was wrong but couldn't put my finger on it either.)

FWIW, I've been thinking about this for over a year and I've yet to find a use-case where storing as serialized data would cause a problem. Which is a moot point related to core, it seems.

Users are shared across multisites.

usermeta would be the place to store user-specific menu options.

No harm, no foul.

[nacin]

Replying to MikeSchinkel:

FWIW, I've been thinking about this for over a year and I've yet to find a use-case where storing as serialized data would cause a problem.

Well, I'm sold.

[sc0ttkclark]

Replying to nacin:

Well, I'm sold.

Not sure if this was necessary.. Seems like a low blow.

As for the ticket, I do agree that the taxonomy issues should be addressed either way. But IMO configuration should never be stored in CPT/CT, core and other plugins should architect in such a way that treasures proper database architecture and separation over ease of API. I speak with large projects in mind, recently which have had to deal with some plugins storing certain types of config as post types and meta, where the DB had millions of rows in posts/postmeta, and was slowing down the page needlessly, even if cached (during generation of initial cached value).

[MikeSchinkel]

Replying to nacin:

Well, I'm sold.

This is why we can't have nice things.

[knutsp]

-1

I have given this suggestion some consideration. No, because it wouldn't solve anything. I would fix the current problem with taxonomies having shared terms. The implementation of menus are quite good, but the implementation of taxonomies are not perfect. Fixing this will solve a lot.

There is a roadmap for taxonomies development, suggested by nacin, after years of discussion. Make it happen now. Terms may be edited, and they must not be shared.

[aaroncampbell]

First, I think we can all agree that the issue of shared terms across taxonomies needs to be fixed. It's a clear win and will definitely help fix some of the oddities that affect menus.

Other than that, I have to say that I'm with Wonderboy, Nacin, and F J on this. I definitely don't see the benefit of storing menus as a serialized array or object. They are notoriously hard to query for, which limits the flexibility of menus quite a bit. Right now you can do things like add another taxonomy and build dynamic menus by pulling menu items from any number of menus. You can also easily do things like pulling all menus that contain a certain post.

Replying to sc0ttkclark:

As for the ticket, I do agree that the taxonomy issues should be addressed either way. But IMO configuration should never be stored in CPT/CT, core and other plugins should architect in such a way that treasures proper database architecture and separation over ease of API. I speak with large projects in mind, recently which have had to deal with some plugins storing certain types of config as post types and meta, where the DB had millions of rows in posts/postmeta, and was slowing down the page needlessly, even if cached (during generation of initial cached value).

The "menu options" are stored as options. Whether to auto-add root-level pages is stored as a regular option and the menu locations are stored as a theme mod, which is just a namespaced option.

Yes, things like the item type, parent, target, etc are stored as post meta, but that means they are able to be included in a query. I can pull a menu and filter out the items that are set to open in a new window...all in the query (and I've done this). It will definitely cause the post meta table to grow, but I'm far less worried about the number of rows than I am the ability to query them.

Maybe I'm not understanding the concern about number of rows. We're currently working on a client site that has over 2.5M rows in the post meta table, and it's still lightning fast. I've worked with post meta tables more than ten times that big and never had any issues. I like that the data is highly available and the ways I can use it are flexible.

Replying to MikeSchinkel:

I used to think that, but after lots of attempts to use CPTs for many things in my own libraries I've move away from that in some cases. If you overload them too much you end up with a very fragile system where it's far too easy to break something when you are working on something else.

I haven't really experienced this, but maybe you can make some tickets that help address the weak or unscalable parts of the CPT system instead? It's definitely not a tool for every purpose, but it actually seems to fit menus quite well.

[mbijon]

Mike, what are your thoughts on using a nested set schema instead? Why specify serialized objects before mapping needed API improvements? Probably a bad way to open discussion though...

Point being:
I think there's a lot of API improvements that are needed for menus, and some may be limited by the current schema. Choosing a schema before planning a final or best-case API seems a little early.

[MikeSchinkel]

Replying to mbijon:

Thanks for commenting.

Mike, what are your thoughts on using a nested set schema instead?

Hmm, not really sure what "a nested set schema" is. Examples?

Why specify serialized objects before mapping needed API improvements? Probably a bad way to open discussion though…

Yes, I do believe I made a tactical error by opening the discussion this way, but it's just the way I think. I've got 30+ career years of indoctrination related to issues where the mantra has always been "Don't bring me a problem, bring me a solution." But for some reason proposing a solution as a ​strawman proposal causes people to react negatively rather than just opening a discussion of alternatives, and I still can't grasp why that is.

OTOH, I have also tried leading with the problem(s) to solve and then I'm told Trac is for code, not for discussions and yet there's no clear place I'm aware of to post discussions on specific issues. So I feel damned if I do, damned if I don't.

Frequently when I post an issue it's usually the last straw for me related to a concern but I don't always remember all the details of the several previous times I've encountered the issues so I can't always give exhaustive reasons; I guess it's my fault that I don't have a better cumulative memory for the details or that I didn't recognize the first time I recognized an issue that it would be recurring.

So I would gladly change how I presenting things if I only knew a better approach that would indeed be more effective. But at this point I'm ​learned helpless.

Anyway, given that you've revised discussion here are the two (2) issues that my proposed solution was trying to solve:

1. The API leaves a lot to be desired, and would IMO benefit significantly from moving to a WP_Menu class and related classes.
2. More importantly, the impedance mismatch between posts used for managing content and posts used for managing Nav Menu Items. I'll elaborate on that to the best of my memory below.

Regarding 2. above, here are the Top 10 reasons why IMO posts don't work well for Nav Menu Items, in increasing order of importance (Note: In general, I advocate people use posts rather than add tables, but experience has taught me that at least for Nav Menu Items, posts are not a good fit):

10. Posts have many fields that are simply not needed for Nav Menu Items, some being to_ping, pinged, post_password, post_content, post_excerpt, post_date, guid, post_mime and others.

9. Nav Menu Items don't have comments.

8. Nav Menu Items use more post meta fields than fields they actually need to use in the wp_posts table: classes, menu_item_parent, object, object_id, orphaned, target, type, url and xfn. Together 10 and 8 make using posts a heavy solution for the use-case.

7. Saving and loading posts fires many hooks, most of which are completely irrelevant for Nav Menu Items.

6. All that hook firing and db access makes the process of using menus in the admin less responsive than it could be.

5. The loading of so many records for one menu on the front takes up time that could be made a lot more efficient without having to implement caching.

4. It's easier for a novice theme or plugin coders to break the menus in strange ways or to incorporate Nav Menu Items into lists of externally published posts because they don't realize the have to filter out 'nav_menu_item' post types in large part because they had no menus when they were building and testing.

3. Workflow that should affect all content posts must always filter out 'nav_menu_item' posts.

2. The 'transition_post_status', 'edit_post', 'post_updated', "save_post_{$post_type}", 'save_post', 'wp_insert_post' hooks (and many others) need to make sure that they filter out 'nav_menu_item' posts which isn't always obvious to everyone.

1. wp_unique_post_slug(); the IMO ill-fated decision to allow values for the post_name field in trash override the values of newly created posts makes Nav Menu Items and other programmatic uses of posts a real data integrity nightmare. I wasted a literal month's worth of coding trying to use the WordPress menus in a system where my code generated the menus, with constant data integrity issues until I finally ripped it out and put in my own menu system and all the issues went away.

P.S. Further, none of the WordPress admin UI for posts is used for menus which is a strong hint that maybe the two things really are different.

FWIW.

I think there's a lot of API improvements that are needed for menus, and some may be limited by the current schema. Choosing a schema before planning a final or best-case API seems a little early.

I probably agree, but I'll need your help to clarify with examples for me to be sure.

[mbijon]

Nested sets are one name for recursive models with left & right ranges, plus possibly a depth column: ​http://en.wikipedia.org/wiki/Nested_set_model. There are a number of places that could make things more-efficient. Menus are one, comments are probably another. The downside is either new tables or violating your item #10 about extra fields in rows, and loads of new model code.

I'm surprised you didn't mention collisions in $priority for add_menu_page(). It's frustrating that we don't have a stack that can handle overlapping priorities.

Regardless, seems like a set of tickets to cover these and a bunch of small commits might at least refactor the API enough that a schema change could work. I'd help with some of those before trying to code a new schema.

[MikeSchinkel]

Replying to mbijon:

Nested sets are one name for recursive models with left & right ranges, plus possibly a depth column: ​http://en.wikipedia.org/wiki/Nested_set_model.

I was not aware of that, but it's definitely an algorithm to consider for future problem solving.

I'm surprised you didn't mention collisions in $priority for add_menu_page(). It's frustrating that we don't have a stack that can handle overlapping priorities.

I just have not run into a use-case where that's been an issue. That's why having experience from so many people is helpful because not everyone experiences all use-cases and without experience it's hard to appreciate why a solution is needed.

Regardless, seems like a set of tickets to cover these and a bunch of small commits might at least refactor the API enough that a schema change could work. I'd help with some of those before trying to code a new schema.

Yeah, agreed.