Make WordPress Core

Opened 13 years ago

Closed 12 years ago

Last modified 12 years ago

#18276 closed enhancement (wontfix)

Implement URL Routing System to "Front-End" WordPress' existing Rewrite System

Reported by: mikeschinkel's profile mikeschinkel Owned by:
Milestone: Priority: normal
Severity: normal Version: 3.2.1
Component: Permalinks Keywords: close
Focuses: Cc:

Description

As per scribu's reply to my comment on "WordPress 3.3 Proposed Scope" I am attaching a plugin I called "WP URL Routes" in file wp-url-routes.php which is a proof-of-concept that works as a plugin today even though it was designed to be integrated into core (i.e. it does not use function name prefixes.)

WP URL Routes uses register_query_var() and register_url_path() functions to build a tree of URL_Node objects where each URL Node ultimately contains the metadata required to represent a path segment where the path '/' is the root URL Node. It is designed so that path matching can be done not only with regular expressions like the WordPress Rewrite system but also using keyed arrays, database lookups, potentially specific callbacks, as well as global hooks. For high traffic systems WP URL Routes could be optimized in a variety of ways including using Memcached and more.

WP URL Routes is designed to be as easy as possible for a themer to add to a theme's function.php in an 'init' hook. Here is what such an 'init' hook might look like to define the oft requested /CATEGORY/POST/ url structure:

add_action( 'init', 'mysite_url_init' );
function mysite_url_init() {
  // Allow categories to be specified at the beginning of a path
  register_url_path( '%category_name%/%name%' );
}

A very important aspect of its architecture is that about the only thing that the WP URL Routes concept overrides is that it effectively replaces $wp->parse_request() by subclassing the WP class (which of course WordPress core need not do.) It actually might be easier just to show you the code for the relevant parts of the WP_Urls_WP class rather than try to explain it (see below):

/*
 * Extends the WP class in WordPress and assigns an instance to the global $wp variable.
 * Notes: This is needed because WordPress does not (yet?) have a hook for $wp->parse_request() as proposed in trac ticket #XXXXX
*/
class WP_Urls_WP extends WP {
  static function on_load() {
    // 'setup_theme' is 1st hook run after WP is created.
    add_action( 'setup_theme', array( __CLASS__, 'setup_theme' ) );
  }
  static function setup_theme() {
    global $wp;
    $wp = new WP_Urls_WP();  // Replace the global $wp
  }
  function parse_request( $extra_query_vars = '' ) {
    if ( apply_filters( 'wp_parse_request', false, $extra_query_vars ) ) {
      WP_Urls::$result = 'routed';
    } else {
      WP_Urls::$result = 'fallback';
      if ( WP_Urls::$fallback ) {
        parent::parse_request($extra_query_vars); // Delegate to WP class
      } else {
        wp_die( 'URL Routing failed.' );
      }
    }
    return;
  }
}
WP_Urls_WP::on_load();

WP URL Routes is currently designed to "front-end" the WordPress Rewrite system so that any URL path patterns defined take precedence over the standard rewrite system but if no URL path patterns match the HTTP request's URL then the WordPress Rewrite system takes over. And it is an option for a developer designing a custom CMS system with WordPress to bypass the WordPress Rewrite system completely on a failed match so they can fully control the URLs of their site (if the WordPress team chooses to use this proof-of-concept as a base for WordPress 3.3 URL routing they could conceivably have a "no-backward-compatibility" mode for URL routing that could be enabled with a constant in /wp-config.php.)

WP URL Routes allows URL paths to be defined using exactly the same URL path format found on the Permalinks page, i.e. %year%/%month%/%day% for example (although I don't think this route is implemented in my plugin just yet.) It also relies heavily on array_merge() to allow for several levels of meta data to finally be merged down to the actual metadata for each URL Node.

Let me illustrate with %pagename%. With the existing WordPress Rewrite system $wp->query_vars ends up with ['pagename'] being set to the URL slug (single or multi-path segments) and ['page'] is set to '' (this latter is unimportant, but we want to match WordPress' Rewrite behavior exactly.) So here is how we might define the %pagename% query variable and the %pagename% path:

add_action( 'init', 'mysite_url_init' );
function mysite_url_init() {
  register_query_var( '%pagename%', array(
    '@validate'       => 'is_valid_page', // Callback ,'@' means don't put into query_var
    '@multi_segment'  => true,            // Because /foo/bar/baz/ is a valid path
    'page'            => '',              // This is match WordPress' behavior
    'pagename'        => '%this%',        // '%this%' gets replaces by current path segement
  ));
  register_url_path( '%pagename%' );
}

But that requires a lot of learning on the themer's part so I hardcoded the meta data for %pagename% into the function register_query_var() on a switch-case statement; here's the case:

case 'pagename':
$defaults = array(
  '@validate'       => 'is_valid_page', // Callback ,'@' means don't put into query_var
  '@multi_segment'  => true,            // Because /foo/bar/baz/ is a valid path
  'page'            => '',              // This is match WordPress' behavior
  'pagename'        => '%this%',        // '%this%' gets replaces by current path segement
);
break;

Which means we can simplify it for the themer to be like this:

add_action( 'init', 'mysite_url_init' );
function mysite_url_init() {
  register_query_var( '%pagename%' );
  register_url_path( '%pagename%' );
}

Of course the plugin can check to see if the query variable '%pagename%' found in the path '%pagename%' has been registered yet and if not register it, so our 'init' hook simply becomes:

add_action( 'init', 'mysite_url_init' );
function mysite_url_init() {
  register_url_path( '%pagename%' );
}

Very easy for the themer, no? Of course, for the person that really needs power they can build all the metadata from scratch, but the functions register_query_var() and __register_url_path() contain all the default metadata for common query variables and common paths.

What you have here is a fully(?) working URL routing engine but only a handful of the standard query variables and standard paths have been defined yet. Remember, this is a proof-of-concept, not something ready to be included into WordPress core (though I'll be happy to help get it ready for core once I get agreement from the team that it is wanted.)

A few other points to argue for this approach:

  1. The core code is working today.


  1. Building a tree of URL path pattern nodes is a very close fit to the structure of the URL path that is it modeling. Can we really do better?
  1. It is very flexible in it's URL matching and does not rely solely on RegEx.


  1. It really fits the WordPress architecture because it's entire goal is to populate $wp->query_vars correctly, and nothing more.
  1. It integrates with existing WordPress architecture with a very tiny amount of changes; only URL rewrites are affected and probably only the lower level hooks.
  1. There will be very few hooks that will need to be deprecated and warnings can be generated for those hooks with WP_DEBUG is defined.


  1. The tree of URL Nodes also provides metadata needed for automated breadbrumb generation and for automated sitemap generation (both for XML Sitemaps and sitemaps for humans.)



To try it the attached plugin copy the wp-url-routes.php file into your site's /wp-content/plugins/ directory and then activate the "WP URL Routes" plugin. Also be sure to define('WP_DEBUG',true); in /wp-config.php

The two URL paths defined for the demo are '%category_name%/%name%' and '%pagename%' so type in any a URL that should match one of these and you should see "URL Routing Result: routed" displayed in the top left corner. What follows is the code for the test config you will find at the bottom of the wp-url-routes.php file:

/*
 * Define OMIT_URL_ROUTES_TEST_CONFIG if you want to omit this test configuration.
 */
if ( ! defined( 'OMIT_URL_ROUTES_TEST_CONFIG') ) {
  add_action( 'init', '_wp_url_routes_test_config' );
  function _wp_url_routes_test_config() {
    register_url_path( '%category_name%/%name%' );
    register_url_path( '%pagename%' );
  }
}


I'd love to see this used as a base to finally "fix" the URL routing in WordPress to make it performant and to provide full flexibility. Again, it's a proof-of-concept so it is just a starting point and we can evolve it significantly if needed. However, if the team is not interested I'll be publishing this on wordpress.org sometime in the next 90-120 days, albeit with a different name. But it really needs to be integrated with core and not be a plugin in order to provide full value to everyone who needs something like this.

Attachments (1)

wp-url-routes.php (23.3 KB) - added by mikeschinkel 13 years ago.
wp-url-routes.php

Download all attachments as: .zip

Change History (15)

@mikeschinkel
13 years ago

wp-url-routes.php

#1 @mikeschinkel
13 years ago

Related #17177, #16692, #16687, #15915, #12935

Also, I'll be at the WordPress Core Contributor Day on August 11 so can help work on this if there is interest.

Last edited 13 years ago by mikeschinkel (previous) (diff)

#2 follow-up: @scribu
13 years ago

Thanks for posting this.

It would be good if you could document all the args that register_query_var() and register_query_path() accept. You need to do this anyway, if you plan to release the plugin eventually. :)

#3 in reply to: ↑ 2 @mikeschinkel
13 years ago

Replying to scribu:

It would be good if you could document all the args that register_query_var() and register_query_path() accept. You need to do this anyway, if you plan to release the plugin eventually. :)

heh. Of course! I am actually working on a higher priority plugin right now but spent 1/2 day preparing this so I could upload here given that with WordPress 3.3 might actually address URL routing and I didn't want to miss the time window to have this considered. Given my current obligations t would be much better if I could do this in a few weeks; do you think that it needs to be done before then?

Last edited 13 years ago by mikeschinkel (previous) (diff)

#4 follow-up: @scribu
13 years ago

Well, according to the WP 3.3 schedule, the last date that this could be commited would be September 7th.

But note that this assumes that a patch is ready to go. The current code is nowhere near commit-ready.

And, since there's no documentation or unit tests, it's hard for others to start working on a patch.

I know that didn't really answer your question, but such is the way of predicting the future. :)

Anyway, since you managed to make this work as a plugin, it means we (assuming you release it on wp.org or github or bitbucket at some point) can take our time to polish and integrate it later, after some real-world usage.

#5 in reply to: ↑ 4 ; follow-up: @mikeschinkel
13 years ago

Replying to scribu:

I know that didn't really answer your question, but such is the way of predicting the future. :)

Sure, I understand. Your reply helps.

Anyway, since you managed to make this work as a plugin, it means we (assuming you release it on wp.org or github or bitbucket at some point) can take our time to polish and integrate it later, after some real-world usage.

What I'd like to see most is to release as a plugin and then get that real world usage you mention, and then integrate into WP core in 3.5 or later. But since it sounded like the team was going to tackle URL routing in 3.3 and if that might be done in a way that provides less flexibility and be incompatible with this approach I wanted to rush it forward. I had planned to turn this into a completed project in the next 90-120 days.

Of course is there is interested I can always work on it in San Fran on the 11th if there is not something else the team would like my help working on.

BTW, is there enough there for you to have any feedback yet?

#6 @scribu
13 years ago

Sure:

  • You use the $wp_urls global, when you could use two static properties on the WP_Urls class.
  • The switch statement doesn't belong in register_query_var(). Put it in a wrapper function.

Suggestion:

Separate attributes - '@validate' etc. - from actual query vars.

#7 in reply to: ↑ 5 @Otto42
13 years ago

Replying to mikeschinkel:

But since it sounded like the team was going to tackle URL routing in 3.3 and if that might be done in a way that provides less flexibility and be incompatible with this approach I wanted to rush it forward. I had planned to turn this into a completed project in the next 90-120 days.

I doubt "URL routing" will be tackled in 3.3. What was being discussed was mainly eliminating or reducing the performance penalty with verbose-page-rules.

See #16687.

#8 @goto10
13 years ago

  • Cc dromsey@… added

#9 @johnbillion
13 years ago

  • Cc johnbillion@… added

#10 @naomicbush
13 years ago

  • Cc naomicbush added

#11 @DrewAPicture
12 years ago

  • Cc xoodrew@… added

#12 follow-up: @sirzooro
12 years ago

  • Cc sirzooro added

Related: #11312, #10722.

Now we are at beginning of 3.4, so we can include this in scope discussion.

#13 in reply to: ↑ 12 @mikeschinkel
12 years ago

  • Keywords close added; dev-feedback has-patch removed
  • Resolution set to wontfix
  • Status changed from new to closed

Replying to sirzooro:

Related: #11312, #10722.

Now we are at beginning of 3.4, so we can include this in scope discussion.

Thanks.

I've done a lot of work with this approach on a client project which gave me a lots of insight. Because of the cons I've found I'm going to close this ticket and attempt to refocus people on ticket #16692 which would be the first thing needed to allow plugins to start working on this problem.

If we can get the proposed 'wp_parse_request' hook added to 3.4 I think we can see a lot of innovation in this space which could then be applied to WordPress starting with version 3.5 or later.

#14 @scribu
12 years ago

  • Milestone Awaiting Review deleted
Note: See TracTickets for help on using tickets.