WordPress.org

Make WordPress Core

Opened 9 years ago

Closed 5 years ago

Last modified 3 years ago

#7652 closed enhancement (fixed)

Merge all atom parsers

Reported by: westi Owned by: rmccue
Milestone: WordPress.org Priority: normal
Severity: normal Version:
Component: Import Keywords: has-patch needs-review gci westi-likes
Focuses: Cc:

Description

Currently we have two Atom Parsers. One for the Blogger Importer and one for our APP implementation.

We need to only have one!

Attachments (7)

merge-atomlibs.diff (6.5 KB) - added by westi 9 years ago.
First pass. Rip it out of the blogger importer and use atomlib, add the extra tag to atomlib
7652.diff (3.9 KB) - added by rmccue 7 years ago.
First pass at the APP create_post() conversion
7652-wpapp.diff (8.8 KB) - added by rmccue 7 years ago.
All of AtomParser converted to SimplePie in wp-app.php
7652-wpapp-done.diff (9.1 KB) - added by rmccue 7 years ago.
wp-app.php should be done now
7652-blogger.diff (18.3 KB) - added by rmccue 7 years ago.
Blogger Importer conversion
7652-separate.diff (9.2 KB) - added by rmccue 7 years ago.
Separate WP_SimplePieAtomPub_Item into class-feed.php
7652-refresh.patch (9.7 KB) - added by kurtpayne 6 years ago.
Refreshed for 3.4

Download all attachments as: .zip

Change History (32)

@westi
9 years ago

First pass. Rip it out of the blogger importer and use atomlib, add the extra tag to atomlib

#1 @westi
9 years ago

  • Keywords has-patch needs testing added
  • Status changed from new to assigned

That is very rough and needs some testing.

#2 @cavemonkey50
9 years ago

  • Cc ron@… added

#3 @westi
9 years ago

  • Keywords needs-patch added; has-patch removed

TODOs:

  • Change AtomLib so that parse can be passed the xml.
  • Change the Blogger importer to use a single AtomParser for the whole feed and iterate over the parsed Entries.

#4 @jacobsantos
9 years ago

  • Milestone changed from 2.7 to 2.8
  • Type changed from defect to enhancement

#5 @ryan
8 years ago

  • Milestone changed from 2.8 to 2.9

#6 @Denis-de-Bernardy
8 years ago

  • Component changed from General to AtomPub
  • Keywords needs testing removed
  • Milestone changed from 2.9 to Future Release

#7 @Denis-de-Bernardy
8 years ago

just for reference, there was some discussion about using simplepie for this one in IRC

#11 @westi
8 years ago

Related tickets: #4555 (MT and TypePad) , #7644 (MT), #7543 (TypePad)

#12 @rmccue
7 years ago

  • Owner changed from westi to rmccue

Taking this for Google Code-In.

I think I'll take the SimplePie route for this, as I have the most experience with it, and I think it's a natural match.

@rmccue
7 years ago

First pass at the APP create_post() conversion

#13 @rmccue
7 years ago

One thing I don't like about this patch is the hack to get it to work with SimplePie. SP expects a normal, Atom syndication feed (with the top-level element being feed), while this is AtomPub. The str_replace hack should work, but there's probably a better way to do it.

@rmccue
7 years ago

All of AtomParser converted to SimplePie in wp-app.php

@rmccue
7 years ago

wp-app.php should be done now

#14 @westi
7 years ago

  • Cc westi added
  • Keywords gci added

This is looking good.

Did you manage to get APE working enough to test this out?

Well done :-)

#15 @rmccue
7 years ago

  • Keywords changed from needs-patch, gci to needs-patch gci

Using the override, I was able to, however, it appears Ape doesn't use the credentials you pass to it. I'll see about fixing this bug in Ape (my Ruby is rather poor), but I doubt they accept patches often.

I'd still like to move this into the unit testing and rewrite the tests in PHP (which should be easy), but that's out-of-scope for this task/bug.

As for the importers, these have now been spun off into their respective plugins, so I'll be taking a look at these next.

#16 @rmccue
7 years ago

Regarding Ape results, the warnings generated are:

  • Summary changes - WP adds the "Read more" link. This is normal.
  • Content changes - Content is trimmed. I believe this is also normal behaviour.
  • "Server discarded foreign markup" - Not sure on this one.
  • Slug doesn't match - Normal for non-rewritten permalinks
  • "Published entry retains dangerous hyperlink: 'javascript:evil'." - kses problem?
  • "Pictures apparently not in collection" - Did not touch this code, so should be fine.

@rmccue
7 years ago

Blogger Importer conversion

#17 @rmccue
7 years ago

  • Keywords has-patch needs-review added; needs-patch removed

This patch converts the Blogger Importer to use SimplePie, and I believe it completes this ticket. Still existing, but out-of-scope for this task are #4555, #7644 and #7543.

One part of this that will need to be tested is the timestamp handling. I'm not entirely sure I nailed that down fully in this importer, and I may have missed something (especially if the timezones from the imported blog and the WP blog don't match).

Due to the impracticality of writing unit tests for these (as doing so would require access to Blogger's servers), the only testing that can be done is manual testing. Unit testing may be possible for the Atom API, and I'll investigate that in the next day until my deadline expires. If I haven't completed that, I'll file a new issue for it. :)

@rmccue
7 years ago

Separate WP_SimplePieAtomPub_Item into class-feed.php

#18 @westi
7 years ago

  • Keywords 3.2-early added

Great work!

#19 @mcepl
7 years ago

  • Cc mcepl added

#20 @westi
6 years ago

  • Keywords westi-likes added; 3.2-early removed

There is great work on this ticket but it is going to need a fair chunk of testing so I don't think we should try and merge it in the 3.2 cycle - if I get time I will look at merging the Blogger importer changes into the plugin though

@kurtpayne
6 years ago

Refreshed for 3.4

#21 @kurtpayne
6 years ago

  • Cc kpayne@… added

Refreshed rmccue's patched for 3.4 and ran Gorilla against it. Looks like everything passed.

C:\Gorilla\bin>gorilla --user=admin --pass=******** --uri=http://127.0.0.1/wordpress-trunk/wp-app.php ServiceDocumentTest
Gorilla is beginning testing on http://127.0.0.1/wordpress-trunk/wp-app.php...


testServiceDocumentExists with data set #0
------------------------------------------
[info] Service document found
[result] success

testCollectionsExist with data set #0
-------------------------------------
[info] Collections found:
    Test Blog Posts accepts application/atom+xml;type=entry
    Test Blog Media accepts image/*, audio/*, video/*
[result] success

testHasJPEGCollection with data set #0
--------------------------------------
[info] JPEG collections found:
    Test Blog Media
[result] success

C:\Gorilla\bin>gorilla --user=admin --pass=******** --uri=http://127.0.0.1/wordpress-trunk/wp-app.php EntryPostsTest
Gorilla is beginning testing on http://127.0.0.1/wordpress-trunk/wp-app.php...


testListPosts
-------------
[info] Post collections found:
    Test Blog Posts
[info] Using Test Blog Posts at http://127.0.0.1/wordpress-trunk/wp-app.php/posts
[info] Checking current
[result] success

testSparsePost
--------------
[info] Testing basics
[info] Posting of new entry reported success, location: http://127.0.0.1/wordpress-trunk/wp-app.php/post/1802
[result] success

testPostFromTemplate with data set #0
-------------------------------------
[info] Creating from Gorilla/lib/tests/data/posts/example-1.xml
[info] Posting of new entry reported success, location: http://127.0.0.1/wordpress-trunk/wp-app.php/post/1804
[info] Examining the new entry as returned in the POST response
[info] Slug was used in server-generated URI
[info] Checking at the new location
[result] success

#22 @Workshopshed
5 years ago

So did WP_SimplePieAtomPub_Item make it into the latest 3.4 builds as I don't see it.

#23 @Otto42
5 years ago

  • Resolution set to fixed
  • Status changed from accepted to closed

Blogger importer now uses SimplePie as of 0.5.

#24 @ocean90
5 years ago

  • Component changed from AtomPub to Import
  • Milestone changed from Future Release to WordPress.org

#25 @chriscct7
3 years ago

#7644 was marked as a duplicate.

Note: See TracTickets for help on using tickets.