WordPress.org

Make WordPress Core

Opened 6 years ago

Closed 6 years ago

Last modified 4 years ago

#6701 closed defect (bug) (invalid)

XmlRpc responses truncated due to UTF-8 BOM

Reported by: aaron.oneal Owned by:
Milestone: Priority: normal
Severity: normal Version: 2.5
Component: XML-RPC Keywords: UTF-8 BOM Live Writer
Focuses: Cc:

Description

Responses for XmlRpc are truncated due to the presence of the UTF-8 BOM. The BOM adds an additional 3 bytes to the beginning of the response, but the content length for the HTTP response is not updated to take that into account. This results in the last 3 bytes of the response getting cut off which results in malformed XML. This causes problems with Windows Live Writer, for example.

This is easily witnessed using NetMon to capture traffic.

  Frame: 
+ Ethernet: Etype = Internet IP (IPv4)
+ Ipv4: Next Protocol = TCP, Packet ID = 18565, Total IP Length = 821
+ Tcp: Flags=...PA..., SrcPort=HTTP(80), DstPort=24091, Len=781, Seq=370821372 - 370822153, Ack=995613184, Win=64808 (scale factor not found)
- Http: Response, HTTP/1.1, Status Code = 200
  - Response: 
     ProtocolVersion: HTTP/1.1
     StatusCode: 200, Ok
     Reason: OK
     ContentLength:  551
     ContentType:  text/xml
     Server:  Microsoft-IIS/6.0
     XPoweredBy:  PHP/5.2.4
     MicrosoftOfficeWebServer:  5.0_Pub
     XPoweredBy:  ASP.NET
     Date:  Sat, 12 Apr 2008 19:37:29 GMT
     Connection:  close
     HeaderEnd: CRLF
  - payload: HttpContentType =  text/xml
   - XmlPayload: 
      
      <?xml version="1.0"?>
    - <methodResponse>
     - <params>
      - <param>
       - <value>
        - <array>
         - <data>
          - <value>
           - <struct>
            - <member>
             - <name>
                isAdmin
                </name>
             - <value>
              - <boolean>
                 1
                 </boolean>
                </value>
               </member>
            + <member>
            - <member>
             - <name>
                blogid
                </name>
             + <value>
               </member>
            + <member>
              </struct>
             </value>
            </data>
           </array>
          </value>
         </param>
        </params>
       </methodRespons

The closing tag is truncated.

I fixed this in my local copy by updating class-IXR.php to add 3 bytes to the string length used to populate the Content-Length.

function output($xml) {
        $xml = '<?xml version="1.0"?>'."\n".$xml;
        $length = strlen($xml)+3;
        header('Connection: close');
        header('Content-Length: '.$length);
        header('Content-Type: text/xml');
        header('Date: '.date('r'));
        echo $xml;
        exit;
    }

Change History (11)

comment:1 josephscott6 years ago

  • Cc josephscott added

We need to find out why your server is adding the UTF-8 BOM to the XML-RPC responses. This is not something to happens on every server (this is the first time I've seen it at all) so having WordPress always add an additional three bytes to the response length calculation isn't a good solution.

comment:2 azaozz6 years ago

You've mentioned Live Writer, could that be something added by it when importing UTF-8 feeds? Most Windows text editors add BOM to UTF-8 files, including Notepad...

In any case it's better to strip the BOM than to increase the Content-Length header. Something like this would work:

$xml = preg_replace("/^\x{feff}/u", "", $xml);

comment:3 jcheng6 years ago

So THAT'S what causes this! I've seen maybe half a dozen WP installations where every XML-RPC response ends with

</methodRespons

Could it be a WP plugin that has a BOM inserted before the opening <?php ??

comment:4 follow-up: aaron.oneal6 years ago

I'm not sure why the BOM is being inserted, but agreed, the length fix isn't a great solution. As for where the BOM is coming from, what you're seeing is what was transferred over the wire, so Live Writer isn't inserting it, that's what the server is sending back in the response.

As for stripping the BOM out of the XML, I will try that, but since it doesn't appear in the code assembling the string, I suspect it won't find one to remove. Could echo be adding it?

comment:5 in reply to: ↑ 4 azaozz6 years ago

Replying to aaron.oneal:

I'm not sure why the BOM is being inserted, but agreed, the length fix isn't a great solution. As for where the BOM is coming from, what you're seeing is what was transferred over the wire, so Live Writer isn't inserting it, that's what the server is sending back in the response.

As for stripping the BOM out of the XML, I will try that, but since it doesn't appear in the code assembling the string, I suspect it won't find one to remove. Could echo be adding it?

Definitely IIS is adding it, not Live Writer. I doubt it echo would add it. Check if any output buffers are used in php (in php.ini), IIS might be adding it when flushing the buffer.

comment:6 josephscott6 years ago

  • Resolution set to invalid
  • Status changed from new to closed

Since this is a problem outside the control of WordPress I'm closing the ticket at this point. If someone does find a solution to this in IIS wouldn't hurt to post a fix here.

comment:7 josephscott6 years ago

  • Milestone 2.7 deleted

comment:8 archon8104 years ago

  • Cc admin@… added

I started experiencing the same issue recently, using Apache and WLW. My response is cut off to "</methodResponse".

No idea what is causing it yet.

comment:9 archon8104 years ago

The culprit turned out to be the latest version of Lazyest gallery. Plugin author contacted.

comment:10 macbrink4 years ago

@archon810
I have just published three items using WLW on my testblog that has Lazyest GAllery installed. No problems whatsoever. Also, I can't think of anything in LG that would interfere with XMLRpc publishing and would add a BOM into the response....

comment:11 macbrink4 years ago

The Lazyest Gallery error has been solved.
It turned out to be a

headers already sent

warning. Because of an extra line after the closing php tag in one of the files in the trunk.

Note: See TracTickets for help on using tickets.