Curl HTTP Posts and 100-continue problem solutions

 

I had to work with a PHP-application using the CURL library to send a HTTP POST request to a lighttpd server or apache server.

 

Strangely enough I seemed unable to get anything back from the server when using PHP and I got the correct answer when I was using wget as a reference.

 

This made me check the lightpd log and I once more (I recommend you to read that entry as this is very much dependent on it) came across the friendly error 417

A quick check with Wireshark confirmed: curl was sending the Expect: 100-continue header.

Personally, I think that 100-continue thing is a good thing and it even seems to me that the curl library is intelligent about it and only does that thing when the size of the data to send is larger than a certain threshold.

 

Also, even though people are complaining about it, I think lighttpd does the right thing. The expect-header is mandatory and if lighttpd doesn't support this particular header, the error 417 is the only viable option.

 

What I think though is that the libraries should detect that automatically.

This is because they are creating a behavior that's not consistent to the other types of request: GET, DELETE and HEAD requests all follow a fire-and-forget paradigm and the libraries employ a 1:1 mapping: Set up the request. Send it. Return the received data.

 

With POST (and maybe PUT), the library changes that paradigm and in fact sends two request to the wire while actually pretending in the interface that it's only sending one request.

If it does that, then it should at least be capable enough to handle the cases where their scheme of transparently changing semantics breaks.

 

If you are doing a POST, and the content length is 1,025 or greater, then curl exploits a feature of http 1.1: 100 (Continue) Status.
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.2.3

* it adds a header, "Expect: 100-continue". 
* it then sends the request head, waits for a 100 response code, then sends the content

Not all web servers support this though.  Various errors are returned depending on the server.  If this happens to you, suppress the "Expect" header with this command:

<?php
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));
?>
See http://www.gnegg.ch/2007/02/the-return-of-except-100-continue/

 

Also, you should check your libcurl version. whether it's the latest 0.7 or not? if not, please use the latest one to your site.