requests library fails to POST properly but urllib succeeds [python]
<p>I am trying to POST a query to the RSCB PDB web services, as outlined <a href="http://www.rcsb.org/pdb/software/rest.do" rel="nofollow">here</a>.</p>

<p>I set up the url, and query as XML:</p>

<pre><code>import urllib.request as urllib
import requests

url = "http://www.rcsb.org/pdb/rest/search"

queryText = """
&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;orgPdbQuery&gt;
&lt;version&gt;B0907&lt;/version&gt;
&lt;queryType&gt;org.pdb.query.simple.ExpTypeQuery&lt;/queryType&gt;
&lt;description&gt;Experimental Method Search: Experimental Method=SOLID-STATE NMR&lt;/description&gt;
&lt;mvStructure.expMethod.value&gt;SOLID-STATE NMR&lt;/mvStructure.expMethod.value&gt;
&lt;/orgPdbQuery&gt;
"""
</code></pre>

<p>I then define two possible ways of POSTing this data:</p>

<pre><code>def query_old_fashioned(url, query_xml):
    req = urllib.Request(url, data=query_xml.encode())
    f = urllib.urlopen(req)
    result = f.read()
    return result.decode()


def query_with_requests(url, query_xml):
    response = requests.post(url, data=query_xml.encode())
    return response.text

# result = query_old_fashioned(url, queryText)
# result = query_with_requests(url, queryText)
</code></pre>

<p>With the first function, using good old fashioned urllib.request, I get the correct result - a list of 4 character strings.</p>

<p>With the second function, which as far as I can tell is doing <em>exactly</em> the same thing, I get a JSP error message HTML returned. This is that error message when displayed in a browser:</p>

<pre><code>type Exception report

message

description The server encountered an internal error that prevented it from fulfilling this request.

exception

java.lang.NullPointerException
    java.util.StringTokenizer.&lt;init&gt;(StringTokenizer.java:199)
    java.util.StringTokenizer.&lt;init&gt;(StringTokenizer.java:221)
    org.rcsb.servlet.RestfulServiceServlet.doPost(RestfulServiceServlet.java:1371)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
    org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    org.apache.struts2.dispatcher.ng.filter.StrutsPrepareAndExecuteFilter.doFilter(StrutsPrepareAndExecuteFilter.java:86)
    org.pdb.util.web.OutOfServiceFilter.doFilter(OutOfServiceFilter.java:91)
    org.pdb.util.web.DOSFilter.doFilter(DOSFilter.java:158)
    org.pdb.util.web.AntiRobotFilter.doFilter(AntiRobotFilter.java:29)
    org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
    org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
    org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
    org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)
note The full stack trace of the root cause is available in the Apache Tomcat/7.0.61 logs.
</code></pre>

<p>I know a little JSP, but I haven not been able to glean a reason for the POST's failure from this error message, nor is it clear to me why requests is failing but the standard library's urllib is succeeding. I've even tried going through the source code of the requests library on github trying to find exactly how requests creates its requests, but I was not successful in this endeavour.</p>

<p>This is using Python 3. I first encountered this problem using Ubuntu, and have since reproduced it on Windows 10.</p>

<p>Any help would be really appreciated.</p>