How set max partitions for Hive query in ipython notebook
<p>I am writing a script currently using Ipython notebook. </p>

<pre><code>import pandas as pd
import pyhs2
import os
import datetime

q1= "set hive.query.max.partition = 3000 ; 
select 'Device_id' as key,
'All Time' as type,
count(distinct a.dev_id) as count
from (select distinct dev_id from DevID
where dev_type = '*****' 
union all 
    select distinct
    key_value_lookup(raw_url, '*****',  '&amp;', '=') as dev_id
    from actions 
    where raw_url like '%*****%'
    and raw_url like '%*****%' 
    and data_date &gt;= '20150901' and data_date &lt;= '20151231') a"

def read_hive(query):
conn = pyhs2.connect(host='*****',
                   port=*****,
                   authMechanism="*****",
                   user='*****',
                   password='*****',
                   database='*****')
cur = conn.cursor()
cur.execute(query)
    #Return column info from query
if cur.getSchema() is None:
    cur.close()
    conn.close()
    return Nonea

columnNames = [a['columnName'] for a in  cur.getSchema()] 
print columnNames
columnNamesStrings = [a['columnName'] for a in  cur.getSchema() if a['type']=='STRING_TYPE'] 
output =  pd.DataFrame(cur.fetch(),columns=columnNames)   

cur.close()
conn.close()
return output
</code></pre>

<p>When calling <code>read_hive(q1)</code>, I receive the following error:</p>

<blockquote>
  <p>FAILED because hive.query.max.partition expects an INT value</p>
</blockquote>

<p>I believe that this is because I am storing the query in a string, but am not completely sure. The query runs perfectly well from Hue.</p>

<p>Does anyone have intuition into the best way to change the maximum amount of partitions? Can this be done in my function?</p>