Expand dataframe with dictionaries
<p>I've got a dataframe that contains a mostly NaN's, but also dictionaries in certain entries. My goal is expanding those dictionaries to columns of the dataframe and keeping their entries on their respective indices. 
This is what a small part of the dataframe looks like. </p>

<pre><code>                                                                       _id  _score 
query                                                        
chrM:g.146T&gt;C                                                            NaN  NaN  
chrM:g.11723C&gt;T                                                          NaN  NaN                                       NaN  NaN   
chrM:g.11813A&gt;G                                                          NaN  NaN   
chrM:g.12140T&gt;A                                                          NaN  NaN   
...                                                                      ...  ...     
chr1:g.11976370T&gt;G         {u'ref': u'T', u'alleles': [{u'allele': u'T', ...  NaN   
chr1:g.12007164A&gt;G                                                       NaN  NaN   
chr1:g.12007165A&gt;G                                                       NaN  NaN 
</code></pre>

<p>So far, I've just managed to pick the keys of each dict and add columns named with those keys:</p>

<pre><code>s ={}
for cols in cols:
    if type(data1[cols].dropna()[0]) == type(s):
       cols_var = np.array(data1[cols].dropna()[0].keys())
       data1 = pandas.concat([data1,pandas.DataFrame(columns=cols_var)])
</code></pre>

<p>Any help or hints on how to do so in an efficient and readable will be much appreciated.</p>

<p>**EDIT: ** this code:</p>

<pre><code>y = pandas.Series((dbsnp.iloc[0]))
print y
</code></pre>

<p>however retrieves something somewhat useful:</p>

<pre><code>allele_origin                   unspecified
alleles                         [{u'allele': u'G'}, {u'allele': u'A'}]      
alt                              A
...                              ...
rsid                             rs201327123
vartype                          snp
dtype: object
</code></pre>

<p>I'll try working from here, other input is much appreciated.</p>