log transformation and index changing in python
<p>I'm writing a helper class function for log transformation. It needs to:</p>

<ol>
<li>Apply log to each column variable. </li>
<li>Name this newly generated variable, "log_variable". For instance, for variable 'InqCnt06', I need to create a new variable called "log_InqCnt06".</li>
<li>Do log(variable_value +1) for values in df[variables] columns that are zero or missing, to avoid getting "-inf" returned. </li>
<li>Find index of original variable.</li>
<li>Add <code>indexplus</code> to the index of the original variable, and make this the index for "log_variable". </li>
</ol>

<p>How would I do <code>log(variable_value +1)</code> for missing or zero values? I found the values using <code>isnull()</code>, could I use the <code>where.isnull()</code> function somehow?</p>

<p>I'm also having difficulty adding original variable index to indexplus, and assigning it to index of log_variable. </p>

<p>Here's my code: </p>

<pre><code>def add_log_transform(df, variable, indexplus = 1):
        print df[variable].isnull()
        log_variable = (np.log(df[variable]))
def main():
    variables_needs_tranform = ['DerogCnt', 'CollectCnt', 'InqCnt06', 'InqTimeLast', 'InqFinanceCnt24', 'TLTimeFirst', 'TLTimeLast', 'TLCnt03', 'TLCnt12', 'TLCnt24', 'TLCnt', 'TLSum', 'TLMaxSum', 'TLDel60Cnt', 'TLBadCnt24', 'TL75UtilCnt', 'TL50UtilCnt', 'TLBalHCPct', 'TLSatPct', 'TLDel3060Cnt24', 'TLDel90Cnt24', 'TLDel60CntAll', 'TLBadDerogCnt', 'TLDel60Cnt24', 'TLOpen24Pct']
util.add_log_transform(df, variables_needs_tranform, indexplus = 1)

if __name__ == "__main__":
        main()
</code></pre>

<p>This is a sample of my df. </p>

<pre><code>    DerogCnt  CollectCnt  InqCnt06  InqTimeLast  InqFinanceCnt24  TLTimeFirst  \
0          1           1         7            1                4          125   
1          1           1         2            1                0          252   
2          0           0         1            1                4          254   
3          0           0         6            3                6          154   
4          0           0         1            0                1          311   
5          1           1         1            1                2          200   
6          2           0         3            1                2          137   
7          0           0         3            2                1          267   
8          0           0         0           16                3          359   
9          1           1         1            6                3          141   
10         0           0         1            4                1          487   
11         0           0         4            1                4           78   
12         3           3         4            0                5          117   
13         2           2         1            0                4          101   
14         1           1         2            1                8          260   
15         0           0         5            1                6          295   
16         0           0         0          NaN                0          223   
17        15           9         3            1                3           14   
18         0           0         3            2                7          345   
19         0           0         4            4                5          145   
20         0           0         3            1                3          222   
21         0           0         1            5                1          354   
22         8           8         0            8                2          111   
23         0           0         1            1                3           56   
24         8           5         6            1               10           16   

TLTimeLast  TLCnt03  TLCnt12  TLCnt24     ...       TL75UtilCnt  \

0            3        1        3        5     ...                 3   
1           18        0        0        2     ...                 1   
2           12        0        1        2     ...                 3   
3            3        1        9       11     ...                 4   
4           17        0        0        1     ...                 2   
5            7        0        1        2     ...                 1   
6           12        0        1        2     ...                 2   
7            2        3        6        8     ...                 1   
8           23        0        0        1     ...                 0   
9            8        0        1        4     ...                 7   
10          12        0        1        6     ...                 9   
11           4        0        7       10     ...                 1   
12          27        0        0        0     ...                 3   
13          12        0        1        1     ...                 3   
14          16        0        0        2     ...                 2   
15           8        0        4       11     ...                 9   
16          18        0        0        1     ...                 2   
17          14        0        0        1     ...                 1   
18           2        2        2        4     ...                 3   
19           3        1        2        2     ...                 2   
20          22        0        0        1     ...                 1   
21           7        0        2       10     ...                 4   
22          24        0        0        1     ...                 0   
23          40        0        0        0     ...                 0   
24           4        0        2        4     ...                 2   

TL50UtilCnt  TLBalHCPct  TLSatPct  TLDel3060Cnt24  TLDel90Cnt24  \
0             4        0.85      0.67               0             0   
1             2        0.48      0.30               0             1   
2             3        0.84      0.67               0             1   
3             5        0.73      0.76               0             1   
4             3        0.88      0.63               0             0   
5             1        0.13      0.25               1             1   
6             2        0.70      0.64               0             0   
7             1        0.47      0.42               0             1   
8             2        0.41      0.69               0             0   
9             7        0.84      0.67               0             0   
10           10        0.77      0.82               0             0   
11            2        0.79      0.74               0             0   
12            3        0.92      0.21               2             3   
13            4        0.90      0.22               3             0   
14            3        0.80      0.35               0             1   
15           12        0.47      0.73               0             0   
16            4        0.89      0.57               3             0   
17            1        0.80      0.00               0             0   
18            3        0.86      0.52               0             0   
19            3        0.59      0.41               0             2   
20            2        0.25      0.62               2             0   
21            5        0.36      0.65               0             1   
22            0        0.38      0.40               0             0   
23            0        0.28      0.20               3             1   
24            3        0.91      0.25               1             1   

TLDel60CntAll  TLBadDerogCnt  TLDel60Cnt24  TLOpen24Pct  

0               1              0             0         0.71  
1               4              2             1         0.50  
2               1              1             1         0.33  
3               1              1             1         1.22  
4               1              0             0         0.20  
5               3              1             1         0.67  
6               0              2             0         0.40  
7               2              1             1         2.00  
8               2              0             0         0.17  
9               0              1             0         0.44  
10              0              0             0         0.46  
11              0              0             0         1.67  
12              7              4             4         0.00  
13              4              1             2         0.17  
14              1              1             1         0.67  
15              0              0             0         0.28  
16              5              0             2         0.17  
17              0             12             0         1.00  
18              0              0             0         0.67  
19              7              2             2         0.25  
20              3              0             1         0.10  
21              1              1             1         0.53  
22              0              4             0         0.50  
23              4              1             3         0.00  
24              1              7             1         1.33  
</code></pre>