<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Interlude...
At this point, I check out the raw_descr sheet and (at a bare minimum) set the keep_flag to indicate whether or not I want to drop the column when processing the dataset. I may also revisit options passed to pd.read_csv() as needed (ex. indicate which column should be used as the index).
Next, I populate the metadata sheet by first ensuring that all columns from the raw_descr sheet are listed under column_name, and then populate the configuration details for each column.
Once I am satisfied with the configuration in the metadata sheet, I load the metadata so that I can pass the most up-to-date processing logic to the clean_data function.
Step1:  APPENDIX
<ASSISTANT_TASK:>
Python Code:
# Get most up-to-date processing logic as configured in the metadata sheet
md = ss.get('metadata', headerrow=1)
# clean df_raw, describe df_clean, create custom sheet range to control where data is written in sheet
df_clean = p.process.clean_data(df_raw, md)
clean_descr = p.create_description(df_clean)
clean_descr_sheet_range = "".join(['clean_descr!A:', (chr(ord('A') + raw_descr.shape[1] - 1))])
# For the sheet range, you can also just pass the sheet name (ex. 'clean_descr')
# if you don't need finite control over where data is written within a sheet
ss.update(clean_descr, clean_descr_sheet_range)  # Check `clean_descr` sheet in spreadsheet to see df_clean description
ss.sheets['clean_descr'] = ss.get('clean_descr')
innocentive_dummified = pd.get_dummies(df_clean)
print(innocentive_dummified.shape)
innocentive_dummified.__name__ = 'innocentive_dummified'
innocentive_dummified_descr = p.create_description(innocentive_dummified)
ss.update(innocentive_dummified_descr, 'dummified_descr')
ss.sheets['dummified_descr'] = ss.get('dummified_descr')
# use a 15% subset of training data to use as an example
subset = innocentive_dummified.sample(n=(int(innocentive_dummified.shape[0]*.15)))
subset.shape
# Pickle dataset for later use
p.utils.pckl(subset, "innocentive_dummified")
Explanation: Interlude...
At this point, I check out the raw_descr sheet and (at a bare minimum) set the keep_flag to indicate whether or not I want to drop the column when processing the dataset. I may also revisit options passed to pd.read_csv() as needed (ex. indicate which column should be used as the index).
Next, I populate the metadata sheet by first ensuring that all columns from the raw_descr sheet are listed under column_name, and then populate the configuration details for each column.
Once I am satisfied with the configuration in the metadata sheet, I load the metadata so that I can pass the most up-to-date processing logic to the clean_data function.
End of explanation
#SOURCE_CSV = os.path.join(os.environ['DATADIR'], 'data.csv.gz')
#SOURCE_CSV = "https://storage.cloud.google.com/prospecting/data/innocentive/InnoCentive_Challenge_9933493_training_data.csv.gz"
Explanation: APPENDIX
End of explanation
<END_TASK>