<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Preprocessing
Step1:  Load the data, in this case a sample data. 
Working with a sample allow us to debug easier and faster!
Step2:  Make a single DataFrame containing all the users
Step3:  Drop useless column(test_users don't have it)
Step4:  Age
One of the most important features when making predictions, as we will see later, is the age. 
Being able to extract information about the users' age is essential to our future predictions.
Some users introduced a year instead of their age, translate this year into their age
Step5:  Set limits to age
Step6:  Fill NaNs with -1 to make it more noticiable
Step7:  The age, is really fine grained. We are going to make bins and fit each user in the proper age group
Step8:  NaNs
With this snippet we can get the number of unknown things from a given user. 
This can be usefull for the classifier when deciding where this user is booking.
Step9:  Date
Cast the dates to proper format
Step10:  Convert to DatetimeIndex
Step11:  Split dates into day, week, month, year
Step12:  Get the difference(time lag) between the date in which the account was created and when it was first active
Step13:  Drop duplicated columns
Step14:  Session Information
There is a lot of information in the sessions.csv file. We will focus on get a count of each user action and some
statistics about their seconds elapsed. Further preprocessing could be benneficial to the final predictions.
Step15:  Frequency Count
This simply counts the number of times an action is made by each user. One thing I learned from this 
competition is to really understand what you want to do and not begin to code an 
over-engineered 
solution(KISS).
Step16:  Elapsed Seconds Stats
Now we are going to extract some information about the seconds elapsed per user.
Since we don't know exactly what is secs_elapsed, I've assumed that are paused between actions
Step17:  Encode categorical features
Step18:  Persistence
Now that we have all the info processed we need to save it!
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import pandas as pd
Explanation: Preprocessing
End of explanation
train_users = pd.read_csv('../data/train_users_sample.csv')
test_users = pd.read_csv('../data/test_users_sample.csv')
sessions = pd.read_csv('../data/sessions_sample.csv')
Explanation: Load the data, in this case a sample data. 
Working with a sample allow us to debug easier and faster!
End of explanation
users = pd.concat([train_users, test_users], axis=0, ignore_index=True)
Explanation: Make a single DataFrame containing all the users:
End of explanation
users.drop('date_first_booking', axis=1, inplace=True)
Explanation: Drop useless column(test_users don't have it)
End of explanation
user_with_year_age_mask = users['age'] > 1000
users.loc[user_with_year_age_mask, 'age'] = 2015 - users.loc[user_with_year_age_mask, 'age']
Explanation: Age
One of the most important features when making predictions, as we will see later, is the age. 
Being able to extract information about the users' age is essential to our future predictions.
Some users introduced a year instead of their age, translate this year into their age:
End of explanation
users.loc[(users['age'] > 100) | (users['age'] < 18), 'age'] = -1
Explanation: Set limits to age:
End of explanation
users['age'].fillna(-1, inplace=True)
Explanation: Fill NaNs with -1 to make it more noticiable:
End of explanation
bins = [-1, 20, 25, 30, 40, 50, 60, 75, 100]
users['age_group'] = np.digitize(users['age'], bins, right=True)
Explanation: The age, is really fine grained. We are going to make bins and fit each user in the proper age group:
End of explanation
users['nans'] = np.sum([
    (users['age'] == -1),
    (users['gender'] == '-unknown-'),
    (users['language'] == '-unknown-'),
    (users['first_affiliate_tracked'] == 'untracked'),
    (users['first_browser'] == '-unknown-')
], axis=0)
Explanation: NaNs
With this snippet we can get the number of unknown things from a given user. 
This can be usefull for the classifier when deciding where this user is booking.
End of explanation
users['date_account_created'] = pd.to_datetime(users['date_account_created'], errors='ignore')
users['date_first_active'] = pd.to_datetime(users['timestamp_first_active'], format='%Y%m%d%H%M%S')
Explanation: Date
Cast the dates to proper format:
End of explanation
date_account_created = pd.DatetimeIndex(users['date_account_created'])
date_first_active = pd.DatetimeIndex(users['date_first_active'])
Explanation: Convert to DatetimeIndex:
End of explanation
users['day_account_created'] = date_account_created.day
users['weekday_account_created'] = date_account_created.weekday
users['week_account_created'] = date_account_created.week
users['month_account_created'] = date_account_created.month
users['year_account_created'] = date_account_created.year
users['day_first_active'] = date_first_active.day
users['weekday_first_active'] = date_first_active.weekday
users['week_first_active'] = date_first_active.week
users['month_first_active'] = date_first_active.month
users['year_first_active'] = date_first_active.year
Explanation: Split dates into day, week, month, year:
End of explanation
users['time_lag'] = (date_account_created.values - date_first_active.values).astype(int)
Explanation: Get the difference(time lag) between the date in which the account was created and when it was first active:
End of explanation
drop_list = [
    'date_account_created',
    'date_first_active',
    'timestamp_first_active'
]
users.drop(drop_list, axis=1, inplace=True)
Explanation: Drop duplicated columns:
End of explanation
sessions.rename(columns = {'user_id': 'id'}, inplace=True)
Explanation: Session Information
There is a lot of information in the sessions.csv file. We will focus on get a count of each user action and some
statistics about their seconds elapsed. Further preprocessing could be benneficial to the final predictions.
End of explanation
action_count = sessions.groupby(['id', 'action'])['secs_elapsed'].agg(len).unstack()
action_type_count = sessions.groupby(['id', 'action_type'])['secs_elapsed'].agg(len).unstack()
action_detail_count = sessions.groupby(['id', 'action_detail'])['secs_elapsed'].agg(len).unstack()
device_type_sum = sessions.groupby(['id', 'device_type'])['secs_elapsed'].agg(sum).unstack()
sessions_data = pd.concat([action_count, action_type_count, action_detail_count, device_type_sum],axis=1)
sessions_data.columns = sessions_data.columns.map(lambda x: str(x) + '_count')
# Most used device
sessions_data['most_used_device'] = sessions.groupby('id')['device_type'].max()
users = users.join(sessions_data, on='id')
Explanation: Frequency Count
This simply counts the number of times an action is made by each user. One thing I learned from this 
competition is to really understand what you want to do and not begin to code an 
over-engineered 
solution(KISS).
End of explanation
secs_elapsed = sessions.groupby('id')['secs_elapsed']
secs_elapsed = secs_elapsed.agg(
    {
        'secs_elapsed_sum': np.sum,
        'secs_elapsed_mean': np.mean,
        'secs_elapsed_min': np.min,
        'secs_elapsed_max': np.max,
        'secs_elapsed_median': np.median,
        'secs_elapsed_std': np.std,
        'secs_elapsed_var': np.var,
        'day_pauses': lambda x: (x > 86400).sum(),
        'long_pauses': lambda x: (x > 300000).sum(),
        'short_pauses': lambda x: (x < 3600).sum(),
        'session_length' : np.count_nonzero
    }
)
users = users.join(secs_elapsed, on='id')
Explanation: Elapsed Seconds Stats
Now we are going to extract some information about the seconds elapsed per user.
Since we don't know exactly what is secs_elapsed, I've assumed that are paused between actions:
End of explanation
categorical_features = [
    'gender', 'signup_method', 'signup_flow', 'language',
    'affiliate_channel', 'affiliate_provider', 'first_affiliate_tracked',
    'signup_app', 'first_device_type', 'first_browser', 'most_used_device'
]
users = pd.get_dummies(users, columns=categorical_features)
Explanation: Encode categorical features
End of explanation
users.set_index('id', inplace=True)
users.loc[train_users['id']].to_csv('../cache/train_users.csv')
users.loc[test_users['id']].drop('country_destination', axis=1).to_csv('../cache/test_users.csv')
Explanation: Persistence
Now that we have all the info processed we need to save it!
End of explanation
<END_TASK>