<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 PyTorch LSTM
Step1:  Paths
Step2:  Config parameters
Step3:  Data Loading
Step4:  experimenting with Fastai
Step5:  Misc / Asides / Notes
Aside 1
Labels are already binary encoded, so no need to numericalize. Therefore use_vocab=False.
<ASSISTANT_TASK:>
Python Code:
import pathlib
import os
import torchtext
# from torchtext.data import Field
from torchtext import data
# import spacy
import pandas as pd
import numpy as np
# from torchtext.data import TabularDataset
Explanation: PyTorch LSTM: GloVe + dropout --- Incomplete
This is a reimplementation of J.Howard's Improved LSTM baseline: GloVe + dropout Kaggle kernel in FastAI/PyTorch. The original kernel manages a private score of 0.09783 tied with 2747/4551 place.
-- Wayne Nixalo
Imports
End of explanation
data_path = pathlib.Path('../../data')
comp_path = pathlib.Path(data_path/'competitions/jigsaw-toxic-comment-classification-challenge')
EMBEDDING_FILE = 'glove/glove.6B.50d.txt'
TRAIN_DATA_FILE= 'train.csv'
TEST_DATA_FILE = 'test.csv'
Explanation: Paths
End of explanation
embed_sz = 50    # embedding vector columns (factors)
max_feat = 20000 # embedding vector rows    (words)
maxlen   = 100   # words in comment to use
Explanation: Config parameters
End of explanation
list_classes = ["toxic", "severe_toxic", "obscene", "threat", "insult", "identity_hate"]
# train = pd.read_csv(comp_path/TRAIN_DATA_FILE)
# test  = pd.read_csv(comp/TEST_DATA_FILE)
# SEE: Aside 1, Aside 2
# TEXT  = Field(sequential=True, tokenize='spacy', lower=True)
TEXT = data.Field(sequential=True, tokenize= lambda x: x.split(), lower=True, )
LABEL = data.Field(sequential=False, use_vocab=False)
# trainval_datafields = [("id",None),("comment_text",TEXT)]
# trainval_datafields.extend((clss, LABEL) for clss in list_classes)
# test_datafields = [("id",None), ("comment_text",TEXT)]
# train_dataset = data.TabularDataset(
#                     path=comp_path/TRAIN_DATA_FILE, format='csv',
#                     skip_header=True, fields=trainval_datafields, 
#                     sort_within_batch=True)
# test_dataset  = data.TabularDataset(
#                     path=comp_path/TEST_DATA_FILE, format='csv',
#                     skip_header=True, fields=test_datafields)
# # TEXT.build_vocab(train_dataset)
Explanation: Data Loading
End of explanation
from fastai.nlp import *
train_df = pd.read_csv(comp_path/TRAIN_DATA_FILE)
# SEE: Aside 3
model = LanguageModelData.from_dataframes(
            path=comp_path, field=TEXT, col="comment_text", 
            train_df=train_df, val_df=train_df, test_df=train_df, 
            bs=64, min_freq=3)
em_sz = 200
nh = 500
nl = 3
opt_fn = partial(optim.Adam, betas=(0.7, 0.99))
learner = model.get_model(opt_fn, em_sz, nh, nl, 
                          dropouti=0.05, dropout=0.05, wdrop=0.1, 
                          dropoute=0.02, dropouth=0.05)
learner.clip = 0.3 # gradient clipping
learner.model.parameters
Explanation: experimenting with Fastai
End of explanation
list_classes = ["toxic", "severe_toxic", "obscene", "threat", "insult", "identity_hate"]
train = pd.read_csv(comp_path/TRAIN_DATA_FILE)
# test  = pd.read_csv(comp/TEST_DATA_FILE)
train[list_classes][55:65]
Explanation: Misc / Asides / Notes
Aside 1
Labels are already binary encoded, so no need to numericalize. Therefore use_vocab=False.
End of explanation
<END_TASK>