<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 L1/L2 Code Along IIA
Step1:  Testing
Above training stopped midway through 8th epoch. Time for testing
Step2:  Experimenting with saving as Pandas.DataFrame.to_feather(.)
Step3:  Another way to create submission file
Step4:  Creating FileLink
Step5:  
<ASSISTANT_TASK:>
Python Code:
%reload_ext autoreload
%autoreload 2
%matplotlib inline
from fastai.imports import *
from fastai.transforms import *
from fastai.conv_learner import *
from fastai.model import *
from fastai.dataset import *
from fastai.sgdr import *
from fastai.plots import *
PATH = "data/dogscats/"
sz=224
ARCH = vgg16
bs = 16
# Uncomment if need to reset precomputed activations
!rm -rf {PATH}tmp
data = ImageClassifierData.from_paths(PATH, bs=bs, tfms=tfms_from_model(ARCH, sz))
learn = ConvLearner.pretrained(ARCH, data, precompute=True)
learn.fit(0.01, 3, cycle_len=1)
tfms = tfms_from_model(ARCH, sz, aug_tfms=transforms_side_on, max_zoom=1.1)
data = ImageClassifierData.from_paths(PATH, tfms=tfms, bs=bs, num_workers=4)
learn = ConvLearner.pretrained(ARCH, data, precompute=True)
learn.fit(lrs=1e-2, n_cycle=2)
learn.save('vgg16_00')  # just in case I run out of memory below
learn.precompute=False
learn.fit(lrs=1e-2, n_cycle=1, cycle_len=1)
learn.save('vgg16_01')
learn.unfreeze()
learn.data.bs = 4 # training ConvNets takes lots of Memory, cut down bs to prevent crashes
lr = np.array([1e-4, 1e-3, 1e-2])
learn.fit(lrs=lr, n_cycle=1, cycle_len=1)
learn.save('vgg16_02')
learn.lr_find()
learn.fit(lrs=lr, n_cycle=3, cycle_len=1, cycle_mult=2)
learn.save('vgg16_03')
learn.fit(lrs=lr, n_cycle=3, cycle_len=3)
learn.save('vgg16_04')
Explanation: L1/L2 Code Along IIA: lesson1.vgg
Replicating: https://github.com/fastai/fastai/blob/master/courses/dl1/lesson1-vgg.ipynb
Image Classification w/ Convolutional Neural Networks
End of explanation
%reload_ext autoreload
%autoreload 2
%matplotlib inline
from fastai.imports import *
from fastai.transforms import *
from fastai.conv_learner import *
from fastai.model import *
from fastai.dataset import *
from fastai.sgdr import *
from fastai.plots import *
PATH = "data/dogscats/"
sz=224
ARCH = vgg16
bs = 16
data = ImageClassifierData.from_paths(PATH, bs=bs, tfms=tfms_from_model(ARCH, sz), 
                                      test_name = 'test1')
# No reason to precompute activations as I'm running a single prediction run on the test set
# Also, since I trained all ConvLayers earlier... where will it know how to compute
# the activations if I haven't loaded the weights yet?
learn = ConvLearner.pretrained(ARCH, data, precompute=False)
# the test data set
len(learn.data.test_dl.dataset)
learn.load('vgg16_04')
# This took about 32 minutes
log_preds = learn.TTA(n_aug=4, is_test=True)[0]
Explanation: Testing
Above training stopped midway through 8th epoch. Time for testing:
End of explanation
log_preds_df = pd.DataFrame(log_preds, columns=['dog','cat'])
log_preds_df.to_feather(PATH + 'results/' + 'log_preds')
# log_preds_df = pd.read_feather(PATH + 'results/' + 'log_preds')
test_preds = np.exp(log_preds)
ids = [f[6:-4] for f in learn.data.test_dl.dataset.fnames]
preds = [np.argmax(pred) for pred in test_preds]
submission = pd.DataFrame({'id': ids, 'label': preds})
Explanation: Experimenting with saving as Pandas.DataFrame.to_feather(.)
End of explanation
submission = pd.DataFrame(preds)
submission.columns = ['label']
submission.insert(0, 'id', ids)
submission.head()
Explanation: Another way to create submission file:
End of explanation
submission.to_csv(PATH + 'subm/' + 'submission_vgg16_04.gz', compression='gzip', index=False)
FileLink(PATH + 'subm/' + 'submission_vgg16_04.gz')
Explanation: Creating FileLink:
End of explanation
%reload_ext autoreload
%autoreload 2
%matplotlib inline
from fastai.imports import *
from fastai.transforms import *
from fastai.conv_learner import *
from fastai.model import *
from fastai.dataset import *
from fastai.sgdr import *
from fastai.plots import *
PATH = "data/dogscats/"
sz=224
ARCH = vgg16
bs = 16
data = ImageClassifierData.from_paths(PATH, bs=bs, tfms=tfms_from_model(ARCH, sz), 
                                      test_name = 'test1')
# No reason to precompute activations as I'm running a single prediction run on the test set
# Also, since I trained all ConvLayers earlier... where will it know how to compute
# the activations if I haven't loaded the weights yet?
learn = ConvLearner.pretrained(ARCH, data, precompute=False)
learn.load('vgg16_04')
log_preds, y = learn.TTA()
accuracy(log_preds, y)
df = pd.read_feather(PATH + 'results/' + 'log_preds')
df.head()
preds = np.array(list(zip(*(df['dog'],df['cat']))))
preds = np.exp(preds)
preds = [np.argmax(pred) for pred in preds]
new_preds = [int(i==0) for i in preds]
new_preds[:10]
ids = [f[6:-4] for f in learn.data.test_dl.dataset.fnames]
submission = pd.DataFrame({'id': ids, 'label': new_preds})
submission.to_csv(PATH + 'subm/' + 'submission_vgg16_04_wtf.gz', compression='gzip', index=False)
FileLink(PATH + 'subm/' + 'submission_vgg16_04_wtf.gz')
log_preds_df = np.array(list(zip(*(df['dog'],df['cat']))))
test_preds = np.exp(log_preds_df)
test_preds = np.clip(test_preds, 0.05, 0.95)
data.classes
labels = test_preds[:,1]
labels[:10]
learn.data.test_dl.dataset.fnames
ids = [f[6:-4] for f in learn.data.test_dl.dataset.fnames]
submission = pd.DataFrame({'id': ids, 'label': labels})
submission.to_csv(PATH + 'subm/' + 'submission_vgg16_04_omg.csv.gz', compression='gzip', index=False)
FileLink(PATH + 'subm/' + 'submission_vgg16_04_omg.csv.gz')
Explanation: 
End of explanation
<END_TASK>