<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <a href="https
Step1:  Environment Preparation
Install bigdl-chronos
You can install the latest pre-release version with chronos support using pip install --pre --upgrade bigdl-chronos[all].
Step2:  Step 0
Step3:  Time series forecasting using Chronos Forecaster
Forecaster Step1. Data transformation and feature engineering using Chronos TSDataset
TSDataset is our abstract of time series dataset for data transformation and feature engineering. Here we use it to preprocess the data.
Step4:  Initialize train, valid and test tsdataset from raw pandas dataframe.
Step5:  Preprocess the datasets. Here we perform
Step6:  Forecaster Step 2
Step7:  Forecaster Step 3
Step8:  Save & restore the forecaster.
<ASSISTANT_TASK:>
Python Code:
#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Explanation: <a href="https://colab.research.google.com/github/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_autots_nyc_taxi.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
Copyright 2016 The BigDL Authors.
End of explanation
# Install latest pre-release version of bigdl-chronos
# Installing bigdl-chronos from pip will automatically install pyspark, bigdl, and their dependencies.
!pip install --pre --upgrade bigdl-chronos
!pip uninstall -y torchtext # uninstall torchtext to avoid version conflict
exit() # restart the runtime to refresh installed pkg
Explanation: Environment Preparation
Install bigdl-chronos
You can install the latest pre-release version with chronos support using pip install --pre --upgrade bigdl-chronos[all].
End of explanation
# download the dataset
!wget https://raw.githubusercontent.com/numenta/NAB/v1.0/data/realKnownCause/nyc_taxi.csv
# load the dataset. The downloaded dataframe contains two columns, "timestamp" and "value".
import pandas as pd
df = pd.read_csv("nyc_taxi.csv", parse_dates=["timestamp"])
Explanation: Step 0: Download & prepare dataset
We used NYC taxi passengers dataset in Numenta Anomaly Benchmark (NAB) for demo, which contains 10320 records, each indicating the total number of taxi passengers in NYC at a corresonponding time spot.
End of explanation
from bigdl.chronos.data import TSDataset
from sklearn.preprocessing import StandardScaler
Explanation: Time series forecasting using Chronos Forecaster
Forecaster Step1. Data transformation and feature engineering using Chronos TSDataset
TSDataset is our abstract of time series dataset for data transformation and feature engineering. Here we use it to preprocess the data.
End of explanation
tsdata_train, tsdata_valid, tsdata_test = TSDataset.from_pandas(df, dt_col="timestamp", target_col="value",
                                                                with_split=True, val_ratio=0.1, test_ratio=0.1)
Explanation: Initialize train, valid and test tsdataset from raw pandas dataframe.
End of explanation
lookback, horizon = 6, 1
scaler = StandardScaler()
for tsdata in [tsdata_train, tsdata_valid, tsdata_test]:
    tsdata.deduplicate()\
          .impute()\
          .gen_dt_feature()\
          .scale(scaler, fit=(tsdata is tsdata_train))\
          .roll(lookback=lookback, horizon=horizon)
Explanation: Preprocess the datasets. Here we perform:
- deduplicate: remove those identical data records
- impute: fill the missing values
- gen_dt_feature: generate feature from datetime (e.g. month, day...)
- scale: scale each feature to standard distribution.
- roll: sample the data with sliding window.
For forecasting task, we will look back 3 hours' historical data (6 records) and predict the value of next 30 miniutes (1 records).
We perform the same transformation processes on train, valid and test set.
End of explanation
from bigdl.chronos.forecaster.tcn_forecaster import TCNForecaster
x, y = tsdata_train.to_numpy()
# x.shape = (num of sample, lookback, num of input feature)
# y.shape = (num of sample, horizon, num of output feature)
forecaster = TCNForecaster(past_seq_len=lookback,  # number of steps to look back
                           future_seq_len=horizon,  # number of steps to predict
                           input_feature_num=x.shape[-1],  # number of feature to use
                           output_feature_num=y.shape[-1],  # number of feature to predict
                           seed=1)
res = forecaster.fit((x, y), epochs=3)
Explanation: Forecaster Step 2: Time series forecasting using Chronos Forecaster
After preprocessing the datasets. We can use Chronos Forecaster to handle the forecasting tasks.
Transform TSDataset to sampled numpy ndarray and feed them to forecaster.
End of explanation
x_test, y_test = tsdata_test.to_numpy()
pred = forecaster.predict(x_test)
pred_unscale, groundtruth_unscale = tsdata_test.unscale_numpy(pred), tsdata_test.unscale_numpy(y_test)
import matplotlib.pyplot as plt
plt.figure(figsize=(24,6))
plt.plot(pred_unscale[:,:,0])
plt.plot(groundtruth_unscale[:,:,0])
plt.legend(["prediction", "ground truth"])
Explanation: Forecaster Step 3: Further deployment with fitted forecaster
Use fitted forecaster to predict test data and plot the result
End of explanation
forecaster.save("nyc_taxi.fxt")
forecaster.load("nyc_taxi.fxt")
Explanation: Save & restore the forecaster.
End of explanation
<END_TASK>