<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Pipeline Match ID Tutorial
Starting at version 1.7, FATE distinguishes sample id(sid) and match id. Sid are unique to each sample entry, while match id corresponds to individual sample source identity. This adaption allows FATE to perform private set intersection on samples with repeated match id. User may choose to create sid by appending uuid to original sample entries at uploading; then module DataTransform will extract true match id for later use. This tutorial walks through a full uploading-training process to demonstrate how to add and train with sid.
install
Pipeline is distributed along with fate_client.
bash
pip install fate_client
To use Pipeline, we need to first specify which FATE Flow Service to connect to. Once fate_client installed, one can find an cmd enterpoint name pipeline
Step1:  Assume we have a FATE Flow Service in 127.0.0.1
Step2:  upload data
Before start a modeling task, the data to be used should be uploaded. 
 Typically, a party is usually a cluster which include multiple nodes. Thus, when we upload these data, the data will be allocated to those nodes.
Step3:  Make a pipeline instance
Step4:  Define partitions for data storage
Step5:  Define table name and namespace, which will be used in FATE job configuration
Step6:  Now, we add data to be uploaded. To create uuid as sid, turn on extend_sid option. Alternatively, set auto_increasing_sid to make extended sid starting at 0.
Step7:  We can then upload data
Step8:  After uploading, we can then start modeling. Here we build a Hetero SecureBoost model the same way as in this demo, but note how specificaiton of DataTransform module needs to be adjusted to crrectly load in match id.
Step9:  Add components to pipeline, in order of execution
Step10:  Now, submit(fit) our pipeline
Step11:  Check data output on FATEBoard or download component output data to see now each data instance has a uuid as sid.
<ASSISTANT_TASK:>
Python Code:
!pipeline --help
Explanation: Pipeline Match ID Tutorial
Starting at version 1.7, FATE distinguishes sample id(sid) and match id. Sid are unique to each sample entry, while match id corresponds to individual sample source identity. This adaption allows FATE to perform private set intersection on samples with repeated match id. User may choose to create sid by appending uuid to original sample entries at uploading; then module DataTransform will extract true match id for later use. This tutorial walks through a full uploading-training process to demonstrate how to add and train with sid.
install
Pipeline is distributed along with fate_client.
bash
pip install fate_client
To use Pipeline, we need to first specify which FATE Flow Service to connect to. Once fate_client installed, one can find an cmd enterpoint name pipeline:
End of explanation
!pipeline init --ip 127.0.0.1 --port 9380
Explanation: Assume we have a FATE Flow Service in 127.0.0.1:9380(defaults in standalone), then exec
End of explanation
from pipeline.backend.pipeline import PipeLine
Explanation: upload data
Before start a modeling task, the data to be used should be uploaded. 
 Typically, a party is usually a cluster which include multiple nodes. Thus, when we upload these data, the data will be allocated to those nodes.
End of explanation
pipeline_upload = PipeLine().set_initiator(role='guest', party_id=9999).set_roles(guest=9999)
Explanation: Make a pipeline instance:
- initiator: 
    * role: guest
    * party: 9999
- roles:
    * guest: 9999
note that only local party id is needed.
End of explanation
partition = 4
Explanation: Define partitions for data storage
End of explanation
dense_data_guest = {"name": "breast_hetero_guest", "namespace": f"experiment"}
dense_data_host = {"name": "breast_hetero_host", "namespace": f"experiment"}
tag_data = {"name": "breast_hetero_host", "namespace": f"experiment"}
Explanation: Define table name and namespace, which will be used in FATE job configuration
End of explanation
data_base = "/workspace/FATE/"
pipeline_upload.add_upload_data(file=os.path.join(data_base, "examples/data/breast_hetero_guest.csv"),
                                table_name=dense_data_guest["name"],             # table name
                                namespace=dense_data_guest["namespace"],         # namespace
                                head=1, partition=partition,                     # data info
                                extend_sid=True,                                 # extend sid 
                                auto_increasing_sid=False)
pipeline_upload.add_upload_data(file=os.path.join(data_base, "examples/data/breast_hetero_host.csv"),
                                table_name=dense_data_host["name"],
                                namespace=dense_data_host["namespace"],
                                head=1, partition=partition,
                                extend_sid=True,
                                auto_increasing_sid=False) 
Explanation: Now, we add data to be uploaded. To create uuid as sid, turn on extend_sid option. Alternatively, set auto_increasing_sid to make extended sid starting at 0.
End of explanation
pipeline_upload.upload(drop=1)
Explanation: We can then upload data
End of explanation
from pipeline.backend.pipeline import PipeLine
from pipeline.component import Reader, DataTransform, Intersection, HeteroSecureBoost, Evaluation
from pipeline.interface import Data
pipeline = PipeLine() \
        .set_initiator(role='guest', party_id=9999) \
        .set_roles(guest=9999, host=10000)
reader_0 = Reader(name="reader_0")
# set guest parameter
reader_0.get_party_instance(role='guest', party_id=9999).component_param(
    table={"name": "breast_hetero_guest", "namespace": "experiment"})
# set host parameter
reader_0.get_party_instance(role='host', party_id=10000).component_param(
    table={"name": "breast_hetero_host", "namespace": "experiment"})
#  set with match id
data_transform_0 = DataTransform(name="data_transform_0", with_match_id=True)
# set guest parameter
data_transform_0.get_party_instance(role='guest', party_id=9999).component_param(
    with_label=True)
data_transform_0.get_party_instance(role='host', party_id=[10000]).component_param(
    with_label=False)
intersect_0 = Intersection(name="intersect_0")
hetero_secureboost_0 = HeteroSecureBoost(name="hetero_secureboost_0",
                                         num_trees=5,
                                         bin_num=16,
                                         task_type="classification",
                                         objective_param={"objective": "cross_entropy"},
                                         encrypt_param={"method": "paillier"},
                                         tree_param={"max_depth": 3})
evaluation_0 = Evaluation(name="evaluation_0", eval_type="binary")
Explanation: After uploading, we can then start modeling. Here we build a Hetero SecureBoost model the same way as in this demo, but note how specificaiton of DataTransform module needs to be adjusted to crrectly load in match id.
End of explanation
pipeline.add_component(reader_0)
pipeline.add_component(data_transform_0, data=Data(data=reader_0.output.data))
pipeline.add_component(intersect_0, data=Data(data=data_transform_0.output.data))
pipeline.add_component(hetero_secureboost_0, data=Data(train_data=intersect_0.output.data))
pipeline.add_component(evaluation_0, data=Data(data=hetero_secureboost_0.output.data))
pipeline.compile();
Explanation: Add components to pipeline, in order of execution:
- data_transform_0 comsume reader_0's output data
- intersect_0 comsume data_transform_0's output data
- hetero_secureboost_0 consume intersect_0's output data
- evaluation_0 consume hetero_secureboost_0's prediciton result on training data
Then compile our pipeline to make it ready for submission.
End of explanation
pipeline.fit()
Explanation: Now, submit(fit) our pipeline:
End of explanation
import json
print(json.dumps(pipeline.get_component("data_transform_0").get_output_data(limits=3), indent=4))
Explanation: Check data output on FATEBoard or download component output data to see now each data instance has a uuid as sid.
End of explanation
<END_TASK>