Massive data exist among user local platforms that usually cannot support deep neural network (DNN) training due to computation and storage resource constraints. Cloud-based training schemes provide beneficial services but suffer from potential privacy risks due to excessive user data collection. To enable cloud-based DNN training while protecting the data privacy simultaneously, we propose to leverage the intermediate representations of the data, which is achieved by splitting the DNNs and deploying them separately onto local platforms and the cloud. The local neural network (NN) is used to generate the feature representations. To avoid local training and protect data privacy, the local NN is derived from pre-trained NNs. The cloud NN is then trained based on the extracted intermediate representations for the target learning task. We validate the idea of DNN splitting by characterizing the dependency of privacy loss and classification accuracy on the local NN topology for a convolutional NN (CNN) based image classification task. Based on the characterization, we further propose PrivyNet to determine the local NN topology, which optimizes the accuracy of the target learning task under the constraints on privacy loss, local computation, and storage. The efficiency and effectiveness of PrivyNet are demonstrated with CIFAR-10 dataset. With the pervasiveness of sensors, cameras, and mobile devices, massive data are generated and stored on local platforms. While useful information can be extracted from the data, the training process can be too computationally intensive that local platforms are not able to support. Cloud-based services provide a viable alternative to enable deep model training but rely on excessive user data collection, which suffers from potential privacy risks and policy violations.To enable the cloud-based training scheme while simultaneously protecting user data privacy, different data pre-processing schemes are proposed. Instead of releasing the original data, transformed representations are usually generated locally and then, uploaded for the target learning tasks. For the intermediate representations to be effective, there are two requirements, i.e. utility and privacy. The utility requirement urges that the target learning task can be accomplished accurately based on the released representations, while the privacy requirement forces the leakage of private information to be constrained within a satisfiable range. Furthermore, the transformation scheme should also be flexible enough for platforms with different computation and storage capabilities and for different types of data, which can be either high dimensional and continuous, like videos or images, or discrete.Related Works Privacy and utility trade-off has been one of the main questions in the privacy research. Different measures of privacy and utility are proposed based on the rate-distortion theory BID18 BID16 du Pin Calmon & Fawaz, 2012) , statistical estimation BID21 , and learnability BID10 . To actively explore the trade-off between privacy and utility, in recent years, many different transformations have been proposed. Syntactic anonymization methods, including k-anonymity BID22 , l-diversity BID14 and t-closeness BID13 , are proposed to anonymize quasiidentifiers and protect sensitive attributes in a static database. However, syntactic anonymization is hard to apply to high-dimensional continuous data because quasi-identifiers and sensitive attributes become hard to define.Differential privacy is proposed to provide a more formal privacy guarantee and can be easily achieved by adding noise BID3 BID4 . However, because differential privacy only prevents an adversary from gaining additional knowledge by inclusion/exclusion of an individual data BID5 , the total information leakage from the released representations is not limited BID8 . Meanwhile, to achieve differential privacy, existing works BID19 BID0 usually require local platforms to get involved in the backward propagation process, which makes it hard to deploy them on lightweight platforms.Non-invertible linear and non-linear transformations are also proposed for data anonymization. Existing linear transformations rely on the covariance between data and labels BID6 or the linear discriminant analysis (LDA) BID24 to filter the training data. However, linear transformations usually suffer from limited privacy protection since the original data can be reconstructed given the released representations. Recently proposed nonlinear transformations based on minimax filter BID8 or Siamese networks BID15 can provide better privacy protection. However, they can only be applied to protect privacy in the inference stage since iteractive training scheme is required between the cloud and local platforms, for which privacy loss becomes very hard to control. Figure 1: The proposed PrivyNet framework: the local NN is derived from pre-trained NNs for feature extraction and the cloud NN is trained for the target learning task. Privacy and utility trade-off is controlled by the topology of the local NN.Contribution To this end, we propose PrivyNet, a flexible DNN training framework to achieve a fine-grained control of the trade-off between privacy and utility. PrivyNet divides a DNN model into two parts and deploys them onto the local platforms and the cloud separately. As shown in Figure 1 , the local NN is used to generate intermediate representations while the cloud NN is trained for the learning task based on the released intermediate representations. The privacy protection is achieved through the transformation realized by the local NN, which is non-linear and consists of different lossy operations, including convolution, pooling, and so on. To avoid local training, we derive the local NN from pre-trained NNs. Our key insight here is that the initial layers of a DNN are usually used to extract general features that are not application specific and can enable different learning tasks. Therefore, by deriving the local NN from pre-trained NNs, a good utility can be achieved since useful features are embedded in the released representations, while privacy can be protected by selecting the topology of the local NN to control the specific features to release. Our main contributions are summarized as follows:• We propose PrivyNet, a novel framework to split DNN model to enable cloud-based training with a fine-grained control of privacy loss.• We characterize the privacy loss and utility of using CNN as the local NN in detail, based on which three key factors that determine the privacy and utility trade-off are identified and compared.• A hierarchical strategy is proposed to determine the topology of the local NN to optimize the utility considering constraints on local computation, storage, and privacy loss.• We verify PrivyNet by using the CNN-based image classification as an example and demonstrate its efficiency and effectiveness. In this section, we provide detailed discussions on the adversarial model adopted in the paper.According to the adversarial model we have defined in Section 3, the transformation induced by the FEN is assumed to be unknown to the attackers. This helps prevent more powerful attacks and enable a better privacy protection. However, because the FEN is derived from the pre-trained NNs, the structure and weights of which are also available to the attackers, we need to provide strategies to protect the anonymity of the FEN. In our framework, we consider the following two methods for the protection of the FEN:• Build a pool of pre-trained NNs to enable FEN derivation from NNs. In our characterization framework, we use VGG16 as an example. The same procedure can be applied to VGG19 BID20 , ResNet BID9 , Inception BID23 . By enlarging the pool, it becomes harder for the attacker to guess how the FEN is derived.• Apply the channel selection procedure to both output channels and intermediate channels.After the channel selection, the number of channels and the subset of selected channels in each layer become unknown to the attackers. Therefore , even if the attackers know the pre-trained NN, from which the FEN is derived, it becomes much harder to guess the channels that form the FEN.One important requirement for the intermediate channel selection is that the utility is not sacrificed and the privacy loss is not increased. We verify the change of privacy and utility empirically. We take the first 6 layers of VGG16, including 4 convolution layers and 2 max-pooling layers, and set the depth of output channel to 8. We use CIFAR-10 dataset and the same ICN and IRN as in Section 2. We first gradually reduce the channel depth of the first convolution layer from 64 to 16. As shown in FIG12 , the privacy and utility are rarely impacted by the change of the channel depth of first convolution layer. Meanwhile, we can observe the dramatic reduction on the runtime. We then gradually reduce the channel depth for each convolution layer. As shown in FIG4 , after we reduce the channel depth for each layer to half of its original depth, we still get similar privacy and utility with a dramatic reduction of the runtime. , 64, 128, 8}, { 32, 32, 128, 8}, {32, 32, 64, 8}, respectively By channel selection for intermediate layers, even if the attackers can know the pre-trained NN that our FEN is derived from, it is still very hard to determine the number of layers for the FEN and the number of channels for each layer. In this way, the anonymity of the FEN can be well protected.
To enable cloud-based DNN training while protecting the data privacy simultaneously, we propose to leverage the intermediate data representations, which is achieved by splitting the DNNs and deploying them separately onto local platforms and the cloud. This paper proposes a technique to privatize data by learning a feature representation that is difficult to use for image reconstruction, but helpful for image classification.