When deep learning is applied to sensitive data sets, many privacy-related implementation issues arise. These issues are especially evident in the healthcare, finance, law and government industries. Homomorphic encryption could allow a server to make inferences on inputs encrypted by a client, but to our best knowledge, there has been no complete implementation of common deep learning operations, for arbitrary model depths, using homomorphic encryption. This paper demonstrates a novel approach, efficiently implementing many deep learning functions with bootstrapped homomorphic encryption. As part of our implementation, we demonstrate Single and Multi-Layer Neural Networks, for the Wisconsin Breast Cancer dataset, as well as a Convolutional Neural Network for MNIST. Our results give promising directions for privacy-preserving representation learning, and the return of data control to users.

 The healthcare, finance, law and government industries often require complete privacy and confidentiality between various stakeholders and partners. With the advent of highly effective AI using deep learning, many real-world tasks can be made more effective and efficient in these industries. However deep learning approaches are seldom performed with privacy preservation in mind, let alone with the encryption of information throughout the entire process.As a result, current deep learning implementations often cannot be used for these confidential applications. Homomorphic Encryption (HE) BID28 offers an opportunity to address the privacy preservation gap, for data processing in general and deep learning in particular. HE can be used to perform computation on encrypted information BID26 , without ever having access to the plaintext information.Our work combines the paradigms of deep learning and homomorphic encryption, allowing improved privacy for existing server-side models , and thus enabling many novel, intelligent, privacy-guaranteeing services.Figure 1: General overview of our privacy-preserving method for deep learning. Encrypted inputs are fed into our hybrid model on the server-side, and this produces encrypted outputs. Our work shows that with the proposed Hybrid Homomorphic Encryption system, almost any production deep learning model can be converted, such that it can process encrypted inputs. Our design also makes it feasible to implement new or bespoke functionality, as the deep learning paradigm evolves.Depending on the value of the problem and the size of the model, this system is already viable for production use. New and updated HE libraries appear frequently, and our code should adapt to any library which implements homomorphic logic gates. Therefore our software could potentially receive "free" performance gains, as the HE paradigm evolves.
We made a feature-rich system for deep learning with encrypted inputs, producing encrypted outputs, preserving privacy. A framework for private deep learning model inference using FHE schemes that support fast bootstrapping and thus can reduce computation time. The paper presents a means of evaluating a neural network securely using homomorphic encryption.