Preserving privacy in machine learning on multi-party datasets is of importance to many domains.
Existing solutions suffer from several critical limitations, such as significantly reduced utility after enforcing differential privacy, excessive communications burden between information fusion center and local participants who contribute data, etc, which severely limit their practical adoption.
In this project,we propose and implement a novel differential private multi-party deep learning framework to address these shortcomings in a more accurate and efficient way.
To be more specific, during the stochastic gradient descent process in training a deep neural network,we focus on the parameters with large gradient values to save privacy budget consumption, and adopt a generalization of the Report-Noisy-Max algorithm in differential privacy to select and release these gradients in order to prevent indirect privacy leakage.
Inspired by the recent novel work, we also limit the shared gradient for each parameter to be one of three floating numbers: −B, 0, and B, where B is the bound for each gradient.
This method can significantly reduce the communication burden without severely affecting accuracy.
We will evaluate the performance of our system on real-world credit card fraud detection dataset with multi-million transactions.