Cryptographic techniques for privacy-preserving machine learning

Cryptographic techniques for privacy-

preserving machine learning

Introduction:

As machine learning becomes increasingly integrated into our everyday

lives, privacy concerns surrounding the use and sharing of personal data

have also increased dramatically. While machine learning can provide

powerful insights, training algorithms on sensitive individual data raises

obvious privacy risks if that data is exposed or misused. Cryptographic

techniques offer compelling solutions to ensure privacy while still enabling

compute a function over their private inputs without revealing anything

except the output. MPC leverages cryptographic protocols like secret sharing

to split and reconstruct values such that no single party learns anything

about the inputs except what is implied by the output. MPC is well-suited for

scenarios where different entities hold distinct pieces of private data that

need to be analyzed together, like healthcare providers collaborating on a

jointly trained model without sharing raw patient records.

An early and Influential application of MPC to machine learning was Privacy-

Preserving Naïve Bayes by Vaidya and Clifton (2002). They demonstrated

how to train a Naïve Bayes classifier over vertically partitioned data (where

size of the circuit being computed. This poses challenges for more complex

machine learning algorithms that may require millions or billions of

calculations. Techniques like garbled circuits and functionality-targeted

protocols have helped address scalability, but MPC remains most practical for

smaller machine learning tasks involving a few parties with modestly sized

datasets. Areas well-suited for MPC include medical data analysis, fraud

detection across financial institutions, and modeling over census or survey

responses. Continued optimization will be needed to apply MPC to larger-

scale deep learning, but the technique offers strong privacy guarantees even

as models become more complex.

enables novel scenarios such as predictive analytics as a cloud service

without sacrificing data privacy.

The first fully homomorphic encryption schemes were proposed in 2009 by

Gentry and later improved by others. Recent advances, like the BGV, BFV,

and CKKS schemes, have improved both theoretical and practical

capabilities. Partially homomorphic schemes like Paillier allow a limited set of

arithmetic operations, while newer fully homomorphic schemes enable

arbitrary computation on encrypted data at the cost of larger ciphertext sizes

and computational overhead. HE remains orders of magnitude slower than

plain computation but continues to become more feasible through

neural network image classification and recommender systems using

homomorphic encryption. Significant challenges remain around managing

ciphertext growth, limiting noise accumulation during recursive

computations, and dealing with non-linearity and minibatching in deep

learning. However, HE shows promise to enable on-device and cloud-based

machine intelligence that respects user privacy through encrypted inference

and learning.

Differential privacy is a promising approach to privacy-preserving machine

learning based not on cryptography but on statistical disclosure limitation.

The goal of differential privacy is to ensure that any statistical analysis

Differential privacy is often achieved through the Laplace or Gaussian

mechanisms which inject randomized noise calibrated to the model’s

sensitivity. Differential privacy has been applied to publish statistical

aggregates, enable interactive data analysis through private query systems,

and even to train machine learning models in a privacy-preserving manner.

Recent work has developed differentially private algorithms for linear and

logistic regression, as well as deep learning using the moments accountant

technique.

Unlike cryptography-based methods, differential privacy does not require

encryption, secret sharing, or trusted set-up assumptions. It can directly

operate on plaintext data, making it easier to integrate into existing

deployment of cryptographic machine learning techniques faces challenges

around efficiency, implementation complexity, and integration with existing

machine learning systems and technologies. Several approaches are being

explored to address these scalability barriers:

- Approximate encryption techniques relax semantic security to trade

privacy for large efficiency gains. Approximate homomorphic

encryption allows much faster, low-precision computation on encrypted

data. Approximate MPC speeds up protocols at the cost of reducing

cryptographic security properties.

- Hardware acceleration moves computation to purpose-built

Frameworks like CrypTen, TAPAS, and PySyft facilitate development of

encrypted programming models suited for machine learning workloads.

- Combining techniques can exploit complementary strengths. Secure

aggregation protocols enable differentially private federated learning

across devices using MPC-style techniques. MPC and homomorphic

encryption are combined in hybrid approaches.

- Approximation and quantization of models enables techniques like

binarized neural networks suitable for cryptographic computation.

Limited-precision and minimum-size reformulations aim to reduce

cryptographic overhead.

As machine learning and artificial intelligence continue to infiltrate more

aspects of life, demand will grow for privacy-respecting techniques that allow

intelligent systems to derive value from sensitive data without compromising

individuals’ control or anonymity. Cryptography is indispensable for building

these next-generation trusted and accountable AI systems:

- Federated learning will help train models across ubiquitous edge

devices in a privacy-preserving decentralized manner. Cryptographic

MPC and secure aggregation can ensure privacy and prevent data

leakage in cross-device collaboration.

- Cryptographic techniques enable personalized and on-device

privacy enable highly distributed collaborative learning.

- AI safety technology like model extraction prevention and output

verification can leverage cryptography for provable guarantees.

Functional encryption enables controlled decryption and evaluation of

confidential models.

user consent and data protection as a foundation of trustworthy, beneficial

progress. Continued advances balancing algorithmic efficiency, security

rigor, and systems integration will determine how far this field progresses to

reconcile data-driven progress with information privacy rights.

Conclusion:

This paper introduced several promising cryptographic techniques for

realizing the vision of privacy-preserving machine learning: secure multiparty

computation, homomorphic encryption, and differential privacy. Each

approach was described at a conceptual level along with representative

applications and challenges. Scaling cryptography for machine learning

that users can genuinely trust with their information. Privacy-preserving

machine learning based on these principles has the potential to drive

beneficial progress while safeguarding rights and empowering individuals in

an increasingly data-driven world.

As machine learning becomes increasingly integrated into our everyday

lives, privacy concerns surrounding the use and sharing of personal data

have also increased dramatically. While machine learning can provide

techniques offer compelling solutions to ensure privacy while still enabling

valuable machine learning tasks. This paper will explore some of the most

prominent cryptographic approaches for privacy-preserving machine

learning, including secure multiparty computation, homomorphic encryption,

and differential privacy.

Secure Multiparty Computation:

Secure multiparty computation (MPC) allows multiple parties to jointly

compute a function over their private inputs without revealing anything

size of the circuit being computed. This poses challenges for more complex

machine learning algorithms that may require millions or billions of

calculations. Techniques like garbled circuits and functionality-targeted

protocols have helped address scalability, but MPC remains most practical for

smaller machine learning tasks involving a few parties with modestly sized

datasets. Areas well-suited for MPC include medical data analysis, fraud

detection across financial institutions, and modeling over census or survey

responses. Continued optimization will be needed to apply MPC to larger-

scale deep learning, but the technique offers strong privacy guarantees even

as models become more complex.

neural network image classification and recommender systems using

Using HE, a party can outsource storage and processing of their encrypted

data to an untrusted third party server, which performs computations and

returns encrypted results without learning the underlying plaintexts. This

enables novel scenarios such as predictive analytics as a cloud service

without sacrificing data privacy.

The first fully homomorphic encryption schemes were proposed in 2009 by

Gentry and later improved by others. Recent advances, like the BGV, BFV,

and CKKS schemes, have improved both theoretical and practical

capabilities. Partially homomorphic schemes like Paillier allow a limited set of

arithmetic operations, while newer fully homomorphic schemes enable

arbitrary computation on encrypted data at the cost of larger ciphertext sizes

and computational overhead. HE remains orders of magnitude slower than

plain computation but continues to become more feasible through

accelerations like GPU, distributed, and approximate computing

neural network image classification and recommender systems using

homomorphic encryption. Significant challenges remain around managing

ciphertext growth, limiting noise accumulation during recursive

computations, and dealing with non-linearity and minibatching in deep

learning. However, HE shows promise to enable on-device and cloud-based

machine intelligence that respects user privacy through encrypted inference

and learning.

Differential Privacy:

depend only weakly on the presence or absence of a particular data point.

Differential privacy is often achieved through the Laplace or Gaussian

mechanisms which inject randomized noise calibrated to the model’s

and even to train machine learning models in a privacy-preserving manner.

Recent work has developed differentially private algorithms for linear and

logistic regression, as well as deep learning using the moments accountant

technique.

around efficiency, implementation complexity, and integration with existing

machine learning systems and technologies. Several approaches are being

explored to address these scalability barriers:

- Approximate encryption techniques relax semantic security to trade

privacy for large efficiency gains. Approximate homomorphic

encryption allows much faster, low-precision computation on encrypted

- Hardware acceleration moves computation to purpose-built

cryptographic coprocessors like Intel SGX enclaves, GPUs, ASICs, and

FPGAs designed to speed up cryptographic operations. Specialized

- Systems integration focuses on compiling machine learning models

- Approximation and quantization of models enables techniques like

Limited-precision and minimum-size reformulations aim to reduce

cryptographic overhead.

intelligent systems to derive value from sensitive data without compromising

individuals’ control or anonymity. Cryptography is indispensable for building

these next-generation trusted and accountable AI systems:

MPC and secure aggregation can ensure privacy and prevent data

leakage in cross-device collaboration.

verification can leverage cryptography for provable guarantees.

Functional encryption enables controlled decryption and evaluation of

cryptography, the machine learning field can help ensure its techniques

develop accountably and respect individuals’ control over sensitive data.

Continued cross-disciplinary work at the intersection of cryptography,

obvious privacy risks if that data is exposed or misused. Cryptographic

techniques offer compelling solutions to ensure privacy while still enabling

valuable machine learning tasks. This paper will explore some of the most

prominent cryptographic approaches for privacy-preserving machine

learning, including secure multiparty computation, homomorphic encryption,

and differential privacy.

Secure Multiparty Computation:

Secure multiparty computation (MPC) allows multiple parties to jointly

compute a function over their private inputs without revealing anything

except the output. MPC leverages cryptographic protocols like secret sharing

jointly trained model without sharing raw patient records.

An early and Influential application of MPC to machine learning was Privacy-

Preserving Naïve Bayes by Vaidya and Clifton (2002). They demonstrated

protocol based on Shamir secret sharing. Other work has extended this

approach to train logistic regression, linear support vector machines, and

neural networks using MPC.

allowing computation directly on encrypted data without decrypting it first.

Using HE, a party can outsource storage and processing of their encrypted

data to an untrusted third party server, which performs computations and

and CKKS schemes, have improved both theoretical and practical

capabilities. Partially homomorphic schemes like Paillier allow a limited set of

arithmetic operations, while newer fully homomorphic schemes enable

arbitrary computation on encrypted data at the cost of larger ciphertext sizes

and computational overhead. HE remains orders of magnitude slower than

plain computation but continues to become more feasible through

accelerations like GPU, distributed, and approximate computing

implementations.

Machine learning on encrypted data through HE allows training machine

learning algorithms like linear regression and deep learning without

decryption. CryptoNets by Dowlin et al. (2016) demonstrated the first deep

neural network trained homomorphically on encrypted data. More recently,

researchers at IBM, Anthropic, and others have demonstrated convolutional

neural network image classification and recommender systems using

machine intelligence that respects user privacy through encrypted inference

and learning.

Differential Privacy:

Differential privacy is a promising approach to privacy-preserving machine

learning based not on cryptography but on statistical disclosure limitation.

The goal of differential privacy is to ensure that any statistical analysis

performed on a dataset does not significantly change if any individual’s data

is removed. Differential privacy adds “noise” to query responses so that they

logistic regression, as well as deep learning using the moments accountant

technique.

Unlike cryptography-based methods, differential privacy does not require

platforms and analytics systems without architectural changes. Differential

privacy provably limits information leakage about individuals in a

composable way, preventing accumulation of privacy risk through model

updates or downstream use. However, optimizing for tight privacy-utility

trade-offs remains an active research area. Differential privacy is well-suited

to large-scale analytics requiring repeated and adaptive access to privacy-

sensitive data.

Scaling Cryptography for Privacy in Machine

Learning:

While cryptography provides strong theoretical privacy guarantees, practical

deployment of cryptographic machine learning techniques faces challenges

around efficiency, implementation complexity, and integration with existing

machine learning systems and technologies. Several approaches are being

explored to address these scalability barriers:

cryptographic security properties.

FPGAs designed to speed up cryptographic operations. Specialized

crypto hardware in smartphones and IoT devices enables on-device

encrypted learning scenarios.

into forms compatible with cryptographic protocols and runtimes.

Frameworks like CrypTen, TAPAS, and PySyft facilitate development of

encrypted programming models suited for machine learning workloads.

encryption are combined in hybrid approaches.

- Approximation and quantization of models enables techniques like

of cryptographic privacy methods will be critical to realizing privacy-

provides a foundation, but practical deployment requires reconciling tight

Future Directions:

As machine learning and artificial intelligence continue to infiltrate more

aspects of life, demand will grow for privacy-respecting techniques that allow

intelligent systems to derive value from sensitive data without compromising

individuals’ control or anonymity. Cryptography is indispensable for building

these next-generation trusted and accountable AI systems:

- Federated learning will help train models across ubiquitous edge

devices in a privacy-preserving decentralized manner. Cryptographic

- Cryptographic techniques enable personalized and on-device

deployment of AI by keeping models and inferences confined to users’

local encrypted data stores. This avoids extracting sensitive personal

jointly from large private corpora without aggregating individually

reduced through innovation, privacy-preserving machine learning will grow

machine intelligence can develop responsibly in alignment with principles of

user consent and data protection as a foundation of trustworthy, beneficial

rigor, and systems integration will determine how far this field progresses to