Awesome Mobile On-Device AI Security

SoK: Attack and Defense Landscape of Mobile On-device AI Systems

Mobile on-device AI systems execute AI models locally through ML frameworks such as LiteRT/TFLite, Core ML, ExecuTorch, ONNX, and hardware-backed accelerators. This repo tracks the security research needed to understand and protect such systems, as the local storage of on-device models introduces new security risks.

Overview of a Mobile On-Device AI system

Contents

• Reading roadmap
• Taxonomy at a glance
• Cross-pillar Security Analysis
• Attacks on MoAI systems
  • Adversarial Attacks
    • Model Similarity Exploitation
    • Gradient Reconstruction
    • Preprocessing Manipulation
  • Backdoor Attacks
    • Payload Injection
    • Model Quantization
    • Image Steganography
  • Adversarial Weight Attacks
  • Model Stealing Attacks
    • Static Analysis
    • Dynamic Analysis
    • Side Channel
  • Energy-latency Attacks
• Defenses for MoAI systems
  • Model Obfuscation
    • Software-level Concealment
    • Hardware-level Concealment
  • Model Authorization
  • Trusted Execution Environments
    • Monolithic Execution
    • Partitioned Execution
    • Obfuscated Offloading
  • Model Watermarking
• Open problems
• Emerging directions

Reading roadmap

New to MoAI security? Start here:

1. Understand the ecosystem. Read empirical studies on deep learning apps and on-device models in Android/iOS apps.
2. Learn the core risk. Study model extraction and model protection papers, because local model residency is the central security shift in MoAI systems.
3. Understand the attack surfaces. Study how MoAI attacks arise across input interfaces, model artifacts, runtime execution, and hardware-backed environments.
4. Connect defenses to the surfaces. Examine how MoAI defenses protect these surfaces across pre-deployment, runtime execution, and post-deployment phases.
5. Look forward. Explore new security challenges in on-device training, on-device GenAI, and agentic MoAI systems.

•  A First Look at Deep Learning Apps on Smartphones
•  A First Look at On-device Models in iOS Apps

•  Mind Your Weight(s): A Large-scale Study on Insufficient ML Model Protection in Mobile Apps

•  Robustness of On-device Models: Adversarial Attack to Deep Learning Models on Android Apps
•  DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection
•  Typhon Unleashed: Practical Adversarial Weight Attacks Against On-Device Deep Learning Models
•  Energy-Latency Attacks to On-Device Neural Networks via Sponge Poisoning

•  ModelObfuscator: Obfuscating Model Information to Protect Deployed ML-based Systems
•  ShadowNet: A Secure and Efficient On-device Model Inference System
•  THEMIS: Towards Practical IP Protection for Post-Deployment On-Device DL Models

Taxonomy at a glance

MoAI security pillar	What it protects	Representative attacks	Representative defenses
User-governed input integrity	The end-to-end integrity of user inputs, from mobile data acquisition to model-input handoff	Adversarial Attacks, Backdoor Attacks, Energy-latency Attacks	-
Device-resident model security	Deployed model artifacts and all post-deployment forms in which models are stored, loaded, transformed, or materialized on devices	Adversarial Attacks, Backdoor Attacks, Adversarial Weight Attacks, Model Stealing Attacks, Energy-latency Attacks	Model Obfuscation, Model Authorization, TEE, Model Watermarking
Device-native environment confinement	Sensitive inference computation and runtime states across the mobile OS, AI runtime, memory subsystem, and hardware-backed execution environments	Model Stealing Attacks, Energy-latency Attacks	Model Obfuscation, TEE

Cross-pillar Security Analysis

Attacks on MoAI systems

Model Similarity Exploitation