Awesome Mobile On-Device AI Security

SoK：移动端设备端人工智能系统的攻防全景

黄宇进，郑鑫，袁兴良，林国耀

移动端设备端人工智能系统通过诸如 LiteRT/TFLite、Core ML、ExecuTorch、ONNX 等机器学习框架以及硬件加速器，在本地执行人工智能模型。本仓库追踪理解和保护此类系统所需的安全研究，因为设备端模型的本地存储引入了新的安全风险。

移动端设备端人工智能系统概览

目录

• 阅读路线图
• 分类概览
• 跨支柱安全分析
• 针对MoAI系统的攻击
  • 对抗性攻击
    • 模型相似性利用
    • 梯度重建
    • 预处理操纵
  • 后门攻击
    • 载荷注入
    • 模型量化
    • 图像隐写
  • 对抗性权重攻击
  • 模型窃取攻击
    • 静态分析
    • 动态分析
    • 侧信道
  • 能耗-延迟攻击
• 针对MoAI系统的防御
  • 模型混淆
    • 软件级隐藏
    • 硬件级隐藏
  • 模型授权
  • 可信执行环境
    • 整体执行
    • 分区执行
    • 混淆卸载
  • 模型水印
• 开放问题
• 新兴方向

阅读路线图

刚接触MoAI安全？从这里开始：

1. 了解生态系统。 阅读关于Android/iOS应用中深度学习应用和设备端模型的实证研究。
2. 学习核心风险。 研究模型提取和模型保护论文，因为本地模型驻留是MoAI系统中安全性的核心转变。
3. 理解攻击面。 研究MoAI攻击如何在输入接口、模型制品、运行时执行和硬件支持的环境中产生。
4. 将防御与攻击面关联。 检查MoAI防御如何在部署前、运行时执行和部署后阶段保护这些面。
5. 展望未来。 探索设备端训练、设备端生成式AI和智能体MoAI系统中的新安全挑战。

•  A First Look at Deep Learning Apps on Smartphones
•  A First Look at On-device Models in iOS Apps

•  Mind Your Weight(s): A Large-scale Study on Insufficient ML Model Protection in Mobile Apps

•  Robustness of On-device Models: Adversarial Attack to Deep Learning Models on Android Apps
•  DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection
•  Typhon Unleashed: Practical Adversarial Weight Attacks Against On-Device Deep Learning Models
•  Energy-Latency Attacks to On-Device Neural Networks via Sponge Poisoning

•  ModelObfuscator: Obfuscating Model Information to Protect Deployed ML-based Systems
•  ShadowNet: A Secure and Efficient On-device Model Inference System
•  THEMIS: Towards Practical IP Protection for Post-Deployment On-Device DL Models

分类概览

跨支柱安全分析

针对MoAI系统的攻击

模型相似性利用

• Robustness of On-device Models: Adversarial Attack to Deep Learning Models on Android Apps [Code]
  _{IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP 2021)}

• Smart App Attack: Hacking Deep Learning Models in Android Apps [Code]
  _{IEEE Transactions on Information Forensics and Security (TIFS 2022)}

• Understanding Real-world Threats to Deep Learning Models in Android Apps [Code]
  _{ACM SIGSAC Conference on Computer and Communications Security (CCS 2022)}

• Cheating Your Apps: Black-box Adversarial Attacks on Deep Learning Apps
  _{Journal of Software: Evolution and Process (JSEP 2024)}

• A First Look at On-device Models in iOS Apps [Code]
  _{ACM Transactions on Software Engineering and Methodology (TOSEM 2024)}

梯度重建

• Investigating White-Box Attacks for On-Device Models [Code]
  _{IEEE/ACM International Conference on Software Engineering (ICSE 2024)}

• TIM: Enabling Large-Scale White-Box Testing on In-App Deep Learning Models [Code]
  _{IEEE Transactions on Information Forensics and Security (TIFS 2024)}

预处理操纵

• Beyond the Model: Data Pre-processing Attack to Deep Learning Models in Android Apps [Code]
  _{Secure and Trustworthy Deep Learning Systems Workshop (SecTL 2023)}

载荷注入

• DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection [Code]
  _{IEEE/ACM International Conference on Software Engineering (ICSE 2021)}

• MalModel: Hiding Malicious Payload in Mobile Deep Learning Models with Black-box Backdoor Attack [Code]
  _{Automated Software Engineering (ASEJ 2026)}

模型量化

• Quantization Backdoors to Deep Learning Commercial Frameworks [Code]
  _{IEEE Transactions on Dependable and Secure Computing (TDSC 2024)}

图像隐写

• Stealthy Backdoor Attack to Real-world Models in Android Apps
  _{arXiv preprint (arXiv 2025)}

• Typhon Unleashed: Practical Adversarial Weight Attacks Against On-Device Deep Learning Models
  _{IEEE Transactions on Dependable and Secure Computing (TDSC 2026)}

静态分析

• A First Look at Deep Learning Apps on Smartphones [Code]
  _{The World Wide Web Conference (WWW 2019)}

• A First Look at On-device Models in iOS Apps [Code]
  _{ACM Transactions on Software Engineering and Methodology (TOSEM 2023)}

• Mind Your Weight(s): A Large-scale Study on Insufficient Machine Learning Model Protection in Mobile Apps [Code]
  _{USENIX Security Symposium (USENIX Security 2021)}

• REDLC: Learning-Driven Reverse Engineering for Deep Learning Compilers
  _{IEEE International Symposium on Software Reliability Engineering (ISSRE 2024)}

动态分析

• Mind Your Weight(s): A Large-scale Study on Insufficient Machine Learning Model Protection in Mobile Apps [Code]
  _{USENIX Security Symposium (USENIX Security 2021)}

• Understanding Real-world Threats to Deep Learning Models in Android Apps [Code]
  _{ACM SIGSAC Conference on Computer and Communications Security (CCS 2022)}

• DeMistify: Identifying On-device Machine Learning Models Stealing and Reuse Vulnerabilities in Mobile Apps [Code]
  _{IEEE/ACM International Conference on Software Engineering (ICSE 2024)}

• Game of Arrows: On the (In-)Security of Weight Obfuscation for On-Device TEE-Shielded LLM Partition Algorithms [Code]
  _{USENIX Security Symposium (USENIX Security 2025)}

侧信道

• Model Extraction Attack against On-Device Deep Learning with Power Side Channel
  _{IEEE International Symposium on Quality Electronic Design (ISQED 2024)}

• DeepCache: Revisiting Cache Side-Channel Attacks in Deep Neural Networks Executables
  _{ACM SIGSAC Conference on Computer and Communications Security (CCS 2024)}

• Energy-Latency Attacks to On-Device Neural Networks via Sponge Poisoning
  _{Secure and Trustworthy Deep Learning Systems Workshop (SecTL 2023)}

针对MoAI系统的防御

软件级隐藏

• ModelObfuscator: Obfuscating Model Information to Protect Deployed ML-Based Systems [Code]
  _{ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2023)}

• DynaMO: Protecting Mobile DL Models through Coupling Obfuscated DL Operators [Code]
  _{IEEE/ACM International Conference on Automated Software Engineering (ASE 2024)}

• Model-less Is the Best Model: Generating Pure Code Implementations to Replace On-Device DL Models [Code]
  _{ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2024)}

硬件级隐藏

• NNSplitter: An Active Defense Solution for DNN Model via Automated Weight Obfuscation [Code]
  _{International Conference on Machine Learning (ICML 2023)}

• A Novel Obfuscation Method Based on Majority Logic for Preventing Unauthorized Access to Binary Deep Neural Networks
  _{Scientific Reports (Sci. Rep. 2025)}

• Securing On-device Transformer with Hardware Binding and Reversible Obfuscation [Code]
  _{Annual Computer Security Applications Conference (ACSAC 2025)}

• MMGuard: Automatically Protecting On-Device Deep Learning Models in Android Apps [Code]
  _{IEEE Security and Privacy Workshops (SPW 2021)}

• Securing On-device Transformer with Hardware Binding and Reversible Obfuscation [Code]
  _{Annual Computer Security Applications Conference (ACSAC 2025)}

整体执行

• Offline Model Guard: Secure and Private ML on Mobile Devices
  _{Design, Automation and Test in Europe Conference (DATE 2020)}

• GuardiaNN: Fast and Secure On-Device Inference in TrustZone Using Embedded SRAM and Cryptographic Hardware
  _{ACM/IFIP International Middleware Conference (Middleware 2022)}

• Secure and Efficient Mobile DNN Using Trusted Execution Environments
  _{ACM Asia Conference on Computer and Communications Security (AsiaCCS 2023)}

• T-Slices: Confidential Execution of Deep Learning Inference at the Untrusted Edge with Arm TrustZone
  _{ACM Conference on Data and Application Security and Privacy (CODASPY 2023)}

• LEAP: TrustZone Based Developer-Friendly TEE for Intelligent Mobile Apps
  _{IEEE Transactions on Mobile Computing (TMC 2022)}

• ASGARD: Protecting On-Device Deep Neural Networks with Virtualization-Based Trusted Execution Environments [Code]
  _{Network and Distributed System Security Symposium (NDSS 2025)}

• TZ-LLM: Protecting On-Device Large Language Models with Arm TrustZone [Code]
  _{European Conference on Computer Systems (EuroSys 2026)}

• FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation
  _{arXiv preprint (arXiv 2026)}

分区执行

• DarkneTZ: Towards Model Privacy at the Edge Using Trusted Execution Environments [Code]
  _{Annual International Conference on Mobile Systems, Applications, and Services (MobiSys 2020)}

• HybridTEE: Secure Mobile DNN Execution Using Hybrid Trusted Execution Environment [Code]
  _{Asian Hardware Oriented Security and Trust Symposium (AsianHOST 2020)}- SecDeep: Secure and Performant On-Device Deep Learning Inference Framework for Mobile and IoT Devices
  _{国际物联网设计与实现会议 (IoTDI 2021)}

• ShadowNet: A Secure and Efficient On-Device Model Inference System for Convolutional Neural Networks [Code]
  _{IEEE安全与隐私研讨会 (S&P 2023)}

• MirrorNet: A TEE-Friendly Framework for Secure On-Device DNN Inference
  _{IEEE/ACM国际计算机辅助设计会议 (ICCAD 2023)}

• TSQP: Safeguarding Real-Time Inference for Quantization Neural Networks on Edge Devices [Code]
  _{IEEE安全与隐私研讨会 (S&P 2025)}

• TEESlice: Protecting Sensitive Neural Network Models in Trusted Execution Environments When Attackers Have Pre-Trained Models
  _{ACM软件工程与方法论汇刊 (TOSEM 2025)}

• TensorShield: Safeguarding On-Device Inference by Shielding Critical DNN Tensors with TEE [Code]
  _{ACM SIGSAC计算机与通信安全会议 (CCS 2025)}

混淆卸载

• GroupCover: A Secure, Efficient and Scalable Inference Framework for On-Device Model Protection Based on TEEs [Code]
  _{国际机器学习会议 (ICML 2024)}

• Game of Arrows: On the (In-)Security of Weight Obfuscation for On-Device TEE-Shielded LLM Partition Algorithms [Code]
  _{USENIX安全研讨会 (USENIX Security 2025)}

• MirageNet: A Secure, Efficient, and Scalable On-Device Model Protection in Heterogeneous TEE and GPU System
  _{arXiv预印本 (arXiv 2026)}

• THEMIS: Towards Practical Intellectual Property Protection for Post-Deployment On-Device Deep Learning Models [Code]
  _{USENIX安全研讨会 (USENIX Security 2025)}

开放问题

以下开放问题总结了我们系统知识综述（SoK）中识别的主要研究缺口。此处保持高层次描述以便读者使用本仓库。更详细的技术讨论请参见论文。

1. 攻击部署的实用性。
   针对设备端模型的对抗攻击在部署后仍然难以实现，因为通常需要控制模型输入、插入对抗扰动，或通过应用重打包修改预处理代码。这些步骤在实际最终用户部署中可能不切实际或容易被检测。

2. 隐蔽的模型修改。
   后门攻击需要在部署后找到除标准训练时投毒之外的新入口点，因为设备端模型通常是只读且仅推理的。关键挑战在于引入隐藏的恶意行为而不产生可观察的模型工件变化。

3. 精确的权重定位。
   对抗性权重攻击暴露了参数级别的完整性风险，但实际部署取决于在庞大的参数搜索空间中定位行为关键权重。这很困难，因为攻击者通常缺乏梯度指导，并且需要在仅修改选定参数的同时保持良性效用。

4. 可靠的模型提取。
   本地模型存储并不能使模型窃取变得简单。实际提取仍然依赖于在自定义加密算法、非标准AI框架和运行时特定加载行为存在的情况下，进行可靠的模型识别、解密和重构。

5. 硬件异构性。
   能量-延迟攻击依赖于中毒激活模式如何与特定设备的执行行为交互。它们可能放大对稀疏敏感的加速器上的延迟和能量消耗，但无法迁移到没有稀疏依赖执行的硬件上。

6. 可执行等价性。
   模型混淆仍需在授权推理期间保持原始预测函数。这种可执行等价性可能暴露可恢复的运行时状态、变换后的权重、算子语义或结构痕迹，从而促成语义、结构或参数恢复。

7. 客户端强制执行。
   模型授权将正确推理与凭证、完整性检查和打包权重恢复绑定。然而，这些检查需在移动堆栈内执行，使得强制执行依赖于可在部署后被逆向工程、重打包、挂钩或插桩的客户端代码。

8. TEE部署可行性。
   TEE防御需要跨模型格式、AI框架、算子库、代理、加速器和CPU/GPU/NPU隔离接口的协调支持。当前移动生态系统仍缺乏广泛采用、对开发者透明的TEE支持推理栈。

9. 水印鲁棒性。
   模型水印支持部署后的所有权验证，但被窃取的模型可能通过框架转换、加密或应用级输入输出中介重新部署。这些变换可以保留良性推理，同时破坏用于验证的触发器响应、置信度模式或输出语义。

新兴方向

除了上述九个开放问题，我们的系统知识综述（SoK）还强调了移动设备端AI（MoAI）安全可能向何方扩展的三个新兴方向。这些方向超越静态、仅推理的设备端模型，转向自适应、生成式和行动导向的MoAI系统。我们在此进行高层次总结。配套论文提供了更详细的动机、威胁面和研究挑战。

当前MoAI安全研究主要侧重于已部署的只读、仅推理模型。设备端训练改变了这一假设，允许模型在本地更新，从而在训练过程中暴露梯度、参数更新和用户数据。这引发了关于本地微调、更新完整性、训练数据暴露、个性化投毒以及最终用户设备上训练时态防御的新问题。

现有MoAI安全研究仍主要集中在视觉基础任务如图像分类上。随着大语言模型（LLM）和生成式模型进入智能手机，MoAI安全必须扩展到提示驱动和内容生成系统。重要挑战包括提示注入、越狱、非预期信息泄露以及设备端LLM的本地上下文泄漏。

MoAI系统正从被动本地推理演变为代理式工作流，将模型与传感器、私有用户数据、应用上下文、操作系统服务和跨应用接口连接起来。这使安全焦点从保护模型工件转向治理上下文到动作的链。未来工作应研究移动上下文溯源、可信用户意图与不可信环境内容的分离、工具和API使用的任务范围权限、敏感动作的确认与回滚，以及代理计划、记忆和动作的审计。