Your developers are already running AI locally: Why on-device inference is the CISO’s new blind spot

Your developers are already running AI locally: Why on-device inference is the CISO’s new blind spot

Shadow AI 2.0: Local Inference Becomes a CISO's New Blind Spot

Section 1 – What happened?

In the last 18 months, the traditional approach to securing generative AI usage in the enterprise has been turned on its head. Security teams, who once focused on controlling browser access and monitoring cloud API calls, are now facing a new challenge. Employees are increasingly running large language models (LLMs) locally on their laptops, offline, and without any obvious network signature. This shift, dubbed "Shadow AI 2.0" or the "bring your own model" (BYOM) era, is making it difficult for security teams to detect and manage potential risks.

Section 2 – Background & Context

The rise of local inference is largely driven by advancements in hardware and software technology. Consumer-grade accelerators, such as those found in high-end laptops, have become powerful enough to run complex LLMs at usable speeds. Additionally, the widespread adoption of quantization, a technique that compresses models into smaller formats, has made it easier for employees to run sensitive workflows locally. The ease of distribution, thanks to open-weight models and user-friendly tooling ecosystems, has also contributed to this shift.

Section 3 – Impact on Swiss SMEs & Finance

For Swiss small and medium-sized enterprises (SMEs) and the finance sector, the emergence of Shadow AI 2.0 poses significant risks. As sensitive data is processed locally, traditional data loss prevention (DLP) tools may not be able to detect and prevent potential data breaches. This could lead to unvetted inference inside devices, which could compromise sensitive information and put companies at risk of non-compliance with regulatory requirements. Furthermore, the lack of visibility into local inference activities may make it challenging for security teams to identify and address potential security threats in a timely manner.

Section 4 – What to Watch

As the BYOM era continues to gain momentum, CISOs and security teams in Switzerland will need to adapt their strategies to address the new risks associated with local inference. This may involve implementing new tools and techniques to detect and manage local inference activities, as well as developing policies and procedures to govern the use of LLMs in the enterprise. Companies that fail to address these risks may find themselves exposed to significant security and compliance risks, making it essential for them to stay vigilant and proactive in the face of this emerging threat.

Source

Original Article: Your developers are already running AI locally: Why on-device inference is the CISO’s new blind spot

Published: April 12, 2026

---

Disclaimer: This article is for informational purposes only and does not constitute financial advice. Consult a licensed financial advisor before making investment decisions.