Large Language Models as a (Bad) Security Norm in the Context of Regulation and Compliance

The use of Large Language Models (LLM) by providers of cybersecurity and digital infrastructures of all kinds is an ongoing development. It is suggested and on an experimental basis used to write the code for the systems, and potentially fed with sensitive data or what would otherwise be considered trade secrets. Outside of these obvious points, this paper asks how AI can negatively affect cybersecurity and law when used for the design and deployment of security infrastructure by its developers. Firstly, the paper discusses the use of LLMs in security, either directly or indirectly, and briefly tackles other types of AI. It then lists norms in cybersecurity, then a range of legal cybersecurity obligations from the European Union, to create a frame of reference. Secondly, the paper describes how LLMs may fail to fulfil both legal obligations and best practice in cybersecurity is given, and the paper ends with some economic and practical consequences for this development, with some notions of solutions as well. The paper finds that using LLMs comes with many risks, many of which are against good security practice, and the legal obligations in security regulation. This is because of the inherent weaknesses of LLMs, most of which are mitigated if replaced with symbolic AI. Both also have issues fulfilling basic traceability obligations and practice. Solutions are secondary systems surrounding LLM based AI, fulfilment of security norms beyond legal requirements and simply not using such technology in certain situations.