AI-Driven Insecurity: Assessing Security Gaps in AI Generated IT Guidance
The increasing reliance on AI-generated technical guidance for IT system configuration introduces significant security risks. This study assesses these risks through a case study: setting up an Apache web server on a Rocky Linux system using instructions from seven AI models. This inquiry also addresses the potential for over-reliance on AI and the possible erosion of cybersecurity skills among IT professionals.
The research demonstrates the variability and potential security gaps in AI-generated instructions by analyzing responses to two carefully designed prompts. The findings highlight that AI models, in their native state, often do not adequately account for cybersecurity best practices, and that security-focused prompts are essential to elicit more secure configuration guidance. These results emphasize the critical need for human oversight, validation, and security expertise in AI-driven IT operations.
SANS_AI_Driven_Insecurity_Assessing_Security_Gaps_in_ AI_Generated_IT_Guidance_Ed_Abbott (PDF, 0.51MB)
13 May 2025Related Content
Risk-Adaptive Data Loss Prevention: Behavioral Intelligence with DLP
Research PaperRisk-Adaptive Data Loss Prevention: Behavioral Intelligence with DLP
- 4 Jun 2026
- Matt Bromiley
Bridging the Gap Between Threat Intelligence and Business Risk
Research PaperThe importance of the threat intelligence function has grown significantly over the years to become a cornerstone of any cybersecurity group.
- 29 May 2026
- Kevin Garvey
Secure By Design: An Exploration of the Application of Generative AI in Threat Modeling Technical Design Documents
Research PaperThis paper explores the efficacy of large language models (LLMs) for creating comprehensive threat models by analyzing technical design documents, particularly when provided with additional contextual information about the product's underlying infrastructure and deployment environment.
- 27 May 2026
Leveraging Large Language Models for Cross-Vendor Firewall Configuration Migration: A Comparative Case Study of Claude and ChatGPT
Research PaperThis paper investigates how two current-generation large language models (LLMs) perform on a single, representative firewall migration task.
- 12 May 2026
Autonomous Defense Induced Disruption: How AI-Driven Automated Response Can Be Manipulated to Disrupt Enterprise Operations
Research PaperThe research highlights the need for governance controls, privilege-aware safeguards, and system-level constraints to prevent autonomous containment from causing operational disruption.
- 12 May 2026
Your Sensitive Data Has Left the Chat: LLMs as Sensitive Data Detectors
Research PaperThis paper seeks to evaluate the hypothesis that language models, large and small, can perform well at sensitive data classification and to offer a solution for companies trying to detect contextually sensitive data in their AI workflows.
- 12 May 2026
Untested: An Overlooked Link in the Software Supply Chain
Research PaperThis research explores test code as an attack surface and takes a first step toward creating a tool to help analysts detect and mitigate malware lurking in test libraries.
- 16 Apr 2026
Cyber Risk Intelligence and Security Posture (CRISP): From Compliance to Threat-Informed Intelligence
Research PaperThis paper presents CRISP (Cyber Risk Intelligence & Security Posture), a platform that automates the transformation of STIG compliance data into threat-informed security intelligence.
- 7 Apr 2026
Leveraging Generative AI for Password Cracking Efficiency Under Resource Constraints
Research PaperThe purpose of this research is to investigate whether generative AI can alleviate the hardware and financial burdens of password cracking (password recovery) while maintaining or even improving cracking success rates.
- 20 Mar 2026
Enhancing Linux Threat Detection: A Sysmon - Based Approach to Identifying Sandworm TTPs
Research PaperLinux systems have become foundational across modern IT enterprises. Threat actors are increasingly targeting Linux systems, including well - known advanced persistent threats (APTs) such as Sandworm.
- 20 Mar 2026
Open-Source National Security Infrastructure for Sweden’s National Security Apparatus
Research PaperThis paper investigates whether core IT infrastructure implemented using open-source software and infrastructure-as-code techniques can achieve compliance with selected information security requirements defined in Chapter 4 of PMFS 2022:1.
- 18 Mar 2026
Detecting AI Pickling
Research PaperThis study examines whether static analysis is a dependable "certification gate" for ingesting third-party, pickle-based AI model artifacts from open-source model hubs into a trusted internal registry.
- 12 Mar 2026
How Many LLMs Does it Take to Classify a Suspicious Email?
Research PaperThis study examines the accuracy, reliability, and operational behavior of three widely available LLMs using a dataset of 2000 human-written emails containing both legitimate and suspicious messages.
- 12 Mar 2026
Autonomous Threat Emulation and Detection Using Agentic AI
Research PaperTraditional threat emulation frameworks struggle to capture the dynamic and adaptive behaviours of modern Advanced Persistent Threats (APTs), leaving defenders reliant on static tests that quickly become obsolete.
- 10 Mar 2026
Evaluating Configurations for Reducing Problematic Emotional Engagement in Enterprise LLM Deployments: Implications for Insider Threat Risk
Research PaperThe risks of Large Language Models (LLMs) include triggering psychological drivers associated with malicious insider threat behavior. This study utilized AWS Bedrock to demonstrate that specific system-level configurations and guardrails can effectively mitigate these risks by reducing problematic human-AI engagement.
- 2 Mar 2026
Configuring Windows 11 Workgroup Computers to CIS Windows 11 L1 and BitLocker Baseline Recommendations Using PowerShell DSC
Research PaperEndpoints are often the first points of cyberattacks. Enterprises would often try to harden them according to established security baselines, such as those published by the Center for Internet Security (CIS).
- 24 Feb 2026
Infrastructure as Code-Driven Group Policy Infrastructure: A Comprehensive Engine for Group Policy Architecture and Enforcement
Research PaperThis study introduces a PowerShell-based Infrastructure as Code (IaC) engine developed to automate the setup and enforcement of a STIG-compliant Group Policy framework.
- 5 Dec 2025
No-Cost Detection of Endpoint Hard Drive Removal
Research PaperThis paper analyzes low-cost detection methods, using existing hard drive counters from Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) and the Windows Registry, for their fidelity in detecting hard drive removal.
- 19 Nov 2025
Defending Vulnerable Populations Against Scams: Effectiveness of Browser Extensions in Mitigating Scammer Attack Chains
Research PaperThis research evaluates the effectiveness of a browser extension as a security control—Grandma’s Guardian—designed for simplicity and accessibility so that even non-technical home users can benefit from enterprise-grade protection.
- 19 Nov 2025
Automating Generative AI Guidelines: Reducing Prompt Injection Risk with 'Shift-Left' MITRE ATLAS Mitigation Testing
Research PaperAutomated testing during the build stage of the AI engineering life cycle can evaluate the effectiveness of generative AI guidelines against prompt injection attacks.
- 7 Nov 2025