AI-Driven Insecurity: Assessing Security Gaps in AI Generated IT Guidance
The increasing reliance on AI-generated technical guidance for IT system configuration introduces significant security risks. This study assesses these risks through a case study: setting up an Apache web server on a Rocky Linux system using instructions from seven AI models. This inquiry also addresses the potential for over-reliance on AI and the possible erosion of cybersecurity skills among IT professionals.
The research demonstrates the variability and potential security gaps in AI-generated instructions by analyzing responses to two carefully designed prompts. The findings highlight that AI models, in their native state, often do not adequately account for cybersecurity best practices, and that security-focused prompts are essential to elicit more secure configuration guidance. These results emphasize the critical need for human oversight, validation, and security expertise in AI-driven IT operations.
SANS_AI_Driven_Insecurity_Assessing_Security_Gaps_in_ AI_Generated_IT_Guidance_Ed_Abbott (PDF, 0.51MB)
13 May 2025Related Content
A New Era in Vulnerability Management: A SANS Review of the Seemplicity Platform
Research PaperIn this paper, Dave Shackleford offers an inside look at Seemplicity, a vendor-agnostic remediation orchestration platform designed to unify vulnerability management across code, cloud, and infrastructure.
- 18 Aug 2025
- Dave Shackleford
Adopting an Offensive Security Posture: Strategies and Best Practices
Research PaperThis paper delves into essential concepts, and offers practical guidance for adopting an offensive security posture.
- 18 Aug 2025
- Jorge Orchilles
Enhanced Decisions with WatsonX: A Look at IBM QRadar Investigation Assistant
Research PaperThis paper examines IBM QRadar Investigation Assistant, an AI-powered tool that enhances SOC performance by streamlining incident triage, automating threat enrichment, and enabling natural language query capabilities.
- 6 Aug 2025
- Matt Bromiley
Balancing On-Prem and Cloud Security Strategic Considerations for Modern Organizations
Research PaperThis paper examines the strategic trade-offs between cloud and on-prem deployments, and the growing trend of consolidating tools into integrated security platforms.
- 30 Jul 2025
- Matt Bromiley
AI-Driven SecOps: Unifying Controls, Automating Response, and Advancing the Modern SOC Using Cortex XSIAM
Research PaperNew research from IDC reveals the tangible business value of rigorous, practitioner-led training from SANS: faster threat detection and response, reduced operational risk, stronger team cohesion, and millions in annual cost savings.
- 29 Jul 2025
- Dave Shackleford
Trust But Verify: Evaluating the Accuracy of LLMs in Normalizing Threat Data Feeds
Research PaperThis paper examines whether Large Language Models (LLMs) can be reliably applied to the normalization of Indicators of Compromise (IOCs) into Structured Threat Information Expression (STIX) format.
- 16 Jul 2025
Evaluating Zero Trust Network Access: A Framework for Comparative Security Testing
Research PaperWhile most evaluations rely on vendor checklists and surface-level comparisons, this white paper takes a different approach: building and applying a hands-on testing framework grounded in NIST SP 800-207 and the CISA Zero Trust Maturity Model.
- 11 Jul 2025
Do AI Coding Assistants Make Bad Coders Worse? A Security Evaluation of GitHub Copilot
Research PaperThis paper examines whether the overall security posture of a project affects the quality of the code produced by Copilot.
- 11 Jul 2025
Defense in Depth: Multiple Layers of Protection Fortifying Your Cyber Defenses
Research PaperDownload this paper and learn how to implement and evolve a Defense-in-Depth (DiD) strategy tailored to your organization’s risk profile, infrastructure, and cloud environment.
- 10 Jul 2025
- Ted Demopoulos
Dropzone AI Can Make Internal SOC Teams More Effective
Research PaperIn this paper, SANS Certified Instructor Mark Jeanmougin examines how Dropzone AI can integrate into existing security stacks and help SOC teams stay focused on high-impact decisions.
- 17 Jun 2025
- Mark Jeanmougin
Beneath the Mask: Can Contribution Data Unveil Malicious Personas in Open-Source Projects?
Research PaperIn February 2024, after building trust over two years with project maintainers by making a significant volume of legitimate contributions, GitHub user "JiaT75" self-merged a version of the XZ Utils project containing a highly sophisticated well-disguised backdoor targeting sshd processes running on systems with the backdoored package installed.
- 13 May 2025
- SANS Institute
Validating the Effectiveness of MITRE Engage and Active Defense
Research PaperThis research examines the impact of Active Defense compared to a traditional security posture when an adversary employs common tactics and techniques to identify high-value targets or exfiltrate sensitive data.
- 29 Mar 2025
Shift Left the Awareness and Detection of Developers Using Vulnerable Open-Source Software Components
Research PaperThe research presented in this paper demonstrates that companies can shift the detection and awareness of developers using vulnerable components left in the early development stages.
- 26 Mar 2025
Leveraging Large Language Models for Security-Focused Code Reviews
Research PaperThis study investigates the potential application of Large Language Models (LLMs) in enhancing software security through automated vulnerability detection during the code review process.
- 26 Mar 2025
Strolling Through the STIG
Research PaperThis research demonstrates how a new tool, Stroll, avoids the additional hardware requirements by living off the land.
- 7 Mar 2025
Building Resilient IoT Devices: Binary Hardening with Yocto and Clang
Research PaperThis paper addresses the critical need for enhanced security in Internet of Things (IoT) devices by evaluating the implementation of binary hardening techniques using Clang security features within the Yocto build environment.
- 3 Mar 2025
Harnessing Entra ID Snapshots for Effective Post-Security Incident Detection and Containment
Research PaperThis research focuses on implementing identity snapshots within Microsoft's Azure Entra ID, demonstrating their potential to significantly enhance the efficiency and effectiveness of post-incident detection and containment.
- 3 Mar 2025
Identifying Advanced Persistent Threat Activity Through Threat-Informed Detection Engineering: Enhancing Alert Visibility in Enterprises
Research PaperAdvanced Persistent Threats (APTs) are among the most challenging to detect in enterprise environments, often mimicking authorized privileged access prior to their actions on objectives. Moving within the environment slowly and quietly, APTs can often persist within the environment for months before detection.
- 20 Feb 2025
Evaluating Modern Network Protocol Fingerprinting: Defending Bastion Hosts in Hostile Networks
Research PaperAdversaries continue to attack the network perimeter and trusted user workstations to gain access to sensitive networks. Modern networks are designed and often mandated to use encrypted communication paths everywhere.
- 6 Feb 2025
MITRE ATT&CK Labeling of Cyber Threat Intelligence via LLM
Research PaperThis paper explores the effectiveness of various online and locally hosted LLMs in classifying an arbitrary statement as containing an MITRE ATT&CK Framework (MAF) technique or not and then producing the technique number if it does.
- 7 Jan 2025