LLM-Assisted Penetration Testing #

Thesis ID: 26-016
Research Proposal: Using Large Language Models to Assist Penetration Testing of Web Applications
Primary focus: Web applications
Optional sub-angle: Software / source-code vulnerability discovery (only with a strong, well-scoped case)
Supervisor: Dr. Emre Süren

Abstract #

Large Language Models (LLMs) are increasingly used as tools in offensive security—not as the target, but as an assistant for finding and exploiting weaknesses in conventional systems. Recent work shows that LLM agents can help probe websites, reason about HTTP interactions, and in some settings exploit known or novel vulnerabilities with limited human guidance.

This topic is about using the AI as a tool. The default scope is web application penetration testing: recon, vulnerability discovery, exploit drafting, and evaluation against realistic targets (labs, intentionally vulnerable apps, or authorized systems). A software/code-focused proposal (e.g. LLM-assisted static analysis or exploit generation from source) is possible if the student makes a strong case and keeps the scope clear.

This is distinct from red teaming AI systems (Thesis 24-08), which asks how to break LLMs themselves.

Details

1. Problem Statement #

Manual web application penetration testing is expert-intensive and hard to scale. LLMs may accelerate reconnaissance, payload generation, traffic interpretation, and multi-step exploitation—but their reliability, coverage, and failure modes are still poorly characterized. This research should rigorously evaluate how well LLMs assist (or automate parts of) web app pentesting, compared to traditional methods and tools.

2. Objectives #

Define a web-app-centric threat model and evaluation methodology (targets, success criteria, safety constraints).
Assess LLMs / LLM agents on tasks such as mapping attack surface, finding common web flaws (e.g. injection, auth/session issues, access control, XSS), and generating or refining exploits.
Measure effectiveness (true/false positives, time-to-finding, severity) against baselines (manual testing, scanners, scripted tools).
Document limitations, hallucinations, and operational risks of LLM-assisted offensive workflows.
Optionally (strong case only): extend or pivot to source-code / software vulnerability discovery with clear metrics and datasets.

3. Research Methodology #

Target environment — Intentionally vulnerable web apps, CTF-style labs, or authorized testbeds; prefer realistic multi-page applications over toy snippets.
Model and agent setup — Commercial and/or open models; optional tool-using agents (browser, HTTP client, scanners) under controlled policies.
Task design — Reproducible scenarios covering common OWASP-style web vulnerabilities and multi-step attack chains.
Evaluation — Compare LLM-assisted workflows to traditional pentesting/scanning on the same targets; record success rates, cost, and human oversight needed.
Ethics and safety — Strict authorization boundaries; no testing of systems without permission; responsible handling of dual-use techniques.

4. Expected Outcomes #

Empirical results on when LLMs help (and when they fail) in web application penetration testing.
A reproducible methodology or prototype workflow for LLM-assisted web pentesting.
Guidance on human-in-the-loop use, limitations, and safe deployment in research settings.
Academic contribution to AI-assisted offensive security.

5. Timeline #

Phase	Duration
Literature review and methodology design	3–4 weeks
Lab / target setup and tooling	2–3 weeks
Experiments and evaluation	2–3 months
Analysis, comparison, refinement	2–3 weeks
Thesis writing and submission	2–3 weeks