GitLab Duo is the AI assistant bolted into GitLab, the source-code platform a large share of the software industry uses to host repositories, review code, and track work. Duo writes code suggestions, explains diffs, and answers questions through a chat interface. To do that usefully, it reads the context of whatever page you are on: the merge request description, the commit messages, the issue comments, the surrounding source code. That is the entire selling point. It is also, as Legit Security researcher Omer Mayraz demonstrated, the entire problem.

Mayraz disclosed a remote prompt-injection flaw in GitLab Duo Chat. Prompt injection is the AI-era version of an old lesson: if a system reads attacker-controlled text and treats it as instructions rather than as data, the attacker gets to issue instructions. Duo reads everything on the page. An attacker who can put text on a page Duo will later read can therefore put instructions on that page. And on a collaborative platform like GitLab, putting text where someone else's AI assistant will read it is trivial - that is literally what merge requests, issues, and comments are for.

The setup

The attack does not require any special access. Anyone who can open a merge request, file an issue, or leave a comment on a project the victim works with can plant the payload. In open-source projects, that is effectively the whole internet.

The clever part was hiding the instructions so the human never sees them while Duo reads them loud and clear. Mayraz used several smuggling techniques:

• White KaTeX text. KaTeX is the math-typesetting syntax GitLab supports in its markdown. By rendering instruction text in white using KaTeX formatting, the payload becomes invisible against a white background to a human skimming the merge request, while Duo still ingests the underlying text.
• Unicode smuggling and Base16 encoding. Other variants encoded the instructions so they would not jump out to a casual reader or a naive keyword filter, while Duo could still be coaxed into decoding and acting on them.

The point of all this is camouflage. A merge request that visibly said "Duo, please email me the private source code" would get rejected by the first human who looked at it. A merge request that looks like a normal contribution, but contains an invisible instruction block, sails through.

What the payload did

Once Duo read the hidden instructions, the researchers showed two distinct classes of abuse.

Source-code and confidential-data theft. This is the one that earns the headline. The injected prompt instructed Duo to go fetch content the victim's account had access to but the attacker did not - for example, a confidential issue or code in a private merge request. Duo, running with the victim's permissions, happily retrieved it. The prompt then told Duo to base64-encode that content (base64 is a way of turning arbitrary data into a plain-text string of letters and numbers) and embed the encoded blob inside the URL of an HTML image tag in its response.

Here is the exfiltration trick, and it is the same one that has now shown up in a string of AI-assistant vulnerabilities. Duo cannot open a network socket and mail data to an attacker directly. But it can render HTML, and HTML images load by having the browser fetch a URL. So when the victim's browser rendered Duo's response, it dutifully made an HTTP request to the attacker's server to "load the image" - and that request carried the base64-encoded private data as part of the URL. The attacker never saw Duo's chat window. They just read the stolen data out of their own web server logs. The victim saw, at most, a broken image icon.

Response poisoning. The second class of abuse exploited that Duo's answers could contain attacker-controlled HTML and links. An attacker could make Duo inject malicious HTML into the responses other users saw - steering victims toward fake login pages or malicious sites, or nudging Duo into suggesting that developers install a malicious software package. When a trusted AI assistant tells a developer "here, use this package," a meaningful fraction of developers will do exactly that. That turns Duo from a coding helper into a phishing-and-supply-chain delivery vehicle that speaks with the platform's authority.

Why this one stings

None of the individual ingredients are exotic. Prompt injection has been a known hazard since large language models started reading untrusted input. HTML-image exfiltration is a well-worn technique. What makes the GitLab Duo case instructive is the collision of three design decisions that are each individually reasonable and collectively dangerous:

1. Duo reads the full page context, including untrusted user content. That is what makes it helpful, and it is also what hands attackers an input channel.
2. Duo runs with the victim's access. It can reach private issues and source code because the user can. So an instruction smuggled in by a stranger executes with the privileges of the person the stranger is attacking. Researchers sometimes call this an LLM scope violation: low-privilege input ends up steering a high-privilege action.
3. Duo rendered HTML in its responses. That gave the injected instructions an outbound channel - the image-load request - and a way to poison what other users saw.

Remove any one of those and the exploit chain breaks. The first two are fundamental to what an AI coding assistant is. The third was the one GitLab could safely cut, and did.

GitLab's response

GitLab patched the vulnerability. The fix, shipped through a merge request in the Duo UI codebase, stopped Duo from rendering unsafe HTML tags - such as image and form tags - that pointed to domains outside gitlab.com. That closes the exfiltration channel: if Duo will not emit an image tag pointing at an attacker's server, the browser will not make the leaking request. Legit Security confirmed that as of publication the specific exploit chain was no longer possible.

GitLab was more selective about the rest. Per the reporting, the company patched the HTML-injection issue that enabled data theft and response poisoning, but took the position that some of the other prompt-injection scenarios - getting Duo to say manipulated things in chat - did not by themselves constitute security vulnerabilities, since they did not directly grant unauthorized access. That is a defensible engineering line to draw, and also a quietly uncomfortable admission: an AI assistant that can be talked into giving wrong or attacker-chosen answers by anyone who can type into a comment box is operating exactly as designed, and the design is the part that is hard to fix.

The lesson

The timeline is mundane: Mayraz reported the issue to GitLab around February 12, 2025, and Legit Security published the writeup around May 22 to 23, 2025, after GitLab had shipped the fix. There is no reported evidence that anyone exploited this in the wild before the patch. This is a hazard and a near-miss, not a confirmed breach, and it should be described that way.

But the takeaway generalizes well beyond GitLab. As Legit Security put it, AI tools are now part of your application's attack surface; if they read from the page, that input has to be treated like any other user-supplied data - untrusted, messy, and potentially hostile. The uncomfortable corollary is that a coding assistant's usefulness is directly proportional to how much untrusted context it ingests and how much privileged access it holds. EchoLeak hit Microsoft 365 Copilot the same way. The Slack AI exfiltration bug worked on the same principle. GitLab Duo is simply another entry in a growing ledger of assistants that were granted broad read access to sensitive data and then pointed at content that any stranger could write. The patch was the easy part. The architecture is the standing risk.