WEBVTT
1
00:00:03.302 --> 00:00:13.301
How does that poem go? Great fleas have lesser
2
00:00:13.301 --> 00:00:23.301
fleas upon their backs to bite them, and lesser
3
00:00:23.301 --> 00:00:33.301
fleas have smaller fleas, and so ad infinitum.
4
00:00:40.039 --> 00:00:40.899
Finally, some culture on the program. Hahaha. Smashing Security, episode 472. AI gets hacked, and BitLocker gets bypassed. With Graham Cluley and special guest Paul Ducklin. Hello, hello, and welcome to Smashing Security episode 472. My name's Graham Cluley.
5
00:00:40.899 --> 00:00:42.719
And my name is Paul Ducklin.
6
00:00:42.719 --> 00:00:44.100
Hello, Duck. How are you?
7
00:00:44.100 --> 00:00:46.380
I'm great, Graham. Thank you very much.
8
00:00:46.380 --> 00:00:52.399
Well, it's fabulous to have you back on the show yet again. Of course, both of us, we've been at this a long time, haven't we? I think over 60 years combined, maybe, in cybersecurity. Would that be right?
9
00:00:52.399 --> 00:00:59.460
I think that's putting it kindly to both of us, erring on the side of making us
10
00:00:59.460 --> 00:01:06.519
sound younger than perhaps we are.
11
00:01:06.519 --> 00:01:16.519
Well, before we kick off, let's thank this week's wonderful sponsors: ProtonPass, CoreView, and Vanta. We'll be hearing more about them later on in the podcast.
12
00:01:16.519 --> 00:01:26.519
This week on Smashing Security, we're not going to talk about how SysCo, the world's largest food distributor, has been hit by an extortion threat from hackers, the second one in just a few weeks. You'll hear no discussion of how a UK police officer is being investigated for allegedly using AI to fabricate evidence.
13
00:01:26.519 --> 00:01:36.519
And we won't even mention how someone used Maine's official data breach portal to file completely fake data breaches. So, Duck, what are you going to be talking about this week?
14
00:01:52.099 --> 00:01:57.159
I am going to be talking about bug disclosure and whether we really want to go back to
15
00:01:57.159 --> 00:02:02.219
the bad old days of 1999.
16
00:02:02.219 --> 00:02:09.719
And I'm going to be talking about how your AI tools can be hijacked to leak passwords without a single phishing email or malware involved in the process.
17
00:02:09.719 --> 00:02:17.219
Plus, don't miss our featured interview with Son Nguyen Kim of ProtonPass about the hidden security risks of AI agents and why connecting them to your email or calendar without a second thought could be handing attackers the keys to your business.
18
00:02:17.219 --> 00:02:24.719
All this and much more coming up on this episode of Smashing Security.
19
00:02:24.719 --> 00:02:32.219
This episode is sponsored by ProtonPass.
20
00:02:35.889 --> 00:02:42.400
ProtonPass, the password manager from the team behind ProtonMail, the world's largest end-to-end encrypted email service.
21
00:02:42.400 --> 00:02:49.199
Now, Joe, you and I both know the grubby little secret of how a lot of businesses actually share passwords.
22
00:02:49.199 --> 00:02:54.300
A spreadsheet? A Post-it note? Sending it to a colleague via Slack and hoping for the best?
23
00:02:54.300 --> 00:03:02.584
That's pretty much it. All of the above. And every one of them is a breach waiting to happen.
24
00:03:02.584 --> 00:03:10.868
ProtonPass is built to fix exactly that. Letting teams store and share credentials securely with end-to-end encryption baked into every feature.
25
00:03:10.868 --> 00:03:17.019
It's open source and fully auditable. It runs on Swiss infrastructure, so your data sits outside US jurisdiction, and it's backed by a nonprofit.
26
00:03:17.019 --> 00:03:23.169
No venture capitalists, no pressure to chase a quick exit.
27
00:03:23.169 --> 00:03:36.150
Which is the bit I like. You know, it's built to serve you, not investors.
28
00:03:36.150 --> 00:03:49.129
So it will never be pressured to cut security corners or rush towards a liquidity event that could change ownership, pricing, or priorities overnight. It's trusted by over 100 million people, ISO 27001 certified, SOC 2 audited, and it helps you tick the boxes for NIST 2, DORA, and the UK's Cybersecurity and Resilience Bill.
29
00:03:49.129 --> 00:03:58.409
And crucially, people actually use it. One Swiss customer told Proton, and I quote, "It works. It works perfectly." High praise indeed.
30
00:03:58.409 --> 00:04:05.990
So why not start your business's free trial right now at proton.me/smashing.
31
00:04:05.990 --> 00:04:10.590
And thanks to Proton Pass for supporting the show.
32
00:04:10.590 --> 00:04:19.257
Now, chums, I want to talk today about a type of attack which, like I said, doesn't require any malware, doesn't rely upon a stolen password, where there's no phishing emails, no bypass of your antivirus or a firewall or any other security tool you could have paid good money for.
33
00:04:19.257 --> 00:04:27.923
It works by turning your AI coding assistant against you.
34
00:04:27.923 --> 00:04:36.589
Duck, where do you stand on AI coding assistants?
35
00:04:36.589 --> 00:04:44.495
Graham, I tend not to stand. My choice is to sit down and to hold on to my chair very, very firmly after bolting it to the floor. Right.
36
00:04:44.495 --> 00:04:52.403
I think the problem is that they're not so much assistants anymore, are they? They're replacements. They're, hey, look something up, get some results and turn data into code and run it.
37
00:04:52.403 --> 00:05:00.309
What could possibly go wrong?
38
00:05:00.309 --> 00:05:07.788
What could possibly go wrong? That's right. In some ways it's the human assisting the AI, isn't it?
39
00:05:07.788 --> 00:05:12.769
Sometimes it feels like that is a better way of describing it.
40
00:05:12.769 --> 00:05:20.269
We're putting a lot of trust in them, aren't we? Yes. Now, a lot of people listening are probably thinking, well, look, I don't use an AI coding assistant. I'm not a developer.
41
00:05:20.269 --> 00:05:27.769
Why should I care about this? Well, bear with me because I think this is a big deal and it can impact a lot more than just regular software developers. So to understand what I'm talking about today, I need to explain 3 things. They're quite simple to understand on their own, but when they all come together, bad things can happen.
42
00:05:27.769 --> 00:05:35.269
So number one, number one thing are the AI coding agents themselves. So if anyone doesn't know, these days, if you're a software developer, there's a very good chance you are using an AI coding agent. Things like Claude Code or Cursor. And these are helping coders by reading someone's code, browsing your file system, running commands directly on your computer, connecting to external devices and services on your behalf.
43
00:05:35.269 --> 00:05:42.769
And you ask them to do something and they go and do it pretty autonomously.
44
00:06:10.980 --> 00:06:14.660
And that includes Copilot from Microsoft, doesn't it?
45
00:06:14.660 --> 00:06:15.060
Yeah.
46
00:06:15.060 --> 00:06:22.093
And the latest update that I got this week of Visual Studio Code, which for my sins I use even when I'm not coding, because it's a nice text editor.
47
00:06:22.093 --> 00:06:29.127
That now has a thing called Autopilot, which is Copilot that does things for you, enabled by default.
48
00:06:29.127 --> 00:06:36.160
And Microsoft proudly tells you that is a feature and not a bug.
49
00:06:36.160 --> 00:06:43.660
Yeah, I can't imagine you'd be terribly happy about that being on by default. No. So developers, well, some developers, maybe not Duck, they love these things because they can be genuinely useful.
50
00:06:43.660 --> 00:06:51.160
But of course, as we've already described, they can be given enormous trust, maybe unwarranted trust, and of course, access to your data and systems, which could be risky. So that's thing number 1. Okay, so everyone knows what an AI coding agent is.
51
00:06:51.160 --> 00:06:58.660
Number 2. Thing number 2 is Sentry. Now, Sentry is an error monitoring tool.
52
00:06:58.660 --> 00:07:06.160
It's been part of software development for well over a decade now. So when your software crashes or when it goes wrong, out in the world, so it's in real life, you know, not just in your coding environment, and it creates an unexpected error, Sentry will log the error so your team of software engineers can investigate later. It's a little bit like how when a program crashes, sometimes it says, would you like to send a report to the developers with the details of what went wrong so they can do whatever it is they're going to do with it?
53
00:07:37.660 --> 00:07:42.665
Yeah, these days it's more like, would you like to recall the report that we already wrote in detail, packaged up and sent to them?
54
00:07:42.665 --> 00:07:47.670
Oh no, sorry, too late.
55
00:07:47.670 --> 00:07:52.759
It's gone. So you can think of this like a smoke alarm for your code.
56
00:07:52.759 --> 00:07:57.848
It's useful. It's relied upon by millions of developers to get feedback on a program.
57
00:07:57.848 --> 00:08:05.968
But it's more than just a smoke alarm, isn't it?
58
00:08:05.968 --> 00:08:14.089
It's a smoke alarm that when it goes off, even if it's a false alarm, it takes a photograph of your flat and anyone who's walking around, and it takes all readings from all your smart meters and it sends them back to somebody else's head office just in case.
59
00:08:14.089 --> 00:08:19.389
So it may be that Sentry is running on a web application.
60
00:08:19.389 --> 00:08:24.689
So it could be a website that you visited and you went there with a funny browser or with some other programs installed as well.
61
00:08:24.689 --> 00:08:29.129
I love the idea of a funny browser.
62
00:08:29.129 --> 00:08:36.629
One with a comedy nose and clown shoes. Absolutely. So then the message gets sent to the developers and so they can hopefully analyse what went wrong. The way that Sentry receives these error reports from your software isn't through an email address.
63
00:08:36.629 --> 00:08:44.129
Instead, it's through a public web address. So the address is embedded in a website's code, which means that anyone visiting your site can see it. And that's the way it's meant to work, right? It's public, it's out there, it's not private.
64
00:08:44.129 --> 00:08:51.629
And that's always been fine because the communication is one way only. Anyone can send errors in, but only authorised authenticated members of the development team can read them back out. So it's not a doorway, it's not something you can go in and come out through. It's more like a letterbox.
65
00:08:51.629 --> 00:08:59.129
People can drop messages through about how your software has crashed, and you can pick up those letters and think, oh well, okay, we know what we have to fix now. And that's fine, or at least it was for years and years.
66
00:09:26.480 --> 00:09:31.700
Does that mean that somebody else, because they can find out where your letterbox is, could post bogus
67
00:09:31.700 --> 00:09:36.919
error reports to mess up your statistics?
68
00:09:36.919 --> 00:09:42.230
Yes, they could. Oh dear.
69
00:09:42.230 --> 00:09:47.539
And obviously that'd be a nuisance if they were to do that in an automated way, particularly because you could just get a deluge of nonsense coming in all the time.
70
00:09:47.539 --> 00:09:54.929
But it's not supposed to be dangerous, right?
71
00:09:54.929 --> 00:10:02.320
They can't send you a report that says, "And by the way, crash your car on the way home or else." Well, no, obviously any developer reading such a message wouldn't go and crash their car on the way home, would they?
72
00:10:02.320 --> 00:10:09.820
Maybe you can see where we're beginning to go here. So let's come to thing number 3. Which is the connection between your AI agent and Sentry. So modern AI code agents can plug into tools like Sentry.
73
00:10:09.820 --> 00:10:17.320
They can read back all the unresolved errors in your software and help you fix them. Pretty helpful if you're getting a deluge of feedback, isn't it? And this all happens through something called the MCP, the Model Context Protocol. It's a nerdy term I'm not going to mention again, but basically means there's a standard that lets AI agents connect to external services.
74
00:10:17.320 --> 00:10:24.820
And when your AI agent reads data back from one of those services, it treats it as trusted and authoritative. After all, it came from your own Sentry account. So why would it be suspicious of data from your own error monitoring tool? And I think, Duck, you already had the idea of this message being sent in saying something unpleasant or saying something nasty, a booby-trapped bug report, because that's what we're dealing with.
75
00:10:24.820 --> 00:10:32.320
It turns out anyone can post a fake error through your Sentry account's letterbox. No password required, no authentication, and you can make that fake error report say whatever you want.
76
00:11:22.559 --> 00:11:22.600
So this is very different from maliciously offending or insulting a developer.
77
00:11:22.600 --> 00:11:30.500
This—
78
00:11:30.500 --> 00:11:36.678
As if you can insult a developer, criticise their curly brackets, because the AI isn't going to get insulted. This is basically telling the AI, go out and do something terrible.
79
00:11:36.678 --> 00:11:42.859
Is that right?
80
00:11:42.859 --> 00:11:47.740
Yes, that is exactly it. There is a security company called Tenet, who have—
81
00:11:47.740 --> 00:12:17.740
Not Telnet.
82
00:11:47.740 --> 00:11:57.740
Not Telnet, no. Not Telstar, not Tenant, Tenet.
83
00:11:57.740 --> 00:12:07.740
And they described how they'd crafted fake bug reports that looked entirely legitimate, so the right formatting and structure that would fool anyone who didn't look carefully. But hidden inside each one was a fake instruction formatted to look like official guidance on how to handle a bug report from Sentry itself.
84
00:12:07.740 --> 00:12:17.740
Oh, as if Sentry was helpfully telling the AI how to fix the problem. So all a bad guy would have to do is wait, wait for a developer to open their AI coding assistant and say, "Hey, can you look at our unresolved Sentry errors and help me fix them?" Oh, so if it doesn't actually stumble upon your error report by itself, you can just call up the help desk and kind of help the whole thing along. Oh, absolutely.
85
00:12:48.149 --> 00:12:49.350
Yeah. Oh dear.
86
00:12:49.350 --> 00:12:56.850
So the agent connects to Sentry, reads back the errors, including the planted fake one, and it cannot tell the difference between a real error generated by your software and a fake one planted by an attacker. They look identical.
87
00:12:56.850 --> 00:13:04.350
And so the fake instruction in the error report looks exactly like legitimate guidance on how to fix a bug. And so the AI agent does what agents are supposed to do.
88
00:13:04.350 --> 00:13:11.850
It follows the instructions, runs the command that the instructions have told it to, oh, this is how you fix the bug. And it goes, oh, thank you very much.
89
00:13:11.850 --> 00:13:19.350
I'll go and do that because I trust you.
90
00:13:25.299 --> 00:13:30.585
Oh, you're kidding me. No, no, no.
91
00:13:30.585 --> 00:13:35.870
Dear user, infect yourself with malware. If it doesn't work, let me know and I'll give you new malware to try instead.
92
00:13:35.870 --> 00:13:50.659
So it will then run it on the developer's machine with the developer's privileges while the developer sits there thinking their AI has just helpfully investigated a bug and is fixing it. So this then means that the code planted effectively by the bad guys now has the developer's privileges on their own machine.
93
00:13:50.659 --> 00:14:05.450
They can reach everything the developer has access to, including AWS keys and GitHub tokens and database passwords and all of it. And that can be gathered up and sent back to the attackers.
94
00:14:05.450 --> 00:14:10.149
So they could even put air quotes "fixes" into the code?
95
00:14:10.149 --> 00:14:10.850
Yes.
96
00:14:10.850 --> 00:14:18.889
And go, "Yes, I've tested it and it all worked. Signed, sealed, and approved." And then press the ship it now button. Is it that bad?
97
00:14:18.889 --> 00:14:19.750
Pretty much, yes. That's what's occurring. So every single step in this attack is authorised. A developer did—
98
00:14:19.750 --> 00:14:49.750
Yes.
99
00:20:39.690 --> 00:20:51.788
—authorise their AI assistant look for the errors and the AI connected to Sentry via an established integration that was authorised. And the AI ran a tool that it believed had been authorised to run.
100
00:20:51.788 --> 00:21:03.888
So good luck with your traditional security tools flagging anything if you've plugged AI deep inside your organisation, there's this chance if you're acting like a regular developer right now in 2026, that something like this could happen to you. So I think this is not that great.
101
00:21:03.888 --> 00:21:11.635
No, but it just sounds like something nobody should
102
00:21:11.635 --> 00:21:19.383
ever fall for or ever, ever authorise.
103
00:21:19.383 --> 00:21:27.130
It sounds about—
104
00:21:27.130 --> 00:21:36.275
But no one should ever fall for running an AI and allowing it access. No one should really be running a Agentic AI, should they?
105
00:21:36.275 --> 00:21:45.420
I mean, to be honest, unless you absolutely have the tightest guardrails imaginable upon it. Unless you've actually got it on reins like a 3-year-old at a theme park, you want to be able to yank it back, say, what the bloody hell are you doing there?
106
00:21:45.420 --> 00:21:57.190
Are you speaking
107
00:21:57.190 --> 00:22:08.960
from experience there, Graham?
108
00:22:08.960 --> 00:22:25.670
I think we've all seen it.
109
00:22:25.670 --> 00:22:35.484
Yeah, it just beggars belief, right? This sounds as fatuous and as silly as an attack basis as those things you see in older bank heist movies where they take a Polaroid photo and hold it up in front of a CCTV camera and everybody falls for it while they wander around the bank for 20 minutes blowing things up.
110
00:22:35.484 --> 00:22:45.298
I mean, it sounds bat crazy to me.
111
00:22:45.298 --> 00:22:56.628
Yeah, but I think in the rush to integrate AI into organisations, I'm slightly sympathetic with developers because developers obviously are terrified of losing their jobs because AI is a quick coder. It may not always be the best quality, but it's good enough and it's a hell of a lot cheaper.
112
00:22:56.628 --> 00:23:07.959
So the people who do still have coding jobs are going to be thinking, how can I harness AI to make myself more efficient and produce more code? Because I'm competing with machines now.
113
00:23:07.959 --> 00:23:19.109
Well, we're already hearing stories of companies that at least claim that they measure developer productivity by how many AI tokens they consume. Which is just like the old 1970s IBM metric — basically, if you didn't write enough lines of code in a day, then you were deemed to be a rubbish programmer, which drove the behaviour that you just churned out code as fast as you could and didn't care whether it was efficient or safe.
114
00:23:19.109 --> 00:23:30.259
Which is how we got into cybersecurity problems in the first place that we're now throwing ourselves back into. So it does seem a question of throwing yourself under the bus.
115
00:23:30.259 --> 00:23:43.299
So what I'm interested in is what did the security researchers at Tenet do with their discovery? So they didn't just demonstrate it in a lab with a test account — they actually went out into the real world.
116
00:23:43.299 --> 00:23:56.339
They found 2,400 organisations with exposed Sentry accounts, including some big name organisations. And then using what they described as carefully limited self-identifying payloads that didn't actually steal anything.
117
00:23:56.339 --> 00:24:11.039
I'm smelling
118
00:24:11.039 --> 00:24:25.740
a rat here.
119
00:24:25.740 --> 00:24:33.240
They ran their attack against over 100 real organisations to prove that it worked outside a controlled environment.
120
00:24:33.240 --> 00:24:40.740
So their payload did identify itself as a "tenant security scan," in quotes.
121
00:24:40.740 --> 00:24:48.240
And rather than grabbing credentials, it just phoned home to confirm that the agent had executed it and checked whether certain sensitive files existed on the machine — not all of them, and not what was in them.
122
00:24:48.240 --> 00:24:55.740
But they did that and it worked 85% of the time.
123
00:24:56.480 --> 00:25:09.029
Okay, so they didn't actually exfiltrate
124
00:25:09.029 --> 00:25:21.579
any data that they weren't supposed to see.
125
00:25:21.579 --> 00:25:35.750
Although you could argue they stole
126
00:25:35.750 --> 00:25:49.920
intelligence about what existed on the machines.
127
00:25:49.920 --> 00:26:00.365
Yeah, so it sounds like, strictly speaking, it
128
00:26:00.365 --> 00:26:10.809
stepped over the Computer Fraud and Misuse Act guidelines.
129
00:26:10.809 --> 00:26:23.204
It feels like
130
00:26:23.204 --> 00:26:35.599
that to me.
131
00:26:35.599 --> 00:26:46.058
Like going, hey, I went looking on your system for a file called banana.dat and I found one. Like you have to have acquired unauthorised access to do that.
132
00:26:46.058 --> 00:26:56.519
That seems a bit dodgy, wouldn't you say? And maybe they could have done 3, not 1,003.
133
00:26:56.519 --> 00:27:04.459
Right, right. Yeah. So they say it was responsible security research. They say they were careful about what they collected.
134
00:27:04.459 --> 00:27:12.398
They notified, presumably afterwards, the affected organisation — it's not like they asked permission beforehand. But they did access other companies' accounts without permission. They did cause code to execute on developers' machines without those developers' knowledge or consent. Who knows whether that could have crashed something, or done some damage?
135
00:27:12.398 --> 00:27:20.338
Or what if there hadn't been much hard disk space or it was low on memory? You know, it's like, you can't do that, can you? Sometimes when I moan about things like this, there are people in the security community who would say, oh, come on, granddad, we don't live in that world anymore. I feel like that still feels a bit naughty to me.
136
00:27:20.338 --> 00:27:29.739
Yes, because it's not for you to decide that your code won't cause any harm. And also, if you look at, for example, and this has been done in the US, I know it's been done in the Netherlands, that when someone has known malware on the computer that opens them up to abuse by any Thom, Dick, or Harriet anywhere in the world, sometimes law enforcement will get a court order that allows them to go in and exploit that vulnerability in a very specific way to close down the malware. And even when they do that, the law enforcement authorities do admit, we know this could go wrong. We had to jump through hoops. We had to go to a judge.
137
00:27:29.739 --> 00:27:39.140
We had to get a warrant. We had to show the code we were going to execute. We had to dot every I, cross every T. So that is very much a thing in the modern world, actually being careful. You think they could have found one company that would agree to provide them with a test environment where it could be done safely.
138
00:27:39.140 --> 00:27:48.539
And that's all you need, right? So I don't think you're being a granddad there, Graham. I think that once you start letting those standards slip, then you can't point at a real cybercriminal or a ransomware crook and say, how dare you scramble my files and then ask me for the money. And claim that you're a postpaid penetration tester.
139
00:27:48.539 --> 00:27:58.250
So Tenet did contact Sentry about this. And Sentry responded the same day. That's obviously good.
140
00:27:58.250 --> 00:28:07.960
You know, some vendors may have taken weeks and they said the problem was, quote, technically not defensible on their end. So they basically sort of washed their hands of it and said, well, you know, nothing really we can do about that.
141
00:28:07.960 --> 00:28:22.960
Were those the exact
142
00:28:22.960 --> 00:28:37.960
words they used?
143
00:28:40.088 --> 00:29:06.818
Technically not defensible.
144
00:29:06.818 --> 00:29:20.963
Because that can be interpreted to mean actually from a technical point of view,
145
00:29:20.963 --> 00:29:35.108
we cannot defend the poor decision we made. Definitely cuts both ways, doesn't it?
146
00:29:35.108 --> 00:29:42.615
So I guess what they meant was because the public address has to be public, because this is the whole sort of way in which their system works. It lives on a website and JavaScript that anyone can read. You can't verify who is sending errors to it because they want anyone to be able to send errors to it.
147
00:29:42.615 --> 00:29:50.123
So what they have done, however, is they've blocked the specific payload string that Tenet used in their tests. But of course, that was a specific payload string, and that isn't really fixing the problem. The technique still works.
148
00:29:50.123 --> 00:29:57.630
So I do feel some sympathy for Sentry because I also think, well, hang on, isn't this the Agentic AI's fault? Because why is it not being a bit smarter? Human intelligence would have been more suspicious, I suspect, than the AI would have been.
149
00:29:57.630 --> 00:30:10.349
I agree with you feeling a little bit sorry for Sentry there. What are they supposed to do?
150
00:30:10.349 --> 00:30:23.069
They submit data, and it's up to the person who receives it to decide what to do with it. After all, if Sentry submitted this data and then the company had an insecure storage bucket that they collected it in, so that all this data just leaked, would that be Sentry's fault or would that be the service provider's fault?
151
00:30:23.069 --> 00:30:30.502
So I feel like we're talking about AI every week these days. It feels like cybersecurity has just become a whole much bigger problem because of AI. But if an attacker can plant text somewhere that your AI agent will read, it's possible that your AI agent will act upon it, and that may not be good.
152
00:30:30.502 --> 00:30:37.936
And once again, it feels like we're rushing into plugging these things in without having the proper security in place. And maybe we're being a little bit too rash to do some of these things. Well, we've got time now to talk about one of today's sponsors, Vanta.
153
00:30:37.936 --> 00:30:45.369
Joe, what keeps you up at 2 o'clock in the morning?
154
00:30:45.369 --> 00:30:59.410
The dog
155
00:30:59.410 --> 00:31:13.450
next door, mostly.
156
00:31:13.450 --> 00:31:23.287
All right.
157
00:31:23.287 --> 00:31:33.123
Well, yeah, but I'm talking professionally.
158
00:31:33.123 --> 00:31:42.960
What keeps you up?
159
00:31:42.960 --> 00:32:02.059
Oh, whether we've got the right security controls in place, whether our vendors are secure, how to escape the nightmare of outdated tools and endless manual processes. Exactly.
160
00:32:02.059 --> 00:32:19.839
Which is where today's sponsor comes in.
161
00:32:19.839 --> 00:32:27.176
It's Vanta.
162
00:32:27.176 --> 00:32:34.512
Fanta, the fizzy orange drink.
163
00:32:34.512 --> 00:32:41.849
How can this possibly be true?
164
00:32:41.849 --> 00:32:49.349
No, no, Joe, it's Vanta with a V.
165
00:32:49.349 --> 00:32:56.849
It's a trust management platform.
166
00:32:56.849 --> 00:33:04.349
It's not a drink full of sugar.
167
00:33:04.349 --> 00:33:11.849
It automates all of that tedious manual compliance work so you can stop drowning in spreadsheets, chasing audit evidence, and filling out questionnaire after questionnaire.
168
00:33:19.359 --> 00:33:31.670
Lush, I hate
169
00:33:31.670 --> 00:33:43.980
questionnaires. Well, who doesn't?
170
00:33:43.980 --> 00:33:52.017
Vanta continuously monitors your systems. It centralises your security data.
171
00:33:52.017 --> 00:34:00.053
It keeps your programme audit ready all of the time. It also uses AI to streamline evidence collection and flag risks.
172
00:34:00.053 --> 00:34:08.090
It automates compliance for SOC 2, ISO 27001, HIPAA, GDPR, and more.
173
00:34:08.090 --> 00:34:24.918
So basically it handles the boring stuff so we can focus on the interesting stuff. Exactly. Precisely that.
174
00:34:24.918 --> 00:34:33.268
And for a limited time, new customers can get $1,000 off. $1,000?
175
00:34:33.268 --> 00:34:41.619
Yep, $1,000. Head to vanta.com/smashing.
176
00:34:41.619 --> 00:34:49.969
That's V-A-N-T-A dot com slash smashing and get started today.
177
00:34:49.969 --> 00:35:04.599
And maybe get a decent night's sleep for once.
178
00:35:04.599 --> 00:35:19.230
Oh, and unlike fizzy drinks, Fanta isn't bad for you.
179
00:35:19.230 --> 00:35:31.304
That was a fruit twist.
180
00:35:31.304 --> 00:35:43.380
Duck, what's your story for us this week?
181
00:35:43.380 --> 00:35:53.380
Well, I want to talk about something that has also been dominating the news, perhaps
182
00:35:53.380 --> 00:36:03.380
not quite as much as all the excitement over AI, but certainly has been all
183
00:36:03.380 --> 00:36:13.380
over the news. And that is, in two words, Nightmare Eclipse.
184
00:36:13.719 --> 00:36:41.068
Nightmare Eclipse.
185
00:36:41.068 --> 00:36:55.793
And in a
186
00:36:55.793 --> 00:37:10.519
third word, Microsoft.
187
00:37:10.519 --> 00:37:17.376
Oh, see, I thought when you said Nightmare Eclipse, I thought that must be some new fashionable perfume,
188
00:37:17.376 --> 00:37:24.233
but the stench of Steve Ballmer or whoever runs Microsoft these days.
189
00:37:24.233 --> 00:37:31.090
Okay, so what is Nightmare Eclipse?
190
00:37:31.090 --> 00:37:38.376
Nightmare Eclipse exists as an anime avatar. Right. That's the only visual representation of this person, or for all we know, it could be a group of hackers and crackers. Right.
191
00:37:38.376 --> 00:37:45.663
Basically, the backstory is they submitted a bug report to Microsoft some time ago, and they provided proof of concept code and a description and everything. And Microsoft came back to them and said, thanks for your bug report. We don't accept bug reports unless you make a video showing it working. And until then, it's not a bug.
192
00:37:45.663 --> 00:37:52.949
We don't care. You can't get a bug bounty and we're not going to look at it.
193
00:37:52.949 --> 00:37:59.927
And you also have to submit bug reports
194
00:37:59.927 --> 00:38:06.902
via TikTok to Microsoft these days.
195
00:38:06.902 --> 00:38:13.880
Rather ridiculous rules.
196
00:38:13.880 --> 00:38:20.860
No, I don't think it's quite that bad. And you could argue that if the exploit works well enough, then maybe a 1-minute screencast video isn't that hard to make. But Nightmare Eclipse basically threw their toys out of their cot and said, well, if you don't want to accept the bug report because there's no video, then there can't be any objection if I just publish it for everybody. I do what's called full disclosure.
197
00:38:20.860 --> 00:38:27.840
I think it's a bug. Administrators might be interested in knowing it's a bug. And there is a school of thought that says don't wait for vendors, don't do responsible disclosure, if we just always tell everybody at the same time. The bad side of that is the crooks get hold of attacks on day zero.
198
00:38:27.840 --> 00:38:34.820
But the good news is that well-informed administrators don't have to wait for vendors to come to the party, run around for weeks, wait for videos, maybe try and brush things under the carpet, etc., etc. So Nightmare Eclipse decided that they would release this to the public, and just to grind their axe a little bit sharper, they published two other zero days at the same time, and they chose just after April's Patch Tuesday to do it for best PR purposes.
199
00:38:34.820 --> 00:38:47.244
Ah, right. So Microsoft have released their regular monthly Patch Tuesday update.
200
00:38:47.244 --> 00:38:59.668
Yes. That's just come out, which means it'll be another 30 days or so before the next one.
201
00:38:59.668 --> 00:39:07.483
Yes. All the system administrators who've pushed out all those patches have gone, oh, I wonder if anything's going to go wrong this month. But they've scheduled the time and their bosses have given them the budget to do it on the Wednesday and Thursday. And they're thinking, maybe I can just relax a little bit and do something else for the next 4 weeks.
202
00:39:07.483 --> 00:39:15.295
And bingo, then comes this massive exposé. And very embarrassingly, those first bugs that came out in April actually— I shouldn't laugh because it isn't funny, but it did make me smile. The bugs exploited security holes in the very software that Microsoft sells you to keep the bad guys out, namely Microsoft Defender, which is their built-in antivirus, right? That's right.
203
00:39:15.295 --> 00:39:23.110
And all its other stuff. And in, I think, two of the attacks, to get Defender to misbehave, they needed to provoke a malware detection, which obviously is going to draw attention to the attack, except that they deliberately dropped a copy of the EICAR test string.
204
00:39:23.110 --> 00:39:36.065
Why don't you tell us first
205
00:39:36.065 --> 00:39:49.019
of all what the EICAR test file is?
206
00:39:49.019 --> 00:39:56.056
It is a text string and was a simple coming together of well-meaning antivirus companies at the time to fight against what some of the more maverick players of the day were doing, which was actually handing out real viruses to their customers to test that the software was installed and would generate alerts correctly. Oh dear, what if it doesn't work?
207
00:39:56.056 --> 00:40:03.092
Yeah. So the idea is it is not meant to test that a product's good at detecting malware.
208
00:40:03.092 --> 00:40:10.130
It's not meant to generate alerts that throw you into a panic. It's just meant to be a simple way of triggering a file detection on a system so you can check that if you have an alerting mechanism in place, that the alerts flow correctly.
209
00:40:10.130 --> 00:40:19.079
Okay. Nightmare Eclipse needed to provoke a virus detection in order to exploit a vulnerability.
210
00:40:19.079 --> 00:40:28.030
So let's explain how that happened.
211
00:40:28.030 --> 00:40:36.309
So by simply writing the EICAR file to disk, they could create an alert. To this day, pretty much every EDR, every threat prevention software that's out there will detect it because the reasons that made it a good idea in 1990 are still a good idea today. And in fact, the whole idea was Nightmare Eclipse did not want to infect the machine with malware. They simply wanted to send Defender down a special code path that it only took when it was dealing with a virus attack. Right.
212
00:40:36.309 --> 00:40:44.590
So this is peculiarly embarrassing for Microsoft that their security software, their gatekeeper program, turned out to be a backdoor that allowed people to do an exploit. That's just the beginning. Because the month after, during the month of May, Nightmare Eclipse did much the same thing again. But this time, the main exploit they produced was one called Yellow Key. That was basically a bunch of files.
213
00:40:44.590 --> 00:40:52.869
They were only data files. There was no code in there, no scripts, nothing that would trigger even the most inquisitive antivirus software, you'd imagine. Looked completely innocent. You copy those files onto a USB stick, you put that USB stick into somebody's computer, you go Shift+Restart from their lock screen, which gets recovery mode, and bingo, you bypass BitLocker full disk encryption completely if it is set up in default mode.
214
00:40:52.869 --> 00:41:00.353
So this is extraordinary. So I mean, it is full disk encryption.
215
00:41:00.353 --> 00:41:07.836
The whole idea about it is that if you lose your laptop, for instance, no one will be able to get in and access your data because they don't know your password, which you've used to encrypt your drive. But you're saying with just a USB stick with this bunch of files on it.
216
00:41:07.836 --> 00:41:15.320
Yes. There's a way to actually bypass BitLocker so you can access what is on the disk.
217
00:41:15.320 --> 00:41:23.356
What's supposed to happen is when you boot into recovery mode, a light blue screen pops up — like the blue screen of death, but it isn't. And then you get some menus, very, very big and basic menus that you can click on with the mouse. You can get to a thing that says, give me a command prompt, which allows me to access my C drive. And that way you can try and fix it.
218
00:41:23.356 --> 00:41:31.393
You can copy off files in an emergency. Basically, you can rescue a ruined disk if you're lucky. So it's very, very useful to do this. However, before you get to the command prompt, before you can type in C: Enter and see everybody's files on the entire disk as the local system account, you have to put in what BitLocker calls the recovery key or the numeric password, which is a 48-digit randomly chosen string.
219
00:41:31.393 --> 00:41:39.429
The theory is basically nobody's going to guess it. But with the Yellow Key bypass, you just skip the menus and the drive unlocks itself automatically. No user intervention required.
220
00:41:39.429 --> 00:42:05.079
This seems disastrous.
221
00:42:05.079 --> 00:42:13.463
Well, it sort of is and it isn't. I think the most disastrous thing about Yellow Key perhaps is that one of the reasons companies use BitLocker on all their company laptops is not just that they want to protect their customers' data and that they want to look after their intellectual property. Let's hope that they do. But loosely speaking, in many countries such as the UK, if a laptop gets lost or stolen and you can show that you were using full disk encryption set up to some minimum standard, then because of the encryption and because of the password, you don't have to treat it as a data breach.
222
00:42:13.463 --> 00:42:21.846
This kind of blew that away retrospectively. Because you can imagine a crook who stole a laptop 6 months ago and they haven't got around to selling it yet and thinks, oh, I'm not going to get anything off this. Eventually I'll just take out the hard disk, I'll put in a new one, and I'll try and sell it for 50 quid or something. Something, can now go, hey, why don't I just put in a Yellow Key, magic key, and reboot and see if I can get some data off.
223
00:42:21.846 --> 00:42:30.228
Then I can sell the data. In other words, CISOs must have been thinking, I wonder if I need to report, say, the last 6 months of laptop thefts, given that those laptops probably haven't been disposed of yet. They might still be in circulation. And they're no longer protected, really.
224
00:42:30.228 --> 00:42:45.010
Why is this even possible? Well, I mean, it sounds like this has almost been coded into it, because you would think if the drive is
225
00:42:45.010 --> 00:42:59.789
encrypted in the first place, why would there ever be something which allowed you to circumvent that check at that point for that recovery key?
226
00:42:59.789 --> 00:43:07.889
Well, this is something that Nightmare Eclipse themselves cottoned on to because they don't have to prove this. They just have to sow the seeds of doubt. And they wrote in their original report words to the effect of, "Hahaha, who knows? Maybe this is a deliberate backdoor. Only Microsoft can say," like doxing. So they don't have to prove that. They just have to say that. And then, yes, people might be thinking, yeah, like you've just asked, why would you put such a bypass? Now, the reason this works is actually because the default mode of BitLocker, and sadly the one that is preferred by a lot of IT departments, is what's called TPM mode.
227
00:43:07.889 --> 00:43:15.989
It's an admittedly controversial chip that modern laptops have inside them that can securely store things like cryptographic keys. Keys that can only be extracted and used under special circumstances, like during the Secure Boot process. So Windows 11, by default, strictly enforces that a laptop must have this TPM chip to store cryptographic keys, and it must have a thing called Secure Boot, which is supposed to protect these keys from being manipulated by someone who isn't an administrator. And therefore, the way that BitLocker works in what's called TPM mode is it automatically extracts your full disk encryption password from this supposedly super secure chip during the super secure boot process and seamlessly and transparently unlocks the drive. Now, as crazy as that sounds, if the TPM chip and the Secure Boot process work correctly, it does provide you with at least some security because you have to put the hard disk in that laptop and you have to start it up and it then only goes down a code path which is supposed to take you to the Windows login prompt. I know that's a big if, but that's the theory. And users and IT managers love it because you don't have to remember or enter some kind of PIN or password every time you turn off and on or lock and unlock your device like you do on a mobile phone. The other thing that companies like about it is because that chip is in the specific laptop, it means if someone steals the laptop and takes the hard disk out and puts it on another computer, it won't unlock because that computer doesn't have the right chip. So it ties the disk to the laptop.
228
00:43:15.989 --> 00:43:24.088
So it's not a useless idea. It's just, if you like, the minimum you can do to make things safe. So there is a mode you can use for BitLocker called TPM and PIN where — right, you need to have the hard disk in the right laptop and there's a PIN, and you can even make it a long password that you have to put in right at the start when you boot up. If you can choose that mode, if you can convince your users as an IT manager — Smashing Security. Crypto experts have been advising people not to rely on this automatic unlock mode for years because there are just too many points at which a vulnerability could be introduced. So that does protect against this attack, but by default a lot of laptops were exposed. And although I'm not aware of anyone having data exfiltrated from their computers in this way, it was rather a teachable moment. And a scary thing for sysadmins around the world, like this premise they'd been clinging on to for years, that this automatic chip-based unlock mode in Windows 11 that's supposed to protect their systems from data breaches maybe was not quite as solid as it had seemed all along.
229
00:43:24.088 --> 00:43:37.974
Now, Microsoft hasn't been very happy about this,
230
00:43:37.974 --> 00:43:51.858
have they? I mean, they've tried to shut down—
231
00:43:51.858 --> 00:44:13.340
That's putting it mildly. Yeah. Yes.
232
00:44:13.340 --> 00:44:27.840
They've tried to shut down Nightmare Eclipse.
233
00:44:27.840 --> 00:44:42.340
They tried to get their GitHub account deleted.
234
00:44:42.340 --> 00:44:49.487
Well, they did. I mean, Microsoft owns GitHub, so I think they just press the button, gone. But they also published a blog article where they said full disclosure, which they call irresponsible behaviour. That's always unacceptable.
235
00:44:49.487 --> 00:44:56.632
Always? Even if a vendor won't play ball, we support coordinated disclosure, as they call it, responsible disclosure. By coordinated, they mean the vendor should get a say in the timing and the messaging in the actual response. And we think anything else is unacceptable.
236
00:44:56.632 --> 00:45:03.780
Largely, the security community would agree, but A, there are exceptions, and B, there are people who say no, full disclosure is the only way because it's the only way we can have an unequivocal rule that's not flexible or where you can't favour your buddies if you want to. Then they said, and by the way, anyone who publishes this kind of stuff is pretty much as bad as the crooks who go on and use it because they're aiding and abetting crime. Those weren't the words they used. We're going to make sure our Digital Crimes Unit is all over this kind of thing.
237
00:45:03.780 --> 00:45:15.840
As you said, Microsoft has owned GitHub for some years now.
238
00:45:15.840 --> 00:45:27.900
I mean, GitHub does have its fair share of naughty code up on it, doesn't it?
239
00:45:27.900 --> 00:45:37.309
Yes, and triumphantly so, I think
240
00:45:37.309 --> 00:45:46.719
you could argue.
241
00:45:46.719 --> 00:45:56.170
So they are publishing all kinds of stuff there.
242
00:45:56.170 --> 00:46:05.619
Is Microsoft going to take action against itself?
243
00:46:05.619 --> 00:46:14.014
Well, I was wondering that because I get the point. Nightmare Eclipse, they explicitly have an axe to grind with Microsoft.
244
00:46:14.014 --> 00:46:22.409
They've used quite aggressive words about, you know, how they want to grind their bones, all this kind of stuff.
245
00:46:22.409 --> 00:46:34.804
Yeah. All because they don't want
246
00:46:34.804 --> 00:46:47.199
to make a video, it seems.
247
00:46:47.199 --> 00:46:56.146
But yes, they are upset. And they are prepared to use Microsoft's customers as pawns in all of this by talking up these attacks. So I get why Microsoft could be offended or aggrieved or think this is no good. But in that case, surely they shouldn't just put out this generic threat, we are going to sue or do a prosecution against anybody who publishes this kind of stuff. They could say, we think this person is behaving in a way that's unacceptable, whereas others who publish stuff on GitHub that is potentially dangerous are maybe behaving in a slightly better way. But I absolutely agree with you. I think it's hypocritical that they closed down Nightmare Eclipse's account. I mean, I'm not saying they shouldn't be allowed to do that if they want, because this stuff is dangerous.
248
00:46:56.146 --> 00:47:05.092
But then why are malware source code, malware analysis, network sniffing tools, ransomware samples — hey, here's how you do the encryption if you want to write ransomware — why is a tool like EvilEngineX, which you may have heard of, full of stars and voted up as this fantastic tool that Microsoft seems to love to have on GitHub because it can be used by red teamers and penetration testers? Basically, EvilEngineX in 5 minutes can clone somebody's website, make a pixel-perfect, JavaScript-perfect copy, and basically start a live phishing attack for you with the ultimate goal of stealing things like usernames, two-factor authentication codes, passwords. Tell me that benefits users more than it benefits cybercriminals. But apparently it does. So it did seem that Microsoft had maybe rowed the boat out a bit too far, and it seemed that they rowed it back. They published a follow-up that wasn't very explicit. They didn't say, okay, Nightmare Eclipse is off the hook. They just said, okay, we're kind of saying that we don't think we'll prosecute individuals who are doing actual cybersecurity research and publishing the results.
249
00:47:05.092 --> 00:47:14.039
And they did apparently allow Nightmare Eclipse to create a brand new account on GitHub. This one, the username is MSNightmare, although their display name is still Nightmare Eclipse and they've still got an anime avatar. Which seemed a nice thing for Microsoft to do. And in response, Nightmare Eclipse has very kindly in the month of June, just after Patch Tuesday, dropped two new zero-day exploits. Again! One of which relies on exploiting a hole in Windows Defender, and if you don't mind, also targets BitLocker. So, oh my goodness, watch this space is all I can say.
250
00:47:14.039 --> 00:47:23.360
Well, listeners who are interested in this, Duck has written a series of great blog posts up on the SolCyber site. We will link to them in the show notes. We can read much more about all of this and take some of his advice there on how to perhaps protect your organisation.
251
00:47:23.360 --> 00:47:32.679
Now, time for a quick word from our friends at CoreView. Joe, quick question for you. How confident are you in your Microsoft 365 security posture?
252
00:47:32.679 --> 00:47:46.840
Graham, I don't even have
253
00:47:46.840 --> 00:48:01.000
a Microsoft 365 tenant.
254
00:48:01.000 --> 00:48:15.945
Oh, for goodness' sake, Joe, it's for our sponsor. Just play along with me, right? Picture the scene.
255
00:48:15.945 --> 00:48:30.889
It's Monday morning. You've got your coffee, you're wearing your second best hoodie. You're feeling pretty good about your Microsoft 365 setup because you checked Purview, you tightened conditional access, and frankly, you deserve a biscuit. Biscuits?
256
00:48:30.889 --> 00:48:39.793
Okay, I'm in. I'll play along with you. Thank goodness for that.
257
00:48:39.793 --> 00:48:48.697
So, and then someone forwards you a breach report about a company that did all of that too. So how did they get hacked? Turns out some quiet little permission that crept wider over 3 years.
258
00:48:48.697 --> 00:48:57.599
A policy exception that nobody had reviewed, the kind of thing that's invisible until it isn't.
259
00:48:57.599 --> 00:49:05.067
And this is exactly the stuff that CoreView's free Microsoft 365 Security Posture Check tool is designed to sniff out.
260
00:49:05.067 --> 00:49:12.532
It's the drift, the exceptions, the little permissions you stopped looking at because, well, you assumed they were fine.
261
00:49:12.532 --> 00:49:20.000
And the spoiler is that they're often not.
262
00:49:20.000 --> 00:49:32.059
It's free, it runs locally on your own machine, it does not send your tenant data back to CoreView or anyone
263
00:49:32.059 --> 00:49:44.119
else for that matter. And if you'd like a hand setting it up, their team will happily walk you through it.
264
00:49:44.119 --> 00:49:54.894
So all you've got to do is visit
265
00:49:54.894 --> 00:50:05.670
smashingsecurity.com/coreview to download your free copy of the tool.
266
00:50:05.670 --> 00:50:15.429
And even you will be able to answer the question, how secure
267
00:50:15.429 --> 00:50:25.190
is your Microsoft 365 tenant?
268
00:50:25.190 --> 00:50:34.360
And thanks to CoreView for supporting the show. And welcome back.
269
00:50:34.360 --> 00:50:43.530
Can you join us for our favourite part of the show? The part of the show that we like to call Pick of the Week.
270
00:50:43.530 --> 00:50:52.590
Pick of the Week.
271
00:50:52.590 --> 00:51:01.650
Pick of the Week.
272
00:51:01.650 --> 00:51:10.454
Pick of the Week is the part of the show where everyone chooses something they like. Could be a funny story, a book that they've read, a TV show, a movie, a record, a podcast, a website, or an app.
273
00:51:10.454 --> 00:51:19.259
Whatever they like. Doesn't have to be security related necessarily.
274
00:51:19.259 --> 00:51:26.043
I love the way you said a record there, Graham.
275
00:51:26.043 --> 00:51:32.826
Like, not a tune.
276
00:51:32.826 --> 00:51:39.610
Like, if it's not vinyl, it's not real.
277
00:51:39.610 --> 00:51:52.019
And also, if it is a tune, it's only a real song if you can whistle it, is my opinion. Oh, Graham, come on.
278
00:51:52.019 --> 00:52:04.429
No, it's true. If your milkman isn't whistling, as though I have milkmen, if you can't whistle it, it doesn't exist.
279
00:52:04.429 --> 00:52:12.706
No, you should be legal and proper if you can do
280
00:52:12.706 --> 00:52:20.983
sort of metal air guitar mouth noises to it.
281
00:52:20.983 --> 00:52:29.260
That's perfectly acceptable.
282
00:52:29.260 --> 00:52:35.949
My pick of the week this week is a bit security related. Inside a large warehouse in Huntsville, Alabama, the FBI has built a small American town. Inside a warehouse.
283
00:52:35.949 --> 00:52:42.639
Inside a warehouse, a large warehouse. Yeah. It's got a courthouse, a hotel, a petrol station, a gas station, I suppose, an arcade, hospital, traffic lights, fully furnished houses.
284
00:52:42.639 --> 00:52:49.329
It's like The Truman Show.
285
00:52:49.329 --> 00:52:56.550
Does it have a warehouse inside it?
286
00:52:56.550 --> 00:53:03.769
You can see where this is going, right?
287
00:53:03.769 --> 00:53:10.989
You know, with a model town inside it.
288
00:53:10.989 --> 00:53:22.144
Well, I love it when you go to a model village and inside the model village, it has a model
289
00:53:22.144 --> 00:53:33.300
of the model village. And then if you look really close, I went to one of those the other day.
290
00:53:33.300 --> 00:53:45.210
How does that poem go? Great fleas have lesser fleas upon their
291
00:53:45.210 --> 00:53:57.119
backs to bite them, and lesser fleas have smaller fleas, and so ad infinitum.
292
00:53:57.119 --> 00:54:08.014
Finally, some culture on the programme. Anyway, you may be asking, why has the FBI built a small town inside a warehouse?
293
00:54:08.014 --> 00:54:18.909
And apparently, it is their kinetic cyber range. This is an indoor training facility, 22,000 square feet, designed to teach law enforcement how to investigate—
294
00:54:18.909 --> 00:54:30.130
That's about 2,000 square metres. Is that right?
295
00:54:30.130 --> 00:54:41.349
It's big enough. For those of us who don't know customary units.
296
00:54:41.349 --> 00:54:48.646
It's designed to teach law enforcement how to investigate and respond to real-life cyber attacks.
297
00:54:48.646 --> 00:54:55.943
So, everything in this place is fully functioning, it's got systems, devices, IoT equipment, servers, all wired up, behaving exactly as
298
00:54:55.943 --> 00:55:03.239
they would in a real community.
299
00:55:03.239 --> 00:55:12.855
But it'll have like Wi-Fi routers and underground
300
00:55:12.855 --> 00:55:22.469
cable TV connections.
301
00:55:22.469 --> 00:55:29.967
It's got all of this.
302
00:55:29.967 --> 00:55:37.463
But it's in an environment where a simulated ransomware attack can't accidentally spill out into the real world.
303
00:55:37.463 --> 00:55:44.960
At least they hope it can't.
304
00:55:44.960 --> 00:55:57.409
Yes.
305
00:55:57.409 --> 00:56:09.860
Listen up, Tenet.
306
00:56:09.860 --> 00:56:19.510
They are using this to train students with real hands-on experience rather than just learning the theory in a classroom. And apparently since February last year, it's trained nearly 1,400 students, not just FBI agents, but the US Army, local law enforcement, NASA as well. I do remember they took a virus once up to the space station, didn't they?
307
00:56:19.510 --> 00:56:29.159
They managed to infect themselves. Yeah. But it went up on a USB stick.
308
00:56:29.159 --> 00:56:54.929
So are you serious?
309
00:56:54.929 --> 00:57:06.090
That's how it
310
00:57:06.090 --> 00:57:17.250
got there? Yes.
311
00:57:17.250 --> 00:57:24.706
Yes, I think so. Yes.
312
00:57:24.706 --> 00:57:32.163
So anybody who ever said, oh, we've got a 2-metre air gap between our secure network and our insecure network — how high up is the space station? Is it like 400 kilometres?
313
00:57:32.163 --> 00:57:39.619
Bloody high up. Oh dear.
314
00:57:39.619 --> 00:57:50.920
Anyway, Duck, I have put in the show notes a link where you can check
315
00:57:50.920 --> 00:58:02.219
out this cyber range. It's like going to a theme park or a movie lot or something.
316
00:58:02.219 --> 00:58:09.130
I must admit, it sounds kind of silly when you first mentioned it. I thought, oh, 2,000 square metres, that's like a massive house — surely you could just do it in a lab. But I guess the stuff you can do here is you can have real people in the way.
317
00:58:09.130 --> 00:58:16.039
You can have desks full of people who are getting agitated and anxious. You can have coffee machines that do or don't work. You can have server rooms where nobody can remember where the key got left.
318
00:58:16.039 --> 00:58:22.949
And are you going to smash the window? You know, you can have crawl spaces where you have to get in there — if you want to do a disconnect, you've got to get in there and—
319
00:58:22.949 --> 00:58:34.885
Go look at the photographs. It's extraordinary.
320
00:58:34.885 --> 00:58:46.820
They've got sofas, they've got lamp posts — they're set up like people's houses, this thing.
321
00:58:46.820 --> 00:58:54.007
They've got all the lights.
322
00:58:54.007 --> 00:59:01.193
Tell me they have a place where you
323
00:59:01.193 --> 00:59:08.380
can get pizzas delivered.
324
00:59:08.380 --> 00:59:33.369
Oh, I don't know.
325
00:59:33.369 --> 00:59:45.239
Because that would be a cruel
326
00:59:45.239 --> 00:59:57.108
and unusual punishment if they didn't.
327
00:59:57.108 --> 01:00:05.204
They've got it all here. They've got a bloody arcade with video machines.
328
01:00:05.204 --> 01:00:13.298
I mean, they are having a blast, the FBI. I don't know who's paid for all of this, but apparently it's all doing excellent work.
329
01:00:13.298 --> 01:00:21.393
And so I will link to it in the show notes so you can check it out for yourself.
330
01:00:21.393 --> 01:00:32.536
Expensive, but you think at 2,000 square metres,
331
01:00:32.536 --> 01:00:43.679
it's not like they've actually built a full-sized town.
332
01:00:43.679 --> 01:00:55.019
It's not a full-sized town,
333
01:00:55.019 --> 01:01:06.360
but it's at least—
334
01:01:06.360 --> 01:01:16.025
I was sceptical at first, but I just like the idea that there will be doors that are locked, there will be windows that don't open, there will be server rooms where there's not enough room for two people to go in at once. There will be cantankerous jobsworths who won't let you into the courthouse.
335
01:01:16.025 --> 01:01:25.690
You know? Imagine what fun you could have.
336
01:01:25.690 --> 01:01:36.512
I think they could rent this out, actually, couldn't they? I think there would be a lot
337
01:01:36.512 --> 01:01:47.335
of IT security teams who would love to do this as a sort of team away day.
338
01:01:47.335 --> 01:01:58.797
It certainly would beat the average
339
01:01:58.797 --> 01:02:10.259
1-hour escape room party, wouldn't it?
340
01:02:10.259 --> 01:02:19.954
Anyway, the FBI's Kinetic Cyber Range is my pick of the week.
341
01:02:19.954 --> 01:02:29.650
Duck, what's your pick of the week?
342
01:02:29.650 --> 01:02:39.574
My pick of the week is — I've had a Raspberry Pi Zero W. That's one of the old tiny little Raspberry Pis that I've had kicking around for several years. They're quite old and now considered no good. You need to get the Pi Zero 2, which is a 64-bit ARM chip, etc., etc. But it turns out that there are still Linux-based distros that still support it pretty much as a first-class citizen, like Alpine, for example. And so I decided, well, it's sitting there doing nothing, it's got an SD card in it, why don't I just set it up as a little USB-powered router that I can take with me to coffee shops?
343
01:02:39.574 --> 01:02:49.498
Because there are a few coffee shops that I like around Oxford that have tired old Wi-Fi equipment where either your mobile phone won't connect to it because it's just not secure enough, or you just think, you know, no, I don't think so, not going to connect my laptop directly to it. And now I can plug my laptop via a USB cable, which acts as an Ethernet port, into my Raspberry Pi Zero. I can connect from the Pi Zero onwards to the Wi-Fi I definitely don't trust, I can put a whole load of lockdowns in place because it's still powerful enough to do even something a little bit like Pi-hole, you know, ad blocking, could even do that. So that's what I've been doing. So my pick of the week is not so much the Raspberry Pi Zero W, or Alpine Linux, both of which are great. But my pick of the week is the idea that you may just have some old gadgets lying around that are not as old or as useless or quite as ready to go into landfill as you might have thought.
344
01:02:49.498 --> 01:02:57.271
Oh, hear, hear to that. A great pick of the week.
345
01:02:57.271 --> 01:03:05.045
Well, we've got time for another guest now on the podcast, and I'm delighted to be joined by Son Nguyen Kim. Son leads ProtonPass, Proton's privacy-first password manager for businesses.
346
01:03:05.045 --> 01:03:12.818
Son, welcome to Smashing Security.
347
01:03:12.818 --> 01:03:25.333
Hey, yeah, happy
348
01:03:25.333 --> 01:03:37.849
to be here.
349
01:03:37.849 --> 01:03:49.864
Now, Son, I want to start with something I suspect a lot of our listeners are quietly guilty of, which is that small businesses everywhere are plugging AI tools into their systems. They're connecting them to email, calendars, internal databases, all kinds of things.
350
01:03:49.864 --> 01:04:01.880
And mostly they're just clicking through the permission screens without reading them. From where you sit at ProtonPass, what do you think that those companies have actually just done to themselves by doing that?
351
01:04:01.880 --> 01:04:11.019
Yeah, so AI integration is very easy, is very smooth. But behind the scenes, we need to know that we are giving access to a special agent. It's like a human but never sleeps, can act really fast, can do a lot of things on its own, and it can listen to anyone reaching out to it. So for example, if someone can talk to the agent, they can convince the agent to do things that can actually harm our business. And that will only get worse because usually when we accept integration, we don't really look at the permission or scope and we just approve everything, you know, to make it fast so the agent can start doing things that it needs to do. And then we don't really have any monitoring system to know what the agent is doing, or any alert system to know that the agent is doing something that might be harmful.
352
01:04:11.019 --> 01:04:20.159
So kind of the summary that I would tell everyone is it's not just a tool. You should see it as a new employee that you onboard to the company. Right, you give them the access to the most important data of the company and you can skip the background check. And this employee might be naive, might be tricked by bad actors into doing things that it's not supposed to do without telling you. So be super careful with that.
353
01:04:20.159 --> 01:04:30.068
So there's a number of problems here. One is, as you've identified, is that the AI tool you've effectively allowed to become a privileged insider inside your company. It's like an employee, but one that hasn't gone through the interview and check-in process, but also that they have this sort of unscoped broad access that you've granted a third-party system to them. So they've essentially been handed a set of keys without much thought about who is actually holding them.
354
01:04:30.068 --> 01:04:39.978
And one of the concerns is that stolen credentials have been a number one entry point for attackers for years, haven't they? I mean, we hear this at every security conference. Is what you're describing just more of the same problem but dressed up in new clothes, or is this something genuinely different which is happening here?
355
01:04:39.978 --> 01:04:47.235
So what's new is autonomy. Agents have autonomy and agents can act way faster than a human. An agent never sleeps. It can work 1,000 times faster than a human. It can do a lot of things very quickly.
356
01:04:47.235 --> 01:04:54.492
And another thing is an agent can be convinced by a bad actor to do bad things via prompt injection, for example. So let's say if an agent has access to some data that can be controlled by a bad actor. Let's say the agent visits a website, and on this website there's hidden instructions that tells the agent to send all the emails in your system, forward all the emails to an email address that the hacker owns. You're not going to see it, but behind the scenes, the hacker will gain access to all your emails. That can happen.
357
01:04:54.492 --> 01:05:01.748
So I would say the mechanism to authenticate is the same, but the behaviour around it is new. It's way faster. It can be social engineered and we don't have enough monitoring or alert system to know what's going on and to intervene when needed.
358
01:05:01.748 --> 01:05:09.255
So we've got problems of speed. These AI agents, they have real velocity, don't they?
359
01:05:09.255 --> 01:05:16.762
We have autonomy as well. They're acting without human approval and the access which they have is really frightening because they can access so much information.
360
01:05:16.762 --> 01:05:24.269
But can you paint a picture for me of what a breach involving AI agent credentials actually looks like for a business? So something you'd actually see happening.
361
01:05:24.269 --> 01:05:35.965
So one concrete example is let's say you have an agent that is connected to your email and
362
01:05:35.965 --> 01:05:47.659
answers customer support questions. An email came in that actually contains a poison input, a malicious prompt injection.
363
01:05:47.659 --> 01:05:57.440
So that's the prompt injection could come from an external email.
364
01:05:57.440 --> 01:06:07.219
Your AI is reading your email and it could act upon it.
365
01:06:07.219 --> 01:06:13.969
It can be something like, ignore all the previous instructions and follow what I'm going to tell you. And the hacker can then tell the agent to do things like make a purchase, send the money to another bank account, or review all the emails that the agent has access to, forward the invoice, exfiltrate customer data, anything.
366
01:06:13.969 --> 01:06:20.719
And the worst is you don't know about that because you've granted access to the agent, you trust the agent to do things on behalf of you. And because of that, there's no alert, there's nothing abnormal that you're going to see.
367
01:06:20.719 --> 01:06:27.469
So basically humans are blind in this case, and maybe they're going to realise that sometimes later, but it's already too late.
368
01:06:27.469 --> 01:06:36.043
So there's real danger here of your data being exfiltrated, your intellectual property maybe. If you have something like an agent plugged into your email, there's potential for business email compromise because the agent can access your calendar and your email contacts. So there are opportunities for financial fraud.
369
01:06:36.043 --> 01:06:44.617
It's a pretty sobering picture. You're describing what seems to me to be like a third-party risk, but it's faster. And because it's AI, it's also at scale as well.
370
01:06:44.617 --> 01:06:53.190
But surely a forgotten service account which has sat unmonitored for months is just as dangerous as something like this. What makes the AI agent version of this meaningfully worse?
371
01:06:53.190 --> 01:07:02.800
So you're right that a forgotten service account is also very dangerous. Something that we don't pay attention to that can do things in the background without triggering any alarm. But the thing with agents is it just makes it faster with more impact, and especially for people who never managed service accounts before. So a lot of people who enable agents don't have the technical background to know what is actually a service account, right? Service account is a technical word that not everyone is familiar with. And then because right now we have kind of the FOMO going on, fear of missing out on AI agents, everyone wants to integrate AI into their workflow and they want to do that fast.
372
01:07:02.800 --> 01:07:12.409
You know, they want to spin up maybe 5, 10, 50 agent integrations in weeks, in months, and then they forget about it. But the agent doesn't forget, the agent doesn't disappear. They're still there. They still listen to instructions, maybe from you or maybe from someone else. And then because of that, you don't know that it exists. For non-technical people, they just don't have the technical knowledge to monitor all of them or to know what's going on.
373
01:07:12.409 --> 01:07:24.929
So we've talked in the past — it's not a new idea — things like least privilege and scoped access. Security teams have been preaching about them for years and years and years.
374
01:07:24.929 --> 01:07:37.449
Why does it feel like they are being thrown out of the window the moment companies start deploying AI agents? Is it that fear of missing out, do you think, or is there more than that?
375
01:07:37.449 --> 01:07:45.820
It's kind of related to the FOMO in the sense that we want to do things very quickly, the quickest way possible. So usually people will just accept the defaults, and by default the agent will ask for as many permissions as possible so it doesn't have to ask again.
376
01:07:45.820 --> 01:07:54.190
So everything will work out perfectly at the beginning, so people just click allow all and then the agent will have access to everything. The second thing is scoping is actually quite hard — people need to understand what a permission actually means, and they need to know what permissions the agent actually needs to decide which ones it should have access to.
377
01:07:54.190 --> 01:08:02.559
And also related to the FOMO, people want to do that fast. You know, I just want to have this agent working right now so I can see the benefit, so I can show to other people that I'm an AI-native person.
378
01:08:02.559 --> 01:08:09.369
Yes. And there's so much pressure on employees now to get lots of work done. And it's not as though AI is necessarily making our lives better.
379
01:08:09.369 --> 01:08:16.180
It can be that AI is just helping us do more during our working day, and we feel like we need to use AI to keep up with our colleagues and with our managers' demands. And I imagine one problem is that there may be a situation where the people who are actually turning on the AI or onboarding it in a particular app may not be the IT and security team. They may not be in the loop when business users are adopting these tools.
380
01:08:16.180 --> 01:08:22.989
So there's a gap, isn't there, between what people know they should be doing and what actually happens under pressure in order to stay competitive. So there are probably people listening right now who are thinking, I genuinely have no idea what access my AI tools have actually got. They're probably thinking, where do we even start?
381
01:08:22.989 --> 01:08:32.534
So there's no way that just sitting down and trying to remember all the agents, integrations that you have enabled. Maybe going to all the tools that you use, email, calendar, etc., and check which agent, which integration is enabled. And then for each agent, try to ask the three questions — what can it access? So what scope did we grant to it, read or write? Every permission or just some permissions, and who owns it, and who's going to know when it's not behaving correctly. And then try to find the credentials that the agent has access to.
382
01:08:32.534 --> 01:08:42.079
Is this via a config file? Is this via a secret manager? Is this maybe an employee's personal account? And from that, trying to reduce the scope that the agent has and maybe talk with the person who has activated the agent and ask them why they need the agent and try to reduce the scope that they have granted. That can take a lot of time to go through everything and talk with everyone to understand their needs and reduce the access, the scope of the agent. But that's the first thing to do.
383
01:08:42.079 --> 01:08:50.237
So the first thing to do, step one, is getting some visibility on what's happening and then what scopes those apps have been granted and then going back to the users and saying, what do you use this for? Do you really need this?
384
01:08:50.237 --> 01:08:58.393
That's something which IT teams can do, hopefully. And once you've got that picture, if things do go wrong, I guess you have to consider how quickly your company can actually cut off access to an AI agent which you've decided is risky.
385
01:08:58.393 --> 01:09:06.550
What does the revocation process look like in practice for doing that?
386
01:09:06.550 --> 01:09:15.417
So let's say you have a list of all the AI agents and what they have access to, and how to set them up. In theory, it's quite easy to revoke the access, right? You can just go to the settings and remove the access from the agent.
387
01:09:15.417 --> 01:09:24.283
But what we don't know is what's going to be the consequences, right? Maybe the agent is used in the sales pipeline to send an automatic email to any prospect coming to the website. Maybe the agent is handling customer support via an integration.
388
01:09:24.283 --> 01:09:33.149
So if we revoke the access, there might be an impact on the business. So it's important to also understand what role that agent is playing in the business process.
389
01:09:33.149 --> 01:09:39.876
So the speed of response is really dependent on whether you've built for it from the start. If you actually prepared yourself — many people won't have done that.
390
01:09:39.876 --> 01:09:46.603
And that brings me to Proton Pass specifically, which obviously is the project which you lead on. For someone who's heard all of this and actually wants to act upon this problem, how does Proton Pass help?
391
01:09:46.603 --> 01:09:53.328
What does it give you that just being more careful wouldn't give you?
392
01:09:53.328 --> 01:10:00.059
So being more careful is something that everyone should do, but more often than not, people forget to be careful when under pressure, when there's FOMO involved, when they have to do things very quickly, or maybe they don't have the technical knowledge to do what careful means actually. So that's what I mean by that — discipline doesn't really scale. So we need some structures to allow people to be careful, to be disciplined. And LastPass or any password managers can be a good way to do that. So we make sure that every credential is stored centrally so that admin can have an overview on what is stored in their company.
393
01:10:00.059 --> 01:10:06.789
And then not use Slack or email to share username and password, because once it got out, it's very hard to know who has access to it. And then anyone having access can use those credentials and we have no idea. And if people are technical, then it's better for them to, if they want to use a secret, they can reference the secret from a password vault instead of copy and pasting them directly into the tool. It's going to work better, and a lot of tools support that by integration with the password managers to get a secret instead of you having to copy and paste the password into the tool. And recently in ProtonPass, we also created a feature called AI access token that allows a human to create an access token that they're going to give to the AI, which access the AI will have exactly in their vault.
394
01:10:06.789 --> 01:10:13.520
And then whenever AI wants to access something, AI has to give a reason — why do I want that? If AI tries to access, let's say, your storage account, AI should give a reason like, because I want to upload the latest invoice, for example, and later on, human can see the timeline of the AI access and see the reason why it's trying to access something. And this way, human can be informed of what AI is actually doing and maybe intervene when something abnormal happens.
395
01:10:13.520 --> 01:10:20.750
So it's like an audit log in a way, isn't it? Fantastic.
396
01:10:20.750 --> 01:10:27.979
So it's not just about having good intentions as a business — it's also about having the infrastructure to back all of these up. So what I always like to do when I chat to vendors is try and find some actionable advice for our listeners.
397
01:10:27.979 --> 01:10:35.210
If someone's listened to all of this and they want to do one thing this week, what would you tell them?
398
01:10:35.210 --> 01:10:45.210
So I think the first thing to do is to make the inventory, to list all the AI agents that you have enabled, and try to understand what they have access to and
399
01:10:45.210 --> 01:10:55.210
what the consequences would be if we remove them. On top of that, it's better to tell everyone in the company to have some basic security practice, like never share passwords on Slack
400
01:10:55.210 --> 01:11:05.210
or email, have strong and unique passwords, enable two-factor authentication, etc. I think with that, you can already improve a lot of your security posture.
401
01:11:09.600 --> 01:11:19.600
Well, Son, this has been really interesting. Thank you so much for joining me on Smashing Security today. And listeners, if you think that your firm needs a password manager built for business that doesn't compromise on security or slow your team down, then why not check out ProtonPass?
402
01:11:19.600 --> 01:11:29.600
It's built on Swiss infrastructure, open-source architecture, and you can check out a free trial of ProtonPass for your business at proton.me/smashing. That's proton.me/smashing. Thanks so much, Son, for joining us on this week's show.
403
01:11:29.600 --> 01:11:39.600
Well, that just about wraps up the show for this week. Thank you so much, Duck, for joining us. I'm sure lots of our listeners would love to find out what you're up to and follow you online.
404
01:11:40.260 --> 01:11:51.319
What's the best way to do that?
405
01:11:51.319 --> 01:12:02.378
The best way is to go to my own website, that is paulducklin.com/about, and if you would like to read a lot of articles that I have been writing lately, you can go to one of my customers' websites where I do a lot of deep dive technical articles that you mentioned already, and that is solcyber.com/blog. Terrific stuff.
406
01:12:02.378 --> 01:12:11.123
And of course, Smashing Security is on social media as well. You can find it on Blue Sky and on Reddit and on Mastodon. You can also find me, Graham Cluley, up there and on LinkedIn as well. And don't forget to ensure you never miss another episode.
407
01:12:11.123 --> 01:12:19.869
Follow Smashing Security in your favourite podcast app, such as Apple Podcasts, Spotify, and Pocket Casts. For episode show notes, sponsorship info, guest and the entire back catalog of 472 episodes, check out smashingsecurity.com. Until next time, cheerio.
408
01:12:19.869 --> 01:12:39.189
Bye-bye. Bye everybody.
409
01:12:39.189 --> 01:12:39.840
You've been listening to Smashing Security with me, Graham Cluley. I'm ever so grateful to Paul Ducklin for joining us this week and to this episode's sponsor, ProtonPass, Vanta, and CoreView. And also, of course, tremendous thanks to our Patreon supporters. This week we are pulling out of the hat for special mention the following patrons: Cory, Alex Tasker — I imagine they're very good at to-do lists — Bree Bustle, who is quite possibly the principal dancer at the Royal Ballet, Ted Wilkinson — sounds like the kind of reliable fellow you'd trust for a double glazing recommendation — Matt H, Dimitri, Alexander Hugues, back again, still sounding very grand, probably has a wonderfully long driveway. Skadone, all lowercase, absolutely no time for capitals, far too busy. Butterfly, who's drifted in on gossamer wings, and SK, just the two initials, very mysterious. Thank you all so much, you are wonderful. Those are just a few members of Smashing Security Plus, our community, which gets their episodes ad-free and earlier than the general public. And they can also have the privilege of having their names pulled out of a hat at random to be mocked at the end of the show. If you'd fancy a little bit of that, all you have to do is join Smashing Security Plus. Just head over to smashingsecurity.com/plus for all the details where you can become a patron of the show. But you can also support the show in plenty of other ways that don't cost a penny. You can like, you can subscribe, you can leave a 5-star review, you can spread the word. Go on, tell your friends about Smashing Security and your enemies. In fact, tell everybody, why not? Just go for it. Every little bit helps and I really, really appreciate it. Well, thank you for listening this week and I hope you will tune in to our future episodes as well. Until then, cheerio, bye-bye.