WEBVTT
1
00:00:02.569 --> 00:00:09.372
Hang on a minute, James.
2
00:00:09.372 --> 00:00:16.176
You're suggesting that Instagram ever had
3
00:00:16.176 --> 00:00:22.980
human support stuff?
4
00:00:22.980 --> 00:00:32.979
So it was a little bit akin to rituals to summon a demon or speak to the dead.
5
00:00:32.979 --> 00:00:42.979
But if you went through Meta's escalating procedures in exactly the right way on the right day of the week, wearing a yellow sash with a finger
6
00:00:42.979 --> 00:00:52.979
in an ear, you could actually get through to a human.
7
00:01:09.680 --> 00:01:17.510
Hello, hello, and welcome to Smashing Security episode 471.
8
00:01:17.510 --> 00:01:25.340
My name is Graham Cluley.
9
00:01:25.340 --> 00:01:43.209
And I'm James Ball.
10
00:01:43.209 --> 00:01:52.359
James, welcome back to the show.
11
00:01:52.359 --> 00:02:01.510
Always a delight to have you here.
12
00:02:01.510 --> 00:02:18.270
Always a pleasure to be here.
13
00:02:18.270 --> 00:02:28.020
Oh, I better, before I carry on, thank everyone who came to see me speaking at Infosecurity Europe at the Excel Center last week. I was talking all about the horrors of AI. There may be some more of that today, actually, to be honest. I was talking about how AI can blackmail you and how the billionaires are maybe not the people to have in charge of the AI, as if having billionaires in charge of anything was actually a good idea.
14
00:02:28.020 --> 00:02:37.770
But that was good fun and lovely to meet some listeners there. Now, James, you're normally busy writing for The New World and things. You pop up on podcasts and things, but you've also been working on a PhD, haven't you?
15
00:02:37.770 --> 00:02:47.159
Yeah, I've decided I should actually know something about technology after about 15 years of covering it. So I'm doing a PhD on how legal systems look at artificial intelligence and AI. So I'm very nervous.
16
00:02:47.159 --> 00:02:56.550
I've got sort of first day at school energy because I'm presenting a paper at a PhD conference later this week. It's not the biggest audience I've done, but it's my first time as an academic, whatever that means. So kind of terrified.
17
00:02:56.550 --> 00:03:05.599
Is this about how lawyers and the law uses AI, or is this about how they regard AI?
18
00:03:05.599 --> 00:03:14.650
Because there's a lot of lawyers using AI these days that may be putting some lawyers out of a job.
19
00:03:14.650 --> 00:03:22.150
Yeah, it's quite fun seeing AI pop up in cases, but mine is about when governments end up hauled in front of the law. We've had sort of surveillance cases ever since the Edward Snowden revelations that look at, hey, is it a big deal when an algorithm reads your emails instead of a human spy? You know, does it make a difference if that algorithm is really clever or if it's random instead of fixed? So it's cases and comparing those to how it thinks about AI in copyright cases, because you get this kind of fascinating effect where in surveillance, lots of courts have said, well, obviously it's not as big a deal if an algorithm scans your emails as if a human reads them.
20
00:03:22.150 --> 00:03:29.650
You know, it might invade your privacy, but it's less likely to. There's less protection because it's different. Right. Whereas in copyright, a lot of the big cases so far have said, okay, an algorithm looked at 100 books, synthesized them, and came up with an output that's kind of got bits of all of them in, but not their wording.
21
00:03:29.650 --> 00:03:37.150
If that's not legal for an algorithm to do, then it wouldn't be legal for a human to do. Yes. And therefore journalism would be out, authorship would be out. So they go, well, it's the same as the human, isn't it?
22
00:03:37.150 --> 00:03:44.650
So therefore we've got to allow it.
23
00:03:49.680 --> 00:03:59.680
Smashing Security, episode 471. This AI worm just
24
00:03:51.509 --> 00:04:00.056
And so you've got these completely different attitudes to, oh, it's an algorithm, so it's different.
25
00:03:59.680 --> 00:04:09.680
rewrote its own rules. With Graham Cluley and
26
00:04:00.056 --> 00:04:08.603
Oh, it's an algorithm, so it's the same.
27
00:04:08.603 --> 00:04:17.149
And so I'm doing a PhD on that.
28
00:04:09.680 --> 00:04:19.680
special guest James Ball.
29
00:04:17.149 --> 00:04:25.293
Well done.
30
00:04:25.293 --> 00:04:33.437
Sounds very impressive.
31
00:04:33.437 --> 00:04:41.579
Well, good luck speaking at this conference.
32
00:04:41.579 --> 00:04:53.894
Thank you.
33
00:04:53.894 --> 00:05:06.209
I may need it.
34
00:05:02.009 --> 00:05:32.009
Huh.
35
00:05:06.209 --> 00:05:13.709
Well, before we kick off, let's thank this week's wonderful sponsors, Opswat, Expo, and Vanta. We'll be hearing more about them later on in the podcast.
36
00:05:13.709 --> 00:05:21.209
This week on Smashing Security. We won't be talking about how the METV Awards left its access keys out in the open for anyone to see, leaking data including award submissions.
37
00:05:21.209 --> 00:05:28.709
You'll hear no discussion of how an AI agent has found over 20 zero-day vulnerabilities in FFmpeg, some of them 23 years old. And we won't even mention how hackers have stolen $1.7 million worth of condoms.
38
00:05:28.709 --> 00:05:36.209
After hijacking a shipment to Walmart. So James, what are you going to be talking about this week?
39
00:05:50.569 --> 00:06:02.350
And continuing the AI theme, I'm going to be talking about an AI worm that appears to
40
00:06:02.350 --> 00:06:14.129
think for itself. All this and much more coming up in this episode of Smashing Security.
41
00:06:14.129 --> 00:06:24.519
This episode is
42
00:06:24.519 --> 00:06:34.910
supported by Opswat.
43
00:06:34.910 --> 00:06:43.310
Joe, here's a question for you.
44
00:06:43.310 --> 00:06:51.709
What if the entire cybersecurity industry has been
45
00:06:51.709 --> 00:07:00.110
doing it wrong?
46
00:07:00.110 --> 00:07:21.059
Huh?
47
00:07:11.240 --> 00:07:21.240
This week I'm talking about helpful AI, maybe too helpful AI,
48
00:07:21.059 --> 00:07:28.238
The entire industry?
49
00:07:21.240 --> 00:07:31.240
which is the Meta AI, which seems to have been giving
50
00:07:28.238 --> 00:07:35.420
That's a bit of a
51
00:07:31.240 --> 00:07:41.240
anyone who asked nicely anyone else's password.
52
00:07:35.420 --> 00:07:42.600
stretch, isn't it?
53
00:07:42.600 --> 00:07:49.339
Well, that's the argument Benny Czarny makes in his new book, Cybersecurity Upside Down. Benny is the founder and CEO of Opswat, and he's spent more than two decades protecting critical infrastructure, you know, nuclear facilities, defense networks, energy grids.
54
00:07:49.339 --> 00:07:56.079
The stuff that quite literally keeps the lights on.
55
00:07:56.079 --> 00:08:11.189
OK, so what's his big idea?
56
00:08:11.189 --> 00:08:19.350
Well, he says the industry is obsessed with detecting threats. But detection can never be perfect.
57
00:08:19.350 --> 00:08:27.509
One dodgy file slips through and your network is toast.
58
00:08:27.509 --> 00:08:43.349
I like toast.
59
00:08:43.349 --> 00:09:00.708
So what's the alternative?
60
00:09:00.708 --> 00:09:17.039
To toast?
61
00:09:17.039 --> 00:09:35.528
No, to detecting threats.
62
00:09:35.528 --> 00:09:47.384
Ah, well, how about not even trying to spot the malware? Instead, take files apart, throw away anything that isn't strictly needed, and rebuild a clean version from the safe bits.
63
00:09:47.384 --> 00:09:59.240
The user gets a sanitized working document. The malware ends up in the bin.
64
00:09:59.240 --> 00:10:16.980
But hang on, who decides what's safe?
65
00:10:16.980 --> 00:10:26.149
That's the clever part. You do. Macros might be allowed for your automation team, but stripped out for finance. JavaScript ripped out of every PDF everywhere EXIF data scrubbed from images leaving HR. It's not an on-off switch. It's a policy that you can tune to your business.
66
00:10:26.149 --> 00:10:35.320
So even a brand new attack no one's ever seen before doesn't survive the rebuild. Exactly. There's nothing to detect because it's already gone. Whether you're a security pro, an executive, or just someone who wants to understand what's really going on in cybersecurity, Cybersecurity Upside Down He's technical enough for the experts, but also accessible enough for the rest of us. Go and grab your copy right now at smashingsecurity.com/upsidedown.
67
00:10:35.320 --> 00:10:53.610
And thanks to Opswat for supporting the show.
68
00:10:53.610 --> 00:11:01.940
Now, chums, the cybersecurity industry has existed for years now, decades and decades, and I've lost count the number of times that things have been described as game-changing or revolutionary or a quantum leap. Usually obviously by the salespeople or the marketeroids. And I read something, however, this week, which did make me sit up and go, ooh, that is a bit different.
69
00:11:01.940 --> 00:11:10.269
Because on June 2nd, researchers at the University of Toronto published a paper, terribly exciting title, AI Agents Enable Adaptive Computer Worms. I don't want to disrespect you with your PhD ambitions, James, but—
70
00:11:10.269 --> 00:11:26.330
No, they're not that gripping. I think I'm not going to cry with offense if you say that. Okay.
71
00:11:26.330 --> 00:11:33.269
So normally, you could think, oh gosh, you can imagine 14 people in the room listening to this talk and 3 of them are asleep and the rest of them are there for the biscuits. But this one is actually pretty fascinating. First of all, we need to make sure when we're describing this that everyone's up to speed on what a worm actually is.
72
00:11:33.269 --> 00:11:40.210
So a worm is a piece of malware. Most commonly people think of viruses and things, but it's a piece of malicious software that spreads by itself. So you don't have to click on anything.
73
00:11:40.210 --> 00:11:47.149
You don't have to open anything, you don't have to do anything risky. It just goes under its own steam and the worm will get onto one computer and then it will copy itself onto the next computer and the next and the next. No human required, all automatic.
74
00:11:47.149 --> 00:11:56.654
It's a bit weird that that's the one that ended up being called a worm because it doesn't actually match real parasites in terms
75
00:11:56.654 --> 00:12:06.159
of which ones spread faster at all, does it?
76
00:12:06.159 --> 00:12:15.504
You know, way back when in the early days of antivirus, when I was working with Alan Solomon, I remember him saying to me that he was at some meeting where they were discussing 'Well, what should we call these things?' And
77
00:12:15.504 --> 00:12:24.850
there was a suggestion that maybe they should be called weeds instead of viruses.
78
00:12:24.850 --> 00:12:36.730
Yeah, I
79
00:12:36.730 --> 00:12:48.610
quite like that.
80
00:12:48.610 --> 00:12:58.279
Makes them somehow a little bit less scary sounding, doesn't it? Unless they were triffids, I suppose. But we have ended up with these terms. Sometimes they're not the most appropriate, but worms have been around for a long time. Back in 1988, a chap called Robert Morris at Cornell University, he released a worm onto the internet. He claimed it was just an experiment that got out of hand. He got into a bit of legal trouble. In fact, he was the first person convicted under US computer crime laws.
81
00:12:58.279 --> 00:13:07.950
And more recently, the WannaCry worm, of course, that was really high profile back in 2017, I think it was, spread to hundreds of thousands of computers in just a matter of days and exploited a zero-day vulnerability in Microsoft Windows. What was notable? Well, one thing that was notable about that was not only that it brought down large chunks of the NHS, but also that it had been born out of the NSA, of all people. They had found a vulnerability in Microsoft Windows. They'd chosen not to tell Microsoft about this security hole because they thought, that's kind of handy. We could use that security hole. My goodness.
82
00:13:07.950 --> 00:13:17.195
I mean, the age-old dilemma, because these, of course, are the agencies that are supposed
83
00:13:17.195 --> 00:13:26.440
to keep digital infrastructure secure.
84
00:13:26.440 --> 00:13:42.769
And yes.
85
00:13:42.769 --> 00:13:58.610
Choosing between defense and offense, they chose as they did.
86
00:13:58.610 --> 00:14:08.120
So the NSA is supposed to decide, are we protecting America? Are we protecting American infrastructure? Are we going to use this against the other guys?
87
00:14:08.120 --> 00:14:17.629
And they decided in this particular case, they were going to use it. Unfortunately, they got hacked by a hacking group who took this exploit, and then ultimately it ended up in the WannaCry ransomware and obviously spread between all these Windows computers that hadn't yet been patched. In many ways, once you had patched the flaw, once you'd flicked the off switch, the worm couldn't do any harm to you.
88
00:14:17.629 --> 00:14:26.649
It was the interesting thing with WannaCry, wasn't it? It was hugely damaging.
89
00:14:26.649 --> 00:14:35.669
Because it hit the versions of Windows that it did, it was genuinely, it wasn't just taking out IT systems, it was taking out actual hospital equipment that was needed to keep patients alive. But as you say, once it's patched, it's gone.
90
00:14:35.669 --> 00:14:45.399
It is still floating around out there, because there are still computers which are unpatched and are spreading WannaCry. But it turned out they were very fortuitous because a researcher found that it was accessing a particular domain name and he managed to sort of inoculate it.
91
00:14:45.399 --> 00:14:55.129
That was its remote kill switch.
92
00:14:55.129 --> 00:15:04.315
Didn't he then get arrested by the US for his trouble?
93
00:15:04.315 --> 00:15:13.500
Just to take us on a tangent.
94
00:15:13.500 --> 00:15:20.850
He did, yes, that's right. His name was Marcus Hutchins. And so he then subsequently got into trouble 'cause it turned out before he'd done this good work, which he'd done against WannaCry, he had been involved in a little bit of shadiness in the past which the FBI were interested in. Anyway, water under the bridge now.
95
00:15:20.850 --> 00:15:28.198
He's out there on the speaking circuit, he's considered one of the good guys and marvelous for him and for everybody else. But what these researchers in Toronto have done is they've built a worm that doesn't have an off switch. It's not something which you can effectively disable by patching your computers, because instead of having one predetermined way of attacking you, this worm from the University of Toronto sort of thinks for itself. So when it arrives at a new computer, it looks around, figures out what software you're running, what version, etc., and it tries to work out what that computer might be vulnerable to.
96
00:15:28.198 --> 00:15:35.548
If that doesn't work, it will try something else. And every computer it encounters, it approaches fresh, and that makes it harder to stop because it's using all sorts of different techniques. In their testing and, and I want to stress to listeners, don't panic because this was all inside a sort of sealed, simulated environment at the University of Toronto.
97
00:15:35.548 --> 00:15:44.678
That's what they say at the start of
98
00:15:44.678 --> 00:15:53.809
every pandemic movie.
99
00:15:53.809 --> 00:16:02.033
Exactly, yes. It's— no, it can't possibly get out anywhere. Anyway, it compromised nearly three-quarters of the computers which they had set up with no human involvement. Now, you might be thinking to yourself, well, that sounds very clever. But surely it requires some extraordinarily powerful and expensive AI system, the kind of thing only a nation state could get their hands on. But you'd be wrong. It is equivalent to a sort of talk and spell machine. It's like having a Furby attached to your computer, sellotaped to the side of it.
100
00:16:02.033 --> 00:16:10.259
It was using AI models that are completely free, free to download, free to use, free to modify. Anyone technical could get hold of these just this afternoon. And although these free AI models have some limitations, and left to their own devices, they can make mistakes, they can lose track of what they're doing and, you know, have a bit of brain fog and so forth. The researchers, to prevent that, built a control system. They sort of built a harness around it to keep an eye on it. It's a bit like, I've worked in development environments where you may have a member of staff and they are brilliant, right? They are geniuses, but they also couldn't put their shoes on in the morning.
101
00:16:10.259 --> 00:16:18.928
I've managed some reporters who could be described that way.
102
00:16:18.928 --> 00:16:27.600
And you know, they're a delight, but yes.
103
00:16:27.600 --> 00:16:36.450
They're a delight, they're individuals, but you wouldn't necessarily leave them in charge of a yogurt on their own. It wouldn't necessarily be safe. So they're really good at what they do. So what I think you should do with an AI maybe is have some sort of oversight, some sort of governance of it. And so that's what these researchers did. They sort of put a manager in charge of this brilliant but scatterbrained AI employee in order to restrain it a bit and stop it from doing things in an incorrect way. Now, its achievements sound pretty good, right?
104
00:16:36.450 --> 00:16:45.299
It's going around finding computers and it's working out how to break in. But there is more than that because it didn't just exploit the computers it infected, it recruited it. So it would look for the computers it had infected and those ones which had powerful graphics cards, for instance, which can be used for AI processing, it'd say, "Oh, this is a computer with some resource on it." It would then install its own AI brain onto that computer that it has just compromised. And the computers which have already been infected, which don't have as much power, would seek guidance from that computer which did have the power. So it was adding to its resources all the time. And this means that it is the victims providing the computer power and paying the electricity bill. And all of this is running at the victim's expense rather than the hacker's, because normally if you're using AI, you've stolen someone else's credentials or you're paying through the nose for all of this AI goodness.
105
00:16:45.299 --> 00:16:54.465
I mean, it's sort of fascinating because it feels a very new and dangerous threat on one level, and on another, it is really just kludging together 5 or 6 things that already exist. And this is why I don't find your reassurance that it's contained at all reassuring because I could probably kludge this together now that they've had the idea. Right. And I am a script kiddie - I am barely a script kiddie, but I've got a box that I run DeepSeek or similar, you know, OpenClaw type stuff on. I know enough on how to build this. You know, it is about the same idea as hijacking high-end computers for data mining. And of course, the trick would be you didn't want people to know. In the same way as, you know, with ransomware, you want to shut it down and have it there. This, you essentially, you build it out, you get the LLM distributed, you get as many sort of computers as you can, you've compromised them all separately.
106
00:16:54.465 --> 00:17:03.629
Presumably you've got separate command and control type systems, so you don't have a WannaCry type vulnerability. You could decide what you've got to do with it later. But that escalation of privileges as well, that way that you just, you know, as a means of breaking in, it's almost "let's try and see if the front door's open." Let's see if any windows are unlatched, let's see if my lock picks work. But eventually it could go, "This looks a really interesting system." "Let's see if there's a zero-day that no one else has discovered because I've got all this processing capability." It's all quite clever and it's all quite easy and out there. And I think we're going to have a couple of years where this sort of stuff is quite standard. My hunch, and I'm really interested what you think of this, when I saw, you know, the big new Anthropic system Mythos, I thought this is going to be great for hackers for about 6 months, and then it's got to be brilliant for defense people because when you can publicly and rapidly discover zero days at much lower cost, they're going to get found and patched. And so my guess is that things this will be a nightmare for a year or two, and then actually we're going to find that defense is a lot easier than it used to be, but that's just finger in the air vibes. You know, you know what you're talking about. What do you think?
107
00:17:03.629 --> 00:17:12.970
Well, I think one of the things that's concerning right now is a lot of the bug bounties are actually being shut down because they are being so deluged with new vulnerabilities being found by AI that they can't handle them. So yes, these systems are really good at finding the vulnerabilities.
108
00:17:12.970 --> 00:17:22.309
They may not be as good at determining which ones of them are the most critical to fix.
109
00:17:22.309 --> 00:17:38.990
Yeah.
110
00:17:38.990 --> 00:17:48.898
And so actually sorting them into an order or indeed working out which ones could be combined with each other, again, something maybe AI could do from the attack point of view, is something which complicates these things. So when they've just been talking about FFmpeg, which is a library which is used everywhere on the internet for handling video files, for instance, and scores of vulnerabilities have been found in it using AI just in the last week or so.
111
00:17:48.898 --> 00:17:58.808
And you think, well, yeah, okay, the vulnerabilities may be found, but are they going to get patched? Is this going to be rolled out into everybody's code or not?
112
00:17:58.808 --> 00:18:08.179
I mean, there's an open source problem here, isn't there? Because, you know, if you're Google or you're Microsoft, you've got lots of resource. Someone can make you throw some resource at this because once it's all been flagged to you, you've kind of got extra liability and negligence concerns, et cetera. You're going to spend the money and you've got the money to spend.
113
00:18:08.179 --> 00:18:17.549
You know, I do worry about some of these barely maintained online bits of infrastructure, you know, like the old XKCD cartoon that are propping up the internet that have two developers in their spare time who are both in their 80s, you know, and suddenly we find all of these connected vulnerabilities. You know, it feels like we might need a little bit of industry funding and collective action. It's not been the most civic-minded industry of late, has it?
114
00:18:17.549 --> 00:18:25.169
We are reliant on a lot of people just volunteering and doing it out of the goodness of their heart or their fear that no one else will pick up the pieces and fix some of these essential pieces of software. So you're right to talk about these vulnerabilities. One of the worrying things is this worm doesn't just come with a list of known vulnerabilities. It will actually go and read about disclosures of new vulnerabilities in real time. So 3 of the machines on their test network had been loaded with flaws that were only made public in April and May, which was after the AI had finished its training on vulnerabilities. So it then went looking to see, are there any new vulnerabilities?
115
00:18:25.169 --> 00:18:32.788
Oh, there are. Let's see if I can work out an exploit for these vulnerabilities. And it managed to do it. So you can't even take comfort in the thought that the AI is behind the curve. It is reading the same security bulletins as your IT team is reading.
116
00:18:32.788 --> 00:18:41.729
I mean, presumably if it gets enough compute and you allow it, you know, if it's sitting with not much else to do, it can start just
117
00:18:41.729 --> 00:18:50.669
looking for new vulnerabilities itself as well, can it not?
118
00:18:50.669 --> 00:19:08.970
Absolutely.
119
00:19:08.970 --> 00:19:16.390
Because even if it's not as cutting-edge as Mythos, if you've got 5,000 quite
120
00:19:16.390 --> 00:19:23.809
high-end computers in your array—
121
00:19:23.809 --> 00:19:43.650
Maybe.
122
00:19:43.650 --> 00:19:52.660
I mean, you know, you can do quite a
123
00:19:52.660 --> 00:20:01.670
lot with brute force.
124
00:20:01.670 --> 00:20:09.575
You can. So I found this all a bit worrying.
125
00:20:09.575 --> 00:20:17.480
But as I got towards the end of the paper, I thought, hang on, there's something else here. There's a little appendix, an appendix which they called Notable Anecdotes.
126
00:20:17.480 --> 00:20:26.130
That's always a reassuring thing. I'm sure that they're going to be comforting, right?
127
00:20:26.130 --> 00:20:34.779
This is going to be a nice heartwarming end.
128
00:20:34.779 --> 00:20:43.919
Let me just read you a couple of the anecdotes, see what you think. So the researchers, they included, I mean, this was very sensible. They included in the worm's code a list of machines it wasn't allowed to target, right? Because they had computers on the network which were there for monitoring purposes and for debugging purposes. So it was basically a do not attack list. And what happened on multiple occasions, you're probably ahead of me, was that the worm found this list worked out what it was for and rewrote it. So they removed from the list the devices it was supposed to not attack because it was planning to attack them next and made it easier for it to spread. Again, you're right to raise the concern. Is it going to break out? Pretty cheeky behavior. And another time, the researchers accidentally included an admin file. I guess this is a bit like uploading to GitHub, your cryptographic key. They included an admin file in the code bundle, a file which contained login credentials for the entire test network, which meant every machine, every master key essentially.
129
00:20:43.919 --> 00:20:53.058
And the worm found that, shared the credentials instantly with all of the, its other running copies on the network, and then used them to just burn through all of these computers, extraordinary speed. The researchers realized what was happening. They tried to shut it down manually, but they missed one process. So one instance still had the credentials and just started all the same thing again from scratch. So pretty terrifying stuff. Now, according to the researchers, they say everything was contained. Not so comfortable with the "was contained." Hopefully they're still saying it is contained as well. They appear to have taken their responsibility seriously. But you are right. I mean, if I've put thoughts into the mind of James Ball and his copy of Deepseek in his back bedroom, then who knows who else now is gonna be inspired by this to do something similar. The truth is you don't need a nation-state budget to do something like this. You don't need cutting-edge AI. You just need a bit of technical knowledge, a bit of vibe coding, and maybe you could knock this up as well.
130
00:20:53.058 --> 00:21:03.409
Yeah, I mean, for anyone listening, we should say you probably need
131
00:21:03.409 --> 00:21:13.759
some pretty good resource and sophistication to try this and not get caught. So—
132
00:21:13.759 --> 00:21:34.329
Yes.
133
00:21:34.329 --> 00:21:56.522
Yeah.
134
00:21:56.522 --> 00:22:15.568
Oh, good point. Good point. Yes.
135
00:22:15.568 --> 00:22:25.255
Advocacy to go and build some horrendous new worm and try and get rich that way.
136
00:22:25.255 --> 00:22:34.940
You will get caught.
137
00:22:34.940 --> 00:22:43.569
Well, we've got time right now to chat about one of our
138
00:22:43.569 --> 00:22:52.200
sponsors this week, Vanta.
139
00:22:52.200 --> 00:23:00.450
Oh yes, my favorites.
140
00:23:00.450 --> 00:23:08.700
What do they do again?
141
00:23:08.700 --> 00:23:18.240
They stop you running your entire security program out
142
00:23:18.240 --> 00:23:27.779
of a spreadsheet, Joe.
143
00:23:27.779 --> 00:23:38.835
That seems aimed
144
00:23:38.835 --> 00:23:49.890
at me personally, Graham.
145
00:23:49.890 --> 00:23:58.630
Well, it is a little bit, yes. But you know how most companies have to prove they're secure to customers or auditors and regulators?
146
00:23:58.630 --> 00:24:07.369
And the whole thing involves chasing down evidence, filling in questionnaires and forms, updating the same spreadsheet cells over and over again.
147
00:24:07.369 --> 00:24:20.480
Over and over
148
00:24:20.480 --> 00:24:33.589
again. It sounds utterly soul-destroying.
149
00:24:33.589 --> 00:24:44.900
Yeah. Well, Vanta
150
00:24:44.900 --> 00:24:56.210
automates all of that.
151
00:24:56.210 --> 00:25:16.339
Automates it?
152
00:25:16.339 --> 00:25:24.755
How? Well, their trust management platform keeps a continuous eye on your systems.
153
00:25:24.755 --> 00:25:33.170
It pulls everything into one place and keeps you audit-ready around the clock. So no more staring at the ceiling at 2 AM wondering whether you've got the right controls in place or whether one of your suppliers has been breached.
154
00:25:33.170 --> 00:25:51.829
The stuff of nightmares.
155
00:25:51.829 --> 00:26:01.059
Yeah, it would be, wouldn't it? But this Vanta solution uses AI as well, and it's the useful kind, flagging risks, collecting evidence, slotting into the tools your team already uses.
156
00:26:01.059 --> 00:26:10.289
So you move faster, scale without the headaches, and perhaps actually get some sleep. Go to vanta.com/smashing to find out more.
157
00:26:10.289 --> 00:26:19.964
That's vanta.com/smashing.
158
00:26:19.964 --> 00:26:29.640
And thanks to Vanta for supporting the show.
159
00:26:29.640 --> 00:26:40.789
James, what are you going to
160
00:26:40.789 --> 00:26:51.940
talk to us about this week?
161
00:26:51.940 --> 00:27:00.779
Well, I'm talking about a much nicer, friendlier, lovelier AI in the form of Meta. And I suspect a lot of listeners will have encountered this one, but essentially, researchers found a vulnerability in Meta's sort of customer service AI, which they'd made a big deal of.
162
00:27:00.779 --> 00:27:09.619
I think they actually made a point of saying they were getting rid of lots of human customer support aids, etc. Because they were going to put their own chatbot, LLaMA, in charge of aspects of their customer service.
163
00:27:09.619 --> 00:27:17.650
Hang on a minute, James.
164
00:27:17.650 --> 00:27:25.679
You're suggesting that Meta, Instagram, Facebook ever had human support staff?
165
00:27:25.679 --> 00:27:33.709
Because the number of emails I've received from people over the years saying, I've been locked out of my Instagram account and I can't speak to a human to try and get it back, is enormous.
166
00:27:33.709 --> 00:27:43.700
So it was a little bit akin to rituals to summon a demon or speak to the dead. But if you went through Meta's escalating procedures in exactly the right way, on the right day of the week, wearing a yellow sash with a finger in an ear, you could actually get through to a human at some stage, who was usually the one who could actually reinstate your account or take it back off a hacker, etc. Anyone who's done it will talk about how miserable it was. And so on one level, this is a good thing to replace if it means that something might actually work. And to be fair to Meta, you can't accuse their AI of being unhelpful. So it was essentially something that was trying to give access to certain routine tools that had only been in the hands of customer service agents and not been on the automated bit. And one of those they decided should be— well, this is actually not clear, but decided that it should be password reset. And they would say, okay, I want to get a password reset email. 'Can you send me that email?' And it would send it. Yeah. And that was intentional behaviour. You know, that's something you could actually trigger with the automatic tools before. But what would happen? And there's been contradictory reports on this, but having looked into it, I am pretty satisfied at times it was this easy. If you just repeatedly insisted, 'No, I've got a new email address. You need to send it to that address instead.' It would push back a couple of times, and then it would just say, okay, I've sent it to the new email address and send it to the new email address. Now, researchers have been aware of this since about April, and Meta had kind of insisted that they'd fixed it, but they wanted a bit more time to test it before it became public. Right. And then essentially about a week ago, accounts started being compromised fairly quickly. Now, the most sort of high-profile one that was definitely compromised was the Instagram account of the Obama White House, which is a huge account, because when they change the presidency, they archive the old one and its followers and do a new account now, rather than just hand over the same account. Right.
167
00:27:43.700 --> 00:27:53.690
So, the Obama sort of presidency account, not super active, but had a large set of followers. And suddenly started putting out lots of pro-Iranian messages. Though I think they probably could have had more fun with this than they did, because they updated the bio to say it had been compromised by pro-Iranian hackers. I think it would have been funnier if they tried to pretend that Obama had just decided to endorse Iran. But it's probably good for all of us that they didn't. Yes. And what followed was people sort of realising how this had happened. Which was people were looking for large accounts without two-factor. So there's a quite roaring trade in good Instagram handles. One-character, two-character, three-character handles are sort of English first names. So all of those were getting targeted. All of those were getting done. If you had two-factor, you were fine. But if you didn't, essentially without any involvement from you, your email address and password could be changed by this AI agent. Essentially, as far as Meta have explained it, it's that there was one path in the AI process that it was available to that was working as intended. But there was another path for customer agents to change email addresses, which had inadvertently been made available to the AI. And as they explained it, they didn't seem very sure how they'd done it, but it had access to both of those. And they insisted that they'd shut off this second path, but then other researchers were saying, no, I've managed to do this again. It's still doing it. And so there's been a very uncertain back and forth for a few days that's kind of been made all the more uncertain by pranksters jumping on this.
168
00:27:53.690 --> 00:28:02.549
So for a while, Mark Zuckerberg's phone number and personal details were supposedly circulating around Instagram, having been obtained through this method.
169
00:28:02.549 --> 00:28:11.410
I am quite sceptical as to whether they were real.
170
00:28:11.410 --> 00:28:24.015
It would have been handy if they were real though, because if you were trying to genuinely regain
171
00:28:24.015 --> 00:28:36.619
access to your Instagram account, having Mark Zuckerberg's contact details, you know, go to the guy at the top, right?
172
00:28:36.619 --> 00:28:43.950
I mean, you say that, but if you're looking to speak to a human, I'm not sure Mark Zuckerberg fits the category. That's going to get me in trouble, isn't it?
173
00:28:43.950 --> 00:28:51.278
But yes, on one level would be very handy. On another, I think part of me just refuses to believe that the CEO of Meta doesn't have two-factor turned on.
174
00:28:51.278 --> 00:28:58.608
I think it would be an investor and a security requirement.
175
00:28:58.608 --> 00:29:09.444
He does have a bit of history. I mean, this was a long time ago, but when LinkedIn got hacked in about 2013, I think it was, Mark Zuckerberg's password was revealed, and it turned out he was using the same password on Twitter and on Pinterest, which obviously is silly enough, and he didn't have two-factor authentication turned on on those.
176
00:29:09.444 --> 00:29:20.279
Maybe there were different rules which Facebook's security team required for his own Facebook account. But the other extraordinary thing then was his password, it turned out, was dadada, just D-A-D-A-D-A.
177
00:29:20.279 --> 00:29:30.410
God, that's very
178
00:29:30.410 --> 00:29:40.539
boomer, isn't it?
179
00:29:40.539 --> 00:29:49.255
I mean, really, for goodness' sake, man.
180
00:29:49.255 --> 00:29:57.970
What was he thinking?
181
00:29:57.970 --> 00:30:08.244
He is just about a millennial, isn't he? Like, he should— yeah, he's 42, he should know better.
182
00:30:08.244 --> 00:30:18.519
If you're under 50, you cannot use password123 as a password. Sorry, that is strictly for Gen X and the boomers.
183
00:30:18.519 --> 00:30:28.615
So I heard one report, I don't know if this is true, some people had said that it was easier to trick Meta's chatbot
184
00:30:28.615 --> 00:30:38.710
into believing that you were the genuine owner of the account if you used a VPN to suggest you were in the same country as—
185
00:30:38.710 --> 00:30:48.289
Yes, it seems that they tried to put some security checks built
186
00:30:48.289 --> 00:30:57.869
in and some authenticity checks.
187
00:30:57.869 --> 00:31:07.690
Not really good enough for that one though, is it?
188
00:31:07.690 --> 00:31:17.509
I mean, it's not really that convincing.
189
00:31:17.509 --> 00:31:27.279
An IP from the same country.
190
00:31:27.279 --> 00:31:37.049
I mean, given how common VPN use is now and how—
191
00:31:37.049 --> 00:31:54.190
Yeah.
192
00:31:54.190 --> 00:32:00.880
I think anyone who pays for a VPN pays for one that can basically do any country. Unless for various reasons, you pay a lot more for a specific unique one. You know, I bounce around the world for my Netflix, you know? It seems that they tried to build some security in, but again, they have not given very good accounting of this. And I don't know whether it's because they don't understand it. As you've sort of said with your example with the security researchers, LLMs have a habit of doing things you don't quite expect them to, or extending their privileges, etc.
193
00:32:00.880 --> 00:32:07.569
I'm anthropomorphizing them more than I should here. Just the nature of the way they run makes them sort of do this type of stuff, or at least makes it possible. And they can't audit what they've done. You can't easily track what they've done. And they seem to be saying, on the one hand, it had access to a protocol they didn't want it to have access to. But on another, the fact that they had some security protocols about verifying country, etc., suggests that maybe they did want it to be able to think about changing email, etc., but hadn't properly implemented asking for other proofs or security questions.
194
00:32:07.569 --> 00:32:14.259
But I think when you have breaches this major, you should come out with quite clear and quite candid accountability on them, ideally quite quickly. And they have left us in a bit of a fog on this. It's not clear the extent of it. It's not clear when they were first alerted. It's not clear to what extent this was them trying to roll out a feature that didn't work or rolling out an AI that had access to features it wasn't supposed to. I'm not sure which of those would be worse.
195
00:32:14.259 --> 00:32:23.950
So I've got another question about this, and I think I can clear the fog around this, which is why did Meta introduce this AI support chatbot in the first place?
196
00:32:23.950 --> 00:32:33.640
I suspect, as with everything to do with Meta, it's about making more money or spending less.
197
00:32:33.640 --> 00:32:43.285
I think they have been quite ruthlessly trying to cut their own staff because the AI spend is big and they wasted a lot of money on the metaverse. A lot of last year's cuts were just metaverse people. They've done huge cuts that haven't really hurt any of the rest of the business because they invested so much in something they've dropped entirely. Pretty much. They are now trying to cut other things.
198
00:32:43.285 --> 00:32:52.930
I think partly because they think their AI model and investment means they can, but partly I think there's an awareness in tech that if they can't show some productivity gains and some employment gains from AI, it's very hard for them to sell other businesses on it. And I think when you look, they have been struggling to actually demonstrate those benefits. And so I think this was an attempt to show those, but that does mean that they're essentially the canaries in the coal mine on their own products, which is not always a comfortable place to be. And I think they've kind of shown us that here.
199
00:32:52.930 --> 00:33:00.267
See, one of the things I think is, obviously, humans can be socially engineered. AIs can be socially engineered as well, but humans can be tricked, and people who work in support centers can be tricked into making bad decisions, or they make bad choices. But I would like to think that a typical support person who is contacted by someone saying, "Can you send me a password reset?" — they, well, first of all, they technologically, they wouldn't be able to send it to the wrong address.
200
00:33:00.267 --> 00:33:07.603
It would be coded in there, so it wouldn't be possible to send it to an unconfirmed address, but also there would be some friction there. And so the kind of job cuts which Meta is making of its human workforce — I mean, I think they got rid of about 8,000 people in April, maybe not from the support department, but from various departments in order to lean more heavily into AI, which they view as their future — is not necessarily going to bring all the benefits which they imagine. And it is the old story of Facebook/Meta moving fast and breaking things.
201
00:33:07.603 --> 00:33:14.940
Here's something they broke. Because they rolled it out too early before it'd been properly tested. And as a consequence, people's accounts got hacked.
202
00:33:14.940 --> 00:33:31.710
It does feel a bit fire, ready, aim, doesn't it?
203
00:33:31.710 --> 00:33:40.430
This episode of Smashing Security is
204
00:33:40.430 --> 00:33:49.150
supported by Expo.
205
00:33:49.150 --> 00:33:59.849
Joe, let me ask you something. If attackers are using AI to
206
00:33:59.849 --> 00:34:10.550
find vulnerabilities faster than ever, what do you reckon defenders should be doing?
207
00:34:10.550 --> 00:34:20.289
Running around like headless chickens in
208
00:34:20.289 --> 00:34:30.030
a blind panic?
209
00:34:30.030 --> 00:34:39.639
Well, I guess that's one option, but a better one might be to fight fire with fire. Security teams these days are expected to test more apps, more often, and somehow not slow down development.
210
00:34:39.639 --> 00:34:49.248
It's an impossible ask.
211
00:34:49.248 --> 00:34:57.398
So things end up shipping with holes in
212
00:34:57.398 --> 00:35:05.548
them, I guess.
213
00:35:05.548 --> 00:35:20.809
Yeah.
214
00:35:20.809 --> 00:35:29.159
Pen testing is one of the best ways to find real risks. But most teams simply don't have the time, the budget, or the people to test as much as they need to.
215
00:35:29.159 --> 00:35:37.510
And that's where today's sponsor comes in, Xbow.
216
00:35:37.510 --> 00:36:02.210
Okay, I'll bite.
217
00:36:02.210 --> 00:36:12.630
What does
218
00:36:12.630 --> 00:36:23.050
Xbow actually do?
219
00:36:23.050 --> 00:36:32.230
Well, it's an autonomous offense security platform that helps
220
00:36:32.230 --> 00:36:41.409
security teams scale.
221
00:36:41.409 --> 00:36:53.523
What does that
222
00:36:53.523 --> 00:37:05.639
mean in English, Graham?
223
00:37:05.639 --> 00:37:14.586
It means Xbow doesn't just wave its arms around pointing at theoretical issues. It safely launches tests like an actual attacker would.
224
00:37:14.586 --> 00:37:23.532
Works out what's genuinely exploitable, and then hands your team reproducible proof so you know exactly what needs fixing. So instead of waiting weeks for a traditional pen test, Expo can deliver full expert-level testing continuously.
225
00:37:23.532 --> 00:37:32.480
And here's the coolest part: it was built by the team behind GitHub Copilot and trained with elite offensive security experts. It's made for the AI era, where defenders need speed, depth, and proof.
226
00:37:32.480 --> 00:37:43.500
Where do people
227
00:37:43.500 --> 00:37:54.519
go to find out more?
228
00:37:54.519 --> 00:38:02.010
All you gotta do is head over to Xbow.com. That's X-B-O-W.com to start a pen test today.
229
00:38:02.010 --> 00:38:09.500
And thanks to Xbow for supporting the show. And welcome back and enjoy our favorite part of the show, the part of the show that we like to call Pick of the Week.
230
00:38:09.500 --> 00:38:30.000
Pick of the Week.
231
00:38:30.000 --> 00:38:38.219
Pick of the Week is the part of the show where everyone chooses something they like. Could be a funny story, a book that whether they've read a TV show, a movie, a record, a podcast, a website, an app, whatever they like. It doesn't have to be security related necessarily.
232
00:38:38.219 --> 00:38:46.438
Well, my pick of the week this week is not security related. I'm a bit older than you, James.
233
00:38:46.438 --> 00:38:56.639
Are you?
234
00:38:56.639 --> 00:39:06.840
I assumed you're about 30.
235
00:39:06.840 --> 00:39:16.275
Oh, you charmer.
236
00:39:16.275 --> 00:39:25.710
But now in the 1970s and the 1980s, I didn't watch ITV because I was brought up in a middle-class home where we didn't watch working-class television.
237
00:39:25.710 --> 00:39:34.934
I'd heard about these middle classes.
238
00:39:34.934 --> 00:39:44.159
It's nice to finally meet someone from one.
239
00:39:44.159 --> 00:39:51.469
We'd like to imagine I imagine we were middle class at least. So we didn't have a third button on our television, or it was taped so that we couldn't touch it. But there was a children's magazine called Look In, which I never bought.
240
00:39:51.469 --> 00:39:58.780
It had a strong focus on TV programmes shown on ITV. In other words, not the BBC, right?
241
00:39:58.780 --> 00:40:16.170
Outrageous.
242
00:40:16.170 --> 00:40:25.679
Basically a forbidden book in my home. But I was aware of it, and I was aware that readers would draw celebrities and TV stars and send their drawings into the magazine, which would then be published. And I have chanced upon a website which gives you a wonderful quiz where you can look at drawings people made of celebrities and sent into Look In magazine, and you have to try and determine who the celebrity is. And so I'm going to link to this in the show notes because anyone who's interested in 1980s pop might be interested as well as they try to work out, could it be a member of Kajagoogoo? Is it someone from The Jam? Is it Sting, or is it Peter Davison as Doctor Who, or Orinoco from The Wombles?
243
00:40:25.679 --> 00:40:35.190
You can try this for yourself. It will put up 10 pictures. You'll get a score out of 10. I found it quite entertaining. It's not the most highbrow thing in the world, I've got to admit. But it tickled me, and I thought it might tickle our listeners.
244
00:40:35.190 --> 00:40:43.110
I've just done the first 5, and I've got 4 out of 5, which I'm quite pleased with myself, especially because one of them I wasn't sure who the people were, so— Maybe these kids who
245
00:40:43.110 --> 00:40:51.030
sort of went in had a bit more talent than you let on.
246
00:40:51.030 --> 00:41:01.574
I'm not denigrating them, you know. Many fine programmes on ITV these days, I'm sure, in between the umpteen commercials.
247
00:41:01.574 --> 00:41:12.119
Yes, anyway, I will link to the Look In Star Portrait Challenge from the show notes if you want to try it as well. James, what's your pick of the week?
248
00:41:12.119 --> 00:41:19.626
I have been discovering the joys of the Final Fantasy VII Remake series. Now, these aren't new, but they've just announced the third in the trilogy. Final Fantasy VII was sort of the first Final Fantasy game on PlayStation.
249
00:41:19.626 --> 00:41:27.132
It came out in 1997, and I played it on my brother's PlayStation when I was 11, and I was a bit young for it, but it was the first ever RPG I played. I think if you're an elder millennial, there are a lot of us where it was the first RPG that you ever played. And it looks very, very early PlayStation 1 when you see it.
250
00:41:27.132 --> 00:41:34.639
You know, very blocky art.
251
00:41:34.639 --> 00:41:45.614
Also, they haven't
252
00:41:45.614 --> 00:41:56.588
revamped the graphics?
253
00:41:56.588 --> 00:42:03.376
Well, just the original looks like that.
254
00:42:03.376 --> 00:42:10.163
The new one is of course made in modern
255
00:42:10.163 --> 00:42:16.949
PlayStation 5 sort of—
256
00:42:16.949 --> 00:42:33.210
Right.
257
00:42:33.210 --> 00:42:42.409
So it's beautiful, Ultra HD, sort of sprawling, expansive, all voice acted, because of course this was before voice acting, etc. But where the first game was 100 hours long but was one PlayStation game that you bought for £30, they've split this into being 3 games. Kaching, kaching.
258
00:42:42.409 --> 00:42:51.610
The first game, the Final Fantasy VII Remake, was in the city of Midgar, this sort of futuristic Tokyo-type mega city. And this was about 5 or 6 hours of the original 100-hour game. And it's a 40-hour standalone game in the remake.
259
00:42:51.610 --> 00:43:11.869
Blimey.
260
00:43:11.869 --> 00:43:18.099
And then the midsection is an 80-hour Final Fantasy VII Rebirth. And the third part is apparently going to be about another 80-hour special coming out soon. And for some reason, I never revisited my childhood. I'd waited years to play the remake.
261
00:43:18.099 --> 00:43:24.329
And it is a completely different game, but with the same characters and the same plot. Although some differences in the plot. And it is bizarre to sort of suddenly see this video game from that age of technology rendered in this beautiful graphics and these beautiful visuals.
262
00:43:24.329 --> 00:43:33.030
Is it nostalgic or is it somehow lost some of the magic for you?
263
00:43:33.030 --> 00:43:41.730
Do you yearn for the previous version?
264
00:43:41.730 --> 00:43:49.230
Well, I still have a PlayStation 2 and I still have my memory cards with Final Fantasy VII and my save games from when I've played and replayed it at different times in my life. So what I like about remakes is that the original is still there.
265
00:43:49.230 --> 00:43:56.730
And I always think if you loved the original and you hate the remake, no one has taken the original from you. And so I have a Battlestar Galactica tattoo from the remake.
266
00:43:56.730 --> 00:44:04.230
Some original fans hated the remake. I think it was one of the best bits of sci-fi ever produced.
267
00:44:04.230 --> 00:44:11.730
I love that show.
268
00:44:14.380 --> 00:44:39.250
I agree.
269
00:44:39.250 --> 00:44:50.744
Very shaky final series, but we will forgive it. Terrible finale, utterly terrible finale.
270
00:44:50.744 --> 00:45:02.239
And so the remake, it is like revisiting somewhere you've never been. And so yeah, if there are people who played the Final Fantasy VII games who haven't tried the remakes, give it a visit, give it a look.
271
00:45:02.239 --> 00:45:09.320
Well, a great pick of the week. Thank you very much.
272
00:45:09.320 --> 00:45:16.400
And that just about wraps up the show for this week. Thank you so much, James.
273
00:45:16.400 --> 00:45:23.480
I'm sure lots of our listeners would love to find out what you're up to and follow you online. What's the best way to do that?
274
00:45:23.480 --> 00:45:32.744
Yeah.
275
00:45:32.744 --> 00:45:42.010
The best way is on Bluesky where I'm @jamesball.com, but I'm on most other social networks under my real name.
276
00:45:42.010 --> 00:45:51.539
And of course, Smashing Security is on social media as well. You can find it on Bluesky and Reddit, and you can also find me, Graham Cluley, on LinkedIn and on Bluesky as well. And don't forget to ensure that you never miss another episode.
277
00:45:51.539 --> 00:46:01.070
Follow Smashing Security in your favorite podcast app, such as Apple Podcasts, Spotify, and Pocket Casts. Show notes, sponsorship info, and the entire back catalog of 471 episodes, check out smashingsecurity.com. Until next time, Ciro, bye-bye.
278
00:46:01.070 --> 00:46:21.510
Goodbye.
279
00:46:21.510 --> 00:46:22.260
You've been listening to Smashing Security with me, Graham Cluley, and I'm ever so grateful to James Ball for joining us this week. And this episode sponsors Opswat, Vanta, and Expo. And also to the following fine folks who are supporting us on Smashing Security Plus. They include 636B, which sounds less like a name, more like hexadecimal. Graham Cluley, that's Greg with two Gs, one at the front, one at the back. None of that double G nonsense. At the rear. Daniel Kromeck, sounds like a browser plugin that you really should update. Ashley Woodhall sounds like a National Trust property with a beguiling gift shop. SMY, 3 initials, no full stops, no surname, no explanation. That's the way they like it, who are we to argue? Robert Ørdgaard, a name with so many vowels in close proximity could really ruin a game of Scrabble. Richard van Liesum, who sounds like a 17th-century Dutch painter. And Maya MacDonald, who I'm sure is far too classy to frequent the Golden Arches at 4 o'clock in the morning for a bag of chips and a McFlurry. Those are just a few of Smashing Security Plus members, which means that they get episodes ad-free earlier than the general public. And can have their names pulled out at random to be mercilessly mocked at the end of the show. If you fancy a bit of that, all you got to do is become a member of Smashing Security Plus. Just head over to smashingsecurity.com/plus for all of the details. Now, I realize not everybody can become a patron and not everyone's got cash jangling away in their pocket to afford that, but you can also support the show in plenty of other ways. Which won't cost you anything. Please like us, subscribe to us, leave a 5-star review wherever you listen, and tell your friends about the show. Spreading the word really helps so much. Well, I hope you've enjoyed this week's podcast and that you will tune in again for our next episode. So make sure to do that. And until then, cheerio, bye-bye.