WEBVTT

00:00:00.640 --> 00:00:04.240
Welcome to episode 43 of the AI

00:00:04.240 --> 00:00:06.720
infrastructure podcast.

00:00:06.720 --> 00:00:08.559
Now, I just want to draw your attention

00:00:08.559 --> 00:00:12.400
to the website debase.com/mpodcast,

00:00:12.400 --> 00:00:15.599
a domain that I own as a minister. And

00:00:15.599 --> 00:00:20.240
the idea is is if uh you miss a podcast

00:00:20.240 --> 00:00:23.039
or you want to find what's in the

00:00:23.039 --> 00:00:26.640
podcast, you can just dip in. There's a

00:00:26.640 --> 00:00:28.640
summary there.

00:00:28.640 --> 00:00:30.400
there's

00:00:30.400 --> 00:00:32.640
uh a whole transcript.

00:00:32.640 --> 00:00:34.800
So don't feel obligated to listen to the

00:00:34.800 --> 00:00:36.000
whole thing. You know, it's just a

00:00:36.000 --> 00:00:37.680
conversation between two geeks

00:00:37.680 --> 00:00:40.559
ultimately and we talk about one thing

00:00:40.559 --> 00:00:42.800
and another. So whatever interests you,

00:00:42.800 --> 00:00:46.160
you can just use debase.com/mpodcast

00:00:46.160 --> 00:00:48.960
as a way of navigating the content that

00:00:48.960 --> 00:00:52.000
we produce. Otherwise, please enjoy this

00:00:52.000 --> 00:00:56.399
podcast, which I've yet to title and

00:00:56.399 --> 00:00:58.719
develop a thumbnail for. And by the way,

00:00:58.719 --> 00:01:00.640
if you have any feedback about the

00:01:00.640 --> 00:01:02.719
editing, because sometimes I've used AI

00:01:02.719 --> 00:01:05.600
to sort of cut out the pauses. This one

00:01:05.600 --> 00:01:09.920
I've kept the pauses and the ums and rs.

00:01:09.920 --> 00:01:11.280
Let me know what you think, what you

00:01:11.280 --> 00:01:14.479
prefer to listen to, and I will make

00:01:14.479 --> 00:01:16.479
adjustments accordingly.

00:01:16.479 --> 00:01:19.680
All the best. Have a great one.

00:01:19.680 --> 00:01:21.360
Can we talk? I actually there was one

00:01:21.360 --> 00:01:23.520
topic I wanted to to to to bring up and

00:01:23.520 --> 00:01:25.920
and it never got uh got brought up and

00:01:25.920 --> 00:01:27.759
it was like two or three months ago when

00:01:27.759 --> 00:01:29.920
people were talking about how the cloud

00:01:29.920 --> 00:01:31.920
app the moment you installed it creates

00:01:31.920 --> 00:01:34.320
this huge reverse tunnel to their

00:01:34.320 --> 00:01:36.240
control center basically.

00:01:36.240 --> 00:01:37.759
>> Well

00:01:37.759 --> 00:01:40.240
in my employer my client they disabled

00:01:40.240 --> 00:01:42.880
that RC feature but yeah carry on.

00:01:42.880 --> 00:01:45.360
>> So what happened is I never used it. I

00:01:45.360 --> 00:01:47.439
think maybe you asked me or like did you

00:01:47.439 --> 00:01:49.680
ever tried slash remote in Claude Code?

00:01:49.680 --> 00:01:50.960
And I was like

00:01:50.960 --> 00:01:51.280
>> see

00:01:51.280 --> 00:01:54.640
>> well I've never gotten to it.

00:01:54.640 --> 00:01:58.079
>> I've never gotten to it but um my son

00:01:58.079 --> 00:01:59.920
got a MacBook and he doesn't know how to

00:01:59.920 --> 00:02:01.600
configure it and I told him to use cloud

00:02:01.600 --> 00:02:03.840
code and suddenly I start getting phone

00:02:03.840 --> 00:02:06.560
notifications on how to set up Roblox on

00:02:06.560 --> 00:02:09.440
MacBook and of course I I I stormed to

00:02:09.440 --> 00:02:11.280
him. I say why are you selling games on

00:02:11.280 --> 00:02:12.560
your MacBook? He's like what are you

00:02:12.560 --> 00:02:14.319
talking about? Well, Claude Code

00:02:14.319 --> 00:02:17.280
magically suddenly connected the RC by

00:02:17.280 --> 00:02:19.520
itself. And then the next thing I know,

00:02:19.520 --> 00:02:22.319
I see, you know, I get a notification

00:02:22.319 --> 00:02:24.959
about my Claude Code on my laptop where I

00:02:24.959 --> 00:02:28.000
never ever enabled remote. Suddenly, my

00:02:28.000 --> 00:02:29.599
phone is pinging me like there's an open

00:02:29.599 --> 00:02:31.440
question in this particular session. And

00:02:31.440 --> 00:02:33.760
I was like, what the hell? You know, I

00:02:33.760 --> 00:02:35.280
never enabled it.

00:02:35.280 --> 00:02:37.519
>> Yeah. Uh

00:02:37.519 --> 00:02:40.080
>> my I think my wife my wife said the same

00:02:40.080 --> 00:02:42.560
thing to me cuz I'm using her her pro

00:02:42.560 --> 00:02:44.879
account and everything everything's

00:02:44.879 --> 00:02:47.680
synced up nicely now like all the cord

00:02:47.680 --> 00:02:48.959
>> so convenient.

00:02:48.959 --> 00:02:50.319
>> It's so good.

00:02:50.319 --> 00:02:52.959
>> Here I am buying surprise gifts for my

00:02:52.959 --> 00:02:56.239
wife.

00:02:56.239 --> 00:02:56.959
>> This is a

00:02:56.959 --> 00:02:58.800
>> She's like she's like watching your chat

00:02:58.800 --> 00:03:00.480
like oh I don't like that.

00:03:00.480 --> 00:03:02.480
>> This is like why people want to use

00:03:02.480 --> 00:03:04.800
Chrome incognito, right? If you wanna if

00:03:04.800 --> 00:03:06.239
you want to buy a surprise for your

00:03:06.239 --> 00:03:06.560
wife.

00:03:06.560 --> 00:03:07.840
>> Yeah.

00:03:07.840 --> 00:03:10.000
>> Actually, that that's one thing that I I

00:03:10.000 --> 00:03:13.519
was chatting to the CISO yesterday about

00:03:13.519 --> 00:03:15.519
is like

00:03:15.519 --> 00:03:19.200
um Okay, you call me out if if this is

00:03:19.200 --> 00:03:21.040
if this is a ridiculous plan. But I was

00:03:21.040 --> 00:03:23.760
saying that we need right now chat logs

00:03:23.760 --> 00:03:25.519
from all the employees is super

00:03:25.519 --> 00:03:27.120
sensitive like only like one or two

00:03:27.120 --> 00:03:29.760
people can view them with permission and

00:03:29.760 --> 00:03:31.440
you know and then of course that

00:03:31.440 --> 00:03:35.680
permission uh is is uh time limited

00:03:35.680 --> 00:03:39.519
what's the expression like it expires

00:03:39.519 --> 00:03:41.840
>> but we really need to use the chat logs

00:03:41.840 --> 00:03:43.360
to improve our processes.

00:03:43.360 --> 00:03:44.799
>> Yeah. Yeah. Yeah. So

00:03:44.799 --> 00:03:46.159
>> that's a big initiative in a lot of

00:03:46.159 --> 00:03:47.680
organizations. The CISO was saying like,

00:03:47.680 --> 00:03:49.200
oh, a lot of people are using this for

00:03:49.200 --> 00:03:50.959
private information like how to deal

00:03:50.959 --> 00:03:53.200
with redundancies and how to deal with

00:03:53.200 --> 00:03:55.680
teammates and things like this. We can't

00:03:55.680 --> 00:03:59.120
we can't have it uh we can't have people

00:03:59.120 --> 00:04:03.120
using the chat logs to to um because of

00:04:03.120 --> 00:04:04.799
that reason, right? Private

00:04:04.799 --> 00:04:06.560
>> ridiculous. And I'm not

00:04:06.560 --> 00:04:09.200
>> Don't they sign a contract that says I'm

00:04:09.200 --> 00:04:12.799
aware that the the the subscription is

00:04:12.799 --> 00:04:14.159
from the company and it is being

00:04:14.159 --> 00:04:16.000
monitored. I signed it. That's all in

00:04:16.000 --> 00:04:18.799
the AI policy. I'm not too sure if

00:04:18.799 --> 00:04:21.280
anyone's [ __ ] read read it, but we

00:04:21.280 --> 00:04:22.880
also even have like, you know, like

00:04:22.880 --> 00:04:25.520
Claude has this uh company announcements

00:04:25.520 --> 00:04:28.479
thing and we say like your stuff is

00:04:28.479 --> 00:04:31.280
logged. Don't put private information in

00:04:31.280 --> 00:04:34.800
here, but people still evidently do it.

00:04:34.800 --> 00:04:37.280
And now like again, I'm blocked. I'm

00:04:37.280 --> 00:04:39.919
blocked. No, but like what does how does

00:04:39.919 --> 00:04:41.919
that impact you? Right. So is there in

00:04:41.919 --> 00:04:44.960
in Europe or in UK a law that says that

00:04:44.960 --> 00:04:48.479
even if you explicitly make an uh what

00:04:48.479 --> 00:04:50.160
is this a disclaimer and the and the

00:04:50.160 --> 00:04:52.160
people sign uh sign the disclaimer you

00:04:52.160 --> 00:04:55.680
cannot use the disclaimer to to

00:04:55.680 --> 00:04:56.800
protect yourself.

00:04:56.800 --> 00:04:58.400
>> Yeah. Absolutely right. But there's some

00:04:58.400 --> 00:05:01.120
people in my organization um who are

00:05:01.120 --> 00:05:04.160
like oh we need to um it's like European

00:05:04.160 --> 00:05:06.960
mentality. We need to look after people.

00:05:06.960 --> 00:05:09.199
>> We need to look after people.

00:05:09.199 --> 00:05:10.320
>> Yeah. Yeah. Of course,

00:05:10.320 --> 00:05:11.840
>> we've done mistakes and now we have to

00:05:11.840 --> 00:05:14.160
like like but come on,

00:05:14.160 --> 00:05:14.560
>> we can

00:05:14.560 --> 00:05:16.720
>> and I think DHH has a lot more to say

00:05:16.720 --> 00:05:18.400
about how Europe likes to look after

00:05:18.400 --> 00:05:21.280
people.

00:05:21.280 --> 00:05:24.639
>> Yeah, we we can't help it that uh

00:05:24.639 --> 00:05:26.160
h

00:05:26.160 --> 00:05:29.840
you want to hear something cool?

00:05:29.840 --> 00:05:33.039
>> Oh, open AI uh Astra.

00:05:33.039 --> 00:05:35.840
>> Astra. No, that's not cool.

00:05:35.840 --> 00:05:36.639
>> I was trying to work.

00:05:36.639 --> 00:05:37.600
>> I mean, it is cool, right?

00:05:37.600 --> 00:05:39.280
>> Is that Is that a new app? Is that a new

00:05:39.280 --> 00:05:41.039
model? I couldn't configure it. Didn't

00:05:41.039 --> 00:05:42.800
really understand.

00:05:42.800 --> 00:05:44.880
So this is the weird thing because when

00:05:44.880 --> 00:05:47.520
OpenAI announced Astra earlier or at

00:05:47.520 --> 00:05:49.120
least there was an announcement saying

00:05:49.120 --> 00:05:51.440
that they will start releasing Astra

00:05:51.440 --> 00:05:54.560
access to a select few organizations

00:05:54.560 --> 00:05:58.400
that Astra has proven to be very uh much

00:05:58.400 --> 00:06:01.199
better than than even you know uh what's

00:06:01.199 --> 00:06:05.440
it called mythos at security um

00:06:05.440 --> 00:06:08.080
hardening and able to to find bugs in

00:06:08.080 --> 00:06:09.840
software so that they are going to

00:06:09.840 --> 00:06:11.919
release it in in like batches.

00:06:11.919 --> 00:06:13.520
Um, I've read that like at the beginning

00:06:13.520 --> 00:06:15.919
of the week or something and then this

00:06:15.919 --> 00:06:17.840
morning you see announcements on Reddit

00:06:17.840 --> 00:06:19.680
like 1 hour till Astra and it's like

00:06:19.680 --> 00:06:21.360
what are they talking about? And then

00:06:21.360 --> 00:06:23.600
finally the announcement is there.

00:06:23.600 --> 00:06:25.039
>> Was down I noticed

00:06:25.039 --> 00:06:27.520
>> and then they're talking about Astra.

00:06:27.520 --> 00:06:30.400
Um, Astra is here. It's amazing. It's

00:06:30.400 --> 00:06:33.440
like terminal bench score is double that

00:06:33.440 --> 00:06:36.720
of Fable or Myth like Fable 5.1 and

00:06:36.720 --> 00:06:39.520
Opus. It's way better. terminal bench is

00:06:39.520 --> 00:06:42.560
um a physics uh

00:06:42.560 --> 00:06:44.800
>> I saw your LinkedIn post about

00:06:44.800 --> 00:06:46.880
benchmarks so I know where this is

00:06:46.880 --> 00:06:49.840
going. No, this is not no that's not I

00:06:49.840 --> 00:06:51.680
don't talk about that the OpenAI

00:06:51.680 --> 00:06:53.600
benchmark. I talk about other benchmark.

00:06:53.600 --> 00:06:56.720
I can I can say very I I have a bone to

00:06:56.720 --> 00:06:58.960
pick with a couple of people uh on on

00:06:58.960 --> 00:07:01.039
the benchmark and they keep hammering it

00:07:01.039 --> 00:07:04.560
on LinkedIn um and they keep repeating

00:07:04.560 --> 00:07:07.840
um you know something ridiculous which

00:07:07.840 --> 00:07:09.599
is the opposite of what I measure but

00:07:09.599 --> 00:07:11.759
okay. Uh no in this case OpenAI

00:07:11.759 --> 00:07:14.960
benchmark I it's true. I think Astra is

00:07:14.960 --> 00:07:17.520
amazing um based on on what I read about

00:07:17.520 --> 00:07:19.440
it, but it's limited to some companies

00:07:19.440 --> 00:07:21.280
and no like only some people have access

00:07:21.280 --> 00:07:23.039
to it. So why do I care?

00:07:23.039 --> 00:07:25.199
>> Exactly. I hate this I hate this uh what

00:07:25.199 --> 00:07:27.599
do you call it? Exclusivity [ __ ]

00:07:27.599 --> 00:07:29.759
>> I mean I mean I don't hate it. It's just

00:07:29.759 --> 00:07:32.319
that I wouldn't hype about it like it's

00:07:32.319 --> 00:07:34.319
not in my hands unless I get my hands on

00:07:34.319 --> 00:07:36.639
it. Sure. Okay. You can tell me it's

00:07:36.639 --> 00:07:37.840
coming but it's not there.

00:07:37.840 --> 00:07:39.680
>> Were seemed to be all about just

00:07:39.680 --> 00:07:42.080
chatting to it. I mean, and and it was

00:07:42.080 --> 00:07:45.280
amazingly fast, like sped up video, like

00:07:45.280 --> 00:07:47.199
draw a circle and instantly appears. I

00:07:47.199 --> 00:07:47.919
mean, in reality, that's

00:07:47.919 --> 00:07:49.919
>> Oh, it's like that. You remember

00:07:49.919 --> 00:07:51.759
>> you remember that last year when Google

00:07:51.759 --> 00:07:53.280
did that video where they talked to

00:07:53.280 --> 00:07:55.280
Gemini and Gemini instantly did the

00:07:55.280 --> 00:07:57.360
thing? It was all fake.

00:07:57.360 --> 00:07:59.120
>> It was all faked.

00:07:59.120 --> 00:08:01.599
>> Yeah, it's in that level, isn't it?

00:08:01.599 --> 00:08:03.599
Yeah. But anyway, when you want to talk

00:08:03.599 --> 00:08:05.599
So, there's We can talk five minutes

00:08:05.599 --> 00:08:08.479
about benchmarks. Um, no, I know

00:08:08.479 --> 00:08:11.039
benchmarks is crazy. Okay. I I need to

00:08:11.039 --> 00:08:12.400
>> No, I'll just say one thing. I'll just I

00:08:12.400 --> 00:08:13.680
have to bump pick. So, you're going to

00:08:13.680 --> 00:08:16.560
have You rent it to me. Now, I get to

00:08:16.560 --> 00:08:18.960
rent. You triggered it. You

00:08:18.960 --> 00:08:20.479
>> benchmarks are so boring.

00:08:20.479 --> 00:08:22.160
>> You can do anything with a benchmark.

00:08:22.160 --> 00:08:24.319
Like my my statistics professor used to

00:08:24.319 --> 00:08:27.199
say, according to statistics, every

00:08:27.199 --> 00:08:30.240
human on earth have one testicle. You

00:08:30.240 --> 00:08:34.320
know, on average, sorry, on average, uh

00:08:34.320 --> 00:08:36.479
the humans on Earth have one testicle

00:08:36.479 --> 00:08:38.959
according to statistics. Right. Um,

00:08:38.959 --> 00:08:40.640
sorry. I I totally botched that one.

00:08:40.640 --> 00:08:41.120
Anyway,

00:08:41.120 --> 00:08:44.080
>> on how I actually I'm struggling to

00:08:44.080 --> 00:08:45.839
understand how that's possible, but

00:08:45.839 --> 00:08:47.200
>> on average

00:08:47.200 --> 00:08:50.080
>> because the average is rounded down.

00:08:50.080 --> 00:08:54.160
>> No, one testicle because female and male

00:08:54.160 --> 00:08:55.839
50/50 about

00:08:55.839 --> 00:08:57.920
>> I I was thinking just men. Okay, that

00:08:57.920 --> 00:08:58.560
makes sense.

00:08:58.560 --> 00:09:00.800
>> No, I said every human on average have

00:09:00.800 --> 00:09:02.240
one testicle.

00:09:02.240 --> 00:09:06.160
>> Yeah. Yeah. I mean I the stats is always

00:09:06.160 --> 00:09:08.560
a thing. Like I think my wife triggered

00:09:08.560 --> 00:09:11.920
me yesterday saying like, "Oh,

00:09:11.920 --> 00:09:14.160
it's it's like when you drive a car,

00:09:14.160 --> 00:09:15.680
you're going to have like one in a

00:09:15.680 --> 00:09:17.600
thousand chance of getting killed or

00:09:17.600 --> 00:09:19.040
something like this per mile or

00:09:19.040 --> 00:09:22.160
something, whatever." And I said, "Yeah,

00:09:22.160 --> 00:09:24.399
but that's an average. Like we're in a

00:09:24.399 --> 00:09:26.800
we're in a really safe car, you know?

00:09:26.800 --> 00:09:30.560
We're not in it's like you can't really

00:09:30.560 --> 00:09:32.800
think that way." Uh anyway,

00:09:32.800 --> 00:09:33.279
>> yeah,

00:09:33.279 --> 00:09:34.959
>> I get I get what you're saying about

00:09:34.959 --> 00:09:36.640
benchmarks.

00:09:36.640 --> 00:09:40.240
I get what you're saying.

00:09:40.240 --> 00:09:42.959
>> The Okay, what I wanted to talk you to

00:09:42.959 --> 00:09:45.040
you about in all seriousness was

00:09:45.040 --> 00:09:47.200
security stuff again because I'm I'm in

00:09:47.200 --> 00:09:48.560
the security team

00:09:48.560 --> 00:09:50.560
>> and security

00:09:50.560 --> 00:09:53.760
>> and you you like me have probably

00:09:53.760 --> 00:09:56.480
wrangled with uh I mean infrastructure

00:09:56.480 --> 00:09:58.560
is more than just deploying stuff with

00:09:58.560 --> 00:10:01.920
Terraform. It's it's also about making

00:10:01.920 --> 00:10:05.279
sure that what we ship is secure and

00:10:05.279 --> 00:10:08.080
we're not and then we're not making our

00:10:08.080 --> 00:10:10.480
infrastructure v vulnerable to attack.

00:10:10.480 --> 00:10:13.519
And a big part of that has always been

00:10:13.519 --> 00:10:16.800
having doing CI/CD work where you maybe

00:10:16.800 --> 00:10:19.040
throw in um

00:10:19.040 --> 00:10:22.480
uh a SAS, you know, source source

00:10:22.480 --> 00:10:24.880
checker, throwing in uh some linting

00:10:24.880 --> 00:10:27.839
rules, whatever. Throwing in uh

00:10:27.839 --> 00:10:30.640
dependabot rules,

00:10:30.640 --> 00:10:33.839
maybe as going as so far as auto

00:10:33.839 --> 00:10:35.760
patching stuff. I don't know how how you

00:10:35.760 --> 00:10:37.360
feel about that stuff,

00:10:37.360 --> 00:10:39.120
>> but that's pretty advanced stuff. like

00:10:39.120 --> 00:10:41.279
Crowd Strike.

00:10:41.279 --> 00:10:43.120
>> I I I don't know what Crowd Strike has

00:10:43.120 --> 00:10:44.640
to offer there. Does it auto patch

00:10:44.640 --> 00:10:46.640
stuff?

00:10:46.640 --> 00:10:48.800
>> I think two years ago, all the major

00:10:48.800 --> 00:10:50.480
airports in the world were down because

00:10:50.480 --> 00:10:52.160
of a Crowd Strike release.

00:10:52.160 --> 00:10:53.519
>> Oh. Oh, yeah. You're right. You're

00:10:53.519 --> 00:10:53.760
right.

00:10:53.760 --> 00:10:56.399
>> An autodeployed patch.

00:10:56.399 --> 00:10:57.839
>> Years ago, that was when I was in

00:10:57.839 --> 00:11:00.000
Singapore, I think.

00:11:00.000 --> 00:11:02.160
>> Yeah. So the point I'm making here is

00:11:02.160 --> 00:11:05.519
that I am very strong uh advocate of

00:11:05.519 --> 00:11:08.399
immutable deploys and promotion of

00:11:08.399 --> 00:11:10.720
images across trust. So in terms of

00:11:10.720 --> 00:11:12.880
patching like when you say auto patching

00:11:12.880 --> 00:11:17.920
to me you need to patch a machine and

00:11:17.920 --> 00:11:20.959
roll it out like you don't have

00:11:20.959 --> 00:11:22.959
you need to test it and then once it

00:11:22.959 --> 00:11:25.120
passes your gates then it drops into

00:11:25.120 --> 00:11:27.120
>> anyway. So, but yeah, I mean you can you

00:11:27.120 --> 00:11:29.120
can you can do that uh but not live,

00:11:29.120 --> 00:11:31.839
right? Autopatching live. No. Um canal

00:11:31.839 --> 00:11:34.640
releases, immutable deploys, rollouts or

00:11:34.640 --> 00:11:35.200
patches,

00:11:35.200 --> 00:11:36.480
>> all that stuff. That's all part of the

00:11:36.480 --> 00:11:37.680
stuff that we've been doing for years,

00:11:37.680 --> 00:11:38.079
I'm sure.

00:11:38.079 --> 00:11:42.000
>> Okay. Yeah, but airports didn't do it.

00:11:42.000 --> 00:11:43.120
>> Now,

00:11:43.120 --> 00:11:46.160
>> or Crowd Strike didn't do it. The

00:11:46.160 --> 00:11:49.279
question was put to me was that oh we we

00:11:49.279 --> 00:11:52.880
have some sast we have some dast and we

00:11:52.880 --> 00:11:55.279
get a lot of alerts nonetheless about

00:11:55.279 --> 00:11:57.120
certain vulnerabilities and our security

00:11:57.120 --> 00:11:59.440
team is overloaded with these sort of

00:11:59.440 --> 00:12:02.480
this sort of toil like keeping these

00:12:02.480 --> 00:12:05.920
things patched chasing down teams to

00:12:05.920 --> 00:12:08.720
make sure the things is all done.

00:12:08.720 --> 00:12:10.079
>> Yeah.

00:12:10.079 --> 00:12:12.800
>> Kai tell me tell me that AI agents can

00:12:12.800 --> 00:12:15.360
do this and do it better. And I was

00:12:15.360 --> 00:12:17.519
like, "Yeah, that that sounds like it

00:12:17.519 --> 00:12:22.160
has potential for AI agents to spawn and

00:12:22.160 --> 00:12:24.160
do things better."

00:12:24.160 --> 00:12:27.279
>> Um, and actually it made me even think

00:12:27.279 --> 00:12:29.200
to myself like there must be a point

00:12:29.200 --> 00:12:33.360
where where deterministic sast and

00:12:33.360 --> 00:12:36.800
dasted or whatever tools are probably

00:12:36.800 --> 00:12:38.639
going to be eclipsed by AI. Maybe not

00:12:38.639 --> 00:12:40.720
today. I don't know if you have any

00:12:40.720 --> 00:12:44.399
feelings about that but surely like

00:12:44.399 --> 00:12:46.399
you know because because sometimes we

00:12:46.399 --> 00:12:49.200
have like really archaic you know sass

00:12:49.200 --> 00:12:50.720
[ __ ]

00:12:50.720 --> 00:12:52.880
to be honest I think you need

00:12:52.880 --> 00:12:55.440
determinist I think AI can be a part of

00:12:55.440 --> 00:12:58.079
generating the rule but I don't see the

00:12:58.079 --> 00:13:00.880
point of spending tokens to apply rules

00:13:00.880 --> 00:13:03.760
AI can help evaluate and deploy and

00:13:03.760 --> 00:13:05.279
propagate deterministic rules

00:13:05.279 --> 00:13:07.440
deterministic rules must be there and

00:13:07.440 --> 00:13:09.600
they can you can you can have a fast and

00:13:09.600 --> 00:13:11.279
a CVE database and you can do an

00:13:11.279 --> 00:13:13.839
evaluation across your fleet and and and

00:13:13.839 --> 00:13:16.240
and determine the impact which today

00:13:16.240 --> 00:13:19.040
might you know take time to accept the

00:13:19.040 --> 00:13:23.360
risk on um on on on some findings and

00:13:23.360 --> 00:13:25.279
set a timeline and I think that's where

00:13:25.279 --> 00:13:27.839
AI can can help you um you know make

00:13:27.839 --> 00:13:29.120
that decision.

00:13:29.120 --> 00:13:31.600
>> I'm basically I'm not saying that um AI

00:13:31.600 --> 00:13:32.880
is going to replace the SAS the

00:13:32.880 --> 00:13:34.959
deterministic procedures. I'm just I'm

00:13:34.959 --> 00:13:35.600
just thinking

00:13:35.600 --> 00:13:39.040
>> literally what you said. Okay, whatever.

00:13:39.040 --> 00:13:41.279
I'm just trying to say that like AI is

00:13:41.279 --> 00:13:43.839
going to surely eclipse them in in their

00:13:43.839 --> 00:13:46.720
potency and effectiveness as a security

00:13:46.720 --> 00:13:47.279
tool.

00:13:47.279 --> 00:13:49.279
>> I mean, they're going to amplify it. And

00:13:49.279 --> 00:13:51.360
maybe that ties into again AI is an

00:13:51.360 --> 00:13:53.920
amplifier. Um AI is amplifying good

00:13:53.920 --> 00:13:57.360
practices and that includes you're not

00:13:57.360 --> 00:13:59.519
going to get better. like if you have a

00:13:59.519 --> 00:14:01.120
complete manual system where humans are

00:14:01.120 --> 00:14:03.920
making are the judgment um factor and

00:14:03.920 --> 00:14:06.880
you're slow down um to to evaluate

00:14:06.880 --> 00:14:08.720
everything. I think everywhere that's

00:14:08.720 --> 00:14:11.279
the case just replacing a human with an

00:14:11.279 --> 00:14:15.839
AI agent is maybe going to give you some

00:14:15.839 --> 00:14:18.000
speed but it's also going to make you

00:14:18.000 --> 00:14:21.040
very like scared about trusting the

00:14:21.040 --> 00:14:22.560
agent is going to make the right call.

00:14:22.560 --> 00:14:24.639
This probabilistic thing um you know

00:14:24.639 --> 00:14:28.240
makes mistakes like humans. Um, so you

00:14:28.240 --> 00:14:30.079
are not going to speed up unless you

00:14:30.079 --> 00:14:32.399
have automated gates and automation in

00:14:32.399 --> 00:14:35.760
place and that ties directly into like

00:14:35.760 --> 00:14:38.959
recent presentations by uh Uber about

00:14:38.959 --> 00:14:41.360
the AISDLC that you also talked about

00:14:41.360 --> 00:14:44.800
before. Um, but another person that did

00:14:44.800 --> 00:14:47.360
a very interesting talk was um I forgot

00:14:47.360 --> 00:14:49.839
the name, Claire. She is part of the

00:14:49.839 --> 00:14:52.240
Kiro team at AWS and that's why I

00:14:52.240 --> 00:14:54.079
follow her on LinkedIn. She also has an

00:14:54.079 --> 00:14:57.519
AI engineering talk. um and she talks

00:14:57.519 --> 00:15:01.760
about how AWS evolved individual teams

00:15:01.760 --> 00:15:04.959
um adopting AI and basically the metrics

00:15:04.959 --> 00:15:07.120
and measurements that they found out

00:15:07.120 --> 00:15:10.079
matter like it's not about

00:15:10.079 --> 00:15:12.160
>> it's a really it's a really good talk

00:15:12.160 --> 00:15:14.560
because she talks about like this team

00:15:14.560 --> 00:15:17.600
adopted AI this is where they were um

00:15:17.600 --> 00:15:19.839
this is how how much it accelerated them

00:15:19.839 --> 00:15:22.160
and this is why they were not a

00:15:22.160 --> 00:15:25.519
reasonable um metric for

00:15:25.519 --> 00:15:27.760
because it was an isolated team of

00:15:27.760 --> 00:15:30.320
highly experienced people with a very

00:15:30.320 --> 00:15:32.480
green field project. So they could not

00:15:32.480 --> 00:15:33.920
use those results and then they have

00:15:33.920 --> 00:15:35.600
like two or three pilot projects like

00:15:35.600 --> 00:15:37.839
that and then each time she extracts the

00:15:37.839 --> 00:15:39.760
information that they measured and then

00:15:39.760 --> 00:15:43.279
she identified the real like um rules

00:15:43.279 --> 00:15:46.160
that you must follow um to adopt or to

00:15:46.160 --> 00:15:49.279
to become faster with AI which is like

00:15:49.279 --> 00:15:51.199
you have to go slow before you go fast.

00:15:51.199 --> 00:15:52.639
You have to put in place the right

00:15:52.639 --> 00:15:54.800
automation and guardrails. Uber talks

00:15:54.800 --> 00:15:57.120
about refactoring their codebase into

00:15:57.120 --> 00:15:59.920
monor repos, adopting basil, setting in

00:15:59.920 --> 00:16:01.440
place all of the things that makes an

00:16:01.440 --> 00:16:03.600
agent efficient. Executing guard rails

00:16:03.600 --> 00:16:05.040
is an important one.

00:16:05.040 --> 00:16:07.920
>> So, and and you know what? Um, so in

00:16:07.920 --> 00:16:10.160
terms of AISDL,

00:16:10.160 --> 00:16:11.440
>> that's exactly what I'm trying to do

00:16:11.440 --> 00:16:13.040
right now at work, right? I'm trying to

00:16:13.040 --> 00:16:15.040
really put in place like get rid of all

00:16:15.040 --> 00:16:17.040
of these manual human, you know, control

00:16:17.040 --> 00:16:18.800
points because putting agents in place

00:16:18.800 --> 00:16:20.720
is not going to solve it. You have to

00:16:20.720 --> 00:16:22.880
refactor and rethink about how you do

00:16:22.880 --> 00:16:24.079
the AIS DLC.

00:16:24.079 --> 00:16:24.560
>> Yeah, exactly.

00:16:24.560 --> 00:16:26.639
>> That fits the agent workflow. I'm going

00:16:26.639 --> 00:16:28.320
to find the other things that she

00:16:28.320 --> 00:16:29.600
brought up.

00:16:29.600 --> 00:16:30.320
>> Where is

00:16:30.320 --> 00:16:32.399
>> How did that Martinelli guy factor in?

00:16:32.399 --> 00:16:36.320
But did she is he a consultant to AWS or

00:16:36.320 --> 00:16:39.839
something? Um so there was a Kiro IDE

00:16:39.839 --> 00:16:43.519
workshop that I participated in back in

00:16:43.519 --> 00:16:46.079
uh January or something like that and

00:16:46.079 --> 00:16:49.199
they then he talked about AI up which is

00:16:49.199 --> 00:16:51.199
his consultancy where he talks about the

00:16:51.199 --> 00:16:53.040
coordination like he's he's mainly

00:16:53.040 --> 00:16:55.199
focused on like a lot of these SDDD like

00:16:55.199 --> 00:16:57.199
specdriven development and and keto IDE

00:16:57.199 --> 00:16:59.360
included is all about single player like

00:16:59.360 --> 00:17:01.120
as if the developer or the engineer

00:17:01.120 --> 00:17:03.199
alone is making those decisions and a

00:17:03.199 --> 00:17:05.520
lot of the actual um requirement are

00:17:05.520 --> 00:17:06.880
about finding the stakeholders ers,

00:17:06.880 --> 00:17:08.160
putting them in the room together to

00:17:08.160 --> 00:17:09.679
actually make the decision. Like the

00:17:09.679 --> 00:17:11.280
expertise lives across multiple people

00:17:11.280 --> 00:17:12.959
that all need to be coordinated. I think

00:17:12.959 --> 00:17:14.400
you talked about that before as well,

00:17:14.400 --> 00:17:18.559
right?

00:17:18.559 --> 00:17:22.160
Hello. Oh, yeah. I I

00:17:22.160 --> 00:17:25.120
I got a Oh, I I actually got permission

00:17:25.120 --> 00:17:28.000
to set up a new Adeps account with so

00:17:28.000 --> 00:17:30.000
that I could get better versed with

00:17:30.000 --> 00:17:32.559
Bedrock and I got a bit of a budget to

00:17:32.559 --> 00:17:34.000
spend. So,

00:17:34.000 --> 00:17:35.919
>> Bedrock or Agent Core? What do you what

00:17:35.919 --> 00:17:36.559
do you mean?

00:17:36.559 --> 00:17:39.600
>> Uh I think I mean I've got I don't know

00:17:39.600 --> 00:17:41.360
100 bucks a month or whatever to spend

00:17:41.360 --> 00:17:42.880
on on AWS.

00:17:42.880 --> 00:17:44.160
>> Better. Okay.

00:17:44.160 --> 00:17:46.000
>> Here I found I found the thread.

00:17:46.000 --> 00:17:48.000
>> So that means I can start using Kro and

00:17:48.000 --> 00:17:49.840
Ango and all that sort of stuff I

00:17:49.840 --> 00:17:50.400
suppose. Right.

00:17:50.400 --> 00:17:53.280
>> So the name I can send you to YouTube

00:17:53.280 --> 00:17:56.640
and we can share it. The name is um

00:17:56.640 --> 00:17:58.880
Claire Legori at AWS.

00:17:58.880 --> 00:18:01.679
>> Oh, I know that lady. I mean I don't

00:18:01.679 --> 00:18:04.960
I've seen her stuff online. So she talks

00:18:04.960 --> 00:18:07.200
about the different

00:18:07.200 --> 00:18:09.440
>> uh pilots that they did and what

00:18:09.440 --> 00:18:11.200
measurements they got but why they were

00:18:11.200 --> 00:18:14.320
not not um not right and then what it

00:18:14.320 --> 00:18:15.919
really is like the things that they

00:18:15.919 --> 00:18:16.640
want.

00:18:16.640 --> 00:18:18.799
>> That's cool. That's cool.

00:18:18.799 --> 00:18:20.640
I mean a big one of course is making

00:18:20.640 --> 00:18:22.000
sure that the agents get the right

00:18:22.000 --> 00:18:24.799
context which is ontology knowledge

00:18:24.799 --> 00:18:28.160
graphs knowledge systems

00:18:28.160 --> 00:18:30.080
like one problem I have at work that

00:18:30.080 --> 00:18:32.320
really frustrated me that I feel like we

00:18:32.320 --> 00:18:35.440
need to measure is that everything needs

00:18:35.440 --> 00:18:37.520
okay I'm in a company right now where

00:18:37.520 --> 00:18:39.760
everything has to be approved okay I can

00:18:39.760 --> 00:18:42.480
I can understand that I am [ __ ]

00:18:42.480 --> 00:18:44.559
improving PRs left right and center

00:18:44.559 --> 00:18:47.440
awesome but then we don't have automerge

00:18:47.440 --> 00:18:49.840
enabled So, it's up to uh other people

00:18:49.840 --> 00:18:52.320
to to hit the the merge button, right? I

00:18:52.320 --> 00:18:52.480
guess

00:18:52.480 --> 00:18:53.600
>> you're on GitHub.

00:18:53.600 --> 00:18:55.280
>> Yeah. And then, of course, there's some

00:18:55.280 --> 00:18:57.360
there's some ambiguity there, like who's

00:18:57.360 --> 00:18:59.200
going to hit the merge button? Like,

00:18:59.200 --> 00:19:00.880
dude, it's your patch.

00:19:00.880 --> 00:19:02.160
>> You're supposed to hit it. Yeah. The

00:19:02.160 --> 00:19:03.600
moment you got your approvals

00:19:03.600 --> 00:19:05.760
>> and like there's people that are

00:19:05.760 --> 00:19:11.039
diddling about that and and like

00:19:11.039 --> 00:19:14.480
those sort of things I want to AI to

00:19:14.480 --> 00:19:16.240
chase for me. Like go and

00:19:16.240 --> 00:19:17.600
>> Yeah. That's exactly the thing that the

00:19:17.600 --> 00:19:19.120
PC that I built.

00:19:19.120 --> 00:19:21.679
>> So the PAC I built is a nudging system.

00:19:21.679 --> 00:19:24.080
So it's um it's a durable workflow that

00:19:24.080 --> 00:19:26.480
helps you carry it was an idea from from

00:19:26.480 --> 00:19:29.120
the team that I'm in um to carry your PR

00:19:29.120 --> 00:19:30.559
because they have so many manual

00:19:30.559 --> 00:19:32.960
approvals. They need two approvals. One

00:19:32.960 --> 00:19:34.880
needs to be from a default reviewer.

00:19:34.880 --> 00:19:38.080
Then there's the whole uh PR branch uh

00:19:38.080 --> 00:19:41.360
validation that takes um you know maybe

00:19:41.360 --> 00:19:44.799
20 minutes to run. And so so you you you

00:19:44.799 --> 00:19:46.880
cannot and every commit you push u

00:19:46.880 --> 00:19:49.440
invalidates of course the approvals. So

00:19:49.440 --> 00:19:53.200
you you can't like chase for approvals.

00:19:53.200 --> 00:19:55.039
>> There is an option in well there is an

00:19:55.039 --> 00:19:57.280
option in GitHub that doesn't remove the

00:19:57.280 --> 00:19:59.039
the approval even if you change it

00:19:59.039 --> 00:19:59.840
subsequently.

00:19:59.840 --> 00:20:01.600
>> Yeah. But like that's that that that's

00:20:01.600 --> 00:20:04.799
not allowed in in this in a in a

00:20:04.799 --> 00:20:06.240
>> Yeah. Well, I can't think a couple of

00:20:06.240 --> 00:20:07.919
things in GitHub could just make a lot

00:20:07.919 --> 00:20:09.600
of problems go away like

00:20:09.600 --> 00:20:11.360
>> Bitbucket.

00:20:11.360 --> 00:20:12.720
>> Oh [ __ ] Oh god.

00:20:12.720 --> 00:20:14.880
>> Yeah. Anyway, um

00:20:14.880 --> 00:20:17.760
>> what I was wanted to say, oh yeah, so we

00:20:17.760 --> 00:20:19.280
have the nudge like one thing that I

00:20:19.280 --> 00:20:22.480
built is a nudge uh system with like

00:20:22.480 --> 00:20:25.440
basically it's um it's a step functions

00:20:25.440 --> 00:20:27.280
workflow, so it's durable and it wakes

00:20:27.280 --> 00:20:30.320
up the you know if it hasn't received

00:20:30.320 --> 00:20:33.039
the approvals. Um it's there's no tokens

00:20:33.039 --> 00:20:34.720
involved in there, but there's like a

00:20:34.720 --> 00:20:39.120
timer and it it uh it reminds people um

00:20:39.120 --> 00:20:40.799
on on an interval and everything's

00:20:40.799 --> 00:20:42.480
configurable. So that that's kind of

00:20:42.480 --> 00:20:44.159
like one of the cool things that that I

00:20:44.159 --> 00:20:46.720
built which is like around the agents um

00:20:46.720 --> 00:20:48.400
a whole workflow system like what people

00:20:48.400 --> 00:20:51.039
use N8 and for and temporal but we want

00:20:51.039 --> 00:20:53.280
to be without having to deploy another

00:20:53.280 --> 00:20:54.720
workflow system. Yeah.

00:20:54.720 --> 00:20:56.559
>> Yeah. And does it what's the medium?

00:20:56.559 --> 00:20:58.720
Slack I guess or something.

00:20:58.720 --> 00:21:01.600
>> Yeah. So so the nut system I built is is

00:21:01.600 --> 00:21:06.400
Slack and um you know uh yeah it's it's

00:21:06.400 --> 00:21:09.360
Slack and reaction and and interactive

00:21:09.360 --> 00:21:11.440
cards and and all of that. That was that

00:21:11.440 --> 00:21:12.880
was that was all really fun. I built

00:21:12.880 --> 00:21:14.960
that like in in a month and uh

00:21:14.960 --> 00:21:16.799
>> a month

00:21:16.799 --> 00:21:19.760
you going to say like an hour.

00:21:19.760 --> 00:21:24.240
>> You have no idea.

00:21:24.240 --> 00:21:26.559
>> You have no idea if you are in aization

00:21:26.559 --> 00:21:28.320
where you can't even get a web hook set

00:21:28.320 --> 00:21:30.400
up. Now the Slack app I have to admit

00:21:30.400 --> 00:21:32.559
was very fast but to get a bit bucket

00:21:32.559 --> 00:21:35.120
web hook that is complicated.

00:21:35.120 --> 00:21:37.600
>> Okay. And then you have to do you have

00:21:37.600 --> 00:21:40.799
to scope it. you have to avoid uh you

00:21:40.799 --> 00:21:42.320
have to make sure it goes to the right

00:21:42.320 --> 00:21:44.640
channels um that is allowed and

00:21:44.640 --> 00:21:46.640
associated with the team. Uh so there's

00:21:46.640 --> 00:21:47.919
a whole bunch of additional stuff but

00:21:47.919 --> 00:21:49.520
that's not even the parts that I that I

00:21:49.520 --> 00:21:51.760
was um I was I was stuck on.

00:21:51.760 --> 00:21:54.000
>> Yeah, sure you can build a something

00:21:54.000 --> 00:21:56.000
real stupid on a on your own slack with

00:21:56.000 --> 00:21:58.400
your own GitHub in in in five minutes

00:21:58.400 --> 00:22:04.240
Kai. But to do it in an organization

00:22:04.240 --> 00:22:06.240
>> okay listen I need I need a break. I'm

00:22:06.240 --> 00:22:07.919
going to run away. There's one more

00:22:07.919 --> 00:22:11.280
thing about CI because you said like um

00:22:11.280 --> 00:22:13.520
aentic CI and and I'm been breaking my

00:22:13.520 --> 00:22:16.320
head about the uh swamp guys there

00:22:16.320 --> 00:22:19.120
saying that you know why are we doing

00:22:19.120 --> 00:22:22.320
why are we running um validations in

00:22:22.320 --> 00:22:25.840
CI/CD that the agents are running they

00:22:25.840 --> 00:22:28.000
they they must run all of these checks

00:22:28.000 --> 00:22:31.520
right this is the feedback loop and they

00:22:31.520 --> 00:22:34.799
said you don't need complicated CI/CD

00:22:34.799 --> 00:22:37.280
systems anymore as long as Whatever the

00:22:37.280 --> 00:22:40.320
agent produces can be signed and like

00:22:40.320 --> 00:22:43.280
has an attest um how you say

00:22:43.280 --> 00:22:46.880
attestification and basically it

00:22:46.880 --> 00:22:49.760
>> yeah it must produce an artifact that

00:22:49.760 --> 00:22:52.000
you can't get unless you've run the test

00:22:52.000 --> 00:22:54.159
suite and then you've you know you

00:22:54.159 --> 00:22:55.919
signed it and then you push that with

00:22:55.919 --> 00:22:57.760
your with your code change and so the

00:22:57.760 --> 00:22:59.919
CI/CD system doesn't run the test it

00:22:59.919 --> 00:23:02.799
just checks that the um attestation is

00:23:02.799 --> 00:23:04.960
there yes the test has been run and and

00:23:04.960 --> 00:23:06.960
and basically that's the gate So it

00:23:06.960 --> 00:23:10.080
doesn't that that completely shifts the

00:23:10.080 --> 00:23:12.640
um CIC CI/CD system to be because

00:23:12.640 --> 00:23:14.400
they're talking about how GitHub is

00:23:14.400 --> 00:23:16.159
dying right with all of these PRs being

00:23:16.159 --> 00:23:17.919
created overloading all of these

00:23:17.919 --> 00:23:20.880
addition.

00:23:20.880 --> 00:23:25.200
>> Yeah. Like I can't help but think CI/CD

00:23:25.200 --> 00:23:27.760
needs to be reinvented because because

00:23:27.760 --> 00:23:29.679
yeah, we we're getting lots of changes

00:23:29.679 --> 00:23:33.679
in and uh and running like one hour test

00:23:33.679 --> 00:23:35.520
suite

00:23:35.520 --> 00:23:38.640
for each change is [ __ ] crazy, right?

00:23:38.640 --> 00:23:40.240
Surely agents can do a better job of

00:23:40.240 --> 00:23:43.440
that. I have I have my reservations

00:23:43.440 --> 00:23:46.000
because so I wanted to post a comment

00:23:46.000 --> 00:23:48.159
and I didn't read his full blog post so

00:23:48.159 --> 00:23:49.840
maybe he addressed it so I I didn't say

00:23:49.840 --> 00:23:51.120
the comment. Maybe I should read the

00:23:51.120 --> 00:23:56.480
post. But for me, a CI/CD system

00:23:56.480 --> 00:23:59.679
is proof that whatever you have on your

00:23:59.679 --> 00:24:01.360
machine works on

00:24:01.360 --> 00:24:02.080
>> exactly that

00:24:02.080 --> 00:24:02.799
>> another machine.

00:24:02.799 --> 00:24:04.240
>> That's a good way of putting it like

00:24:04.240 --> 00:24:06.240
it's not not my machine problem or

00:24:06.240 --> 00:24:09.039
you've said it before works. It's not a

00:24:09.039 --> 00:24:10.640
works on my machine problem anymore.

00:24:10.640 --> 00:24:13.760
>> Yeah. If I if I I have set up a repo and

00:24:13.760 --> 00:24:17.120
my CI/CD is green and that means I if

00:24:17.120 --> 00:24:18.400
somebody comes to me and says it doesn't

00:24:18.400 --> 00:24:20.960
work on my machine I said well there

00:24:20.960 --> 00:24:23.200
right there is a complete automated

00:24:23.200 --> 00:24:26.080
setup of a a complete fresh machine that

00:24:26.080 --> 00:24:28.320
is provided by GitHub all of the steps

00:24:28.320 --> 00:24:29.679
are there I mean of course you could

00:24:29.679 --> 00:24:31.760
build your self-hosted runner with all

00:24:31.760 --> 00:24:33.600
of your in software installed but even

00:24:33.600 --> 00:24:35.279
then you will have a definition of that

00:24:35.279 --> 00:24:37.919
runner with all the software it requires

00:24:37.919 --> 00:24:40.240
so if you're not able to like do that

00:24:40.240 --> 00:24:42.240
there's There's an automated setup

00:24:42.240 --> 00:24:44.960
script and a fully automated test suite

00:24:44.960 --> 00:24:47.440
that validates that it works not just on

00:24:47.440 --> 00:24:49.600
my machine but also on that machine and

00:24:49.600 --> 00:24:52.480
and and if you're using agents and

00:24:52.480 --> 00:24:53.840
they're running all the tests and

00:24:53.840 --> 00:24:55.760
they're signing an attestation that they

00:24:55.760 --> 00:24:57.840
have done so and then submit the patch

00:24:57.840 --> 00:24:59.679
and then the CI/CD system doesn't

00:24:59.679 --> 00:25:02.880
validate um the actual test run but just

00:25:02.880 --> 00:25:04.960
validates that you know yes the

00:25:04.960 --> 00:25:08.720
attestation is val valid and um

00:25:08.720 --> 00:25:10.960
therefore the the tests uh the the patch

00:25:10.960 --> 00:25:13.840
can be merged. Um

00:25:13.840 --> 00:25:16.080
it makes me wonder.

00:25:16.080 --> 00:25:18.400
So I I don't quite understand this. So

00:25:18.400 --> 00:25:22.480
is it is it is Claude making a claim

00:25:22.480 --> 00:25:26.000
that the test ran in its own environment

00:25:26.000 --> 00:25:28.720
and then and then there's a test that

00:25:28.720 --> 00:25:31.039
and then this attestation means that it

00:25:31.039 --> 00:25:33.679
doesn't need to run.

00:25:33.679 --> 00:25:35.679
>> Yeah, I I haven't read the full blog

00:25:35.679 --> 00:25:37.440
post. So the way I understand it is

00:25:37.440 --> 00:25:40.480
>> double guessing what it says. the I I

00:25:40.480 --> 00:25:42.799
think I I've read the summaries and I've

00:25:42.799 --> 00:25:44.960
I think started reading it, but um I

00:25:44.960 --> 00:25:46.720
just, you know, my my head was just

00:25:46.720 --> 00:25:49.360
racing on some of the uh thoughts and I

00:25:49.360 --> 00:25:52.000
just in my head the way it works, maybe

00:25:52.000 --> 00:25:54.240
I'm wrong, uh maybe he addresses all of

00:25:54.240 --> 00:25:55.840
my concerns and maybe I should go back.

00:25:55.840 --> 00:25:58.320
So Paul's tech from um from Swamp.

00:25:58.320 --> 00:25:58.640
>> Okay.

00:25:58.640 --> 00:26:02.240
>> Um says that the way I understand it,

00:26:02.240 --> 00:26:04.799
you have a what they also do with swamp

00:26:04.799 --> 00:26:06.480
is they have a they built something

00:26:06.480 --> 00:26:08.480
weird that they said is the future of

00:26:08.480 --> 00:26:09.760
how you build software. But to be

00:26:09.760 --> 00:26:11.440
honest, Claude Code doesn't need any of

00:26:11.440 --> 00:26:13.520
that. What what they build you you you

00:26:13.520 --> 00:26:15.760
build your software um suite with with

00:26:15.760 --> 00:26:17.039
your unit test. And I mean obviously

00:26:17.039 --> 00:26:18.320
when you run unit test you have a

00:26:18.320 --> 00:26:20.320
coverage report, you have all kinds of

00:26:20.320 --> 00:26:22.159
additional reports that that gets

00:26:22.159 --> 00:26:24.720
generated, right? And if you then sign

00:26:24.720 --> 00:26:27.360
it um

00:26:27.360 --> 00:26:29.039
and make sure that there's no way for

00:26:29.039 --> 00:26:31.919
the agent to mock it and fake it, maybe

00:26:31.919 --> 00:26:33.840
maybe it calls out to a separate service

00:26:33.840 --> 00:26:37.440
to I don't know how but it's certified.

00:26:37.440 --> 00:26:39.600
Let's let's assume that there is a

00:26:39.600 --> 00:26:42.480
system that the agent cannot game that

00:26:42.480 --> 00:26:44.320
ensures that if this if the agent has

00:26:44.320 --> 00:26:44.720
run

00:26:44.720 --> 00:26:46.640
>> the certification means that it's done

00:26:46.640 --> 00:26:50.720
some due diligence before. Yes, it it

00:26:50.720 --> 00:26:52.880
it's he has ran he has tested his code

00:26:52.880 --> 00:26:54.559
and in the comments people were going

00:26:54.559 --> 00:26:56.799
well wasn't like in our company we used

00:26:56.799 --> 00:26:58.799
to have this social rule that you

00:26:58.799 --> 00:27:01.279
shouldn't be wasting CI minutes the rule

00:27:01.279 --> 00:27:02.960
was that you need to test your code and

00:27:02.960 --> 00:27:05.360
if you submit a PR um yeah

00:27:05.360 --> 00:27:06.880
>> I get that I get that

00:27:06.880 --> 00:27:09.360
>> because CI minutes were expensive so we

00:27:09.360 --> 00:27:11.600
would not um you know we would we would

00:27:11.600 --> 00:27:13.760
uh actually as a social rule within our

00:27:13.760 --> 00:27:16.240
company 30 30 engineers committing code

00:27:16.240 --> 00:27:19.360
um we would expect people to sell like

00:27:19.360 --> 00:27:22.720
you know honors uh you know say that

00:27:22.720 --> 00:27:24.400
like I've I've built something and I've

00:27:24.400 --> 00:27:25.600
ran all the tests.

00:27:25.600 --> 00:27:26.080
>> Exactly.

00:27:26.080 --> 00:27:26.799
>> Um

00:27:26.799 --> 00:27:32.159
>> yeah so CI/CD systems today are

00:27:32.159 --> 00:27:33.679
sometimes

00:27:33.679 --> 00:27:35.919
not really possible to run locally right

00:27:35.919 --> 00:27:38.320
and and and sometimes

00:27:38.320 --> 00:27:40.960
GitHub is a great example. Yeah, because

00:27:40.960 --> 00:27:42.720
of the way that the GitHub actions YAML

00:27:42.720 --> 00:27:44.559
work and all that it and and then you

00:27:44.559 --> 00:27:46.000
have Dagger and all the solutions that

00:27:46.000 --> 00:27:47.279
are trying to make it so that you can

00:27:47.279 --> 00:27:49.679
run it anywhere no matter your machine.

00:27:49.679 --> 00:27:50.240
>> Yeah.

00:27:50.240 --> 00:27:52.720
>> Uh but but but

00:27:52.720 --> 00:27:55.039
another thing that I feel like like when

00:27:55.039 --> 00:27:58.159
I when I use cloud I have Mi and Mi

00:27:58.159 --> 00:28:00.240
allows me to run like for example I

00:28:00.240 --> 00:28:02.240
maintain CDK terrain and we support

00:28:02.240 --> 00:28:05.440
Terraform 1.7 all the way up to 1.16.

00:28:05.440 --> 00:28:07.279
>> Are you using Mi? I only just learned

00:28:07.279 --> 00:28:09.360
about Mi the other day.

00:28:09.360 --> 00:28:12.000
all your your CLIs and whatever.

00:28:12.000 --> 00:28:14.080
>> It's really useful. It's it's like when

00:28:14.080 --> 00:28:17.200
you used to use uh TFN like terraform or

00:28:17.200 --> 00:28:19.360
node version manager, you have so many

00:28:19.360 --> 00:28:21.520
like for every single tool that you use

00:28:21.520 --> 00:28:24.159
and for Python you then use uh VN

00:28:24.159 --> 00:28:25.360
virtual amps.

00:28:25.360 --> 00:28:28.880
>> Exactly. Exactly. Now this one

00:28:28.880 --> 00:28:30.399
>> there was a real gap. There was a real

00:28:30.399 --> 00:28:31.120
gap.

00:28:31.120 --> 00:28:33.679
>> Yeah. Mises cross everything. So it's a

00:28:33.679 --> 00:28:35.520
bit like I I mean people that use Nyx

00:28:35.520 --> 00:28:37.440
say like yeah but I heard that with Nyx.

00:28:37.440 --> 00:28:39.679
What are you talking about? I had it

00:28:39.679 --> 00:28:43.360
with next 20 years ago.

00:28:43.360 --> 00:28:46.399
And then I I I I provide a workshop with

00:28:46.399 --> 00:28:48.159
uh with me and I say this is what meat

00:28:48.159 --> 00:28:52.080
does. And then somebody [ __ ] they don't

00:28:52.080 --> 00:28:53.600
they're never going to watch this. They

00:28:53.600 --> 00:28:56.320
said like oh let me put this into Nick.

00:28:56.320 --> 00:28:58.399
And you know what happened for a 4hour

00:28:58.399 --> 00:29:00.880
workshop. He didn't do any exercise. He

00:29:00.880 --> 00:29:03.440
spent four hours trying to to to set up

00:29:03.440 --> 00:29:06.159
the sample repo into Nick. And initially

00:29:06.159 --> 00:29:07.760
I entertained him, but when it was like

00:29:07.760 --> 00:29:09.440
half the workshop done, I was like,

00:29:09.440 --> 00:29:11.200
"Yeah, you kind of missed the workshop

00:29:11.200 --> 00:29:12.399
though.

00:29:12.399 --> 00:29:16.480
>> Stop. Please stop.

00:29:16.480 --> 00:29:18.159
>> Shit."

00:29:18.159 --> 00:29:21.039
>> Anyway, Mi is awesome. Mi allows you to

00:29:21.039 --> 00:29:23.120
to run your test matrix.

00:29:23.120 --> 00:29:24.880
>> Yeah, I got you. Oh, yeah. So, you got

00:29:24.880 --> 00:29:26.080
your test matrix

00:29:26.080 --> 00:29:27.919
>> and it's across versions, which I don't

00:29:27.919 --> 00:29:31.200
think you usually can. See, Mi probably

00:29:31.200 --> 00:29:35.200
solves the crazy GitHub actions YAML,

00:29:35.200 --> 00:29:36.880
doesn't it? Because everything's set up.

00:29:36.880 --> 00:29:38.559
Mis is a way of setting up your

00:29:38.559 --> 00:29:40.799
developer environment, right? So, I had

00:29:40.799 --> 00:29:44.640
this PR uh for CDK terrain that fixed um

00:29:44.640 --> 00:29:46.320
some

00:29:46.320 --> 00:29:48.080
some change of behavior that was

00:29:48.080 --> 00:29:50.640
introduced with Terraform version 1.15.

00:29:50.640 --> 00:29:54.320
So, on 1.14 it worked. on 1.15 it

00:29:54.320 --> 00:29:55.520
stopped working because they changed

00:29:55.520 --> 00:29:58.000
some behavior and somebody submitted a

00:29:58.000 --> 00:29:59.360
pull request based off of an issue

00:29:59.360 --> 00:30:04.399
report. Um and then the Hermes bot ident

00:30:04.399 --> 00:30:09.120
read the code and said that it fixes A

00:30:09.120 --> 00:30:12.399
but it doesn't fix B. And I was like is

00:30:12.399 --> 00:30:15.440
B new and and then and then it says no B

00:30:15.440 --> 00:30:17.919
B also worked the same way with the old

00:30:17.919 --> 00:30:20.480
behavior. Um and I said okay can you do

00:30:20.480 --> 00:30:24.000
a full test suite meaning you run

00:30:24.000 --> 00:30:28.159
version 1.14 A B validate that both of

00:30:28.159 --> 00:30:30.000
them work then

00:30:30.000 --> 00:30:35.600
>> run terapform 1.15 on the PR head um

00:30:35.600 --> 00:30:39.520
confirm that like across across master

00:30:39.520 --> 00:30:41.120
like latest release didn't have to be

00:30:41.120 --> 00:30:43.919
master uh latest stable release versus

00:30:43.919 --> 00:30:47.760
1.14 1.15 and then PR head versus 1.15

00:30:47.760 --> 00:30:48.640
1.14

00:30:48.640 --> 00:30:51.840
plus A versus B. Um, and and it

00:30:51.840 --> 00:30:54.000
basically build up a whole test harness

00:30:54.000 --> 00:30:55.840
with everything with MIS. I told it use

00:30:55.840 --> 00:30:57.200
MIS because it has

00:30:57.200 --> 00:30:58.799
>> But you're going to get a permutation

00:30:58.799 --> 00:31:00.399
explosion, aren't you? You're going to

00:31:00.399 --> 00:31:02.320
be testing different versions with

00:31:02.320 --> 00:31:02.559
different

00:31:02.559 --> 00:31:04.159
>> versions. And we do. And the problem is

00:31:04.159 --> 00:31:06.159
that we have the CI/CD workflow that

00:31:06.159 --> 00:31:09.360
that has 350 tests uh that we inherited

00:31:09.360 --> 00:31:11.600
and used to take 5 hours to run, but um

00:31:11.600 --> 00:31:13.120
thanks to community contributors,

00:31:13.120 --> 00:31:15.760
somebody completely uh spent like two or

00:31:15.760 --> 00:31:17.760
3 weeks and he got it down to under an

00:31:17.760 --> 00:31:19.120
hour. We're still running the full test

00:31:19.120 --> 00:31:21.039
suite. He just rearranged it. He just

00:31:21.039 --> 00:31:23.120
put in early exit conditions and all

00:31:23.120 --> 00:31:25.120
that. So, he really really made it much

00:31:25.120 --> 00:31:27.039
better. Like the PRs run way faster. The

00:31:27.039 --> 00:31:29.279
the merge to master runs way faster.

00:31:29.279 --> 00:31:30.880
>> I love those changes.

00:31:30.880 --> 00:31:32.320
>> Yeah. Well, yeah, it's a really nice

00:31:32.320 --> 00:31:34.720
thing to do until you you have like 355

00:31:34.720 --> 00:31:36.559
jobs to look at. And I'm like, "Oh,

00:31:36.559 --> 00:31:39.039
man." And he just he just kept hammering

00:31:39.039 --> 00:31:41.519
at it, you know. He and every PR this

00:31:41.519 --> 00:31:43.919
was beautiful. This also he was using

00:31:43.919 --> 00:31:46.720
Clark, but every PR of him was just like

00:31:46.720 --> 00:31:49.519
just 20 lines, 30 lines stiff, very easy

00:31:49.519 --> 00:31:52.559
to review, very small, very focused.

00:31:52.559 --> 00:31:54.480
>> That's really not easy with bloody opus

00:31:54.480 --> 00:31:55.360
in the [ __ ] mix.

00:31:55.360 --> 00:31:58.559
>> No, I mean it was also three months ago.

00:31:58.559 --> 00:32:02.880
Uh but the the the the question is um I

00:32:02.880 --> 00:32:04.240
want to know how exactly he did it

00:32:04.240 --> 00:32:05.919
because then I I had like a whole bunch

00:32:05.919 --> 00:32:09.120
of like markdown to control the uh spec

00:32:09.120 --> 00:32:11.760
driven development and he asked me like

00:32:11.760 --> 00:32:14.000
can you remove all the markdown because

00:32:14.000 --> 00:32:17.200
it impacts my my uh my my workflow my

00:32:17.200 --> 00:32:19.360
cloud workflow uh and I have

00:32:19.360 --> 00:32:21.600
>> this this is a a guy in your in your

00:32:21.600 --> 00:32:23.600
employee in your company that you work

00:32:23.600 --> 00:32:25.120
>> no no no no this is the open source

00:32:25.120 --> 00:32:26.960
project so this is a guy that came along

00:32:26.960 --> 00:32:28.640
asks very very politely

00:32:28.640 --> 00:32:31.200
Guys, do you mind if I spend some time

00:32:31.200 --> 00:32:32.799
optimizing DCI? We're like,

00:32:32.799 --> 00:32:34.320
>> okay.

00:32:34.320 --> 00:32:36.960
>> Okay, send me the links, man. I'm I I

00:32:36.960 --> 00:32:38.960
want to I want to celebrate this guy. I

00:32:38.960 --> 00:32:40.799
want to learn from this guy myself.

00:32:40.799 --> 00:32:42.640
>> Yeah, he did it. And you could tell it

00:32:42.640 --> 00:32:44.960
was cloud, but he controlled it so well.

00:32:44.960 --> 00:32:47.039
And um you know, compare his PRs to my

00:32:47.039 --> 00:32:49.279
PRs. I'm embarrassed. Uh

00:32:49.279 --> 00:32:51.360
>> I I was like, god damn, I can do so much

00:32:51.360 --> 00:32:53.039
better than what I'm doing right now.

00:32:53.039 --> 00:32:54.559
But

00:32:54.559 --> 00:32:55.600
anyway,

00:32:55.600 --> 00:32:58.960
>> yeah, I I I I need to work out there's a

00:32:58.960 --> 00:33:01.840
few things on my to-do list. I I want

00:33:01.840 --> 00:33:04.240
when Claude makes a PR, I want it to be

00:33:04.240 --> 00:33:07.519
draft because by default it makes it an

00:33:07.519 --> 00:33:09.519
open PR and then my colleagues are going

00:33:09.519 --> 00:33:11.679
like, what is this? And like I'm working

00:33:11.679 --> 00:33:13.279
on it.

00:33:13.279 --> 00:33:14.080
>> Yeah.

00:33:14.080 --> 00:33:17.679
>> And uh and it has the most insane commit

00:33:17.679 --> 00:33:20.480
messages nowadays. It's like more co

00:33:20.480 --> 00:33:22.559
more more [ __ ] in the commit message

00:33:22.559 --> 00:33:24.799
than there was in the diff and like

00:33:24.799 --> 00:33:26.399
[ __ ] [ __ ] up, man.

00:33:26.399 --> 00:33:27.919
>> I don't know. Something happened right

00:33:27.919 --> 00:33:30.080
right now. It uses comments as journals.

00:33:30.080 --> 00:33:32.640
It puts comment It puts diffs that are

00:33:32.640 --> 00:33:34.559
massive. I mean, it puts comments that

00:33:34.559 --> 00:33:36.640
are massive.

00:33:36.640 --> 00:33:38.159
>> I think everyone's complaining.

00:33:38.159 --> 00:33:40.799
>> I need I need a way. You you know I have

00:33:40.799 --> 00:33:43.600
this lock sensor to limit the lines to

00:33:43.600 --> 00:33:46.080
code but it doesn't catch the crazy

00:33:46.080 --> 00:33:48.159
messages and the commit and the commit

00:33:48.159 --> 00:33:50.559
messages. Okay, I just added a new rule

00:33:50.559 --> 00:33:53.840
to my u my my fable work like main

00:33:53.840 --> 00:33:57.039
agents and it follows it has like the

00:33:57.039 --> 00:34:00.320
operand uh modus operandi rules are that

00:34:00.320 --> 00:34:02.240
and and I haven't put this in steering

00:34:02.240 --> 00:34:04.080
rules but like I need to mine and dream

00:34:04.080 --> 00:34:05.919
like you just said but any what it what

00:34:05.919 --> 00:34:08.240
it has right now is that uh medium tasks

00:34:08.240 --> 00:34:10.879
are background opus agents and I never

00:34:10.879 --> 00:34:13.359
hit the limit. So dynamic workflows with

00:34:13.359 --> 00:34:18.000
opus implementers, opus verifiers and um

00:34:18.000 --> 00:34:19.760
only very small patch fixes it does

00:34:19.760 --> 00:34:21.839
itself. And the new rule is you run a

00:34:21.839 --> 00:34:25.359
background opus comment um um fixer that

00:34:25.359 --> 00:34:27.520
basically has a few rules like comments

00:34:27.520 --> 00:34:30.159
are not journals. The the history lives

00:34:30.159 --> 00:34:32.480
in the comets. So you do not put like

00:34:32.480 --> 00:34:35.119
the full history of why a line of code

00:34:35.119 --> 00:34:37.200
turns out to be that line of code by

00:34:37.200 --> 00:34:40.399
saying on on uh September 25 we

00:34:40.399 --> 00:34:42.240
identified that it didn't work. So we

00:34:42.240 --> 00:34:43.679
tried and this and then we tried that

00:34:43.679 --> 00:34:45.599
and now we are this is why this this

00:34:45.599 --> 00:34:47.359
value this line right here and you have

00:34:47.359 --> 00:34:49.919
like 10 lines and I said no more than

00:34:49.919 --> 00:34:52.480
three lines like maximum five

00:34:52.480 --> 00:34:54.480
>> comments.

00:34:54.480 --> 00:34:56.320
I need to fix this and I'm going to fix

00:34:56.320 --> 00:34:57.920
this [ __ ] right now because this is

00:34:57.920 --> 00:35:00.000
the problem that's bothering me right

00:35:00.000 --> 00:35:02.400
now besides needing the toilet. Okay,

00:35:02.400 --> 00:35:05.440
thanks Vincent for the for the chat. I

00:35:05.440 --> 00:35:08.240
don't know if this will be a