WEBVTT

00:00:00.000 --> 00:00:01.840
Yeah, so this is episode 30. I think

00:00:01.840 --> 00:00:03.840
this is quite a milestone. We've chatted

00:00:03.840 --> 00:00:06.640
30 times about AI infrastructure.

00:00:06.640 --> 00:00:09.800
I hope not to repeat it too early.

00:00:09.800 --> 00:00:12.640
I don't think repetition is a bad thing

00:00:12.640 --> 00:00:14.920
necessarily. I I I like I had the

00:00:14.920 --> 00:00:16.600
thought this morning our brains are

00:00:16.600 --> 00:00:19.040
probably changing and I was just

00:00:19.040 --> 00:00:20.560
thinking like wouldn't you wouldn't you

00:00:20.560 --> 00:00:23.480
say AI requires some spinning of plates

00:00:23.480 --> 00:00:25.680
like I'm hopping in between sessions all

00:00:25.680 --> 00:00:27.920
the time which is that the opposite you

00:00:27.920 --> 00:00:31.040
know sage advice of focusing on a

00:00:31.040 --> 00:00:33.560
particular problem and doing it well.

00:00:33.560 --> 00:00:35.240
Anyway. Yeah, I think that's one of the

00:00:35.240 --> 00:00:36.800
discussions that we had like what are

00:00:36.800 --> 00:00:39.480
the skills that you need as an engineer

00:00:39.480 --> 00:00:41.800
in an in the AI future. You need to be

00:00:41.800 --> 00:00:44.880
able to like you said switch contexts

00:00:44.880 --> 00:00:47.240
more actively, be able to critically

00:00:47.240 --> 00:00:50.080
review an architectural design and kind

00:00:50.080 --> 00:00:52.440
of predict or know. Like they say on one

00:00:52.440 --> 00:00:54.400
side it kills the junior engineers

00:00:54.400 --> 00:00:57.240
because you um as a junior maybe you

00:00:57.240 --> 00:00:58.640
don't know yet what's going to work and

00:00:58.640 --> 00:01:01.160
what fails. I did I did notice that I'm

00:01:01.160 --> 00:01:03.240
I'm handing off my brain to the AI a lot

00:01:03.240 --> 00:01:05.680
more even in area I mean I guess more in

00:01:05.680 --> 00:01:08.520
areas my brain. It's more in areas that

00:01:08.520 --> 00:01:10.520
you're not familiar with, right? If

00:01:10.520 --> 00:01:12.440
you're very familiar it's a funny quote

00:01:12.440 --> 00:01:14.400
where they say like it's amazing like if

00:01:14.400 --> 00:01:17.360
I work with with ChatGPT if it goes into

00:01:17.360 --> 00:01:18.760
a domain I know very well it makes so

00:01:18.760 --> 00:01:20.040
many mistakes but if I'm learning

00:01:20.040 --> 00:01:21.560
something new it's amazing it knows

00:01:21.560 --> 00:01:23.160
everything and I just can't trust it

00:01:23.160 --> 00:01:25.760
fully. What is it What what you said

00:01:25.760 --> 00:01:28.120
about switching context I almost thought

00:01:28.120 --> 00:01:30.000
you were talking about AI because

00:01:30.000 --> 00:01:32.800
because when I when I think of um I know

00:01:32.800 --> 00:01:34.240
what you mean you you're you're saying

00:01:34.240 --> 00:01:36.760
that us humans need to switch context

00:01:36.760 --> 00:01:39.560
but at the same time each one of my AI

00:01:39.560 --> 00:01:42.760
sessions feel like if if anything I'm

00:01:42.760 --> 00:01:44.600
curating the context you know I'm making

00:01:44.600 --> 00:01:47.160
sure that it's got the right it's in the

00:01:47.160 --> 00:01:49.680
right directory it's in the it's got the

00:01:49.680 --> 00:01:52.800
right claw.md it's got the right MCP if

00:01:52.800 --> 00:01:54.400
I was going to go down that way. You

00:01:54.400 --> 00:01:55.640
know what I mean like each one of my

00:01:55.640 --> 00:01:58.280
folders is like curated with the right

00:01:58.280 --> 00:02:00.720
context for it to do the job. Yeah. I

00:02:00.720 --> 00:02:02.200
mean that's I had an interesting

00:02:02.200 --> 00:02:03.920
discussion a few days ago for an

00:02:03.920 --> 00:02:06.800
organization that are looking for like

00:02:06.800 --> 00:02:09.640
platform and platform engineering and

00:02:09.640 --> 00:02:11.920
and my opinion is that for platform

00:02:11.920 --> 00:02:14.240
engineering as we were the enablers for

00:02:14.240 --> 00:02:17.520
CICD and DevOps and and Agile that our

00:02:17.520 --> 00:02:20.360
role now extends towards AI and so I I'm

00:02:20.360 --> 00:02:21.960
having conversations with organizations

00:02:21.960 --> 00:02:23.640
where I'm saying if you're looking at me

00:02:23.640 --> 00:02:25.800
as just doing you know business as usual

00:02:25.800 --> 00:02:28.680
I see and get ups get up. I mean that's

00:02:28.680 --> 00:02:30.160
not what I'm looking for, right? I can

00:02:30.160 --> 00:02:32.120
do that. I'm going to drive cloud to do

00:02:32.120 --> 00:02:33.520
it because I've done it so many times

00:02:33.520 --> 00:02:36.240
and it's like short-term I can output

00:02:36.240 --> 00:02:38.040
text but long-term it's not a viable

00:02:38.040 --> 00:02:39.520
strategy because the model's going to

00:02:39.520 --> 00:02:41.480
get so good to get it all of it context

00:02:41.480 --> 00:02:44.160
it needs. And I'm not just going to be a

00:02:44.160 --> 00:02:46.040
basic context provider finding all of

00:02:46.040 --> 00:02:47.880
the like you know connections.

00:02:47.880 --> 00:02:50.560
>> provider. Long-term I want to be you

00:02:50.560 --> 00:02:52.000
know you know what what what what the

00:02:52.000 --> 00:02:54.920
whole idea of DevOps and Agile is we are

00:02:54.920 --> 00:02:56.880
a feedback loop. We provide feedback

00:02:56.880 --> 00:02:59.080
loops you know continuous improvements.

00:02:59.080 --> 00:03:01.840
The whole idea of of DevOps is to close

00:03:01.840 --> 00:03:04.680
that loop so that we are able to source

00:03:04.680 --> 00:03:07.240
the relevant information and measure

00:03:07.240 --> 00:03:08.880
what matters so that we know the

00:03:08.880 --> 00:03:11.040
effectiveness and and that mindset of

00:03:11.040 --> 00:03:12.440
DevOps you know comes with the Dora

00:03:12.440 --> 00:03:14.240
metrics comes with measuring

00:03:14.240 --> 00:03:16.400
effectiveness of of teams like in a lot

00:03:16.400 --> 00:03:18.640
of organizations we've been relegated to

00:03:18.640 --> 00:03:20.480
do pure ops

00:03:20.480 --> 00:03:21.520
a little more than a system

00:03:21.520 --> 00:03:23.640
administrator and maybe a little bit of

00:03:23.640 --> 00:03:25.600
a expert log reader.

00:03:25.600 --> 00:03:27.640
>> business as usual is the way I

00:03:27.640 --> 00:03:29.520
understand it. Yeah, we are like log

00:03:29.520 --> 00:03:31.560
readers. Like people come to us with

00:03:31.560 --> 00:03:33.360
errors and we go like you didn't read

00:03:33.360 --> 00:03:35.880
the logs. We read the logs and we go

00:03:35.880 --> 00:03:36.840
like

00:03:36.840 --> 00:03:38.600
>> I can use cloudwatch obviously you

00:03:38.600 --> 00:03:40.240
can't.

00:03:40.240 --> 00:03:42.960
So so our role has been to surface

00:03:42.960 --> 00:03:44.920
information. If we're not relegated to

00:03:44.920 --> 00:03:47.240
to a title change that's still the old

00:03:47.240 --> 00:03:49.080
system admin Quick [clears throat] side

00:03:49.080 --> 00:03:51.040
note when I was at the AWS summit last

00:03:51.040 --> 00:03:53.120
Wednesday in London I couldn't help but

00:03:53.120 --> 00:03:56.600
think observability was one of the most

00:03:56.600 --> 00:04:00.000
popular sort of third-party providers.

00:04:00.000 --> 00:04:01.640
You know there there was about like you

00:04:01.640 --> 00:04:06.560
know honeycomb data dog new relic um

00:04:06.560 --> 00:04:08.640
I think there's a new one called uh

00:04:08.640 --> 00:04:10.680
Bronte. There was about there was about

00:04:10.680 --> 00:04:13.040
10 of them. Yeah. I think they they need

00:04:13.040 --> 00:04:15.440
to reposition themselves as like context

00:04:15.440 --> 00:04:17.200
providers for AI, right? Like a lot of

00:04:17.200 --> 00:04:19.239
the honeycomb talks from Charity Majors

00:04:19.239 --> 00:04:20.920
on LinkedIn it's like well it's what

00:04:20.920 --> 00:04:22.240
we've been doing all the time, haven't

00:04:22.240 --> 00:04:23.920
we? We've been providing you all of the

00:04:23.920 --> 00:04:26.960
data for your services in in operations

00:04:26.960 --> 00:04:28.440
and now it's no longer needs to be

00:04:28.440 --> 00:04:30.600
serviced to towards humans it needs to

00:04:30.600 --> 00:04:31.760
go towards

00:04:31.760 --> 00:04:33.680
>> towards the agents, yeah. agents. This

00:04:33.680 --> 00:04:36.000
whole context provider it feels like a

00:04:36.000 --> 00:04:39.080
meme but it's bloody true. But

00:04:39.080 --> 00:04:40.880
I I I think that's an important

00:04:40.880 --> 00:04:42.960
evolution, right? I don't think many

00:04:42.960 --> 00:04:45.480
DevOps engineers had the opportunity to

00:04:45.480 --> 00:04:47.920
take on more of like a platform type of

00:04:47.920 --> 00:04:51.000
role of like I create the measurable

00:04:51.000 --> 00:04:53.040
like I create the data dog monitors. I

00:04:53.040 --> 00:04:54.520
help you define your runbooks.

00:04:54.520 --> 00:04:55.760
Ultimately we're not the one that's

00:04:55.760 --> 00:04:57.000
supposed to write the runbooks. We're

00:04:57.000 --> 00:04:58.400
the ones that's supposed to put the

00:04:58.400 --> 00:05:01.200
systems in place and dashboards in place

00:05:01.200 --> 00:05:02.480
so that others when they build a

00:05:02.480 --> 00:05:04.960
solution can build those runbooks. And

00:05:04.960 --> 00:05:07.160
if they didn't then we were embedded and

00:05:07.160 --> 00:05:08.600
then we would be the ones like site

00:05:08.600 --> 00:05:10.840
reliability team engineers responsible

00:05:10.840 --> 00:05:12.720
for highlighting if you want to go live

00:05:12.720 --> 00:05:14.560
with this feature this is probably what

00:05:14.560 --> 00:05:16.480
you need to be aware of. This is where

00:05:16.480 --> 00:05:18.200
we need to put some you know monitors

00:05:18.200 --> 00:05:20.360
around and how we we we measure the

00:05:20.360 --> 00:05:23.240
effectiveness of the solution. So I mean

00:05:23.240 --> 00:05:25.920
the idea has been shared responsibility

00:05:25.920 --> 00:05:28.080
and enabling others to take on that

00:05:28.080 --> 00:05:29.920
responsibility not taking over the

00:05:29.920 --> 00:05:31.720
responsibility from them, right? Our

00:05:31.720 --> 00:05:33.640
role was to enable them to to put in

00:05:33.640 --> 00:05:35.480
place the systems like observability

00:05:35.480 --> 00:05:38.680
platforms like CICD checks that give us

00:05:38.680 --> 00:05:40.640
the trust and gives us the ability to to

00:05:40.640 --> 00:05:42.040
to

00:05:42.040 --> 00:05:42.920
to pull out the information.

00:05:42.920 --> 00:05:45.120
>> On that role I mean you you define that

00:05:45.120 --> 00:05:47.480
role pretty well I think. Though perhaps

00:05:47.480 --> 00:05:49.120
I've done the wrong thing in in past

00:05:49.120 --> 00:05:51.360
projects or maybe I just just got roped

00:05:51.360 --> 00:05:54.280
into it. Like I can't help but think in

00:05:54.280 --> 00:05:56.640
a lot of my I'm just thinking of all the

00:05:56.640 --> 00:05:58.840
clients that I've worked with

00:05:58.840 --> 00:06:00.600
in my in my consultancy

00:06:00.600 --> 00:06:03.760
job. Many times I've not even though my

00:06:03.760 --> 00:06:06.440
title is infrastructure

00:06:06.440 --> 00:06:08.640
infrastructure lead I think technically.

00:06:08.640 --> 00:06:11.720
I I've I've gone into I've wore

00:06:11.720 --> 00:06:13.360
different hats in the sense that I've

00:06:13.360 --> 00:06:15.200
I've got my I've I've went into full

00:06:15.200 --> 00:06:17.800
development mode on some features of the

00:06:17.800 --> 00:06:20.160
product, right? And I never got and yeah

00:06:20.160 --> 00:06:21.880
I've played like maybe the security

00:06:21.880 --> 00:06:23.840
champion you know information security

00:06:23.840 --> 00:06:27.160
dude. But the same but like in the job

00:06:27.160 --> 00:06:30.200
that I've done I would say that I've

00:06:30.200 --> 00:06:33.480
I've [snorts] not had the

00:06:33.480 --> 00:06:36.960
opportunity to to wear one hat and wear

00:06:36.960 --> 00:06:39.760
it well. Just I'm actually

00:06:39.760 --> 00:06:42.040
having to wear many hats

00:06:42.040 --> 00:06:45.120
in my typical job.

00:06:45.120 --> 00:06:46.960
And hence I've I feel like sometimes the

00:06:46.960 --> 00:06:49.480
focus has been lost and I've never

00:06:49.480 --> 00:06:52.160
proudly you know

00:06:52.160 --> 00:06:54.000
on you know

00:06:54.000 --> 00:06:56.800
100% on getting the Dora metrics done in

00:06:56.800 --> 00:06:58.680
a way because I was like you know

00:06:58.680 --> 00:07:00.760
juggling some other balls. I mean to be

00:07:00.760 --> 00:07:02.360
honest for an organization usually they

00:07:02.360 --> 00:07:03.720
don't want you to spend time on that

00:07:03.720 --> 00:07:06.160
anyway, right? They're like what's your

00:07:06.160 --> 00:07:07.960
how are you contributing business value?

00:07:07.960 --> 00:07:09.560
Like I mean

00:07:09.560 --> 00:07:11.640
If you're being questioned as a platform

00:07:11.640 --> 00:07:13.240
team how are you contributing business

00:07:13.240 --> 00:07:13.600
value?

00:07:13.600 --> 00:07:16.400
>> Yeah I mean that's the age-old question.

00:07:16.400 --> 00:07:19.720
Right. Then then the platform team

00:07:19.720 --> 00:07:21.880
will will When things go well like oh

00:07:21.880 --> 00:07:24.360
what did you guys do? Oh my lord.

00:07:24.360 --> 00:07:26.720
Yeah. And then the worst is when you're

00:07:26.720 --> 00:07:28.600
most valued if you're inside a startup

00:07:28.600 --> 00:07:31.160
and it's failing all over the place and

00:07:31.160 --> 00:07:33.400
everyone is depending on you to keep the

00:07:33.400 --> 00:07:35.880
system running. Yeah. And

00:07:35.880 --> 00:07:37.800
>> I I hate it when you like

00:07:37.800 --> 00:07:40.680
you know John was putting out fires over

00:07:40.680 --> 00:07:43.600
the weekend like he's such a hero. Like

00:07:43.600 --> 00:07:46.440
John doesn't know what he's doing.

00:07:46.440 --> 00:07:49.520
But but I think so for for AI future

00:07:49.520 --> 00:07:51.000
like when I'm looking at roles now when

00:07:51.000 --> 00:07:52.960
I'm talking because I am talking to

00:07:52.960 --> 00:07:56.000
companies I I am asking them like I got

00:07:56.000 --> 00:07:57.320
an interesting question which they ask

00:07:57.320 --> 00:07:58.440
what is it that you want to do and what

00:07:58.440 --> 00:07:59.760
is something that you absolutely don't

00:07:59.760 --> 00:08:02.280
want to do if we were to like hire you.

00:08:02.280 --> 00:08:03.760
>> That's a good question to ask anyone.

00:08:03.760 --> 00:08:07.000
Yeah and I said well I'm okay with being

00:08:07.000 --> 00:08:10.040
I can drive cloud to to do your Argo CD

00:08:10.040 --> 00:08:13.160
migration from helm add-on to a EKS

00:08:13.160 --> 00:08:15.720
managed add-on but it will obviously be

00:08:15.720 --> 00:08:18.040
cloud and it will obviously be me

00:08:18.040 --> 00:08:19.760
generating the plan getting alignment

00:08:19.760 --> 00:08:21.640
with you guys and then just having cloud

00:08:21.640 --> 00:08:23.960
deliver it and driving cloud and that's

00:08:23.960 --> 00:08:25.800
not a long-term strategy. I can do that.

00:08:25.800 --> 00:08:28.080
I don't mind doing that but for my role

00:08:28.080 --> 00:08:30.600
it must be I would rather be the person

00:08:30.600 --> 00:08:32.520
that helps cuz I I talked to a lot of

00:08:32.520 --> 00:08:34.120
organizations that said they have these

00:08:34.120 --> 00:08:36.000
phases of AI adoption. Initially a

00:08:36.000 --> 00:08:37.400
little bit reluctant we don't want to

00:08:37.400 --> 00:08:39.960
give it access to our critical systems

00:08:39.960 --> 00:08:42.440
then everybody goes all in maybe they

00:08:42.440 --> 00:08:44.400
got cursor licenses or they start

00:08:44.400 --> 00:08:46.040
rolling out Claude Code or teams

00:08:46.040 --> 00:08:48.080
independently adopt these tools anyway

00:08:48.080 --> 00:08:49.520
like as an organization you can't stop

00:08:49.520 --> 00:08:51.920
them because they are so productive and

00:08:51.920 --> 00:08:53.960
they save so much time that teams go in

00:08:53.960 --> 00:08:55.480
a wild sprawl and every team does it

00:08:55.480 --> 00:08:57.960
differently and then after now 1 year

00:08:57.960 --> 00:09:00.440
that was 2025 now 2026 a lot of the

00:09:00.440 --> 00:09:02.480
organizations are like we have a wild

00:09:02.480 --> 00:09:04.320
sprawl of tools. Some teams are

00:09:04.320 --> 00:09:05.800
efficient some teams are giving their

00:09:05.800 --> 00:09:08.480
product managers AI access and they are

00:09:08.480 --> 00:09:10.200
releasing to production and we have a

00:09:10.200 --> 00:09:12.120
huge slope problem. We have reliability

00:09:12.120 --> 00:09:13.400
issues. These systems haven't been

00:09:13.400 --> 00:09:15.400
properly designed and we need to put in

00:09:15.400 --> 00:09:17.360
place guardrails right now. We need to

00:09:17.360 --> 00:09:19.200
get that under control. A lot of

00:09:19.200 --> 00:09:21.120
organizations are at that at that stage

00:09:21.120 --> 00:09:23.160
where like we've gone into a wild west

00:09:23.160 --> 00:09:25.520
and we have no control and I'm like well

00:09:25.520 --> 00:09:27.360
It's so easy to happen. It's so easy to

00:09:27.360 --> 00:09:29.720
happen. I've seen it myself already a

00:09:29.720 --> 00:09:31.560
couple of times almost. I've seen it

00:09:31.560 --> 00:09:34.880
with my own work in a way.

00:09:34.880 --> 00:09:35.800
Yeah I

00:09:35.800 --> 00:09:38.480
we have to fail in order to succeed. I'm

00:09:38.480 --> 00:09:40.160
worried because you told me that you had

00:09:40.160 --> 00:09:42.640
scenario where you trusted it and then

00:09:42.640 --> 00:09:45.200
it it ended up giving biting you in the

00:09:45.200 --> 00:09:47.480
back. I've had a scenario where I'm I'm

00:09:47.480 --> 00:09:49.720
like I said handing over the the brain.

00:09:49.720 --> 00:09:51.840
It's doing work that I have like a grasp

00:09:51.840 --> 00:09:53.680
of what we're trying to do and the AI is

00:09:53.680 --> 00:09:56.560
going off on these massive quests of

00:09:56.560 --> 00:09:57.920
we're going to do this thing and that

00:09:57.920 --> 00:10:00.040
thing and I'm like, well, kind of makes

00:10:00.040 --> 00:10:01.760
sense, but I'm not 100% sure, but

00:10:01.760 --> 00:10:03.240
honestly can't be bothered. And I'm

00:10:03.240 --> 00:10:05.680
like, this is going to bite me. So, I I

00:10:05.680 --> 00:10:07.400
have the one agent It's going to bite

00:10:07.400 --> 00:10:10.080
the whole industry. Like, during the AWS

00:10:10.080 --> 00:10:11.560
keynote, I mentioned to you over

00:10:11.560 --> 00:10:13.400
WhatsApp, I I've got to put the slide in

00:10:13.400 --> 00:10:15.000
somewhere in the description or

00:10:15.000 --> 00:10:17.080
somewhere just so I'm not making this

00:10:17.080 --> 00:10:20.480
up. The AWS partner with a

00:10:20.480 --> 00:10:23.800
uh with a local company called Motorway.

00:10:23.800 --> 00:10:25.680
And they were like boasting that they

00:10:25.680 --> 00:10:27.800
can ship a million lines of code. Now,

00:10:27.800 --> 00:10:29.600
with Kiro, they're shipping a million

00:10:29.600 --> 00:10:32.960
lines of code per month now. And I mean

00:10:32.960 --> 00:10:35.000
The industry is I'm just I'm just so

00:10:35.000 --> 00:10:37.760
flabbergasted when people say that.

00:10:37.760 --> 00:10:39.160
I mean, a lot of people go like saying

00:10:39.160 --> 00:10:41.120
"Swans is key lock a measurement of

00:10:41.120 --> 00:10:43.840
productivity." Lines of code has a never

00:10:43.840 --> 00:10:46.000
been a measurement of productivity.

00:10:46.000 --> 00:10:50.160
>> And like, how AWS stand up and and like

00:10:50.160 --> 00:10:52.000
and like slap this guy on the back

00:10:52.000 --> 00:10:53.800
saying this is where you want to be.

00:10:53.800 --> 00:10:55.360
Yeah, I feel like it feels as a horrible

00:10:55.360 --> 00:10:58.640
thing. It's like a broken trust. Uh

00:10:58.640 --> 00:11:01.680
it's not cool and it's really really not

00:11:01.680 --> 00:11:02.880
cool.

00:11:02.880 --> 00:11:04.360
Quite a lot of people in the like the in

00:11:04.360 --> 00:11:07.280
the in the classic Agile DevOps uh space

00:11:07.280 --> 00:11:09.960
where we're quite

00:11:09.960 --> 00:11:12.520
you know, heavy on culture, heavy on

00:11:12.520 --> 00:11:14.760
measure measurable outcomes, um on

00:11:14.760 --> 00:11:17.240
defining what you want to measure and

00:11:17.240 --> 00:11:18.920
validating that they are the the proper

00:11:18.920 --> 00:11:21.200
metrics. So, in my circle on LinkedIn,

00:11:21.200 --> 00:11:22.880
there's quite a lot of people that are

00:11:22.880 --> 00:11:25.360
going like heavily

00:11:25.360 --> 00:11:26.760
against what you just mentioned, like

00:11:26.760 --> 00:11:28.200
sales standing up and saying "Ah,

00:11:28.200 --> 00:11:30.600
they're shipping so much code." Which is

00:11:30.600 --> 00:11:31.880
like, what do you mean you're shipping

00:11:31.880 --> 00:11:34.040
code, but isn't the purpose to ship ship

00:11:34.040 --> 00:11:36.120
features and measure that we're actually

00:11:36.120 --> 00:11:38.320
like getting the results that we're

00:11:38.320 --> 00:11:39.000
looking for?

00:11:39.000 --> 00:11:40.760
>> the guy the guy that was on stage was

00:11:40.760 --> 00:11:42.400
like the director of engineering, which

00:11:42.400 --> 00:11:45.320
is just just absolutely scary. I mean, I

00:11:45.320 --> 00:11:46.480
don't want to throw shade to the guy. I

00:11:46.480 --> 00:11:48.360
mean, maybe maybe it is working out for

00:11:48.360 --> 00:11:50.600
him, but like I I didn't

00:11:50.600 --> 00:11:52.800
get the

00:11:52.800 --> 00:11:55.120
I didn't get the the upside of of

00:11:55.120 --> 00:11:56.640
shipping a million lines of code. Maybe

00:11:56.640 --> 00:11:57.040
maybe they were

00:11:57.040 --> 00:11:59.360
>> director of engineering using key lock

00:11:59.360 --> 00:12:01.760
as a metric and standing up upstage and

00:12:01.760 --> 00:12:03.280
being proud, then I guess you're going

00:12:03.280 --> 00:12:05.200
to get a lot of backlash. Yeah, I

00:12:05.200 --> 00:12:06.640
believe I I believe I

00:12:06.640 --> 00:12:09.520
No, no one like if it was if the

00:12:09.520 --> 00:12:11.160
auditorium was full with people like me,

00:12:11.160 --> 00:12:13.680
there would have been an audible groan

00:12:13.680 --> 00:12:16.320
or sigh. But, everyone was like, "Yeah,

00:12:16.320 --> 00:12:18.880
this is amazing." Like, no, this not

00:12:18.880 --> 00:12:21.560
amazing. And and I did have a chat with

00:12:21.560 --> 00:12:24.000
an AWS guy that I know for some time. I

00:12:24.000 --> 00:12:27.240
won't name I won't name names. And he

00:12:27.240 --> 00:12:28.800
we were we were talking about in the

00:12:28.800 --> 00:12:30.920
context of serverless stuff. And like,

00:12:30.920 --> 00:12:33.480
one thing that was amazing about and

00:12:33.480 --> 00:12:35.440
still is amazing about serverless about

00:12:35.440 --> 00:12:38.880
lambda functions is that it sort of like

00:12:38.880 --> 00:12:41.600
encouraged engineering to be small,

00:12:41.600 --> 00:12:43.839
right? And I'm missing the word

00:12:43.839 --> 00:12:46.360
containable. Not containable, but like a

00:12:46.360 --> 00:12:49.120
unit of work that could be easily

00:12:49.120 --> 00:12:50.920
understood. Lambdas are brilliant for

00:12:50.920 --> 00:12:53.640
that. Of course, you can do what I do is

00:12:53.640 --> 00:12:56.000
shove in a monolithic

00:12:56.000 --> 00:12:59.320
binary into a lambda context, but

00:12:59.320 --> 00:13:00.720
but you know what I mean. Like, lambda

00:13:00.720 --> 00:13:03.360
is actually like such a breath of fresh

00:13:03.360 --> 00:13:06.120
air in the industry to keep things

00:13:06.120 --> 00:13:07.960
manageable. But, I mean

00:13:07.960 --> 00:13:09.640
>> it's it's almost like Heroku, right?

00:13:09.640 --> 00:13:11.600
Like, the whole problem of of people

00:13:11.600 --> 00:13:13.480
building stuff was that how do we get

00:13:13.480 --> 00:13:15.800
this in front of a lot of people? And

00:13:15.800 --> 00:13:17.720
then if you're scaling, then you have

00:13:17.720 --> 00:13:19.720
problems of like, what is my monthly

00:13:19.720 --> 00:13:21.520
cost of running this solution? How do I

00:13:21.520 --> 00:13:23.480
make it highly available? And then you

00:13:23.480 --> 00:13:25.800
have solutions like Heroku and lambda

00:13:25.800 --> 00:13:27.160
where it's like you don't need to worry

00:13:27.160 --> 00:13:28.600
about that. You just define your

00:13:28.600 --> 00:13:30.920
functionality and you get it deployed.

00:13:30.920 --> 00:13:32.600
And then you have larger engineering

00:13:32.600 --> 00:13:34.440
teams which are like, we build this

00:13:34.440 --> 00:13:37.680
monstrosity of hundreds lambdas and API

00:13:37.680 --> 00:13:39.760
gateways and we have no idea like we

00:13:39.760 --> 00:13:41.080
would have been better off with building

00:13:41.080 --> 00:13:44.360
a you know, a monolithic API and

00:13:44.360 --> 00:13:46.440
deploying it and that's way more

00:13:46.440 --> 00:13:49.200
easier to understand and manage and run

00:13:49.200 --> 00:13:51.320
locally and deploy. I mean, there's

00:13:51.320 --> 00:13:52.960
there's pros and cons. There's This is

00:13:52.960 --> 00:13:54.200
one of the questions they asked me as

00:13:54.200 --> 00:13:55.640
well, like from you what have you

00:13:55.640 --> 00:13:57.400
learned? What are some of the mistakes

00:13:57.400 --> 00:13:59.480
that you would avoid if you would start

00:13:59.480 --> 00:13:59.880
now?

00:13:59.880 --> 00:14:01.960
>> interview question. Yeah, but like I

00:14:01.960 --> 00:14:04.240
felt it's like on one way I was saying,

00:14:04.240 --> 00:14:06.880
well, you know, don't need sometimes

00:14:06.880 --> 00:14:08.920
these like serverless solutions and you

00:14:08.920 --> 00:14:10.839
can get away like you know, the DHH type

00:14:10.839 --> 00:14:12.760
of a spiel of like you can get away with

00:14:12.760 --> 00:14:14.800
an instance and you can scale out quite

00:14:14.800 --> 00:14:17.920
quite strongly early stage with a simple

00:14:17.920 --> 00:14:18.800
virtual machine

00:14:18.800 --> 00:14:20.800
>> machine under the desk. Yeah.

00:14:20.800 --> 00:14:22.480
Um you can ship fast with that as well.

00:14:22.480 --> 00:14:24.560
You can avoid, you know, mono poly repo

00:14:24.560 --> 00:14:27.160
setups and go mono repo and or you don't

00:14:27.160 --> 00:14:28.720
even need a mono repo.

00:14:28.720 --> 00:14:30.480
>> because you want the agents to to

00:14:30.480 --> 00:14:32.400
develop quickly and avoid all this

00:14:32.400 --> 00:14:33.200
costly

00:14:33.200 --> 00:14:35.079
>> even talking about like in the AI world.

00:14:35.079 --> 00:14:36.959
And then I said, well, and then on the

00:14:36.959 --> 00:14:38.440
other side what I've learned because

00:14:38.440 --> 00:14:40.280
I've joined a lot of Usually you they

00:14:40.280 --> 00:14:42.240
hire DevOps or site reliability when

00:14:42.240 --> 00:14:43.839
they're having scaling issues, right? If

00:14:43.839 --> 00:14:45.600
they are starting to have more teams,

00:14:45.600 --> 00:14:47.120
they're starting to slow down in

00:14:47.120 --> 00:14:48.880
shipping features or they have

00:14:48.880 --> 00:14:50.680
compliance requirements. And now

00:14:50.680 --> 00:14:52.880
suddenly they need to be SOC 2 compliant

00:14:52.880 --> 00:14:54.839
or whatever and then they realize that

00:14:54.839 --> 00:14:56.440
every team is doing their own thing. One

00:14:56.440 --> 00:14:58.760
team's running on Google Cloud with

00:14:58.760 --> 00:15:00.120
their

00:15:00.120 --> 00:15:01.760
analytical database and then another

00:15:01.760 --> 00:15:04.200
team is using SageMaker on AWS and

00:15:04.200 --> 00:15:06.079
nobody has any control and now suddenly

00:15:06.079 --> 00:15:08.640
they need to run all like control access

00:15:08.640 --> 00:15:10.200
to production data and they need to roll

00:15:10.200 --> 00:15:11.640
out uh privileged access management

00:15:11.640 --> 00:15:13.720
solutions. And how do you do that like

00:15:13.720 --> 00:15:16.280
from from um like a platform team

00:15:16.280 --> 00:15:17.720
perspective or DevOps? Like, if you're

00:15:17.720 --> 00:15:19.560
embedded within the small teams. So, I

00:15:19.560 --> 00:15:19.959
said like

00:15:19.959 --> 00:15:21.800
>> have infrastructure as code and like,

00:15:21.800 --> 00:15:23.400
well, what what is this Terraform stuff?

00:15:23.400 --> 00:15:24.480
You got to teach them.

00:15:24.480 --> 00:15:26.360
>> So, this was my answer. I started saying

00:15:26.360 --> 00:15:28.160
like you need to find a balance between

00:15:28.160 --> 00:15:29.920
depending because I was talking to a

00:15:29.920 --> 00:15:32.320
sensitive industry. So, I think your

00:15:32.320 --> 00:15:34.000
compliance was important for them and I

00:15:34.000 --> 00:15:35.680
said this is actually quite a big of a

00:15:35.680 --> 00:15:37.240
trouble if you're trying to do this on a

00:15:37.240 --> 00:15:39.160
startup scale. But then I I really

00:15:39.160 --> 00:15:41.040
regret because over the course of the

00:15:41.040 --> 00:15:43.520
conversation, what I really think I

00:15:43.520 --> 00:15:45.280
learned after, you know, the call, I

00:15:45.280 --> 00:15:46.959
stopped and then I thought like, you

00:15:46.959 --> 00:15:48.520
know, what I really think I learned is

00:15:48.520 --> 00:15:50.600
that now you have so like, what is the

00:15:50.600 --> 00:15:52.320
thing that you usually miss when you go

00:15:52.320 --> 00:15:54.240
into an organization? They hire you,

00:15:54.240 --> 00:15:55.920
they say "We want to be able to stand up

00:15:55.920 --> 00:15:57.760
an ephemeral environment. We want to be

00:15:57.760 --> 00:15:59.720
able to test if new feature development

00:15:59.720 --> 00:16:01.760
completely individually and stand up

00:16:01.760 --> 00:16:03.200
everything." And then you're like,

00:16:03.200 --> 00:16:05.120
"Yeah, but do you have database seats?

00:16:05.120 --> 00:16:06.760
Do you have the ability to stand up an

00:16:06.760 --> 00:16:08.760
environment on its own?" Like, all of

00:16:08.760 --> 00:16:11.200
this is usually missing. I don't know. I

00:16:11.200 --> 00:16:12.760
don't know in your client engagements,

00:16:12.760 --> 00:16:14.800
but most of the organizations I joined

00:16:14.800 --> 00:16:16.680
have absolutely no way of creating an

00:16:16.680 --> 00:16:18.800
ephemeral environment or doing an

00:16:18.800 --> 00:16:21.000
integration testing environment.

00:16:21.000 --> 00:16:23.120
And and and I it did kind of come out of

00:16:23.120 --> 00:16:25.120
the whole conversation because I did say

00:16:25.120 --> 00:16:26.920
like if today with AI adoption and AI

00:16:26.920 --> 00:16:29.280
agents having the ability to create an

00:16:29.280 --> 00:16:30.680
ephemeral environment or to do an

00:16:30.680 --> 00:16:32.200
end-to-end integration test or to set up

00:16:32.200 --> 00:16:34.280
your Playwright scripts is absolutely

00:16:34.280 --> 00:16:35.880
crucial. And it's also the first thing

00:16:35.880 --> 00:16:37.640
and we talked about this. It's the first

00:16:37.640 --> 00:16:39.200
thing that you can employ AI agents on

00:16:39.200 --> 00:16:41.720
to brownfield environment. So, this is

00:16:41.720 --> 00:16:43.800
what I should have answered and I regret

00:16:43.800 --> 00:16:44.280
after the call.

00:16:44.280 --> 00:16:45.480
>> I hate it when someone asks you a

00:16:45.480 --> 00:16:48.760
question and then you know you've asked

00:16:48.760 --> 00:16:51.400
answered it wrong 10 minutes or in the

00:16:51.400 --> 00:16:53.200
middle of the night is for me rather.

00:16:53.200 --> 00:16:54.720
Okay, I need to take my kids to school.

00:16:54.720 --> 00:16:56.839
Maybe we can resume the conversation,

00:16:56.839 --> 00:16:58.240
but yeah, I need to take my kids to

00:16:58.240 --> 00:16:59.920
school now. So, it's a 20-20 minute

00:16:59.920 --> 00:17:02.040
call. I'll catch up with you later. Bye.

00:17:02.040 --> 00:17:03.760
So, are you having off? Are you having

00:17:03.760 --> 00:17:05.600
off, sorry? For the weekend? I mean, for

00:17:05.600 --> 00:17:08.240
like Labor Day? Labor Day? First of May?

00:17:08.240 --> 00:17:10.160
Next week? I'm not [clears throat] sure,

00:17:10.160 --> 00:17:11.679
actually.

00:17:11.679 --> 00:17:13.640
I should

00:17:13.640 --> 00:17:16.320
UK holidays?

00:17:16.320 --> 00:17:18.760
Um

00:17:18.760 --> 00:17:21.199
No, I think it's 25th of May is the next

00:17:21.199 --> 00:17:24.520
holiday in in the UK. Yeah. In In

00:17:24.520 --> 00:17:26.640
Vietnam, 30th of April is King Hung

00:17:26.640 --> 00:17:28.960
holiday and 1st of May is Labor Day. I

00:17:28.960 --> 00:17:32.200
think in a lot of the ex communist

00:17:32.200 --> 00:17:34.480
countries, Labor Day is very important.

00:17:34.480 --> 00:17:36.400
It's a very important Russian kind of

00:17:36.400 --> 00:17:37.200
celebration.

00:17:37.200 --> 00:17:39.800
>> Russian? A USSR type of celebration.

00:17:39.800 --> 00:17:41.960
>> Yeah, I sometimes think to myself like

00:17:41.960 --> 00:17:44.400
with the with AI and what does labor

00:17:44.400 --> 00:17:46.040
mean?

00:17:46.040 --> 00:17:47.640
There doesn't seem to be any of that

00:17:47.640 --> 00:17:49.200
highbrow

00:17:49.200 --> 00:17:51.600
>> question. There There doesn't seem to

00:17:51.600 --> 00:17:54.120
highbrow political philosophy being

00:17:54.120 --> 00:17:56.880
applied to AI, but like in our industry,

00:17:56.880 --> 00:17:59.800
the churn or the uh

00:17:59.800 --> 00:18:01.880
the fear is real. Like, people losing

00:18:01.880 --> 00:18:03.600
their jobs and everything like that.

00:18:03.600 --> 00:18:06.800
It's it's pretty real. The workers uh

00:18:06.800 --> 00:18:09.679
rights seem to be not on the forefront

00:18:09.679 --> 00:18:12.040
of discussions.

00:18:12.040 --> 00:18:12.960
Yeah. They're being

00:18:12.960 --> 00:18:15.200
>> Yeah, it's tough it's tough these times.

00:18:15.200 --> 00:18:16.600
There was a post on Reddit the other day

00:18:16.600 --> 00:18:18.640
of an engineering manager saying he's

00:18:18.640 --> 00:18:20.679
part of these organizations that are of

00:18:20.679 --> 00:18:22.679
course claiming productivity gains with

00:18:22.679 --> 00:18:25.320
the layoffs and adoption of AI, but that

00:18:25.320 --> 00:18:27.400
there is a resurgence in hiring senior

00:18:27.400 --> 00:18:31.080
engineers because AI turns out to be

00:18:31.080 --> 00:18:33.400
harder to control than they imagine. And

00:18:33.400 --> 00:18:34.919
then a second thing that he mentioned

00:18:34.919 --> 00:18:37.640
was that a lot of the employees are not

00:18:37.640 --> 00:18:40.919
happy, are you know, using AI to do the

00:18:40.919 --> 00:18:42.960
work because they have to and they're

00:18:42.960 --> 00:18:44.360
not resigning because they're afraid

00:18:44.360 --> 00:18:46.000
that they can't get another job. So,

00:18:46.000 --> 00:18:47.919
they're all just begrudgingly like

00:18:47.919 --> 00:18:49.640
burning out and

00:18:49.640 --> 00:18:51.800
ending AI. Yeah.

00:18:51.800 --> 00:18:52.960
I

00:18:52.960 --> 00:18:56.000
There's a part of me that that loves AI

00:18:56.000 --> 00:18:58.040
because, you know, I think it's new and

00:18:58.040 --> 00:19:00.679
cool and it's really fun. But, as a

00:19:00.679 --> 00:19:03.760
47-year-old experienced dev and just I

00:19:03.760 --> 00:19:05.760
guess talking with my peers, I guess

00:19:05.760 --> 00:19:09.200
that the fear does trickle down. The FUD

00:19:09.200 --> 00:19:11.520
does affect me. For me, like I said,

00:19:11.520 --> 00:19:14.080
I've had very positive and very scary

00:19:14.080 --> 00:19:16.120
experiences. Like, I have half of my

00:19:16.120 --> 00:19:19.200
brain afraid sometimes where it I feel

00:19:19.200 --> 00:19:20.960
like AI is going off on its own and I'm

00:19:20.960 --> 00:19:23.679
like, yeah, I'm not 100% sure what

00:19:23.679 --> 00:19:25.520
you're doing, but makes sense to me.

00:19:25.520 --> 00:19:27.679
But, I've also had scenarios where like

00:19:27.679 --> 00:19:30.440
I told you, there was this report of a

00:19:30.440 --> 00:19:32.800
problem and I saw that the screenshot

00:19:32.800 --> 00:19:34.880
was shared and I I had a cursory glance

00:19:34.880 --> 00:19:36.440
at the screenshot and it was in an area

00:19:36.440 --> 00:19:38.520
that I had worked on for redacting

00:19:38.520 --> 00:19:40.720
sensitive information from Datadog agent

00:19:40.720 --> 00:19:44.040
logs. And I had set up like log

00:19:44.040 --> 00:19:46.880
redaction like regular patterns on the

00:19:46.880 --> 00:19:50.080
agents to to prevent this issue and

00:19:50.080 --> 00:19:51.440
apparently some of the logs were not

00:19:51.440 --> 00:19:53.800
redacted. So, I had assumed that it was

00:19:53.800 --> 00:19:56.080
in the area that I had worked on. But

00:19:56.080 --> 00:19:58.160
the screenshot showed So I worked on

00:19:58.160 --> 00:20:01.720
like an EC2 instance deployed agent and

00:20:01.720 --> 00:20:04.080
I I told the AI give it all the context

00:20:04.080 --> 00:20:06.160
that I have like hey there's these log

00:20:06.160 --> 00:20:07.720
forwarding

00:20:07.720 --> 00:20:10.200
um regular expressions to detect the

00:20:10.200 --> 00:20:12.680
type of sensitive information to redact

00:20:12.680 --> 00:20:14.760
it and so on. They're in this area,

00:20:14.760 --> 00:20:17.760
they're in that area and Opus went and

00:20:17.760 --> 00:20:19.800
said well the screenshot shows that the

00:20:19.800 --> 00:20:21.720
agent on the EC2 instance is redacting

00:20:21.720 --> 00:20:23.600
the data correctly. The other screenshot

00:20:23.600 --> 00:20:25.400
is from a lambda. So it's coming from

00:20:25.400 --> 00:20:26.720
the data dog lambda forwarder

00:20:26.720 --> 00:20:29.000
integration in your AWS account. Which I

00:20:29.000 --> 00:20:31.200
had totally not given any context about.

00:20:31.200 --> 00:20:33.240
And it had gone ahead and said like I

00:20:33.240 --> 00:20:35.560
can see that you have a Terraform module

00:20:35.560 --> 00:20:38.240
that manages your data dog integrations

00:20:38.240 --> 00:20:39.960
and it's deployed across all these AWS

00:20:39.960 --> 00:20:42.240
accounts. It's straight passing through

00:20:42.240 --> 00:20:44.480
the cloud formation stack configuration.

00:20:44.480 --> 00:20:46.000
So there's a Terraform variable for

00:20:46.000 --> 00:20:48.280
every cloud formation stack input and

00:20:48.280 --> 00:20:49.800
there's a parameter that allows you to

00:20:49.800 --> 00:20:51.880
set the data dog lambda forwarder

00:20:51.880 --> 00:20:54.080
redaction like regular expressions as

00:20:54.080 --> 00:20:56.720
well. So not only did it like identify

00:20:56.720 --> 00:20:58.240
that I was giving it the completely

00:20:58.240 --> 00:21:00.040
wrong direction. Cuz I told it it's

00:21:00.040 --> 00:21:02.280
agents deployed on a instances and it

00:21:02.280 --> 00:21:03.840
says no it's the data dog lambda

00:21:03.840 --> 00:21:05.520
forwarder that's deployed inside your

00:21:05.520 --> 00:21:06.640
AWS account.

00:21:06.640 --> 00:21:08.960
>> because of of its efficacy sort of

00:21:08.960 --> 00:21:10.480
thing. No in this case I wasn't scared.

00:21:10.480 --> 00:21:11.840
It's like I said I have scary

00:21:11.840 --> 00:21:13.440
experiences where I have it handed it

00:21:13.440 --> 00:21:15.480
off and I have no idea and I have

00:21:15.480 --> 00:21:17.040
wonderful experiences where I give it

00:21:17.040 --> 00:21:18.560
the wrong direction and it comes back to

00:21:18.560 --> 00:21:20.840
me pushes back against me and the only

00:21:20.840 --> 00:21:22.360
and also goes a step further

00:21:22.360 --> 00:21:24.360
highlighting exactly what I need to do.

00:21:24.360 --> 00:21:26.520
So we I know and I know this very well

00:21:26.520 --> 00:21:28.040
because I've done all of this Terraform

00:21:28.040 --> 00:21:30.560
configuration. So everything it said I

00:21:30.560 --> 00:21:34.000
knew 100% was correct because I had

00:21:34.000 --> 00:21:35.880
written that and carefully cross it like

00:21:35.880 --> 00:21:38.120
over the last 3 years. So I knew very

00:21:38.120 --> 00:21:39.800
well what it was talking about. I didn't

00:21:39.800 --> 00:21:41.480
know that there was a variable for the

00:21:41.480 --> 00:21:42.880
cloud formation stack to configure your

00:21:42.880 --> 00:21:44.960
data dog lambda forwarders. Obviously

00:21:44.960 --> 00:21:46.760
the problem is in the lambda's logging

00:21:46.760 --> 00:21:49.280
sensitive details to cloud watch cuz

00:21:49.280 --> 00:21:50.960
forwarding them and then removing them

00:21:50.960 --> 00:21:52.720
from data dog doesn't remove them from

00:21:52.720 --> 00:21:54.360
cloud watch. They're still there, right?

00:21:54.360 --> 00:21:56.760
So that's the second And then again the

00:21:56.760 --> 00:21:58.320
agent and this is what I was thinking it

00:21:58.320 --> 00:21:59.760
helps to know the terminology because

00:21:59.760 --> 00:22:02.240
the agent was like as a defense in death

00:22:02.240 --> 00:22:03.760
we're going to make sure that the

00:22:03.760 --> 00:22:05.520
forwarder doesn't send the sensitive

00:22:05.520 --> 00:22:07.679
data. But the real fix is obviously to

00:22:07.679 --> 00:22:09.280
go back to the product teams and these

00:22:09.280 --> 00:22:10.679
particular lambda's like it even

00:22:10.679 --> 00:22:12.679
highlighted these are the lambda's

00:22:12.679 --> 00:22:15.240
writing sensitive data. These are the

00:22:15.240 --> 00:22:16.520
ones that you need to go back to the

00:22:16.520 --> 00:22:18.480
product teams to tell them not to log

00:22:18.480 --> 00:22:20.320
this data. Because they're still in

00:22:20.320 --> 00:22:22.400
cloud watch. But we're also going to

00:22:22.400 --> 00:22:24.200
deploy a fix so that they're not being

00:22:24.200 --> 00:22:25.600
indexed. So it's like

00:22:25.600 --> 00:22:27.679
>> But but it's getting to a stage where

00:22:27.679 --> 00:22:29.880
where it's going above and beyond and

00:22:29.880 --> 00:22:32.080
it's becoming the master of us in a way.

00:22:32.080 --> 00:22:33.640
It's like you didn't think about this

00:22:33.640 --> 00:22:36.720
you know like like you become the junior

00:22:36.720 --> 00:22:39.480
getting schooled by the AI. And and and

00:22:39.480 --> 00:22:41.280
that's where you you're easily then hand

00:22:41.280 --> 00:22:43.240
off. Like in this particular case it

00:22:43.240 --> 00:22:45.040
proven above and beyond that it knows

00:22:45.040 --> 00:22:46.480
exactly what it's doing. Because I was

00:22:46.480 --> 00:22:48.600
very well aware of what is the setup and

00:22:48.600 --> 00:22:50.520
I know that what it says is correct. And

00:22:50.520 --> 00:22:52.040
I was telling it it asked me a couple of

00:22:52.040 --> 00:22:53.520
clarifying questions. Do you want me to

00:22:53.520 --> 00:22:55.360
deploy this first to dev and then

00:22:55.360 --> 00:22:56.920
propagate it? And I was like yes we need

00:22:56.920 --> 00:22:58.840
to propagate it. And then we have like

00:22:58.840 --> 00:23:00.679
what I I call trunk based development

00:23:00.679 --> 00:23:03.000
with Terraform modules. So we have these

00:23:03.000 --> 00:23:06.480
environments using the live module like

00:23:06.480 --> 00:23:08.720
from head of branch. Trunk based

00:23:08.720 --> 00:23:11.360
basically. So if I'm bumping because I

00:23:11.360 --> 00:23:13.280
have two problems. One is the cloud

00:23:13.280 --> 00:23:14.679
formation stack that deploys the lambda

00:23:14.679 --> 00:23:16.920
needs to be updated to include the

00:23:16.920 --> 00:23:19.080
regular expression to mask out sensitive

00:23:19.080 --> 00:23:21.200
data so that the lambda doesn't send the

00:23:21.200 --> 00:23:23.040
sensitive data down to the indexing in

00:23:23.040 --> 00:23:25.160
data dog's third party API. And then the

00:23:25.160 --> 00:23:27.160
second thing is I have a runtime. So

00:23:27.160 --> 00:23:29.760
this has been left to a little bit on

00:23:29.760 --> 00:23:31.360
its own and it needs to be updated and

00:23:31.360 --> 00:23:32.880
there's a lambda runtime update that

00:23:32.880 --> 00:23:35.240
needs to happen. So I told it we need to

00:23:35.240 --> 00:23:37.280
do this as well and it was like yeah but

00:23:37.280 --> 00:23:38.560
that's a separate issue like you

00:23:38.560 --> 00:23:39.679
shouldn't put this in the same pull

00:23:39.679 --> 00:23:41.520
request, right? We need to keep this

00:23:41.520 --> 00:23:44.000
dedicated on the on the issue at hand.

00:23:44.000 --> 00:23:46.560
And I was like yes but if we're going to

00:23:46.560 --> 00:23:49.000
make a change to the lambda to the

00:23:49.000 --> 00:23:51.120
cloud formation stack the runtime is

00:23:51.120 --> 00:23:53.040
going to be blocked, right? If we make a

00:23:53.040 --> 00:23:54.600
reconfiguration it's going to redeploy

00:23:54.600 --> 00:23:55.679
the lambda the lambda's going to be

00:23:55.679 --> 00:23:57.360
blocked because the Python runtime ran

00:23:57.360 --> 00:23:59.960
out end of life. After a certain support

00:23:59.960 --> 00:24:02.840
period lambda will block any roll out.

00:24:02.840 --> 00:24:04.440
It will keep the lambda's running and it

00:24:04.440 --> 00:24:05.960
will not provide any security patches

00:24:05.960 --> 00:24:07.440
for them. But if you try to make a

00:24:07.440 --> 00:24:09.840
change it will fail. So I mean I didn't

00:24:09.840 --> 00:24:11.520
even say it in that many details. I said

00:24:11.520 --> 00:24:13.000
we need to update the runtime and then

00:24:13.000 --> 00:24:15.920
it says based on the research it did

00:24:15.920 --> 00:24:17.320
says like yes you're right because the

00:24:17.320 --> 00:24:19.600
runtime is too old and if we try to make

00:24:19.600 --> 00:24:21.160
a change to cloud formation it's going

00:24:21.160 --> 00:24:23.080
to fail because the lambda runtime will

00:24:23.080 --> 00:24:24.600
be blocked. Like we won't be able to

00:24:24.600 --> 00:24:26.640
reconfigure the lambda and and we need

00:24:26.640 --> 00:24:28.400
to update the runtime as well. So we

00:24:28.400 --> 00:24:29.800
will include it into this pull request

00:24:29.800 --> 00:24:31.320
as well. So it's always very careful

00:24:31.320 --> 00:24:33.640
like are we going to scope our work?

00:24:33.640 --> 00:24:35.280
Like are we going to keep it within the

00:24:35.280 --> 00:24:37.440
same scope? Because actually Opus now is

00:24:37.440 --> 00:24:40.080
very aggressive in like this is not my

00:24:40.080 --> 00:24:41.840
problem. This is not related to my

00:24:41.840 --> 00:24:43.480
change and I'm not going to make a

00:24:43.480 --> 00:24:45.160
change to that. And then I have to push

00:24:45.160 --> 00:24:47.480
it. I say actually in this case I want

00:24:47.480 --> 00:24:50.480
you to also do that on top of that. And

00:24:50.480 --> 00:24:53.240
I like it. I like it. I've noticed that

00:24:53.240 --> 00:24:56.520
in my last vibing session where it was

00:24:56.520 --> 00:24:57.560
like

00:24:57.560 --> 00:24:58.920
like I'm going to set up a different

00:24:58.920 --> 00:25:01.320
issue for this and like no and then we

00:25:01.320 --> 00:25:03.520
had this little fight. It's like what

00:25:03.520 --> 00:25:05.840
the what's going on here? Yeah but I

00:25:05.840 --> 00:25:07.400
mean this is a perfect scenario where

00:25:07.400 --> 00:25:08.960
the models have become and the harness

00:25:08.960 --> 00:25:11.200
is so good that it's giving it system

00:25:11.200 --> 00:25:13.320
instructions the model is defensive and

00:25:13.320 --> 00:25:15.080
is able to

00:25:15.080 --> 00:25:16.679
you know do this in a vibe

00:25:16.679 --> 00:25:18.960
vibing manner, right? I on one side I I

00:25:18.960 --> 00:25:21.320
have a Jira ticket that records the

00:25:21.320 --> 00:25:23.040
issue and I told it like I pasted the

00:25:23.040 --> 00:25:24.880
screenshot in this chat and I was and I

00:25:24.880 --> 00:25:26.960
know that it basic C4 encodes your

00:25:26.960 --> 00:25:28.720
screenshots and stores them inside JSON

00:25:28.720 --> 00:25:30.400
as well. And I was like can you take the

00:25:30.400 --> 00:25:32.320
screenshot that I sent to you and put

00:25:32.320 --> 00:25:34.679
them inside there? And it says no

00:25:34.679 --> 00:25:36.679
because there is a leaked password in it

00:25:36.679 --> 00:25:37.840
and I'm not going to post the leaked

00:25:37.840 --> 00:25:39.679
password on the Jira ticket. And I was

00:25:39.679 --> 00:25:42.200
like god damn. So I then opened the

00:25:42.200 --> 00:25:44.280
screenshot marked redacted the leaked

00:25:44.280 --> 00:25:46.160
password and I said I'll paste it it

00:25:46.160 --> 00:25:47.360
myself.

00:25:47.360 --> 00:25:49.160
That is crazy.

00:25:49.160 --> 00:25:50.640
It is crazy, right? I mean they made it

00:25:50.640 --> 00:25:52.120
a lot more defensive.

00:25:52.120 --> 00:25:54.800
Okay let's maybe talk about one thing

00:25:54.800 --> 00:25:57.160
that came up on my radar was this whole

00:25:57.160 --> 00:26:00.240
new lever with with saying

00:26:00.240 --> 00:26:01.840
the thinking level. Is it the right

00:26:01.840 --> 00:26:04.160
term? Yeah they added extra high high

00:26:04.160 --> 00:26:07.280
now in Or they have max now which is

00:26:07.280 --> 00:26:08.600
>> Or the ultra think it's been there for a

00:26:08.600 --> 00:26:09.760
while, right? If you type it then it

00:26:09.760 --> 00:26:11.280
becomes a regular

00:26:11.280 --> 00:26:12.280
>> ultra think.

00:26:12.280 --> 00:26:14.320
>> Ultra plan is like a second planning

00:26:14.320 --> 00:26:16.200
session. I wish I kind of wish that they

00:26:16.200 --> 00:26:18.920
didn't introduce these variables because

00:26:18.920 --> 00:26:20.840
like because in that blog post where

00:26:20.840 --> 00:26:23.240
they talk about how they messed up. I

00:26:23.240 --> 00:26:24.960
don't know if you saw that one. What one

00:26:24.960 --> 00:26:27.080
of the issues was that they basically

00:26:27.080 --> 00:26:29.000
changed the

00:26:29.000 --> 00:26:31.679
the thinking level from high to medium

00:26:31.679 --> 00:26:33.679
or something. And yeah like isn't there

00:26:33.679 --> 00:26:35.760
like an adage when it comes to software

00:26:35.760 --> 00:26:38.080
engineering? Like the more freaking

00:26:38.080 --> 00:26:40.080
variables you give the more the more

00:26:40.080 --> 00:26:42.120
surface area your API has the more

00:26:42.120 --> 00:26:43.720
likely things are going to go wrong. I

00:26:43.720 --> 00:26:45.600
mean

00:26:45.600 --> 00:26:47.360
if they can't get it right like what is

00:26:47.360 --> 00:26:49.280
it what is it going to be it's like how

00:26:49.280 --> 00:26:51.760
many developer teams are going to be

00:26:51.760 --> 00:26:55.360
perhaps just confused and befuddled by

00:26:55.360 --> 00:26:58.240
someone who specified extra high versus

00:26:58.240 --> 00:27:00.480
high versus medium. I mean it's going to

00:27:00.480 --> 00:27:02.080
going to have probably different results

00:27:02.080 --> 00:27:03.200
and everyone's just going to be in

00:27:03.200 --> 00:27:04.840
disarray. It's going to be a mess.

00:27:04.840 --> 00:27:06.480
That's what a lot of people are saying,

00:27:06.480 --> 00:27:08.800
right? I haven't read but I've seen it

00:27:08.800 --> 00:27:11.240
posted like 20 times on LinkedIn and

00:27:11.240 --> 00:27:13.320
Reddit every single one. And Anthropic

00:27:13.320 --> 00:27:15.480
has released a postmortem on their

00:27:15.480 --> 00:27:18.000
degradation on why performance was bad

00:27:18.000 --> 00:27:20.040
and and they talked about the caching

00:27:20.040 --> 00:27:22.080
issues that we talked to you community

00:27:22.080 --> 00:27:23.800
kind of highlighted on its own. I

00:27:23.800 --> 00:27:26.320
honestly I think I've been I've been

00:27:26.320 --> 00:27:28.400
vibe coding a lot with it because of all

00:27:28.400 --> 00:27:30.440
of this like concerns that people have

00:27:30.440 --> 00:27:31.880
been raising. So I've been a little bit

00:27:31.880 --> 00:27:34.000
more hands-on with like small little

00:27:34.000 --> 00:27:36.840
vibe plans instead of like long long

00:27:36.840 --> 00:27:40.080
lift um I'm not giving it a 20 task. One

00:27:40.080 --> 00:27:41.840
of the issues was that it was it was

00:27:41.840 --> 00:27:43.800
clearing up its context from from

00:27:43.800 --> 00:27:45.320
previous memories. But that wouldn't I

00:27:45.320 --> 00:27:46.560
don't think that would have affect

00:27:46.560 --> 00:27:48.800
affect me with beads. Yeah but I feel

00:27:48.800 --> 00:27:50.000
like there's

00:27:50.000 --> 00:27:52.440
there's a whole bunch of schadenfreude.

00:27:52.440 --> 00:27:54.640
Like people are going overboard. I'm

00:27:54.640 --> 00:27:56.200
really tired of the community just

00:27:56.200 --> 00:27:58.800
constantly complaining. Like back in

00:27:58.800 --> 00:28:00.880
November you were barely able you felt

00:28:00.880 --> 00:28:02.600
like you couldn't use it and then you

00:28:02.600 --> 00:28:04.720
said December was amazing. Like I was on

00:28:04.720 --> 00:28:06.679
a honeymoon and it was crazy and then

00:28:06.679 --> 00:28:08.640
everybody has these these what do you

00:28:08.640 --> 00:28:10.720
call nostalgic posts about I'm

00:28:10.720 --> 00:28:13.040
reminiscing about how amazing my January

00:28:13.040 --> 00:28:16.320
February life was. I I don't care. I

00:28:16.320 --> 00:28:18.160
think I think it's I think it's very

00:28:18.160 --> 00:28:20.920
human in a way to complain. I think it's

00:28:20.920 --> 00:28:23.320
not necessarily the worst thing ever as

00:28:23.320 --> 00:28:25.720
long as it's obviously constructive. And

00:28:25.720 --> 00:28:27.159
I think it has been constructive for the

00:28:27.159 --> 00:28:29.960
most part. Isn't it? I don't know. Like

00:28:29.960 --> 00:28:31.760
I mean there's a few constructive posts

00:28:31.760 --> 00:28:33.919
like where the director of AMD has some

00:28:33.919 --> 00:28:35.280
factual data.

00:28:35.280 --> 00:28:36.800
Some of the GitHub issues that have been

00:28:36.800 --> 00:28:38.560
linked with the Anthropic team responds

00:28:38.560 --> 00:28:41.000
to. They have been very technical and

00:28:41.000 --> 00:28:43.480
detailed. This is also the feedback of

00:28:43.480 --> 00:28:45.400
Hashimoto, right? He says like he's

00:28:45.400 --> 00:28:47.280
getting technical reports by people that

00:28:47.280 --> 00:28:49.800
never wrote Zig's programming language

00:28:49.800 --> 00:28:52.240
but because of the AI is able to surface

00:28:52.240 --> 00:28:53.840
all of these details. They're now

00:28:53.840 --> 00:28:56.159
getting these super detailed bug reports

00:28:56.159 --> 00:28:57.440
and I think Anthropic is getting the

00:28:57.440 --> 00:28:59.400
same. But I think at the same time

00:28:59.400 --> 00:29:00.840
Anthropic is also pushing back. They're

00:29:00.840 --> 00:29:02.560
like well you know a lot of these

00:29:02.560 --> 00:29:04.840
extended thinking or chain of thoughts

00:29:04.840 --> 00:29:06.320
reasoning are not part of your

00:29:06.320 --> 00:29:08.760
issues.json. So you're saying that the

00:29:08.760 --> 00:29:11.000
model is like being overcharged in token

00:29:11.000 --> 00:29:13.080
usage but all of those thinking tokens

00:29:13.080 --> 00:29:14.520
are spent on the service side and you're

00:29:14.520 --> 00:29:15.960
not seeing the full picture. They're

00:29:15.960 --> 00:29:19.080
having these AIs run loose on client

00:29:19.080 --> 00:29:21.280
side information that they have and then

00:29:21.280 --> 00:29:22.679
of course the AI is great in making

00:29:22.679 --> 00:29:24.159
assumptions and comes up with all of

00:29:24.159 --> 00:29:26.400
these stories of why it's going wrong

00:29:26.400 --> 00:29:28.240
and then they have to go and debunk and

00:29:28.240 --> 00:29:30.280
every single one of them. I mean yes

00:29:30.280 --> 00:29:32.440
there's there's signal and there's noise

00:29:32.440 --> 00:29:35.159
and I think that that's the hard bit for

00:29:35.159 --> 00:29:37.440
Anthropic to go and filter out and

00:29:37.440 --> 00:29:40.240
correlate what the community is saying.

00:29:40.240 --> 00:29:41.800
And then of course there's a whole slew

00:29:41.800 --> 00:29:43.679
of people that are saying well it's all

00:29:43.679 --> 00:29:46.320
slop it's all AI code that Anthropic

00:29:46.320 --> 00:29:47.360
are doing.

00:29:47.360 --> 00:29:48.440
>> to I to

00:29:48.440 --> 00:29:50.240
like think the best of people's

00:29:50.240 --> 00:29:52.640
intentions in a way. Because as we as we

00:29:52.640 --> 00:29:54.120
started, you know, we were talking about

00:29:54.120 --> 00:29:56.280
Workers Day. I mean, we're we're humans,

00:29:56.280 --> 00:29:58.760
we're workers, we have feelings, and AI

00:29:58.760 --> 00:30:01.040
is moving so quickly. It it can't I

00:30:01.040 --> 00:30:02.400
think we're in a very privileged

00:30:02.400 --> 00:30:04.440
position because we can experiment a bit

00:30:04.440 --> 00:30:05.440
in our

00:30:05.440 --> 00:30:07.920
>> Yeah. uh AI AI infrastructure field. A

00:30:07.920 --> 00:30:10.760
lot of people are perhaps relying on the

00:30:10.760 --> 00:30:12.840
Claude code for their business, and then

00:30:12.840 --> 00:30:14.400
of course they might just go, "Well,

00:30:14.400 --> 00:30:16.080
actually this this makes me think like

00:30:16.080 --> 00:30:17.920
some people do complain

00:30:17.920 --> 00:30:18.560
at

00:30:18.560 --> 00:30:20.040
um

00:30:20.040 --> 00:30:22.960
a little bit too ferociously. And I

00:30:22.960 --> 00:30:24.760
sometimes think to myself like, "I wish

00:30:24.760 --> 00:30:26.120
they gave the context of what they're

00:30:26.120 --> 00:30:27.640
trying to do." Like, you know, "I want

00:30:27.640 --> 00:30:29.520
to I want a refund. This is balls."

00:30:29.520 --> 00:30:31.800
Like, what what what were you doing? I

00:30:31.800 --> 00:30:33.160
was What were you doing that was so

00:30:33.160 --> 00:30:34.080
important and time-sensitive? I had

00:30:34.080 --> 00:30:35.760
three accounts and I cancelled them all.

00:30:35.760 --> 00:30:38.320
I'm using ChatGPT now. Yeah, okay.

00:30:38.320 --> 00:30:39.800
>> Like, what were you doing that was so

00:30:39.800 --> 00:30:42.040
important and time-sensitive? Like, what

00:30:42.040 --> 00:30:43.680
were you what Like, are you are you

00:30:43.680 --> 00:30:46.040
working on assignment with Like, what

00:30:46.040 --> 00:30:48.160
what Like, I see at the bottom of their

00:30:48.160 --> 00:30:49.640
post they have that that /

00:30:49.640 --> 00:30:52.040
feedback command. I I do I do think as

00:30:52.040 --> 00:30:55.640
humans it's like we we need to like have

00:30:55.640 --> 00:30:57.600
some norms when it comes to

00:30:57.600 --> 00:30:59.280
>> Well, I think what people don't realize

00:30:59.280 --> 00:31:01.640
and also if you look at how Anthropic is

00:31:01.640 --> 00:31:03.360
controlling access to these

00:31:03.360 --> 00:31:05.360
subscriptions is that they don't realize

00:31:05.360 --> 00:31:07.160
how heavily subsidized this is, right? I

00:31:07.160 --> 00:31:08.280
mean, this is something that we've

00:31:08.280 --> 00:31:10.600
always been saying. These subscriptions

00:31:10.600 --> 00:31:12.240
we're in an in the heyday of AI,

00:31:12.240 --> 00:31:14.400
everything is subsidized. Right now you

00:31:14.400 --> 00:31:16.520
get for a certain amount of money you

00:31:16.520 --> 00:31:19.400
get a certain service that you know, a

00:31:19.400 --> 00:31:21.840
long time was the the best of the best

00:31:21.840 --> 00:31:24.400
AI model available. Open AI obviously is

00:31:24.400 --> 00:31:26.720
trying to undercut them and is trying to

00:31:26.720 --> 00:31:29.960
offer more access, more tokens, maybe

00:31:29.960 --> 00:31:32.320
throws more raw power to it to to to

00:31:32.320 --> 00:31:34.960
lure you away. You're you're getting all

00:31:34.960 --> 00:31:37.120
these Chinese and then Google sometimes

00:31:37.120 --> 00:31:39.200
coming in with with with a some some

00:31:39.200 --> 00:31:40.840
solution to try and lure away some of

00:31:40.840 --> 00:31:42.880
that accounts so that they can talk to

00:31:42.880 --> 00:31:44.800
the board of how many subscriptions that

00:31:44.800 --> 00:31:47.120
they have and increased the subscription

00:31:47.120 --> 00:31:49.800
count. But people are just extremely

00:31:49.800 --> 00:31:52.360
spoiled right now and acting like,

00:31:52.360 --> 00:31:54.080
"Yeah, I mean, I've you know, I've been

00:31:54.080 --> 00:31:56.800
able to build my business with this and

00:31:56.800 --> 00:31:58.360
it's completely failed me in the last

00:31:58.360 --> 00:32:00.360
month. It wasn't able to do the simplest

00:32:00.360 --> 00:32:02.080
of things. And of course it's all AI

00:32:02.080 --> 00:32:04.040
slop and I can't do it myself. Not me,

00:32:04.040 --> 00:32:05.720
right? That's what people post on on on

00:32:05.720 --> 00:32:08.000
Reddit. But like

00:32:08.000 --> 00:32:09.480
The thing the thing that bothers me is

00:32:09.480 --> 00:32:12.080
that people make it sound like this

00:32:12.080 --> 00:32:15.880
Claude thing has enabled them to build a

00:32:15.880 --> 00:32:18.240
life support system that you know,

00:32:18.240 --> 00:32:21.560
basically help I don't know some some

00:32:21.560 --> 00:32:24.080
very critical need. But at the same

00:32:24.080 --> 00:32:28.000
time, I don't actually see

00:32:28.000 --> 00:32:30.240
or may maybe I'm probably just building

00:32:30.240 --> 00:32:32.120
a very hypothetical example here, but

00:32:32.120 --> 00:32:34.600
like if Claude code and all this AGI

00:32:34.600 --> 00:32:37.280
stuff is so great, what do we have to

00:32:37.280 --> 00:32:39.200
show for it other than people just

00:32:39.200 --> 00:32:42.400
whining and going mental? Yeah, I mean,

00:32:42.400 --> 00:32:45.000
I think this is one of the aspects of an

00:32:45.000 --> 00:32:46.960
If you're going to rely on an AI, I

00:32:46.960 --> 00:32:48.720
think you have to be realistic that you

00:32:48.720 --> 00:32:51.160
are relying on a subsidized token

00:32:51.160 --> 00:32:54.360
industry that you're at um a privileged

00:32:54.360 --> 00:32:56.360
position. That if it works out for you,

00:32:56.360 --> 00:32:58.120
that's great. I mean, obviously somebody

00:32:58.120 --> 00:32:59.920
has to hold them accountable. They're

00:32:59.920 --> 00:33:01.640
complaining about it's bait and switch.

00:33:01.640 --> 00:33:04.400
You're luring us in. Now they removed

00:33:04.400 --> 00:33:06.679
Claude code from the pro account level.

00:33:06.679 --> 00:33:09.400
>> like can you can you think of a business

00:33:09.400 --> 00:33:12.520
that has like, "Hey, thanks to Claude,

00:33:12.520 --> 00:33:15.720
now we are you know, now we've 10x." I I

00:33:15.720 --> 00:33:18.040
mean, I can't think of a public maybe I

00:33:18.040 --> 00:33:18.560
don't know what to say.

00:33:18.560 --> 00:33:21.000
>> that I wouldn't be able to build so many

00:33:21.000 --> 00:33:23.400
things without all of the Like like I

00:33:23.400 --> 00:33:25.920
said, a lot of the stuff that I I kick

00:33:25.920 --> 00:33:27.720
off with, you know, spec ledgers and

00:33:27.720 --> 00:33:28.160
some specs.

00:33:28.160 --> 00:33:29.720
>> like are you you're not like the

00:33:29.720 --> 00:33:32.120
greatest is all I mean, I don't want to

00:33:32.120 --> 00:33:34.679
put you down, but like like

00:33:34.679 --> 00:33:37.120
I'm not making any business or money out

00:33:37.120 --> 00:33:37.560
of the process.

00:33:37.560 --> 00:33:39.520
>> Yeah, like I I I can't green field fun

00:33:39.520 --> 00:33:40.920
projects for me. Some people say they

00:33:40.920 --> 00:33:43.200
have like 7K recurring rent monthly

00:33:43.200 --> 00:33:44.679
recurring revenue. Yeah, exactly.

00:33:44.679 --> 00:33:46.040
building something small with it. I

00:33:46.040 --> 00:33:48.200
mean, I think the biggest issue is and I

00:33:48.200 --> 00:33:49.600
think what I can see is that those

00:33:49.600 --> 00:33:51.040
people have built something that they

00:33:51.040 --> 00:33:53.840
cannot maintain because they have

00:33:53.840 --> 00:33:55.640
something and they rely heavily on the

00:33:55.640 --> 00:33:57.640
performance of the model to be able to

00:33:57.640 --> 00:33:59.679
maintain it. And and that gets that's

00:33:59.679 --> 00:34:02.560
the point that I wanted you to make or

00:34:02.560 --> 00:34:04.640
us to make. They built something they

00:34:04.640 --> 00:34:06.560
can't maintain. This goes back to what I

00:34:06.560 --> 00:34:08.200
was saying about the AWS summit. When

00:34:08.200 --> 00:34:09.399
you're producing a million lines of

00:34:09.399 --> 00:34:12.159
code, you've built something that you

00:34:12.159 --> 00:34:14.800
can't maintain. On your own like I mean,

00:34:14.800 --> 00:34:16.800
at the same time people are saying like

00:34:16.800 --> 00:34:21.240
Gemma 4 on an M4 Mac Mini or even some

00:34:21.240 --> 00:34:22.760
of the models like Gemma you mean the

00:34:22.760 --> 00:34:24.760
Yeah, Gemma from Google that is now a

00:34:24.760 --> 00:34:26.879
local LLM that you can run.

00:34:26.879 --> 00:34:28.800
>> one you mean? A lot of those self-hosted

00:34:28.800 --> 00:34:30.480
ones like even if suddenly all the

00:34:30.480 --> 00:34:32.960
prices go up, um you know, you cannot

00:34:32.960 --> 00:34:37.080
use Opus 4.7 on an cloud Claude Code or

00:34:37.080 --> 00:34:38.679
cloud pro plan anymore. You can you have

00:34:38.679 --> 00:34:40.919
to pay max you have to be close to $200

00:34:40.919 --> 00:34:43.320
to get access even to a decent model. I

00:34:43.320 --> 00:34:45.919
mean, decent if you want to $200 isn't

00:34:45.919 --> 00:34:47.359
much in the scheme of things, but carry

00:34:47.359 --> 00:34:49.473
on. You're very privileged to say that.

00:34:49.473 --> 00:34:50.200
>> [laughter]

00:34:50.200 --> 00:34:52.200
>> $200 is a lot for a lot of people per

00:34:52.200 --> 00:34:55.120
month just on the AI. That's a lot. But

00:34:55.120 --> 00:34:57.240
come on, what is the average hourly rate

00:34:57.240 --> 00:34:59.320
of a developer? Not that much anymore, I

00:34:59.320 --> 00:35:00.680
would say. I think that's that has

00:35:00.680 --> 00:35:02.040
collapsed. I think

00:35:02.040 --> 00:35:04.040
if you if you go look at at at

00:35:04.040 --> 00:35:06.040
engineering job posts and rates, they

00:35:06.040 --> 00:35:08.760
have really really collapsed a lot. $200

00:35:08.760 --> 00:35:10.880
a month is not nothing. Definitely not

00:35:10.880 --> 00:35:13.320
like in Vietnam or is the

00:35:13.320 --> 00:35:15.040
I'm just typing this into Gemini. So

00:35:15.040 --> 00:35:16.880
anyway, about like building something

00:35:16.880 --> 00:35:19.320
you can't maintain, I do think that's

00:35:19.320 --> 00:35:21.000
absolutely a reality for me. I have a

00:35:21.000 --> 00:35:22.440
lot of projects where I absolutely have

00:35:22.440 --> 00:35:23.960
no idea what is the code look like and I

00:35:23.960 --> 00:35:25.920
don't care because my unit tests pass,

00:35:25.920 --> 00:35:27.400
my integration tests pass, the thing

00:35:27.400 --> 00:35:29.480
works the way I want it to work, and I

00:35:29.480 --> 00:35:32.280
don't I really don't care. But then if

00:35:32.280 --> 00:35:34.120
the Claude code is this, you know, phase

00:35:34.120 --> 00:35:35.480
where I can't really do any work with

00:35:35.480 --> 00:35:37.600
it, then I will be in trouble because I

00:35:37.600 --> 00:35:38.920
can't really change much about it

00:35:38.920 --> 00:35:40.800
because it's so But at the same

00:35:40.800 --> 00:35:42.920
time, I believe that if that's really

00:35:42.920 --> 00:35:45.160
the case that I cannot get done what I

00:35:45.160 --> 00:35:46.440
want to get done and I haven't faced

00:35:46.440 --> 00:35:48.000
that issue to be honest. I have always

00:35:48.000 --> 00:35:49.400
been able to get it to do what I wanted

00:35:49.400 --> 00:35:51.160
to do, so I'm not in that boat at the

00:35:51.160 --> 00:35:52.840
moment and knock wood I don't want to be

00:35:52.840 --> 00:35:54.760
in that boat. I can't believe I'm not

00:35:54.760 --> 00:35:56.400
saying that those testimonies are wrong.

00:35:56.400 --> 00:35:58.440
They probably are in those situations,

00:35:58.440 --> 00:36:01.440
but um I still feel that there will be a

00:36:01.440 --> 00:36:03.760
future because I will be able to use a

00:36:03.760 --> 00:36:05.120
local LLM.

00:36:05.120 --> 00:36:06.920
Um I hear that their performance is

00:36:06.920 --> 00:36:08.720
getting more better and better. I don't

00:36:08.720 --> 00:36:10.000
understand why you're

00:36:10.000 --> 00:36:11.640
like you're saying that

00:36:11.640 --> 00:36:13.480
that you're not dependent

00:36:13.480 --> 00:36:14.120
on Anthropic.

00:36:14.120 --> 00:36:16.720
>> am dependent. Okay, say we are dependent

00:36:16.720 --> 00:36:18.400
on a on a model whatever.

00:36:18.400 --> 00:36:19.720
>> I'm dependent on it for a lot of

00:36:19.720 --> 00:36:21.640
projects. Not not my day-to-day job. I'm

00:36:21.640 --> 00:36:23.120
still confident I can do what I need to

00:36:23.120 --> 00:36:24.880
But still, how do you it doesn't really

00:36:24.880 --> 00:36:27.640
help you with that whole question you've

00:36:27.640 --> 00:36:30.320
built Is it a question? The statement of

00:36:30.320 --> 00:36:33.080
you've built something with AI that you

00:36:33.080 --> 00:36:35.680
that now you're struggling to maintain.

00:36:35.680 --> 00:36:36.960
Are you are you are you saying that

00:36:36.960 --> 00:36:39.320
like, "Actually no, I can maintain it

00:36:39.320 --> 00:36:41.200
because I can run a

00:36:41.200 --> 00:36:42.720
I mean, I I hear a lot of people saying

00:36:42.720 --> 00:36:43.960
that they built something and they can't

00:36:43.960 --> 00:36:45.880
maintain it. I did not face that

00:36:45.880 --> 00:36:48.000
problem. So far AI has been able to do

00:36:48.000 --> 00:36:49.640
what I wanted to do. I haven't faced

00:36:49.640 --> 00:36:51.280
those issues. Like I said, knock wood. I

00:36:51.280 --> 00:36:52.680
don't know if it's because of the way

00:36:52.680 --> 00:36:54.280
that I manage the context for it. I

00:36:54.280 --> 00:36:56.480
don't know what allows me to maintain it

00:36:56.480 --> 00:36:58.720
or if I'm able to engineering like break

00:36:58.720 --> 00:37:00.000
down the problem. Maybe I'm a more

00:37:00.000 --> 00:37:01.640
technical user. Yeah. But I have not

00:37:01.640 --> 00:37:03.680
faced the issue that I cannot use AI to

00:37:03.680 --> 00:37:05.840
maintain it. I do face the issue that if

00:37:05.840 --> 00:37:07.760
I don't have AI, I'm pretty sure I can't

00:37:07.760 --> 00:37:08.520
maintain

00:37:08.520 --> 00:37:10.120
>> I see what you're saying. I I thought I

00:37:10.120 --> 00:37:11.280
thought you're making a general

00:37:11.280 --> 00:37:13.400
statement that like when you build with

00:37:13.400 --> 00:37:14.920
AI, you're going to have an an

00:37:14.920 --> 00:37:17.080
inevitable maintenance problem. I think

00:37:17.080 --> 00:37:18.840
what you were saying really was that

00:37:18.840 --> 00:37:20.680
when you build with AI, you become

00:37:20.680 --> 00:37:22.680
dependent on AI, and this is why you

00:37:22.680 --> 00:37:24.440
need to look at self-hosted models as a

00:37:24.440 --> 00:37:26.160
backup or something. Okay, okay. That's

00:37:26.160 --> 00:37:28.120
what I said also. I mean, I said that I

00:37:28.120 --> 00:37:29.960
currently do not have a problem with AI

00:37:29.960 --> 00:37:31.400
getting the things done that I wanted to

00:37:31.400 --> 00:37:33.160
get done. A lot of people say that they

00:37:33.160 --> 00:37:35.200
have are at the stage with Claude code

00:37:35.200 --> 00:37:37.400
that they send one prompt, say hello,

00:37:37.400 --> 00:37:40.600
and 70 and 20% of their their 5-hour

00:37:40.600 --> 00:37:42.400
session budget is gone. This type of

00:37:42.400 --> 00:37:43.560
statements I have not faced.

00:37:43.560 --> 00:37:46.320
>> Oh yeah, like anyone who tries I well,

00:37:46.320 --> 00:37:48.280
this was a couple of weeks back, but I I

00:37:48.280 --> 00:37:50.040
think I I signed up a friend for the

00:37:50.040 --> 00:37:52.320
free version, and then of course he said

00:37:52.320 --> 00:37:54.640
hello and and something else, and then

00:37:54.640 --> 00:37:57.920
that the free version was already gone.

00:37:57.920 --> 00:37:59.359
Which is I think why it makes sense they

00:37:59.359 --> 00:38:01.760
remove access on those

00:38:01.760 --> 00:38:04.520
subscriptions because and I think they

00:38:04.520 --> 00:38:05.880
responded to the complaints of the

00:38:05.880 --> 00:38:07.880
community because community is saying,

00:38:07.880 --> 00:38:10.000
"Oh, you're taking things away that used

00:38:10.000 --> 00:38:11.040
to be there." And they said, "Well,

00:38:11.040 --> 00:38:12.480
those things didn't exist when those

00:38:12.480 --> 00:38:14.080
subscriptions were created. We gave you

00:38:14.080 --> 00:38:15.920
a $20 per month subscription."

00:38:15.920 --> 00:38:18.200
Originally with Claude code you get like

00:38:18.200 --> 00:38:20.560
maybe one or two hours of usage out of

00:38:20.560 --> 00:38:22.040
it and then you're out of your 5-hour

00:38:22.040 --> 00:38:25.200
session budget with Sonnet 3.5

00:38:25.200 --> 00:38:27.080
and maybe with Sonnet 4 and then some

00:38:27.080 --> 00:38:29.200
Haiku that worked fine. Now we're on

00:38:29.200 --> 00:38:32.400
Opus 4.7, the token cost is so high,

00:38:32.400 --> 00:38:34.120
they make it the default model, and they

00:38:34.120 --> 00:38:35.720
feel like people are not getting good

00:38:35.720 --> 00:38:38.840
enough quality uh out of the product.

00:38:38.840 --> 00:38:40.320
And they're saying, "Look, we're we're

00:38:40.320 --> 00:38:41.840
just simply not going to advertise

00:38:41.840 --> 00:38:44.280
Claude code as as an as a as an offering

00:38:44.280 --> 00:38:46.440
of this pro plan anymore because with

00:38:46.440 --> 00:38:48.520
$20 a month you're not going to get a

00:38:48.520 --> 00:38:49.800
good experience out of Claude code." And

00:38:49.800 --> 00:38:50.560
then

00:38:50.560 --> 00:38:53.040
I think it's expectations yeah. probably

00:38:53.040 --> 00:38:54.960
getting quite high. Like, I was actually

00:38:54.960 --> 00:38:56.520
thinking to myself like I'm I'm just

00:38:56.520 --> 00:38:59.480
looking at Hacker News here that GPT 5.5

00:38:59.480 --> 00:39:01.720
is out.

00:39:01.720 --> 00:39:03.800
Uh

00:39:03.800 --> 00:39:05.040
I mean, surely we're going to get to a

00:39:05.040 --> 00:39:06.520
point where maybe they're not going to

00:39:06.520 --> 00:39:07.280
even

00:39:07.280 --> 00:39:09.359
tell you what the new models are

00:39:09.359 --> 00:39:12.080
or something because well, A, people are

00:39:12.080 --> 00:39:15.320
like using it through a back door to do

00:39:15.320 --> 00:39:16.800
their things. It gets to a point where

00:39:16.800 --> 00:39:19.720
like expectations are so high, do they

00:39:19.720 --> 00:39:21.960
do they want to like release a new

00:39:21.960 --> 00:39:24.040
version to like make everyone's head

00:39:24.040 --> 00:39:26.040
spin or they just want to hide it?

00:39:26.040 --> 00:39:28.120
>> They have to, you know, because Claude

00:39:28.120 --> 00:39:30.800
just launched 4.7, right? And then Open

00:39:30.800 --> 00:39:33.520
AI announces their competitor 5.5, so

00:39:33.520 --> 00:39:36.240
>> but I I do feel it's a bit of a charade

00:39:36.240 --> 00:39:38.000
or a bit of a show. You don't think

00:39:38.000 --> 00:39:40.200
those are like really newly trained

00:39:40.200 --> 00:39:41.920
models that are

00:39:41.920 --> 00:39:44.160
new capabilities. They are publishing

00:39:44.160 --> 00:39:46.400
book benchmarks each time about the code

00:39:46.400 --> 00:39:47.360
bench.

00:39:47.360 --> 00:39:49.480
It's funny that every model now released

00:39:49.480 --> 00:39:51.560
is always saying this model excels as

00:39:51.560 --> 00:39:54.080
coding tasks. That's the same with 5.5

00:39:54.080 --> 00:39:54.200
again.

00:39:54.200 --> 00:39:56.080
>> Exactly. It's getting a bit samey. It's

00:39:56.080 --> 00:39:57.840
getting a bit samey, but I'm sure they

00:39:57.840 --> 00:39:59.200
are improving. I'm sure they are

00:39:59.200 --> 00:40:00.960
improving. Though there's more there's

00:40:00.960 --> 00:40:02.880
more to the model numbers now, isn't

00:40:02.880 --> 00:40:04.800
there? There's the harness, there's I

00:40:04.800 --> 00:40:06.600
think Open AI is significantly

00:40:06.600 --> 00:40:09.280
subsidizing their GPT models. I don't

00:40:09.280 --> 00:40:10.440
know if they've

00:40:10.440 --> 00:40:12.400
I mean a lot more than Anthropic. I

00:40:12.400 --> 00:40:14.320
think Anthropic has lost a lot of their

00:40:14.320 --> 00:40:16.480
margin with the Department of Force in a

00:40:16.480 --> 00:40:19.120
situation and they're having a huge

00:40:19.120 --> 00:40:20.960
influx of customers and they're having

00:40:20.960 --> 00:40:23.000
an inference problem. So they are not

00:40:23.000 --> 00:40:25.200
being able to subsidize as much as Open

00:40:25.200 --> 00:40:27.280
AI, which is still having the Department

00:40:27.280 --> 00:40:29.440
of War contract, unless they have really

00:40:29.440 --> 00:40:32.240
built such a good model that is doing

00:40:32.240 --> 00:40:34.040
inference much cheaper and more

00:40:34.040 --> 00:40:35.600
efficient, but I don't believe it. I

00:40:35.600 --> 00:40:37.840
think Anthropic is like cutting back and

00:40:37.840 --> 00:40:39.440
that's why they're making the plans

00:40:39.440 --> 00:40:41.640
changes and they're having to compete

00:40:41.640 --> 00:40:43.520
and they have to uphold a certain level

00:40:43.520 --> 00:40:45.520
of quality and people have been super

00:40:45.520 --> 00:40:47.560
critical.

00:40:47.560 --> 00:40:49.480
Yeah, that makes sense. It absolutely

00:40:49.480 --> 00:40:51.080
makes sense. I guess we're living

00:40:51.080 --> 00:40:53.080
through living through a phase right now

00:40:53.080 --> 00:40:55.280
and it's cool it's cool. It's cool

00:40:55.280 --> 00:40:56.480
nonetheless.

00:40:56.480 --> 00:41:00.160
It's just It's just that it's full of uh

00:41:00.160 --> 00:41:01.560
I feel like I'm

00:41:01.560 --> 00:41:03.600
I'm very out of touch. I'm also like by

00:41:03.600 --> 00:41:06.720
default on my on my Claude Code Opus 4.7

00:41:06.720 --> 00:41:09.200
is running on extra high and I read the

00:41:09.200 --> 00:41:11.240
effort levels and like extra high is

00:41:11.240 --> 00:41:12.560
when you have to do some serious

00:41:12.560 --> 00:41:13.880
thinking in the midst of an

00:41:13.880 --> 00:41:15.840
implementation. Whereas if I'm using

00:41:15.840 --> 00:41:18.000
Spec Ledger, I have it run on extra high

00:41:18.000 --> 00:41:19.480
to do the research and come up with the

00:41:19.480 --> 00:41:21.760
implementation plan and generate a task.

00:41:21.760 --> 00:41:24.080
And but then I switch it to lower effort

00:41:24.080 --> 00:41:26.240
because why would you need it on extra

00:41:26.240 --> 00:41:28.560
high and then it really redefines this

00:41:28.560 --> 00:41:30.640
task because when it's on extra high, it

00:41:30.640 --> 00:41:32.320
actually goes and says, "Well, this task

00:41:32.320 --> 00:41:33.680
is kind of stupid. I'm not going to do

00:41:33.680 --> 00:41:35.080
that." So you have to have find those

00:41:35.080 --> 00:41:36.360
balances.

00:41:36.360 --> 00:41:37.720
I think that the

00:41:37.720 --> 00:41:40.160
when it comes to my my own workflow with

00:41:40.160 --> 00:41:42.200
AI, the thing that I still haven't

00:41:42.200 --> 00:41:44.680
figured out really is that I I'm still

00:41:44.680 --> 00:41:46.600
using beads like a caveman. I'm still

00:41:46.600 --> 00:41:48.960
using caveman like a caveman.

00:41:48.960 --> 00:41:51.120
Are you using caveman? No, no, no. No,

00:41:51.120 --> 00:41:52.600
I'm just joking. That whole caveman

00:41:52.600 --> 00:41:54.880
thing is a total meme. Now, the thing

00:41:54.880 --> 00:41:56.480
that I'm struggling with Claude code

00:41:56.480 --> 00:41:59.040
especially is that we you know, that we

00:41:59.040 --> 00:42:02.000
do the BD ready and we implement a task

00:42:02.000 --> 00:42:03.880
and then even even if it has no

00:42:03.880 --> 00:42:05.920
questions for me, it then stops and

00:42:05.920 --> 00:42:07.840
says, "Do you want me to continue with

00:42:07.840 --> 00:42:09.600
the next thing?" Yes, I want you to

00:42:09.600 --> 00:42:11.040
continue. Do I have

00:42:11.040 --> 00:42:12.800
>> I just showed you to put this in a BD

00:42:12.800 --> 00:42:15.360
ready loop with a while? I mean I just

00:42:15.360 --> 00:42:16.800
told you that that

00:42:16.800 --> 00:42:19.760
that having So, okay, so beats for me

00:42:19.760 --> 00:42:21.600
and I have the equivalent of beats but

00:42:21.600 --> 00:42:23.280
in a simplified version without all of

00:42:23.280 --> 00:42:25.120
the extra whistles all the bells that

00:42:25.120 --> 00:42:26.800
that Stevie got added. It's a little bit

00:42:26.800 --> 00:42:28.840
crazy. So I've got a very simplified

00:42:28.840 --> 00:42:31.360
version of beats, which is more of like

00:42:31.360 --> 00:42:34.000
a persistent task graph across sessions,

00:42:34.000 --> 00:42:35.960
which Claude Code by the way also is

00:42:35.960 --> 00:42:37.240
trying to build as well. They have the

00:42:37.240 --> 00:42:38.840
ability to create task across sessions

00:42:38.840 --> 00:42:40.520
as well and Claude Code has task

00:42:40.520 --> 00:42:41.800
dependencies as well built into the

00:42:41.800 --> 00:42:44.000
harnesses too. But I also had the same

00:42:44.000 --> 00:42:45.800
problem where like you said it it goes

00:42:45.800 --> 00:42:47.280
off and does a couple of beats ready

00:42:47.280 --> 00:42:49.560
tasks and then after a while it stops

00:42:49.560 --> 00:42:50.920
and says, "Okay, I've completed this and

00:42:50.920 --> 00:42:52.840
I want to continue." But I just told you

00:42:52.840 --> 00:42:53.840
I showed you at the start of the

00:42:53.840 --> 00:42:55.800
recording it was running for 35 minutes

00:42:55.800 --> 00:42:58.000
at that time and when I came back now

00:42:58.000 --> 00:43:00.040
after having a bagel, it's done. It's

00:43:00.040 --> 00:43:03.520
implemented 24 of 24 MVP task. Five

00:43:03.520 --> 00:43:05.600
phases have been completed. It says I

00:43:05.600 --> 00:43:07.440
haven't validated it yet.

00:43:07.440 --> 00:43:09.600
Like I just told you like at the start

00:43:09.600 --> 00:43:12.840
of the call I updated all my prompts to

00:43:12.840 --> 00:43:14.400
have it read and so basically the

00:43:14.400 --> 00:43:16.760
prompts are like get the task graph, get

00:43:16.760 --> 00:43:18.360
the phases and the MVP task that you

00:43:18.360 --> 00:43:19.480
need to do.

00:43:19.480 --> 00:43:21.160
And then I tell it

00:43:21.160 --> 00:43:23.000
No, stop. Then I tell it to actually

00:43:23.000 --> 00:43:26.480
create with its to-do tool the actual

00:43:26.480 --> 00:43:28.400
task. So it says in its particular

00:43:28.400 --> 00:43:30.680
session also because these are 1 million

00:43:30.680 --> 00:43:32.400
token windows now. In this particular

00:43:32.400 --> 00:43:34.200
session it say it it actually created a

00:43:34.200 --> 00:43:37.080
task graph on top of the beats graph. It

00:43:37.080 --> 00:43:38.760
knows that after it has it has done

00:43:38.760 --> 00:43:40.800
this, the next task is going to be that,

00:43:40.800 --> 00:43:42.600
which is just a pointer to the full task

00:43:42.600 --> 00:43:44.400
description and definition of done that

00:43:44.400 --> 00:43:46.200
it has created. And then it goes off the

00:43:46.200 --> 00:43:47.960
task list. It just keeps going. It

00:43:47.960 --> 00:43:49.920
doesn't stop because it has a a task

00:43:49.920 --> 00:43:51.680
graph and it Do you Do you

00:43:51.680 --> 00:43:54.600
Do you Do you spin the task off in an

00:43:54.600 --> 00:43:55.640
agent or something?

00:43:55.640 --> 00:43:57.200
>> No, no, no, still not doing that because

00:43:57.200 --> 00:43:59.200
I had bad experience with that and and I

00:43:59.200 --> 00:44:01.680
like your styles your graphs, which are

00:44:01.680 --> 00:44:04.280
sorry, your stages slide that you share,

00:44:04.280 --> 00:44:06.280
which is like um you start with live

00:44:06.280 --> 00:44:08.480
coding plan plan modes and you go

00:44:08.480 --> 00:44:11.880
towards like building hooks and like you

00:44:11.880 --> 00:44:14.960
know, giving it in like inline like with

00:44:14.960 --> 00:44:17.120
system prompts adjustments while it's

00:44:17.120 --> 00:44:18.720
working with hooks and then you go

00:44:18.720 --> 00:44:22.359
towards agent like agent background

00:44:22.359 --> 00:44:24.000
and then you go to agents teams as like

00:44:24.000 --> 00:44:25.960
the full master of orchestration. I'm

00:44:25.960 --> 00:44:28.160
not at that at all. I'm at hooks. I

00:44:28.160 --> 00:44:31.280
should write a hook to do a BD ready.

00:44:31.280 --> 00:44:32.840
You've given me an idea.

00:44:32.840 --> 00:44:35.320
>> that. Beats has like a whole bunch of

00:44:35.320 --> 00:44:36.840
stuff it puts inside your context. It

00:44:36.840 --> 00:44:39.400
has like landing the plane. It has like

00:44:39.400 --> 00:44:41.000
when you when you do a compact, it's

00:44:41.000 --> 00:44:43.040
it's injects a whole message inside the

00:44:43.040 --> 00:44:45.000
compact and says, "We're we're doing a

00:44:45.000 --> 00:44:46.320
compact, but after that you have to do

00:44:46.320 --> 00:44:48.320
the beats." Anyway, but like I said, I

00:44:48.320 --> 00:44:51.080
use beats similar thing for like the

00:44:51.080 --> 00:44:53.600
full five phase with like three phases

00:44:53.600 --> 00:44:56.240
for MVP uh implementation plan, which

00:44:56.240 --> 00:44:58.320
includes like dotting the eyes and

00:44:58.320 --> 00:44:59.840
polish phase and all that, which is all

00:44:59.840 --> 00:45:01.920
extra stuff. But in this case, I

00:45:01.920 --> 00:45:04.720
launched the prompt the spec ledger or

00:45:04.720 --> 00:45:06.520
spec it implementation prompt, which

00:45:06.520 --> 00:45:09.240
tells it to read the spec, read the

00:45:09.240 --> 00:45:11.480
plan, the task, everything and it picked

00:45:11.480 --> 00:45:12.880
up and it asked me when it started. It

00:45:12.880 --> 00:45:13.960
says, "What do you want me to do? Do you

00:45:13.960 --> 00:45:16.240
want me to just focus on the MVP tasks?"

00:45:16.240 --> 00:45:18.359
And I said, "Just do the MVP task." And

00:45:18.359 --> 00:45:20.720
then it created 24

00:45:20.720 --> 00:45:21.640
I'm going to do this. I'm going to do

00:45:21.640 --> 00:45:22.640
this. I'm going to do this. I'm going to

00:45:22.640 --> 00:45:23.800
do this. I'm going to do this and it

00:45:23.800 --> 00:45:26.000
didn't stop until all of them were done.

00:45:26.000 --> 00:45:27.440
It ran more than 30 minutes. I think

00:45:27.440 --> 00:45:29.120
close to an hour and it says, "I have

00:45:29.120 --> 00:45:30.520
completed all the MVP. I have

00:45:30.520 --> 00:45:31.960
implemented all the phase features,"

00:45:31.960 --> 00:45:33.800
which is in this case it's building a

00:45:33.800 --> 00:45:35.400
a web dashboard

00:45:35.400 --> 00:45:38.840
for like a node graph visualization.

00:45:38.840 --> 00:45:41.160
Does your to-do you know

00:45:41.160 --> 00:45:43.240
use the full description? No, it it it

00:45:43.240 --> 00:45:46.280
it had like I'm going to do task SL1556

00:45:46.280 --> 00:45:48.120
or whatever. And then I guess when it I

00:45:48.120 --> 00:45:49.560
haven't really looked at what exactly

00:45:49.560 --> 00:45:51.440
the content is of the to-do list, but it

00:45:51.440 --> 00:45:53.359
says, "I'm going to work on this item."

00:45:53.359 --> 00:45:55.520
And when it reads that item, it sees a

00:45:55.520 --> 00:45:57.280
full definition of done of each item.

00:45:57.280 --> 00:45:59.400
But I tell you right now, it's asking me

00:45:59.400 --> 00:46:01.400
to do an adversarial review of the

00:46:01.400 --> 00:46:03.720
implementation. I'm telling you this is

00:46:03.720 --> 00:46:05.480
actually going to read and this is where

00:46:05.480 --> 00:46:07.880
you have Spec Ledger as as a good thing

00:46:07.880 --> 00:46:09.720
because I have a very well-defined

00:46:09.720 --> 00:46:11.120
definition of done and what I'm trying

00:46:11.120 --> 00:46:13.080
to do. And if I run this adversarial

00:46:13.080 --> 00:46:16.200
review, 90% it will tell me it says it's

00:46:16.200 --> 00:46:18.680
completed, but it hasn't done ABCD of

00:46:18.680 --> 00:46:21.040
the definition of done. 90% every single

00:46:21.040 --> 00:46:22.040
time.

00:46:22.040 --> 00:46:24.240
I'm just trying to summarize this last

00:46:24.240 --> 00:46:25.760
conversation. It just sounds like you're

00:46:25.760 --> 00:46:28.680
pretty excited that you have got a

00:46:28.680 --> 00:46:30.480
better workflow going with Spec Ledger,

00:46:30.480 --> 00:46:31.920
which is great, which is great.

00:46:31.920 --> 00:46:33.800
>> Yeah, I can show you like

00:46:33.800 --> 00:46:36.840
the where it's at right now. Yeah. Well

00:46:36.840 --> 00:46:38.160
screen here.

00:46:38.160 --> 00:46:39.480
>> Well well it's

00:46:39.480 --> 00:46:40.920
I mean maybe

00:46:40.920 --> 00:46:42.280
I was thinking of of changing the

00:46:42.280 --> 00:46:43.880
conversation to the the other thing you

00:46:43.880 --> 00:46:45.760
were mentioning about the or the So it

00:46:45.760 --> 00:46:48.800
says here implementation I'll zoom in.

00:46:48.800 --> 00:46:51.000
It says here

00:46:51.000 --> 00:46:53.320
it has completed all the MVP phases

00:46:53.320 --> 00:46:56.400
phases five to nine. Um it has deferred

00:46:56.400 --> 00:46:58.040
something. So it's already telling me

00:46:58.040 --> 00:46:59.480
that has deferred something, which it

00:46:59.480 --> 00:47:00.840
always does.

00:47:00.840 --> 00:47:02.280
Interesting.

00:47:02.280 --> 00:47:03.760
I don't know what the slices part is. I

00:47:03.760 --> 00:47:06.040
think it's because this is a basically a

00:47:06.040 --> 00:47:08.000
Terraform state graph visualization

00:47:08.000 --> 00:47:09.040
thing

00:47:09.040 --> 00:47:11.960
for from fog and it's the serve command

00:47:11.960 --> 00:47:14.600
is going to serve a JavaScript web page

00:47:14.600 --> 00:47:17.800
that uses Sigma JS to visualize it

00:47:17.800 --> 00:47:19.240
because apparently it can render a lot

00:47:19.240 --> 00:47:21.480
of nodes like tons of nodes in with

00:47:21.480 --> 00:47:23.800
WebGL in the browser. So it allows me to

00:47:23.800 --> 00:47:25.680
drag all of the Terraform states around

00:47:25.680 --> 00:47:27.600
with all the edges attached to it.

00:47:27.600 --> 00:47:29.359
So it has completed all of them.

00:47:29.359 --> 00:47:31.320
And one of the things that I really like

00:47:31.320 --> 00:47:33.760
is I think because of the task graph and

00:47:33.760 --> 00:47:35.520
like handoff type of setup of the

00:47:35.520 --> 00:47:38.240
prompts right now is it is very

00:47:38.240 --> 00:47:40.520
forceful now. It's forcing you into this

00:47:40.520 --> 00:47:42.760
form into this workflow where it says,

00:47:42.760 --> 00:47:44.160
"I have you know, the agent says

00:47:44.160 --> 00:47:45.960
everything is done. You need to run a

00:47:45.960 --> 00:47:48.440
checkpoint." And the checkpoint

00:47:48.440 --> 00:47:51.720
is going to do

00:47:51.720 --> 00:47:53.800
a full coherence check, but it's going

00:47:53.800 --> 00:47:55.680
to validate its own work, which is

00:47:55.680 --> 00:47:56.680
always going to be like, "Yeah, yeah,

00:47:56.680 --> 00:47:58.200
yeah, it's all good. I haven't done

00:47:58.200 --> 00:47:59.600
this, but it's fine." It's always going

00:47:59.600 --> 00:48:02.000
to do that. And then if I say yes, do

00:48:02.000 --> 00:48:03.600
that. So it's it's forcing me into this

00:48:03.600 --> 00:48:05.480
flow. When it has finished doing a full

00:48:05.480 --> 00:48:07.080
review of what it has completed against

00:48:07.080 --> 00:48:08.520
the code and everything it's written,

00:48:08.520 --> 00:48:10.600
it's then going to propose to launch an

00:48:10.600 --> 00:48:12.880
agent in an empty context with no

00:48:12.880 --> 00:48:15.040
information other than here is the task

00:48:15.040 --> 00:48:16.960
list and the definition of done. Here is

00:48:16.960 --> 00:48:18.840
the code that has been completed.

00:48:18.840 --> 00:48:21.160
Now validate it. And that agent in a

00:48:21.160 --> 00:48:23.560
completely fresh context is absolutely

00:48:23.560 --> 00:48:25.880
every single time going to be brutal.

00:48:25.880 --> 00:48:27.680
It's always going to go like, "You know,

00:48:27.680 --> 00:48:29.160
the agent says that everything's been

00:48:29.160 --> 00:48:31.280
completed, but this definition of done

00:48:31.280 --> 00:48:33.840
is clearly not A few minutes ago I asked

00:48:33.840 --> 00:48:36.600
you about using agents, but you you you

00:48:36.600 --> 00:48:38.040
said that you don't use agents cuz you

00:48:38.040 --> 00:48:39.520
had a bad experience, but of course this

00:48:39.520 --> 00:48:41.800
this particular agent the adversary

00:48:41.800 --> 00:48:44.240
agent you're now saying to me Yes. is

00:48:44.240 --> 00:48:46.800
actually pretty amazing. If I do agents,

00:48:46.800 --> 00:48:48.000
I'm not running them that much in

00:48:48.000 --> 00:48:51.200
parallel, but I am using more of like or

00:48:51.200 --> 00:48:53.280
like a planning agent that helps build

00:48:53.280 --> 00:48:55.080
the plan and then an implementation

00:48:55.080 --> 00:48:57.200
agent that implements against the plan

00:48:57.200 --> 00:49:00.000
and then a adversarial review agent that

00:49:00.000 --> 00:49:01.880
validates the work of the implementation

00:49:01.880 --> 00:49:03.920
agent against the plan. And you need to

00:49:03.920 --> 00:49:05.400
run that one in an in a in a separate

00:49:05.400 --> 00:49:07.440
context because again, if you're asking

00:49:07.440 --> 00:49:10.000
self review, it's always going to say,

00:49:10.000 --> 00:49:11.600
"I haven't done it, but it's okay

00:49:11.600 --> 00:49:13.680
because blah blah blah." And the

00:49:13.680 --> 00:49:14.960
adversarial review is going to be

00:49:14.960 --> 00:49:16.920
brutal. It's going to be it's not done

00:49:16.920 --> 00:49:20.960
whatever excuse is not valid. Well, I I

00:49:20.960 --> 00:49:23.880
I like to pretend that I have a fresh

00:49:23.880 --> 00:49:26.680
context when I when I do a BD ready, but

00:49:26.680 --> 00:49:28.560
the fact of the matter is is that I do

00:49:28.560 --> 00:49:31.480
keep my context open because it's just

00:49:31.480 --> 00:49:33.560
easier to work that way and and probably

00:49:33.560 --> 00:49:35.520
I get better results. And I think it's

00:49:35.520 --> 00:49:37.400
the same way that you're working. You're

00:49:37.400 --> 00:49:38.960
basically

00:49:38.960 --> 00:49:42.280
uh leveraging a a large context to to to

00:49:42.280 --> 00:49:45.200
uh strike off your your to-do's, which

00:49:45.200 --> 00:49:47.680
which isn't um the optimal way, right?

00:49:47.680 --> 00:49:50.400
The optimal way would be each to-do it

00:49:50.400 --> 00:49:53.040
in a separate agent. But I'm sure we'll

00:49:53.040 --> 00:49:54.640
get there and like, you know, one step

00:49:54.640 --> 00:49:57.320
at a time, I suppose. Yeah, so the idea

00:49:57.320 --> 00:49:59.160
idea originally with with Speck it as

00:49:59.160 --> 00:50:00.520
well as it always identifies

00:50:00.520 --> 00:50:02.920
parallelizable tasks where it says these

00:50:02.920 --> 00:50:04.240
tasks can be done in parallel. They're

00:50:04.240 --> 00:50:06.840
in different areas of the code base and

00:50:06.840 --> 00:50:09.880
very early on like back in October when

00:50:09.880 --> 00:50:12.560
sub-agents were early in Claude Code, I

00:50:12.560 --> 00:50:15.440
told it very ambitiously, go and spin

00:50:15.440 --> 00:50:18.000
off like five parallel agents to do all

00:50:18.000 --> 00:50:19.760
of these parallel tasks.

00:50:19.760 --> 00:50:22.120
And every single time it would come back

00:50:22.120 --> 00:50:24.520
and it would have implemented uh

00:50:24.520 --> 00:50:27.520
a utility shared function over and over.

00:50:27.520 --> 00:50:29.000
It would have fought It would There

00:50:29.000 --> 00:50:30.880
would have been assumptions where like

00:50:30.880 --> 00:50:33.280
agent A is going to work on task A and

00:50:33.280 --> 00:50:36.080
agent B is on B, but then agent B would

00:50:36.080 --> 00:50:37.400
have made assumptions and like

00:50:37.400 --> 00:50:40.000
implemented mocks against task A saying

00:50:40.000 --> 00:50:41.760
like, "Oh, the code isn't there, so I'm

00:50:41.760 --> 00:50:43.600
going to build these additional mocks to

00:50:43.600 --> 00:50:45.480
to simulate this work." That's what I

00:50:45.480 --> 00:50:47.240
used to do. I think they've fixed that a

00:50:47.240 --> 00:50:49.800
lot. I haven't tried it since. But it

00:50:49.800 --> 00:50:51.120
used to do that all the time and it was

00:50:51.120 --> 00:50:53.280
like at the time I was not on Max Plan,

00:50:53.280 --> 00:50:55.880
so a $20 per month gives me like 2 hours

00:50:55.880 --> 00:50:57.760
of effective usage and I don't want to

00:50:57.760 --> 00:50:59.640
have 2 hours of crap.

00:50:59.640 --> 00:51:01.240
Cuz if after 2 hours it gives me all of

00:51:01.240 --> 00:51:03.120
these duplicate implementations with

00:51:03.120 --> 00:51:03.720
mocks like

00:51:03.720 --> 00:51:06.120
>> to button and say that I'm not I'm not a

00:51:06.120 --> 00:51:09.120
huge fan of parallelization. Like

00:51:09.120 --> 00:51:11.640
what like the Is there a real need for

00:51:11.640 --> 00:51:13.480
that? I I thought the

00:51:13.480 --> 00:51:16.760
I think the the rough wagon loop to

00:51:16.760 --> 00:51:19.200
doing things sequentially is is fine.

00:51:19.200 --> 00:51:21.320
>> See, so it has completed. So what it has

00:51:21.320 --> 00:51:22.720
done in the prompt, which is a

00:51:22.720 --> 00:51:24.480
checkpoint prompt, which is not the

00:51:24.480 --> 00:51:26.240
Claude Code checkpoint, but my Speck

00:51:26.240 --> 00:51:28.000
Ledger checkpoint, it has found some

00:51:28.000 --> 00:51:29.720
divergences. It says it's testing the

00:51:29.720 --> 00:51:32.320
wrong scenario. As I said, 90% chance

00:51:32.320 --> 00:51:34.160
there's a whole bunch of issues. Uh and

00:51:34.160 --> 00:51:35.680
if you don't run this adversarial

00:51:35.680 --> 00:51:37.680
review, you will get crap code and you

00:51:37.680 --> 00:51:39.600
will find problems all the time. And it

00:51:39.600 --> 00:51:42.000
has this done. It's a self-evaluation.

00:51:42.000 --> 00:51:43.400
And it says, "Oh, none of these are

00:51:43.400 --> 00:51:46.080
important." But then it also goes ahead

00:51:46.080 --> 00:51:48.840
and um says that right now, uh do you

00:51:48.840 --> 00:51:50.120
want to launch an independent

00:51:50.120 --> 00:51:52.280
adversarial review? And it has actually

00:51:52.280 --> 00:51:54.560
I have embedded the adversarial agent's

00:51:54.560 --> 00:51:57.360
prompt I I can look at your

00:51:57.360 --> 00:51:57.800
your agent.

00:51:57.800 --> 00:52:00.000
>> Yeah, it's actually a an issue that a

00:52:00.000 --> 00:52:01.920
feature request that I have opened for

00:52:01.920 --> 00:52:03.760
myself, uh I mean, for Cloud in the

00:52:03.760 --> 00:52:06.120
future, is to actually ship in agents

00:52:06.120 --> 00:52:07.840
because right now the the agent's the

00:52:07.840 --> 00:52:10.200
adversarial review prompt is baked in

00:52:10.200 --> 00:52:12.560
the in other prompt. So this this agent

00:52:12.560 --> 00:52:13.920
is going to launch another agent with

00:52:13.920 --> 00:52:14.840
the prompt. In reality

00:52:14.840 --> 00:52:16.760
>> walk me through this, Vincent. You you

00:52:16.760 --> 00:52:18.840
you spin off the adversarial review. It

00:52:18.840 --> 00:52:21.360
basically is brutal with the work that's

00:52:21.360 --> 00:52:23.400
been done. Now what happens? How do you

00:52:23.400 --> 00:52:25.640
rectify the

00:52:25.640 --> 00:52:27.760
the the issues that raised by the

00:52:27.760 --> 00:52:29.560
Is it Is it just another bunch of to-do

00:52:29.560 --> 00:52:31.440
lists? Yeah, the adversarial will come

00:52:31.440 --> 00:52:34.000
back with um some probably find the same

00:52:34.000 --> 00:52:36.160
issues and then also some some other

00:52:36.160 --> 00:52:38.160
ones. Usually it will highlight that

00:52:38.160 --> 00:52:40.080
some definition of done or some some

00:52:40.080 --> 00:52:42.160
task has been closed where the

00:52:42.160 --> 00:52:43.600
definition of done has not been checked

00:52:43.600 --> 00:52:45.520
off. I don't know. This was back in 4.6.

00:52:45.520 --> 00:52:47.560
I don't know if 4.7 does the same thing,

00:52:47.560 --> 00:52:49.840
but 4.6 would just close the just say

00:52:49.840 --> 00:52:51.240
it's done, but actually didn't check it

00:52:51.240 --> 00:52:51.760
off.

00:52:51.760 --> 00:52:54.280
>> the the task. So the adversarial agent

00:52:54.280 --> 00:52:55.880
will just come back with a report. So

00:52:55.880 --> 00:53:00.920
they they generate

00:53:00.920 --> 00:53:01.040
um

00:53:01.040 --> 00:53:03.360
>> Oh. They they generate here, but the way

00:53:03.360 --> 00:53:05.560
that it calls these reports are I'm not

00:53:05.560 --> 00:53:06.760
very happy with, but like it did a

00:53:06.760 --> 00:53:10.040
divergence review on this time and uh it

00:53:10.040 --> 00:53:11.880
basically wrote this down. I don't think

00:53:11.880 --> 00:53:13.840
the adversarial agent will read this.

00:53:13.840 --> 00:53:15.320
Could it could be It could that it reads

00:53:15.320 --> 00:53:17.160
this and just goes off this.

00:53:17.160 --> 00:53:18.400
Yeah, I hope not.

00:53:18.400 --> 00:53:21.080
>> like like many teams, you know, I've

00:53:21.080 --> 00:53:23.400
I've worked on some teams where

00:53:23.400 --> 00:53:25.640
where the the card would go back into

00:53:25.640 --> 00:53:28.400
the um the Kanban, reopened. And then

00:53:28.400 --> 00:53:29.640
I've worked in some teams where like,

00:53:29.640 --> 00:53:32.720
"Nah, okay. I mean, we've done this.

00:53:32.720 --> 00:53:34.920
Let's just start a new issue cuz we

00:53:34.920 --> 00:53:37.760
found this issue in this after we

00:53:37.760 --> 00:53:39.680
deployed it or something like that." So

00:53:39.680 --> 00:53:42.040
I guess I guess all this the way that

00:53:42.040 --> 00:53:44.960
you're defining your own um you're

00:53:44.960 --> 00:53:47.920
defining your own your own workflow here

00:53:47.920 --> 00:53:50.280
and and that's perfectly fine, in my

00:53:50.280 --> 00:53:52.840
opinion. You're you're And that's what

00:53:52.840 --> 00:53:55.080
you That's what every business needs to

00:53:55.080 --> 00:53:57.560
do in a way, to find their own find what

00:53:57.560 --> 00:53:59.160
works for them

00:53:59.160 --> 00:54:01.280
in their problem space. And this is what

00:54:01.280 --> 00:54:02.920
you're doing. Um That's what every

00:54:02.920 --> 00:54:04.680
business did for the last year and now

00:54:04.680 --> 00:54:06.240
have like one team that's able to

00:54:06.240 --> 00:54:07.920
deliver decent code and another team

00:54:07.920 --> 00:54:09.520
that's delivering slop and has a massive

00:54:09.520 --> 00:54:10.640
amount of

00:54:10.640 --> 00:54:12.800
quality issues. So I think we're past

00:54:12.800 --> 00:54:15.160
that. I think we're past the point where

00:54:15.160 --> 00:54:17.040
every team is on their own coming up

00:54:17.040 --> 00:54:18.840
with their own workflows and so on. I

00:54:18.840 --> 00:54:20.200
think we're at the stage that we need to

00:54:20.200 --> 00:54:23.840
start measuring what works and go back

00:54:23.840 --> 00:54:26.760
and and and actually do things with

00:54:26.760 --> 00:54:28.920
metric like driven and not just like how

00:54:28.920 --> 00:54:31.280
much KLOC we ship, but uh how many

00:54:31.280 --> 00:54:32.840
efficiencies do we have and how many

00:54:32.840 --> 00:54:34.440
times did we have to roll back a

00:54:34.440 --> 00:54:36.000
deployment, you know, that sort of

00:54:36.000 --> 00:54:37.359
stuff.

00:54:37.359 --> 00:54:39.240
Yeah, but but but but there's It's got

00:54:39.240 --> 00:54:42.080
to be uh the

00:54:42.080 --> 00:54:43.480
Yeah, I'm with you. I mean, there's

00:54:43.480 --> 00:54:45.400
there's some maybe some top-down things

00:54:45.400 --> 00:54:46.640
you can do.

00:54:46.640 --> 00:54:49.680
Here.

00:54:49.680 --> 00:54:51.520
I mean, wouldn't you say wouldn't you

00:54:51.520 --> 00:54:53.800
say, Vincent, that that a lot of teams

00:54:53.800 --> 00:54:56.080
need to to find their own way? They need

00:54:56.080 --> 00:54:59.400
to go They need to adopt AI from a

00:54:59.400 --> 00:55:01.680
grassroots level as well, you know.

00:55:01.680 --> 00:55:02.960
Yeah, absolutely. And I think that's the

00:55:02.960 --> 00:55:05.120
most um important thing. Like if you're

00:55:05.120 --> 00:55:06.760
in a platform team, the biggest

00:55:06.760 --> 00:55:08.200
resistance against a centralized

00:55:08.200 --> 00:55:09.920
platform team by most product teams

00:55:09.920 --> 00:55:11.320
within an organization is like, "You're

00:55:11.320 --> 00:55:12.760
going to guys You guys going to be the

00:55:12.760 --> 00:55:13.960
bottleneck. You're going to tell me what

00:55:13.960 --> 00:55:15.480
to do. I want to be able to do my own

00:55:15.480 --> 00:55:16.920
thing. I want to be able to explore this

00:55:16.920 --> 00:55:18.920
framework and this tool on my own." I

00:55:18.920 --> 00:55:20.760
mean, every single organization to have

00:55:20.760 --> 00:55:23.280
that tension between um

00:55:23.280 --> 00:55:24.640
we have a certain way of doing things

00:55:24.640 --> 00:55:26.000
that we want to standardize so that

00:55:26.000 --> 00:55:27.720
people can move between teams and and

00:55:27.720 --> 00:55:29.000
things are aligned so that we can be

00:55:29.000 --> 00:55:30.680
better in compliance. And on the other

00:55:30.680 --> 00:55:32.640
side, everyone is free to explore the

00:55:32.640 --> 00:55:34.800
tools and find the best way of working

00:55:34.800 --> 00:55:35.280
and

00:55:35.280 --> 00:55:36.720
>> Yeah, there's there's a real conflict

00:55:36.720 --> 00:55:38.200
there, isn't there? Like I've I've seen

00:55:38.200 --> 00:55:39.040
it

00:55:39.040 --> 00:55:40.359
uh on a on a couple of clients where

00:55:40.359 --> 00:55:41.880
like where

00:55:41.880 --> 00:55:44.040
like so someone with the best intentions

00:55:44.040 --> 00:55:45.359
like, "Hey, I'm coming up with a

00:55:45.359 --> 00:55:48.240
framework." And then and then he

00:55:48.240 --> 00:55:50.560
>> pushback. or she inevitably says,

00:55:50.560 --> 00:55:52.640
"Everyone needs to use our framework so

00:55:52.640 --> 00:55:56.120
that we we pick the Jira card up and we

00:55:56.120 --> 00:55:58.120
implement it to our coding standards."

00:55:58.120 --> 00:55:59.160
And then like

00:55:59.160 --> 00:56:00.280
and then people going like, "No, I don't

00:56:00.280 --> 00:56:01.840
want to use your your framework. I want

00:56:01.840 --> 00:56:03.280
to do this my way."

00:56:03.280 --> 00:56:05.440
>> I mean, this tension's been in DevOps

00:56:05.440 --> 00:56:06.600
for ages, right? And they always say

00:56:06.600 --> 00:56:08.840
DevOps the culture of DevOps is a

00:56:08.840 --> 00:56:11.120
bottoms-up and up-top-down. Whereas you

00:56:11.120 --> 00:56:12.680
build the best tool and people adopt

00:56:12.680 --> 00:56:13.960
them because they're good, not because

00:56:13.960 --> 00:56:16.840
they're forced to use them. So that's

00:56:16.840 --> 00:56:19.720
part of the idea of DevOps culture is to

00:56:19.720 --> 00:56:22.960
um you know, find buy-in and bring those

00:56:22.960 --> 00:56:24.600
tools that you want to build with for

00:56:24.600 --> 00:56:26.640
for the other people to benefit them.

00:56:26.640 --> 00:56:28.520
That you work with them to to to see

00:56:28.520 --> 00:56:29.840
where it fits in.

00:56:29.840 --> 00:56:30.680
>> to

00:56:30.680 --> 00:56:33.320
to to do that approach, right? Because

00:56:33.320 --> 00:56:35.040
there's so much tooling, there's so many

00:56:35.040 --> 00:56:37.440
ideas there that that top-down like,

00:56:37.440 --> 00:56:39.640
"Hey, use Speck Ledger in our company."

00:56:39.640 --> 00:56:41.359
It's It's not going to work, is it? I

00:56:41.359 --> 00:56:44.359
mean,

00:56:44.359 --> 00:56:46.359
Yeah, absolutely. Um

00:56:46.359 --> 00:56:49.080
I It's a tough one, isn't it?

00:56:49.080 --> 00:56:51.720
Yeah, I mean, the way that I see it

00:56:51.720 --> 00:56:52.880
We're going to go through some sort of

00:56:52.880 --> 00:56:55.920
trough of disillusionment or some chaos

00:56:55.920 --> 00:57:00.440
before before uh real practical

00:57:00.440 --> 00:57:02.960
standards really emerge in a way. What I

00:57:02.960 --> 00:57:04.760
hear the most in organizations that are

00:57:04.760 --> 00:57:06.680
adopting AI heavily is that they they

00:57:06.680 --> 00:57:09.359
have a whole framework, right? To to

00:57:09.359 --> 00:57:12.160
basically share brown bags, shared

00:57:12.160 --> 00:57:15.080
repositories, skills registries, agents

00:57:15.080 --> 00:57:17.600
registries. We are sharing an agent that

00:57:17.600 --> 00:57:19.600
is able to pull all the context needed

00:57:19.600 --> 00:57:20.960
and that is able to help you with your

00:57:20.960 --> 00:57:23.400
work. We have agents that can live where

00:57:23.400 --> 00:57:25.359
you live, inside Slack, inside

00:57:25.359 --> 00:57:27.120
Confluence. And we are making these

00:57:27.120 --> 00:57:28.760
available across the organization and

00:57:28.760 --> 00:57:30.640
it's up to you to use them or not, but

00:57:30.640 --> 00:57:32.600
we're advocating for them and we're

00:57:32.600 --> 00:57:34.000
we're promoting them across the

00:57:34.000 --> 00:57:36.480
organizations and we're doing brown bags

00:57:36.480 --> 00:57:37.840
and we have

00:57:37.840 --> 00:57:39.920
um an actual effort. Uh but I guess if

00:57:39.920 --> 00:57:41.000
you do that again, you're going to have

00:57:41.000 --> 00:57:46.680
the same

00:57:46.680 --> 00:57:47.600
That's always a problem with the

00:57:47.600 --> 00:57:50.120
platform or an internal workflow team.

00:57:50.120 --> 00:57:52.600
It's like, "How are you improving our

00:57:52.600 --> 00:57:54.160
workflow?" No matter where you work,

00:57:54.160 --> 00:57:55.640
they're always going to ask the question

00:57:55.640 --> 00:57:57.800
and that's where then you can grope

00:57:57.800 --> 00:58:00.760
grasp at DORA metrics or similar and

00:58:00.760 --> 00:58:02.600
say, "Well, we are gathering all the

00:58:02.600 --> 00:58:04.000
information." And this is actually

00:58:04.000 --> 00:58:05.240
something interesting because I have

00:58:05.240 --> 00:58:07.400
some friends uh that I used to work with

00:58:07.400 --> 00:58:09.840
and they're saying, "We've we've got

00:58:09.840 --> 00:58:11.920
backstage now. We didn't have it when I

00:58:11.920 --> 00:58:14.920
was there. Um we've got backstage and we

00:58:14.920 --> 00:58:15.880
have set up Spotify

00:58:15.880 --> 00:58:17.760
>> Spotify backstage, right? Yeah, Spotify

00:58:17.760 --> 00:58:20.120
backstage, which is inherently

00:58:20.120 --> 00:58:22.440
considered very hard to to deploy

00:58:22.440 --> 00:58:24.000
because it's a bunch of TypeScript

00:58:24.000 --> 00:58:26.000
plugins and they're saying like, "It's

00:58:26.000 --> 00:58:28.120
you you There's a lot of like this

00:58:28.120 --> 00:58:30.800
Roadie and other hosted backstage

00:58:30.800 --> 00:58:33.120
solutions that are saying backstage

00:58:33.120 --> 00:58:34.600
sounds great, right? It's like

00:58:34.600 --> 00:58:36.160
self-hosted.

00:58:36.160 --> 00:58:37.760
But once you deploy it, then you need to

00:58:37.760 --> 00:58:39.240
build all your plugins, then you need to

00:58:39.240 --> 00:58:41.560
integrate it within all your systems. I

00:58:41.560 --> 00:58:43.080
think today with AI, that's not an an

00:58:43.080 --> 00:58:45.120
issue, right? Integrating it with an

00:58:45.120 --> 00:58:46.960
existing system and go and find the

00:58:46.960 --> 00:58:49.080
information where it lives is so much

00:58:49.080 --> 00:58:51.160
easier with AI and to have it built a

00:58:51.160 --> 00:58:53.120
TypeScript extension into backstage is

00:58:53.120 --> 00:58:54.680
so much easier with AI. You can

00:58:54.680 --> 00:58:56.400
literally can give the task to an AI

00:58:56.400 --> 00:58:58.480
agent, it goes and builds the thing. And

00:58:58.480 --> 00:59:01.080
and so I think today you don't face the

00:59:01.080 --> 00:59:03.280
same problems that you would if you were

00:59:03.280 --> 00:59:05.680
to roll out backstage 3 years ago where

00:59:05.680 --> 00:59:07.240
you need to be a TypeScript expert, you

00:59:07.240 --> 00:59:08.560
need to learn how to write backstage

00:59:08.560 --> 00:59:10.120
plugins, you need to know how to connect

00:59:10.120 --> 00:59:11.400
them to everything.

00:59:11.400 --> 00:59:14.720
>> was was to be an internal developer

00:59:14.720 --> 00:59:16.000
platform, a dashboard that you can

00:59:16.000 --> 00:59:18.160
connect all the information to.

00:59:18.160 --> 00:59:21.200
standards, right? Yeah, the idea was to

00:59:21.200 --> 00:59:23.280
give it a remove friction. To deploy a

00:59:23.280 --> 00:59:25.440
dashboard that that you can connect your

00:59:25.440 --> 00:59:28.000
DataDog dashboards into, that you can

00:59:28.000 --> 00:59:29.880
connect your Confluence space into, that

00:59:29.880 --> 00:59:31.680
you you connect your GitHub resource

00:59:31.680 --> 00:59:33.640
into, that. Every team, whatever tool

00:59:33.640 --> 00:59:35.440
they're using, can integrate into it.

00:59:35.440 --> 00:59:36.560
So, no matter if you're part of the

00:59:36.560 --> 00:59:38.040
team, you can go and get like a little

00:59:38.040 --> 00:59:39.880
dashboard overview of like, "Okay, so

00:59:39.880 --> 00:59:42.240
this is the This is the commercial

00:59:42.240 --> 00:59:45.080
platform team, and they have their

00:59:45.080 --> 00:59:46.240
source code live across these

00:59:46.240 --> 00:59:49.000
repositories. They are in the adoption

00:59:49.000 --> 00:59:52.480
of Python 3.14. Uh They have mostly

00:59:52.480 --> 00:59:54.000
upgraded their code base. They're about

00:59:54.000 --> 00:59:56.400
70% of adoption because we can see that

00:59:56.400 --> 00:59:57.680
as the metrics are being pulled out of

00:59:57.680 --> 00:59:59.840
repositories." So, this If you have a

00:59:59.840 --> 01:00:01.360
internal developer platform like that

01:00:01.360 --> 01:00:03.160
Backstage, which a long time people have

01:00:03.160 --> 01:00:04.480
said this is the future of platform

01:00:04.480 --> 01:00:05.520
engineering, you need to have an

01:00:05.520 --> 01:00:07.480
internal developer platform. I think

01:00:07.480 --> 01:00:09.760
today with AI, you can then connect MCPs

01:00:09.760 --> 01:00:11.600
for it. And I was like, "I hate MCPs."

01:00:11.600 --> 01:00:13.040
But then my friend who then he says

01:00:13.040 --> 01:00:14.640
like, "Well, MCPs can be remotely

01:00:14.640 --> 01:00:16.680
hosted. Therefore, everyone always gets

01:00:16.680 --> 01:00:18.280
the up-to-date information. You need to

01:00:18.280 --> 01:00:20.280
You don't need to distribute your CLI

01:00:20.280 --> 01:00:22.120
and your skills to every every person.

01:00:22.120 --> 01:00:23.840
An MCP can be a remote server that you

01:00:23.840 --> 01:00:26.280
remotely update, and then you connect

01:00:26.280 --> 01:00:27.800
that to your Backstage, and then

01:00:27.800 --> 01:00:30.440
everybody can just immediately via their

01:00:30.440 --> 01:00:33.080
agents, they get the context for this

01:00:33.080 --> 01:00:34.440
particular team. They know where the

01:00:34.440 --> 01:00:36.360
code lives. They know where the DataDog

01:00:36.360 --> 01:00:38.000
alerts are. They know which workflows

01:00:38.000 --> 01:00:39.920
matter for deployment." I think you're

01:00:39.920 --> 01:00:44.520
painting a picture of

01:00:44.520 --> 01:00:47.640
of an enterprise having a harmonious

01:00:47.640 --> 01:00:50.160
coding standards driven by some

01:00:50.160 --> 01:00:51.320
dashboard

01:00:51.320 --> 01:00:52.720
>> path page.

01:00:52.720 --> 01:00:54.400
I think that's the hardest bit bit about

01:00:54.400 --> 01:00:56.560
golden paths is like it is no Nobody

01:00:56.560 --> 01:00:59.480
wants to adopt a golden path exists.

01:00:59.480 --> 01:01:01.440
Like there's no purpose. It's very hard

01:01:01.440 --> 01:01:03.680
to build a golden path. And once you

01:01:03.680 --> 01:01:06.040
have it, and once you start deploying

01:01:06.040 --> 01:01:08.120
and managing your your your product

01:01:08.120 --> 01:01:10.200
efforts with golden paths, you just have

01:01:10.200 --> 01:01:11.480
to update the golden path, and then you

01:01:11.480 --> 01:01:13.240
can keep rolling it out. So, the picture

01:01:13.240 --> 01:01:14.520
that I've been painting is a picture

01:01:14.520 --> 01:01:16.600
that be painting to me by people that in

01:01:16.600 --> 01:01:18.400
an organization that I've left. And

01:01:18.400 --> 01:01:20.480
they're saying that this has been a very

01:01:20.480 --> 01:01:23.240
successful adoption of AI and and MCPs

01:01:23.240 --> 01:01:26.000
within their organization. And I do not

01:01:26.000 --> 01:01:27.680
know the details, but this is what they

01:01:27.680 --> 01:01:29.480
say, and I can see that that that's

01:01:29.480 --> 01:01:31.600
something I can play devil's advocate.

01:01:31.600 --> 01:01:33.160
I'm I'm

01:01:33.160 --> 01:01:35.640
almost of the opinion

01:01:35.640 --> 01:01:38.360
Okay, I'm just maybe opinions aside,

01:01:38.360 --> 01:01:41.280
couldn't we expect a world of chaos

01:01:41.280 --> 01:01:43.520
because a lot of people have strong

01:01:43.520 --> 01:01:44.800
People are going to probably get

01:01:44.800 --> 01:01:47.480
stronger opinions in order to deliver

01:01:47.480 --> 01:01:50.040
with AI.

01:01:50.040 --> 01:01:51.800
You know

01:01:51.800 --> 01:01:53.560
and with these strong opinions, there's

01:01:53.560 --> 01:01:55.880
going to be a conflict of

01:01:55.880 --> 01:01:58.240
of settling on to

01:01:58.240 --> 01:02:00.880
a golden path that's going to apply to

01:02:00.880 --> 01:02:03.080
to a team of teams, you know, like

01:02:03.080 --> 01:02:05.000
there's going to be conflicts because

01:02:05.000 --> 01:02:06.480
maybe the team doesn't want to work with

01:02:06.480 --> 01:02:07.920
the other team because like they know

01:02:07.920 --> 01:02:09.360
better or they want to do things

01:02:09.360 --> 01:02:10.560
differently.

01:02:10.560 --> 01:02:14.280
I do feel that agents that AI empowers

01:02:14.280 --> 01:02:17.440
at least myself to do what I want to do

01:02:17.440 --> 01:02:20.680
and and take up and take up more scope

01:02:20.680 --> 01:02:22.680
and be more ambitious. And now that's

01:02:22.680 --> 01:02:24.400
going to be a conflict in a in a typical

01:02:24.400 --> 01:02:26.720
enterprise because

01:02:26.720 --> 01:02:27.960
because

01:02:27.960 --> 01:02:28.960
there's going to be

01:02:28.960 --> 01:02:31.680
you know, like we we want to do this.

01:02:31.680 --> 01:02:33.600
And I think the conflict, I'm not sure

01:02:33.600 --> 01:02:35.000
if you touched on that, but the conflict

01:02:35.000 --> 01:02:36.800
on that I also heard from other people

01:02:36.800 --> 01:02:38.840
is that they went all in on AI, and then

01:02:38.840 --> 01:02:40.760
they are generating so much code that

01:02:40.760 --> 01:02:43.120
there's a bottleneck. Um and I think we

01:02:43.120 --> 01:02:44.320
mentioned that before as well.

01:02:44.320 --> 01:02:45.720
>> going to be

01:02:45.720 --> 01:02:47.440
a bottleneck in terms of like review.

01:02:47.440 --> 01:02:48.800
Like you're putting out too much code.

01:02:48.800 --> 01:02:49.680
We don't know if this is

01:02:49.680 --> 01:02:51.720
>> Exactly. The There's the PR like like

01:02:51.720 --> 01:02:53.840
PRs are are well known to be

01:02:53.840 --> 01:02:56.760
problematic, and you can't expect

01:02:56.760 --> 01:02:59.080
uh

01:02:59.080 --> 01:03:01.440
stack sets or whatever to save you,

01:03:01.440 --> 01:03:04.360
right? A lot of PRs are going to

01:03:04.360 --> 01:03:07.240
If If you expect a PR to be the way to

01:03:07.240 --> 01:03:08.680
distribute knowledge in inside an

01:03:08.680 --> 01:03:10.760
enterprise, I don't see it really

01:03:10.760 --> 01:03:12.480
especially in the volume that agents

01:03:12.480 --> 01:03:14.640
produce. Yeah. I can't help but think

01:03:14.640 --> 01:03:16.640
we're we're going to head for quite

01:03:16.640 --> 01:03:18.800
tough times, really. But But this is

01:03:18.800 --> 01:03:20.760
also where I don't I think Codium just

01:03:20.760 --> 01:03:23.000
raised like 10 million

01:03:23.000 --> 01:03:25.040
I don't know how many on maybe a I don't

01:03:25.040 --> 01:03:26.920
know how many dollars, but a lot of

01:03:26.920 --> 01:03:28.800
these AI agentic code review systems

01:03:28.800 --> 01:03:30.640
now, they are raising tons of money

01:03:30.640 --> 01:03:33.120
because And I'm using one for SpecLedger

01:03:33.120 --> 01:03:34.240
as well.

01:03:34.240 --> 01:03:36.480
Um whereas like I just now I did the

01:03:36.480 --> 01:03:38.720
full close to an hour implementation of

01:03:38.720 --> 01:03:40.560
the MVP path, and it highlighted that I

01:03:40.560 --> 01:03:42.280
need to do a manual verification because

01:03:42.280 --> 01:03:43.640
there's no Playwright script. I think I

01:03:43.640 --> 01:03:45.680
explicitly put because I'm building a

01:03:45.680 --> 01:03:47.480
browser and a web app, and I put

01:03:47.480 --> 01:03:49.240
Playwright as a deferred task for

01:03:49.240 --> 01:03:52.440
another feature to to do full end-to-end

01:03:52.440 --> 01:03:54.520
testing. So, the AI is blocking. It

01:03:54.520 --> 01:03:57.200
says, "You need to validate it." Um but

01:03:57.200 --> 01:03:59.080
then I've done the adversarial review,

01:03:59.080 --> 01:04:00.800
which came back and said, "There's a

01:04:00.800 --> 01:04:02.160
couple of like I don't think the

01:04:02.160 --> 01:04:04.440
adversarial review actually came up with

01:04:04.440 --> 01:04:06.480
a lot of like high findings. I'm

01:04:06.480 --> 01:04:08.560
surprised." It says like, "There's a bad

01:04:08.560 --> 01:04:10.240
equality test missing according to the

01:04:10.240 --> 01:04:12.160
data model. You should have a specific

01:04:12.160 --> 01:04:13.480
test that you had that hasn't been

01:04:13.480 --> 01:04:15.359
built." And so on.

01:04:15.359 --> 01:04:16.880
Uh but then what I usually get from

01:04:16.880 --> 01:04:18.880
Codium

01:04:18.880 --> 01:04:20.760
And it's really good. Codium Yeah,

01:04:20.760 --> 01:04:22.640
there's quite a few. There's like Grapp

01:04:22.640 --> 01:04:24.480
Tile, Codium.

01:04:24.480 --> 01:04:25.200
Um

01:04:25.200 --> 01:04:26.720
One of them is doing a a ton of

01:04:26.720 --> 01:04:28.840
advertisement, so that always makes me

01:04:28.840 --> 01:04:30.280
super suspicious of their actual

01:04:30.280 --> 01:04:32.040
solution. If you If you need that much

01:04:32.040 --> 01:04:34.040
advertising to sell me your solution,

01:04:34.040 --> 01:04:35.600
it's probably I'm sure there's

01:04:35.600 --> 01:04:37.960
going to be a lot of invention with with

01:04:37.960 --> 01:04:39.760
new tools

01:04:39.760 --> 01:04:42.320
Where is it? Oh, here it is. No, but to

01:04:42.320 --> 01:04:44.160
be honest, I think this is the only

01:04:44.160 --> 01:04:46.080
solution. I mean,

01:04:46.080 --> 01:04:47.800
we're going to Humans are clearly the

01:04:47.800 --> 01:04:49.920
bottleneck. We're going to have a way

01:04:49.920 --> 01:04:52.600
more output of code generated

01:04:52.600 --> 01:04:54.200
PRs. Like you said, there's going to be

01:04:54.200 --> 01:04:56.160
a huge battle. People are are going to

01:04:56.160 --> 01:04:58.359
complain about the quality. I definitely

01:04:58.359 --> 01:05:00.480
have a lot of like complaints when I

01:05:00.480 --> 01:05:02.880
have somebody else use AI and send in

01:05:02.880 --> 01:05:04.880
code that I do not trust their code.

01:05:04.880 --> 01:05:06.680
Whereas I generate code and I trust my

01:05:06.680 --> 01:05:08.560
code because I believe that I've put in

01:05:08.560 --> 01:05:10.040
place the mechanisms to catch the

01:05:10.040 --> 01:05:11.600
problems. But then

01:05:11.600 --> 01:05:13.240
as I use the product, I find a lot of

01:05:13.240 --> 01:05:15.359
errors down the line as I because I

01:05:15.359 --> 01:05:17.000
built it for myself. And then I said,

01:05:17.000 --> 01:05:19.400
"Oh, I thought I I I thought I I said it

01:05:19.400 --> 01:05:21.080
specifically to do it correctly, and it

01:05:21.080 --> 01:05:22.185
did it wrong."

01:05:22.185 --> 01:05:22.359
>> [laughter and gasps]

01:05:22.359 --> 01:05:24.760
>> Be extra careful mean on mistakes.

01:05:24.760 --> 01:05:26.080
But But

01:05:26.080 --> 01:05:28.040
Yeah, we definitely cannot trust the

01:05:28.040 --> 01:05:31.440
output of our peers and their AI.

01:05:31.440 --> 01:05:34.840
Okay, say you have an amazing amazing AI

01:05:34.840 --> 01:05:36.080
uh

01:05:36.080 --> 01:05:38.720
way of reviewing your code. But still

01:05:38.720 --> 01:05:41.400
the problem remains that that your team

01:05:41.400 --> 01:05:44.920
is going to potentially be unaligned.

01:05:44.920 --> 01:05:47.680
And I feel like no one's No one I I keep

01:05:47.680 --> 01:05:50.280
mentioning this like a broken record. If

01:05:50.280 --> 01:05:51.560
I'm sure I've mentioned this hundreds of

01:05:51.560 --> 01:05:53.720
times before now. It's like I I still

01:05:53.720 --> 01:05:55.880
don't understand how you're supposed to

01:05:55.880 --> 01:05:58.040
collaborate with your team. The teams of

01:05:58.040 --> 01:06:01.320
anything need to be smaller

01:06:01.320 --> 01:06:03.920
to improve communication.

01:06:03.920 --> 01:06:06.320
And If If you If you're working in a big

01:06:06.320 --> 01:06:08.600
team like five, seven people banging out

01:06:08.600 --> 01:06:11.120
code with AI, it's I don't see it

01:06:11.120 --> 01:06:12.640
working. Maybe I just lack the

01:06:12.640 --> 01:06:14.760
imagination.

01:06:14.760 --> 01:06:16.440
So, where people are That's definitely

01:06:16.440 --> 01:06:18.000
the million-dollar question, right?

01:06:18.000 --> 01:06:19.280
That's the problem that everyone's

01:06:19.280 --> 01:06:21.600
facing. We've heavily adopted AI. Some

01:06:21.600 --> 01:06:22.960
people stand up proudly that they've

01:06:22.960 --> 01:06:23.720
been

01:06:23.720 --> 01:06:25.840
shipping tons of code, which we disagree

01:06:25.840 --> 01:06:27.840
with the metrics that they use.

01:06:27.840 --> 01:06:29.440
Um but

01:06:29.440 --> 01:06:30.920
a lot of them are coming back from that

01:06:30.920 --> 01:06:32.680
saying that some teams deliver quality,

01:06:32.680 --> 01:06:34.359
other teams do not. So, we need to find

01:06:34.359 --> 01:06:35.840
a way to align the teams so that they

01:06:35.840 --> 01:06:38.280
can deliver the same amount of quality.

01:06:38.280 --> 01:06:40.120
How do we validate the quality? We've

01:06:40.120 --> 01:06:41.800
talked about it several times. Do we

01:06:41.800 --> 01:06:45.080
find a way to have executable specs like

01:06:45.080 --> 01:06:47.200
behavior-driven development?

01:06:47.200 --> 01:06:47.800
Um

01:06:47.800 --> 01:06:51.640
you know, using Cucumber to define the

01:06:51.640 --> 01:06:54.600
what what we want Are specs formally

01:06:54.600 --> 01:06:57.040
specific enough to capture all of the

01:06:57.040 --> 01:06:58.880
demands so that we can execute them and

01:06:58.880 --> 01:07:01.080
that we can have enough trust in into

01:07:01.080 --> 01:07:02.840
the code being built? Are we becoming

01:07:02.840 --> 01:07:04.680
too reliant on AI? Are we unable to

01:07:04.680 --> 01:07:06.560
maintain it because of the

01:07:06.560 --> 01:07:07.720
stupid has generated?

01:07:07.720 --> 01:07:09.200
>> Surely you would consider some teams

01:07:09.200 --> 01:07:11.520
might want to do spec-it driven

01:07:11.520 --> 01:07:13.960
development. A spec-driven development

01:07:13.960 --> 01:07:15.960
uh driven development. Some teams might

01:07:15.960 --> 01:07:18.000
want to do like test-driven

01:07:18.000 --> 01:07:19.680
uh driven development. Some Some teams

01:07:19.680 --> 01:07:21.120
might want to do

01:07:21.120 --> 01:07:23.040
some other workflow, you know? And there

01:07:23.040 --> 01:07:25.040
will definitely be an a battle on some

01:07:25.040 --> 01:07:27.040
teams arguing that their way is better,

01:07:27.040 --> 01:07:28.560
and they can trust it better, and they

01:07:28.560 --> 01:07:29.880
don't want to adopt the other teams'

01:07:29.880 --> 01:07:31.400
way, and you're going to have a a

01:07:31.400 --> 01:07:33.400
divergence across the organizations on

01:07:33.400 --> 01:07:35.160
how the code needs to be validated. It

01:07:35.160 --> 01:07:37.000
will also that have very strong depend

01:07:37.000 --> 01:07:38.760
on the industry and the level of risk

01:07:38.760 --> 01:07:41.400
that the industry that the company is in

01:07:41.400 --> 01:07:43.440
that it But we haven't even talked about

01:07:43.440 --> 01:07:45.280
the recent leaks. Vercel being

01:07:45.280 --> 01:07:47.160
compromised because of What is it?

01:07:47.160 --> 01:07:49.160
Context 7?

01:07:49.160 --> 01:07:50.720
GitHub being compromised and then

01:07:50.720 --> 01:07:52.400
attackers being able to exfiltrate all

01:07:52.400 --> 01:07:54.359
of the Vercel source code now being sold

01:07:54.359 --> 01:07:56.480
on the black market. Lovable having of

01:07:56.480 --> 01:07:58.440
complete

01:07:58.440 --> 01:08:01.480
leak of their platform.

01:08:01.480 --> 01:08:04.560
Basically, if you share or if you

01:08:04.560 --> 01:08:06.480
if some tenants make made some code

01:08:06.480 --> 01:08:09.040
public, then there was a way to

01:08:09.040 --> 01:08:11.240
get access to other tenants' private

01:08:11.240 --> 01:08:12.800
code or something. There was a leak. And

01:08:12.800 --> 01:08:14.520
I have to say, I had the exact same

01:08:14.520 --> 01:08:16.440
problem with SpecLedger

01:08:16.440 --> 01:08:18.240
after I introduced a new feature where

01:08:18.240 --> 01:08:19.600
it assumed a certain level of

01:08:19.600 --> 01:08:21.319
permissions or a certain level

01:08:21.319 --> 01:08:23.400
authorization against the user. But

01:08:23.400 --> 01:08:25.200
because of the new feature, those

01:08:25.200 --> 01:08:27.160
assumptions didn't weren't right

01:08:27.160 --> 01:08:28.680
anymore. So, even though the feature

01:08:28.680 --> 01:08:30.520
spec and the development was all good,

01:08:30.520 --> 01:08:32.000
and it worked, and it passed the the

01:08:32.000 --> 01:08:34.480
validation like our manual verification,

01:08:34.480 --> 01:08:36.120
it turned out that it changed a

01:08:36.120 --> 01:08:38.120
fundamental assumption in the security

01:08:38.120 --> 01:08:40.960
model. And as a result, um you were able

01:08:40.960 --> 01:08:42.200
to

01:08:42.200 --> 01:08:45.520
to bypass some authorization um guards

01:08:45.520 --> 01:08:47.440
on on the back end on the Next.js app.

01:08:47.440 --> 01:08:49.920
So, this type of problem can sneak in so

01:08:49.920 --> 01:08:51.440
easily, and it happens in Vercel, and it

01:08:51.440 --> 01:08:53.799
happens in Lovable. Lovable being

01:08:53.799 --> 01:08:56.280
billions of dollars and used by uh I

01:08:56.280 --> 01:08:58.920
don't know, huge companies um to to

01:08:58.920 --> 01:09:00.880
build their internal platform uh

01:09:00.880 --> 01:09:03.520
solutions quickly. And there's a clear

01:09:03.520 --> 01:09:05.560
security problem. There's a clear

01:09:05.560 --> 01:09:07.040
quality problem.

01:09:07.040 --> 01:09:11.359
Um and there's a clear Yeah, um

01:09:11.359 --> 01:09:13.040
problem. Well, the If you can solve the

01:09:13.040 --> 01:09:15.080
quality problem of AI,

01:09:15.080 --> 01:09:17.240
the thing that concerns me is that

01:09:17.240 --> 01:09:19.240
there's going to be a lots of problems

01:09:19.240 --> 01:09:22.319
that require some

01:09:22.319 --> 01:09:24.520
But most of these problems have Most of

01:09:24.520 --> 01:09:26.000
these problems are valid with users as

01:09:26.000 --> 01:09:27.960
well with humans as well, right? I mean

01:09:27.960 --> 01:09:33.400
agents are just slightly better humans.

01:09:33.400 --> 01:09:34.960
>> you know, I'm not a huge fan of using

01:09:34.960 --> 01:09:36.720
Jira and sprints. Don't get me wrong,

01:09:36.720 --> 01:09:39.080
but like there's there's an ebb there's

01:09:39.080 --> 01:09:41.680
a flow there's a familiarity with that

01:09:41.680 --> 01:09:44.160
sort of way of developing software and

01:09:44.160 --> 01:09:46.279
now we're going to go into a stage where

01:09:46.279 --> 01:09:47.640
where there's going to be a lot of

01:09:47.640 --> 01:09:51.000
conflict no doubt and and essentially

01:09:51.000 --> 01:09:52.600
people are going to lose people probably

01:09:52.600 --> 01:09:54.080
potentially going to lose their jobs

01:09:54.080 --> 01:09:55.280
because people are going to say now I

01:09:55.280 --> 01:09:57.320
don't want to do it that way basically.

01:09:57.320 --> 01:09:58.600
>> I mean

01:09:58.600 --> 01:10:02.080
I think I think it's true that

01:10:02.080 --> 01:10:04.880
because of the speed at which AI works a

01:10:04.880 --> 01:10:06.600
lot of the mechanisms that we have

01:10:06.600 --> 01:10:08.320
because I was about to say we have all

01:10:08.320 --> 01:10:10.040
the mechanisms in place with human

01:10:10.040 --> 01:10:12.720
review and with human quality control is

01:10:12.720 --> 01:10:14.800
the same thing. You must have your CICD

01:10:14.800 --> 01:10:16.400
security checks shift left whatever

01:10:16.400 --> 01:10:18.080
practices that you had you have to still

01:10:18.080 --> 01:10:19.280
apply them.

01:10:19.280 --> 01:10:22.320
But the volume output of AI agents make

01:10:22.320 --> 01:10:24.760
a lot of these original frameworks too

01:10:24.760 --> 01:10:26.520
much of a bottleneck that a lot of

01:10:26.520 --> 01:10:29.280
organizations are bypassing them or are

01:10:29.280 --> 01:10:30.760
shortcutting them

01:10:30.760 --> 01:10:33.120
or are just simply ignoring them because

01:10:33.120 --> 01:10:34.920
of the bottleneck that they are posing

01:10:34.920 --> 01:10:36.600
and they want to move fast and AI is

01:10:36.600 --> 01:10:39.200
doing has passed has been doing a good

01:10:39.200 --> 01:10:40.280
enough job long enough

01:10:40.280 --> 01:10:43.120
>> Yeah, like if if if you trust it.

01:10:43.120 --> 01:10:45.280
If you said to me

01:10:45.280 --> 01:10:46.400
that

01:10:46.400 --> 01:10:47.720
you we're we're enforcing some

01:10:47.720 --> 01:10:49.120
organizational standards and the

01:10:49.120 --> 01:10:50.920
standards would be something like you

01:10:50.920 --> 01:10:53.200
know, we we're going to do it like a QA

01:10:53.200 --> 01:10:55.840
phase and things like that. I can get

01:10:55.840 --> 01:10:58.440
behind that. I can't get behind when

01:10:58.440 --> 01:11:00.640
people say okay now we're going to have

01:11:00.640 --> 01:11:01.960
a

01:11:01.960 --> 01:11:03.640
everyone use you know

01:11:03.640 --> 01:11:07.280
you you know one two three QA or some

01:11:07.280 --> 01:11:11.840
some QA tool our QA tool like to do this

01:11:11.840 --> 01:11:13.760
thing. Like I I'm not too sure I can get

01:11:13.760 --> 01:11:15.640
behind that.

01:11:15.640 --> 01:11:17.200
You know what I mean? It's like the

01:11:17.200 --> 01:11:19.320
difference between

01:11:19.320 --> 01:11:21.080
sort of like a high level in principle

01:11:21.080 --> 01:11:23.400
what you want to do as opposed to like

01:11:23.400 --> 01:11:26.120
hey, here's some tool that now we think

01:11:26.120 --> 01:11:28.040
should be using now. Here's a model that

01:11:28.040 --> 01:11:29.680
everyone should be using. It's great.

01:11:29.680 --> 01:11:32.919
It's called GPT 3.0.

01:11:32.919 --> 01:11:33.400
>> [laughter]

01:11:33.400 --> 01:11:35.360
>> And that's and that's the standard that

01:11:35.360 --> 01:11:37.440
everyone needs to use. It's like oh no.

01:11:37.440 --> 01:11:39.160
Did you see also from Uber that they are

01:11:39.160 --> 01:11:41.040
like they blew through their AI budget

01:11:41.040 --> 01:11:42.560
and it's not even half half the year

01:11:42.560 --> 01:11:44.760
gone. They're like fire all the staff.

01:11:44.760 --> 01:11:46.680
They put in place a bunch of AI

01:11:46.680 --> 01:11:49.200
workflows and halfway down the year

01:11:49.200 --> 01:11:51.200
they're like we threw our budget. We

01:11:51.200 --> 01:11:53.120
cannot afford the AI tools anymore. Like

01:11:53.120 --> 01:11:55.189
we have to rehire humans.

01:11:55.189 --> 01:11:55.600
>> [laughter]

01:11:55.600 --> 01:11:56.760
>> Well, there's

01:11:56.760 --> 01:11:58.400
there's lots of talk about yeah token

01:11:58.400 --> 01:12:01.120
maxing is the is the meme, right?

01:12:01.120 --> 01:12:02.520
Where you have leaderboards and

01:12:02.520 --> 01:12:04.240
companies.

01:12:04.240 --> 01:12:06.160
Yeah, this quote to quote from Jensen as

01:12:06.160 --> 01:12:07.600
well saying if you have an engineer and

01:12:07.600 --> 01:12:10.000
you're paying him 200,000 dollars a year

01:12:10.000 --> 01:12:11.800
and he's not spending 50 like I don't

01:12:11.800 --> 01:12:12.600
know what was the number.

01:12:12.600 --> 01:12:14.520
>> Oh yeah, that's the other one.

01:12:14.520 --> 01:12:17.040
20% of this annual budget on In the

01:12:17.040 --> 01:12:19.160
enterprise in my my experience people

01:12:19.160 --> 01:12:21.120
are struggling just to get access still.

01:12:21.120 --> 01:12:22.200
People are just struggling to get

01:12:22.200 --> 01:12:24.040
access. We we have to like pinch

01:12:24.040 --> 01:12:26.320
ourselves Vincent but and and and who

01:12:26.320 --> 01:12:29.480
anyone who's listening we are pioneers

01:12:29.480 --> 01:12:31.960
here. We are pioneers. I'm I'm

01:12:31.960 --> 01:12:34.000
I'm sorry to say put the put your FOMO

01:12:34.000 --> 01:12:36.080
aside. We are basically pioneers at this

01:12:36.080 --> 01:12:37.280
point.

01:12:37.280 --> 01:12:38.440
We're

01:12:38.440 --> 01:12:41.480
So just chill. Just chill a little bit.

01:12:41.480 --> 01:12:43.840
Chill. Take stock.

01:12:43.840 --> 01:12:45.680
I thought you were going to like when

01:12:45.680 --> 01:12:48.080
you said we are and then I would like

01:12:48.080 --> 01:12:50.240
the huge Invincible title screen would

01:12:50.240 --> 01:12:52.040
pop up.

01:12:52.040 --> 01:12:52.920
I don't know I've been watching

01:12:52.920 --> 01:12:54.560
Invincible and so one of the things they

01:12:54.560 --> 01:12:56.720
do is at the start of the episode they

01:12:56.720 --> 01:12:59.280
will always frame a sentence and where

01:12:59.280 --> 01:13:01.760
it Invincible is then one of the like

01:13:01.760 --> 01:13:04.120
word plays that completes the sentence.

01:13:04.120 --> 01:13:05.720
Somebody would say like you think I am

01:13:05.720 --> 01:13:06.640
and then

01:13:06.640 --> 01:13:09.520
Invincible pops up and say we are

01:13:09.520 --> 01:13:10.280
pioneers.

01:13:10.280 --> 01:13:12.160
>> Is it a Disney thing or Oh yes, it's a

01:13:12.160 --> 01:13:14.560
Disney thing. It's a very gory it's a

01:13:14.560 --> 01:13:18.200
bit like The Boys on Amazon Prime.

01:13:18.200 --> 01:13:19.240
Invincible.

01:13:19.240 --> 01:13:21.600
>> catch a series called

01:13:21.600 --> 01:13:24.160
the Mr. In-Between on Disney.

01:13:24.160 --> 01:13:26.000
It's really good. Mr. In-Between on

01:13:26.000 --> 01:13:28.440
Disney? I stopped paying for Disney. It

01:13:28.440 --> 01:13:30.920
was it wasn't worth it. It's

01:13:30.920 --> 01:13:32.560
it's quite cheap here with the with the

01:13:32.560 --> 01:13:34.560
ads. It's like five bucks. Okay, let

01:13:34.560 --> 01:13:36.400
let's let's end it there. I I think

01:13:36.400 --> 01:13:38.240
we've talked for a solid hour if not

01:13:38.240 --> 01:13:40.560
more and I need to session alone and you

01:13:40.560 --> 01:13:42.080
have two more of 20 minutes each so

01:13:42.080 --> 01:13:44.240
that's like 1 hour and 40 minutes. I

01:13:44.240 --> 01:13:45.760
need to

01:13:45.760 --> 01:13:47.640
Have you ever done AWS certifications? I

01:13:47.640 --> 01:13:49.520
need to do mine next week. Oh god.

01:13:49.520 --> 01:13:51.320
>> Can't you let AI do it for you? No, you

01:13:51.320 --> 01:13:51.800
can't.

01:13:51.800 --> 01:13:54.640
>> You can you can ask Claude to build a

01:13:54.640 --> 01:13:56.560
battle cards for you. Like in just all

01:13:56.560 --> 01:13:58.640
those material. I've been actually

01:13:58.640 --> 01:14:00.040
vibing

01:14:00.040 --> 01:14:01.120
um

01:14:01.120 --> 01:14:03.080
I've I've shown you command and control

01:14:03.080 --> 01:14:04.080
on my mobile, right?

01:14:04.080 --> 01:14:05.960
>> Flash cards not battle cards. Command

01:14:05.960 --> 01:14:07.120
control yes.

01:14:07.120 --> 01:14:09.520
I've prompted it just to just to just to

01:14:09.520 --> 01:14:12.520
test me and I've been doing tests like

01:14:12.520 --> 01:14:14.960
that on my mobile which is nice cuz the

01:14:14.960 --> 01:14:18.520
tutorial Dojo online is is on the mobile

01:14:18.520 --> 01:14:20.680
web is a mess. Okay, let's end the

01:14:20.680 --> 01:14:22.800
conversation there for now. I thought I

01:14:22.800 --> 01:14:24.880
thought it was good. I mean

01:14:24.880 --> 01:14:26.400
I feel like we're just rehashing old

01:14:26.400 --> 01:14:28.840
topics sometimes but I think that's the

01:14:28.840 --> 01:14:29.200
part

01:14:29.200 --> 01:14:30.400
>> in the way that the ecosystem is

01:14:30.400 --> 01:14:31.720
constantly changing and some of the

01:14:31.720 --> 01:14:33.640
topics we've raised just prove to be

01:14:33.640 --> 01:14:35.960
true over and over like all of these

01:14:35.960 --> 01:14:37.120
Yeah. We didn't even talk about the

01:14:37.120 --> 01:14:39.720
supply chain attack now against check

01:14:39.720 --> 01:14:42.080
marks and it even impacting Bitwarden

01:14:42.080 --> 01:14:44.560
CLI which is like the secret management

01:14:44.560 --> 01:14:46.400
solution now being impacted by a supply

01:14:46.400 --> 01:14:49.080
chain attack. Today my my GitHub token

01:14:49.080 --> 01:14:51.040
just expired and somebody was like I

01:14:51.040 --> 01:14:53.200
know reminding me and I was like I'm not

01:14:53.200 --> 01:14:55.120
giving a long live token. I don't want

01:14:55.120 --> 01:14:56.840
the project to die because of a supply

01:14:56.840 --> 01:14:59.440
chain attack. I want multiple small I

01:14:59.440 --> 01:15:00.800
still have to get rid of the token

01:15:00.800 --> 01:15:02.960
though. Yeah, it's funny. We had that

01:15:02.960 --> 01:15:05.240
conversation a a few episodes ago when

01:15:05.240 --> 01:15:07.040
you were talking about

01:15:07.040 --> 01:15:09.160
the NPM what do you call it max age and

01:15:09.160 --> 01:15:10.880
all that sort of stuff and I've just

01:15:10.880 --> 01:15:12.960
been seeing in in my internal stuff like

01:15:12.960 --> 01:15:14.840
how how people are just cottoning on to

01:15:14.840 --> 01:15:17.240
this practice. I was like okay or you

01:15:17.240 --> 01:15:18.600
know, some people are talking about it

01:15:18.600 --> 01:15:20.680
but it's great because

01:15:20.680 --> 01:15:22.000
uh you

01:15:22.000 --> 01:15:24.160
Renovate bot apparently also has this

01:15:24.160 --> 01:15:26.440
now as a flag that you can you have to

01:15:26.440 --> 01:15:28.160
or you can set up your renovate to not

01:15:28.160 --> 01:15:30.440
accept any updates that are younger than

01:15:30.440 --> 01:15:32.520
a certain amount or a certain amount of

01:15:32.520 --> 01:15:35.000
days on the on the registry even for

01:15:35.000 --> 01:15:36.560
Terraform modules apparently. So that's

01:15:36.560 --> 01:15:38.000
interesting. I mean I still hate

01:15:38.000 --> 01:15:40.080
Terraform modules as a as a mechanisms

01:15:40.080 --> 01:15:41.600
for your infrastructure as code but as

01:15:41.600 --> 01:15:43.360
long as they're around it only makes

01:15:43.360 --> 01:15:45.520
sense that people keep duplicating any

01:15:45.520 --> 01:15:47.240
sane package manager features and

01:15:47.240 --> 01:15:49.240
rolling it out out into this stupid

01:15:49.240 --> 01:15:51.000
niche infrastructure as code packaging

01:15:51.000 --> 01:15:53.560
mechanism. I I hate Terraform modules.

01:15:53.560 --> 01:15:55.840
Okay. Why why do you need a like a

01:15:55.840 --> 01:15:57.960
separate package manager for Terraform

01:15:57.960 --> 01:15:59.520
modules and then you have to duplicate

01:15:59.520 --> 01:16:00.440
all of this stuff that you just

01:16:00.440 --> 01:16:02.400
mentioned against supply chain attacks?

01:16:02.400 --> 01:16:04.480
Yeah, yeah. Okay, this is kind of like

01:16:04.480 --> 01:16:07.440
premature defense because somebody was

01:16:07.440 --> 01:16:09.680
saying like when it hits when the supply

01:16:09.680 --> 01:16:11.600
chain attack hits on a Terraform module

01:16:11.600 --> 01:16:13.560
at least you should have this already in

01:16:13.560 --> 01:16:15.640
place before it hits, right? That's the

01:16:15.640 --> 01:16:18.200
point. Anyway, there's lots to do lots

01:16:18.200 --> 01:16:20.240
to talk about lots to talk about again I

01:16:20.240 --> 01:16:22.560
feel though. Anyway, thanks thanks

01:16:22.560 --> 01:16:24.160
Vincent for your time. I'm going to wrap

01:16:24.160 --> 01:16:26.440
it up here. If you want this far please

01:16:26.440 --> 01:16:29.400
like please subscribe please review

01:16:29.400 --> 01:16:32.320
please email us about something you

01:16:32.320 --> 01:16:35.520
would prefer us to talk about and yeah.

01:16:35.520 --> 01:16:37.680
Toodle toodle pip. Bye.

01:16:37.680 --> 01:16:39.720
>> Bye.