WEBVTT

00:00:01.199 --> 00:00:04.080
Welcome to episode 42

00:00:04.080 --> 00:00:06.879
of the AI infrastructure podcast with

00:00:06.879 --> 00:00:11.200
Vincent Dismat and myself Kai Hendry

00:00:11.200 --> 00:00:14.080
talking about AI workflows in the

00:00:14.080 --> 00:00:17.199
context of provisioning infrastructure.

00:00:17.199 --> 00:00:19.840
Today's topic is about agent idees. It's

00:00:19.840 --> 00:00:22.880
kind of like the next step over using

00:00:22.880 --> 00:00:25.840
claw code or Codex in your terminal. I

00:00:25.840 --> 00:00:28.400
think it has some potential. I've just

00:00:28.400 --> 00:00:29.679
been using it only for a couple of days

00:00:29.679 --> 00:00:31.279
and we're talking about that amongst

00:00:31.279 --> 00:00:33.120
other things. I hope you enjoy it.

00:00:33.120 --> 00:00:35.840
Please like, please rate the podcast on

00:00:35.840 --> 00:00:40.960
Spotify, Apple, Music, uh YouTube,

00:00:40.960 --> 00:00:44.480
comment, you know the drill. Thanks and

00:00:44.480 --> 00:00:45.600
enjoy.

00:00:45.600 --> 00:00:51.200
>> I have been using AI and Orca and

00:00:51.200 --> 00:00:53.280
>> Oh, share about Orca. Sorry, I couldn't

00:00:53.280 --> 00:00:56.719
see your uh breakdown, but you've used

00:00:56.719 --> 00:00:59.920
Herder. I've used Semox. So maybe you

00:00:59.920 --> 00:01:03.520
can show Orca and explain how you like

00:01:03.520 --> 00:01:05.040
it compared to Herder.

00:01:05.040 --> 00:01:06.240
>> Well, to be honest, I haven't really

00:01:06.240 --> 00:01:10.000
used Herder in anger in all honesty.

00:01:10.000 --> 00:01:11.600
>> I saw that you were trying to do a

00:01:11.600 --> 00:01:14.240
comparison between two agents. Kind of

00:01:14.240 --> 00:01:16.560
boring. Just show me like something

00:01:16.560 --> 00:01:17.520
interesting.

00:01:17.520 --> 00:01:19.439
>> Oh

00:01:19.439 --> 00:01:22.640
Okay. Well, let me just share my screen.

00:01:22.640 --> 00:01:24.640
I mean, I've only really used it for the

00:01:24.640 --> 00:01:28.240
like two days solidly. solidly two days

00:01:28.240 --> 00:01:32.880
is that a lot that's not much is it? So

00:01:32.880 --> 00:01:35.040
for example,

00:01:35.040 --> 00:01:37.200
what do I like about it? Okay, so one

00:01:37.200 --> 00:01:41.200
thing I like about it is is that

00:01:41.200 --> 00:01:44.880
it's supports all the the different

00:01:44.880 --> 00:01:47.520
agents easily. I like how it's like kind

00:01:47.520 --> 00:01:50.479
of terminal based in a way like the tabs

00:01:50.479 --> 00:01:52.399
are are kind of terminals.

00:01:52.399 --> 00:01:53.759
>> Looks like CMX

00:01:53.759 --> 00:01:56.079
>> and then it's easy to start a new work

00:01:56.079 --> 00:01:58.399
tree with a different thing

00:01:58.399 --> 00:02:00.960
>> and a different configuration. It has

00:02:00.960 --> 00:02:03.759
has mobile so that's that's a killer

00:02:03.759 --> 00:02:06.159
feature for me. So that so that so now I

00:02:06.159 --> 00:02:08.640
can you know check what's going on my

00:02:08.640 --> 00:02:09.599
mobile

00:02:09.599 --> 00:02:11.440
>> but that means you have to pay their cl

00:02:11.440 --> 00:02:14.400
for their cloud service. I I use the the

00:02:14.400 --> 00:02:17.200
local one, but the trouble is

00:02:17.200 --> 00:02:19.200
>> But it must still have like some type of

00:02:19.200 --> 00:02:22.239
control plane for your mobile app to

00:02:22.239 --> 00:02:24.959
connect to your to your MacBook, right?

00:02:24.959 --> 00:02:27.599
Some type of reverse tunnel.

00:02:27.599 --> 00:02:30.640
>> Well, I mean, I I manage my own LAN and

00:02:30.640 --> 00:02:32.879
I use Tail Scale, so I don't use their

00:02:32.879 --> 00:02:34.319
their stuff.

00:02:34.319 --> 00:02:36.720
>> Oh, so your so so is there requirement

00:02:36.720 --> 00:02:39.280
that your mobile phone and the Mac are

00:02:39.280 --> 00:02:41.280
on the same network?

00:02:41.280 --> 00:02:42.560
>> Exactly. on LAN.

00:02:42.560 --> 00:02:45.440
>> Yes. So, basically I just use the LAN

00:02:45.440 --> 00:02:46.480
and it works.

00:02:46.480 --> 00:02:48.800
>> Oh, perfect. So, you can set up a VPN

00:02:48.800 --> 00:02:51.280
and then you can be out of home and they

00:02:51.280 --> 00:02:53.280
can still be on a network on a virtual

00:02:53.280 --> 00:02:54.080
network.

00:02:54.080 --> 00:02:56.080
>> Yeah. The there is a there is a big

00:02:56.080 --> 00:02:57.760
downside that I haven't really figured

00:02:57.760 --> 00:02:59.040
out is that um

00:02:59.040 --> 00:03:00.959
>> so somebody Well, when they replay it,

00:03:00.959 --> 00:03:02.959
this code will be will no longer be

00:03:02.959 --> 00:03:03.840
valid, right?

00:03:03.840 --> 00:03:05.760
>> I hope so.

00:03:05.760 --> 00:03:07.760
>> Sorry. One big downside you were saying

00:03:07.760 --> 00:03:09.840
>> one big downside is that uh this is like

00:03:09.840 --> 00:03:11.680
a what do you call it? first world

00:03:11.680 --> 00:03:14.560
problem. But like I use Orca on this

00:03:14.560 --> 00:03:18.080
machine and I use and then when I'm in

00:03:18.080 --> 00:03:21.920
my in my home which is it's just like 20

00:03:21.920 --> 00:03:25.280
m that way. I use another MacBook Pro.

00:03:25.280 --> 00:03:27.760
So the trouble is is that is that like

00:03:27.760 --> 00:03:30.400
I've paired this with my I've paired my

00:03:30.400 --> 00:03:32.319
phone with the other Orca. I haven't

00:03:32.319 --> 00:03:33.840
paired it with this Orca. So I don't

00:03:33.840 --> 00:03:36.080
know if I don't know. I I guess I could

00:03:36.080 --> 00:03:37.120
just set up a new one.

00:03:37.120 --> 00:03:38.480
>> It's a paid feature. I

00:03:38.480 --> 00:03:40.879
>> I'm sure this will work to be honest.

00:03:40.879 --> 00:03:43.120
They must have a business model and some

00:03:43.120 --> 00:03:44.720
type of paid

00:03:44.720 --> 00:03:47.120
>> I can see two hosts. Oh, it does work.

00:03:47.120 --> 00:03:50.159
It does work. I can see two hosts now.

00:03:50.159 --> 00:03:54.080
But but the problem is is that is that

00:03:54.080 --> 00:03:56.879
well I mean it's it's I'm just confusing

00:03:56.879 --> 00:03:58.480
things a bit. But like I basically work

00:03:58.480 --> 00:04:00.080
from two laptops is what I'm trying to

00:04:00.080 --> 00:04:02.400
say. So okay. So the the mobile

00:04:02.400 --> 00:04:05.200
integration is great. Awesome. The other

00:04:05.200 --> 00:04:08.879
thing that I that's quite nice is that

00:04:08.879 --> 00:04:11.360
for example actually I haven't tested

00:04:11.360 --> 00:04:13.840
this thing just I'm just working with

00:04:13.840 --> 00:04:16.160
Can you see?

00:04:16.160 --> 00:04:19.840
Oh no I can't see I I'm using SSH on one

00:04:19.840 --> 00:04:22.079
of my hosts here. I was hoping that it

00:04:22.079 --> 00:04:26.000
would do the image pushing stuff. It

00:04:26.000 --> 00:04:29.280
doesn't seem to work.

00:04:29.280 --> 00:04:31.520
Oh that sucks. Okay that's another

00:04:31.520 --> 00:04:33.040
negative point. I'm just finding all the

00:04:33.040 --> 00:04:34.560
negatives here.

00:04:34.560 --> 00:04:37.759
like the SSH doesn't work to send images

00:04:37.759 --> 00:04:40.400
across and then other than that it's

00:04:40.400 --> 00:04:42.400
just it's just quite tasteful like you

00:04:42.400 --> 00:04:45.759
can see the files

00:04:45.759 --> 00:04:49.120
the sessions the git changes yeah the g

00:04:49.120 --> 00:04:50.800
I haven't got any git changes to show

00:04:50.800 --> 00:04:52.960
you but it's quite the uh the pager to

00:04:52.960 --> 00:04:56.960
to to view uh diffs is quite nice and

00:04:56.960 --> 00:04:59.680
you can you can comment on them so you

00:04:59.680 --> 00:05:01.600
so you can do things

00:05:01.600 --> 00:05:03.840
>> direct the agent that's really nice I Is

00:05:03.840 --> 00:05:06.080
this the future of an IDE? Then

00:05:06.080 --> 00:05:07.759
>> I think this is getting damn close to

00:05:07.759 --> 00:05:09.600
it. I mean, this is this is something I

00:05:09.600 --> 00:05:12.400
wish for like a month or two ago and I

00:05:12.400 --> 00:05:14.080
should have just got off my ass in some

00:05:14.080 --> 00:05:16.080
ways to it's I guess everything like

00:05:16.080 --> 00:05:18.080
this is in hindsight. It's like I wanted

00:05:18.080 --> 00:05:20.720
this a couple of months ago and now it's

00:05:20.720 --> 00:05:24.400
here and um is this the future? Well, it

00:05:24.400 --> 00:05:26.560
doesn't it certainly has a lot of

00:05:26.560 --> 00:05:28.560
features that are better than I mean a

00:05:28.560 --> 00:05:31.840
lot more than CMOX. CMOX is quite old

00:05:31.840 --> 00:05:33.120
and I haven't really been paying

00:05:33.120 --> 00:05:35.120
attention to new capabilities.

00:05:35.120 --> 00:05:36.800
>> It does have like like one thing that

00:05:36.800 --> 00:05:38.560
impressed me is that like when I when I

00:05:38.560 --> 00:05:41.039
create a new work tree for example here

00:05:41.039 --> 00:05:43.680
on one of my projects it it sort of

00:05:43.680 --> 00:05:46.400
automatically you know gets the git

00:05:46.400 --> 00:05:47.520
>> you call that

00:05:47.520 --> 00:05:49.680
>> the the

00:05:49.680 --> 00:05:51.759
automatically gets the git issues. Sorry

00:05:51.759 --> 00:05:54.080
I'm really slow. And then for example,

00:05:54.080 --> 00:05:56.800
yeah, I want to work on issue here with

00:05:56.800 --> 00:06:00.800
say Codex for a change and then booms.

00:06:00.800 --> 00:06:01.759
>> That's nice.

00:06:01.759 --> 00:06:03.759
>> And it it doesn't like do too much

00:06:03.759 --> 00:06:06.720
trickery in a way. Like for example, it

00:06:06.720 --> 00:06:08.880
just pre-populates it with the the issue

00:06:08.880 --> 00:06:12.479
number and then I go doink. So it's just

00:06:12.479 --> 00:06:13.039
little

00:06:13.039 --> 00:06:14.800
>> that sent a message. Mhm.

00:06:14.800 --> 00:06:17.440
>> It's just just a tiny bit of polish. You

00:06:17.440 --> 00:06:19.360
know what I mean? It's just like does

00:06:19.360 --> 00:06:21.520
what you expect it to do. It's just

00:06:21.520 --> 00:06:25.600
tasteful. Does it support like u reviews

00:06:25.600 --> 00:06:29.039
of markdown files or like in a rendered

00:06:29.039 --> 00:06:31.600
markdown view? Does it have artifact

00:06:31.600 --> 00:06:35.120
support that you can easily

00:06:35.120 --> 00:06:37.280
maybe get an like an overview of a

00:06:37.280 --> 00:06:38.960
composition of modules that you can

00:06:38.960 --> 00:06:40.160
review?

00:06:40.160 --> 00:06:42.240
>> I haven't tried that but I mean but it

00:06:42.240 --> 00:06:45.840
it's just it is just as you can see a

00:06:45.840 --> 00:06:48.560
clawed code. I mean I could I could just

00:06:48.560 --> 00:06:51.360
run artifacts here, couldn't I? or what?

00:06:51.360 --> 00:06:53.440
>> Yeah, but I'm interested in like what

00:06:53.440 --> 00:06:55.759
happens when the artifact

00:06:55.759 --> 00:06:58.319
uh is available uh like created by the

00:06:58.319 --> 00:07:00.319
session. Does it open an external

00:07:00.319 --> 00:07:03.840
browser? I would imagine and then um I

00:07:03.840 --> 00:07:05.360
guess

00:07:05.360 --> 00:07:07.680
giving feedback on because we were

00:07:07.680 --> 00:07:10.720
looking at a while ago people are trying

00:07:10.720 --> 00:07:12.479
to break down work with planning

00:07:12.479 --> 00:07:14.080
artifacts.

00:07:14.080 --> 00:07:15.840
Markdown isn't cutting it anymore. We're

00:07:15.840 --> 00:07:17.840
moving more and more towards HTML. Those

00:07:17.840 --> 00:07:20.479
HTML documents need to receive comments

00:07:20.479 --> 00:07:21.280
in line.

00:07:21.280 --> 00:07:21.680
>> Oh, wow.

00:07:21.680 --> 00:07:23.280
>> And that needs to go back to the agent,

00:07:23.280 --> 00:07:23.919
right?

00:07:23.919 --> 00:07:25.520
>> Oh, wow. Yeah. Yeah, I think I know what

00:07:25.520 --> 00:07:28.080
you mean. No, I haven't I haven't got I

00:07:28.080 --> 00:07:31.280
haven't got as far as doing multiplayer

00:07:31.280 --> 00:07:33.440
HTML,

00:07:33.440 --> 00:07:34.400
if that's what you'd say.

00:07:34.400 --> 00:07:36.080
>> Yeah, I haven't played multiplayer. I I

00:07:36.080 --> 00:07:39.759
think multiplayer HTML or or MDX, which

00:07:39.759 --> 00:07:42.479
is Markdown with React components. So

00:07:42.479 --> 00:07:45.039
you can build reusable component library

00:07:45.039 --> 00:07:47.680
that intuitively fit within a markdown

00:07:47.680 --> 00:07:49.440
file but that get rendered interactively

00:07:49.440 --> 00:07:51.919
within a browser window. I really look

00:07:51.919 --> 00:07:54.960
forward to this type of interactivity.

00:07:54.960 --> 00:07:58.720
Imagine that you have a an ontology or a

00:07:58.720 --> 00:08:01.759
glossery of a sequence diagram component

00:08:01.759 --> 00:08:04.720
model and then you just tell the agent

00:08:04.720 --> 00:08:08.240
build it and it uses the you know like a

00:08:08.240 --> 00:08:09.680
React component

00:08:09.680 --> 00:08:11.599
>> inside the markdown and then when you

00:08:11.599 --> 00:08:14.319
view it in in a browser the React

00:08:14.319 --> 00:08:15.840
renders as an actually interactive

00:08:15.840 --> 00:08:17.520
sequence diagram

00:08:17.520 --> 00:08:20.160
>> that allows you to plan uh nicely,

00:08:20.160 --> 00:08:20.720
right?

00:08:20.720 --> 00:08:23.440
>> Yeah. Okay. Get get alignment and things

00:08:23.440 --> 00:08:25.680
like that. Yeah, that sounds quite cool.

00:08:25.680 --> 00:08:28.240
>> So, in in a way, I think Orcai looks

00:08:28.240 --> 00:08:31.520
interesting compared to more like

00:08:31.520 --> 00:08:34.159
personal agents like Kiro Crew that we

00:08:34.159 --> 00:08:36.560
looked at a while ago. I think I do not

00:08:36.560 --> 00:08:38.560
sure if I was able to show it back then,

00:08:38.560 --> 00:08:40.880
but with Kiro Crew, the way that the

00:08:40.880 --> 00:08:43.599
artifacts is integrated. It's not a

00:08:43.599 --> 00:08:46.160
terminal shell, but Kiro Crew under the

00:08:46.160 --> 00:08:49.760
under the hood can run Codex well if if

00:08:49.760 --> 00:08:52.480
or can run Kiro terminal and you can

00:08:52.480 --> 00:08:54.080
also get access to the terminal and it

00:08:54.080 --> 00:08:56.480
all runs in a remote machine. So

00:08:56.480 --> 00:09:00.399
currently I I'm able to use my Kir crew

00:09:00.399 --> 00:09:02.800
that's running on my uh desktop at home

00:09:02.800 --> 00:09:05.519
>> and I'm able to tunnel to it from work

00:09:05.519 --> 00:09:08.399
through a mic or like a nano instance

00:09:08.399 --> 00:09:10.640
running in AWS that is running tail

00:09:10.640 --> 00:09:13.040
scale as well. So, I'm actually using it

00:09:13.040 --> 00:09:15.600
as a jump host. So, I'm sshing out of

00:09:15.600 --> 00:09:17.519
work u or wherever I am.

00:09:17.519 --> 00:09:19.600
>> You can set up Orca, I believe in the

00:09:19.600 --> 00:09:21.839
documentation, on a on a on a remote VPS

00:09:21.839 --> 00:09:23.519
if you wanted to, by the way.

00:09:23.519 --> 00:09:24.880
>> But I I don't want to set it up on a

00:09:24.880 --> 00:09:26.800
remote VPS like I have a powerful

00:09:26.800 --> 00:09:28.320
hardware at home. It doesn't really

00:09:28.320 --> 00:09:30.240
matter like cuz ultimately I just told

00:09:30.240 --> 00:09:32.880
cloud look uh I want the jump host in

00:09:32.880 --> 00:09:35.279
the cloud that has a fixed IP. So, I pay

00:09:35.279 --> 00:09:37.680
for what is it? how much US dollar you

00:09:37.680 --> 00:09:40.320
pay for a for um

00:09:40.320 --> 00:09:43.600
>> elastic IP in AWS and then you you just

00:09:43.600 --> 00:09:44.959
jump towards it

00:09:44.959 --> 00:09:46.959
>> I don't see the point I mean my my home

00:09:46.959 --> 00:09:50.560
internet is I've got like static IP it's

00:09:50.560 --> 00:09:52.240
very easy for me to jump back into my

00:09:52.240 --> 00:09:53.839
home internet so

00:09:53.839 --> 00:09:55.920
>> if you have a static IP yeah for me

00:09:55.920 --> 00:09:58.959
>> I'm hash or tail scale even

00:09:58.959 --> 00:10:01.040
>> yeah well that's a problem if your

00:10:01.040 --> 00:10:04.959
device is um is is controlled by by um

00:10:04.959 --> 00:10:07.839
you know network, it's easier to SSH

00:10:07.839 --> 00:10:10.880
out. Funny thing is when I told cloud to

00:10:10.880 --> 00:10:13.200
set this up, I was talking about working

00:10:13.200 --> 00:10:15.279
from hotel rooms and working from coffee

00:10:15.279 --> 00:10:18.079
shops and and it was like and working

00:10:18.079 --> 00:10:20.399
from office and it it actually set up a

00:10:20.399 --> 00:10:22.399
whole bunch of like constraints because

00:10:22.399 --> 00:10:25.519
it it runs SSH on 443. So it's an it

00:10:25.519 --> 00:10:28.800
looks like an HTTPS session, but within

00:10:28.800 --> 00:10:31.519
there's a tunnel actually because a lot

00:10:31.519 --> 00:10:34.480
of offices will block port 22.

00:10:34.480 --> 00:10:36.959
>> Yeah. But but tail scale gets around all

00:10:36.959 --> 00:10:38.160
that stuff.

00:10:38.160 --> 00:10:41.040
>> No, but that again if you listen to what

00:10:41.040 --> 00:10:43.440
I said at the beginning which is of that

00:10:43.440 --> 00:10:45.920
sentence which is a device that's

00:10:45.920 --> 00:10:48.160
controlled you cannot just install any

00:10:48.160 --> 00:10:49.440
networking on top of

00:10:49.440 --> 00:10:51.600
>> see okay right

00:10:51.600 --> 00:10:53.839
>> if I try to install the tail scale

00:10:53.839 --> 00:10:55.600
client

00:10:55.600 --> 00:10:57.200
immediately gets blocked by

00:10:57.200 --> 00:10:58.320
>> Oh really?

00:10:58.320 --> 00:11:00.800
>> Yeah. So if you can create an SSH tunnel

00:11:00.800 --> 00:11:03.519
it's pretty simple. That's that's one

00:11:03.519 --> 00:11:06.240
issue that I have with with work and and

00:11:06.240 --> 00:11:09.440
clients is that is that uh the mix of

00:11:09.440 --> 00:11:11.839
devices and because like I want to

00:11:11.839 --> 00:11:14.079
leverage my home infrastructure to do my

00:11:14.079 --> 00:11:16.720
work and I do for the most part like

00:11:16.720 --> 00:11:20.160
like you know I do get a laptop from

00:11:20.160 --> 00:11:22.800
work but like the monitors, the mobile

00:11:22.800 --> 00:11:27.120
phone, the ISP, all that stuff is is in

00:11:27.120 --> 00:11:29.440
my domain, right? And why can't I

00:11:29.440 --> 00:11:31.600
leverage it? It's it's insane if I can't

00:11:31.600 --> 00:11:35.360
leverage my own network and my own

00:11:35.360 --> 00:11:38.959
my own just to basically I just want to

00:11:38.959 --> 00:11:41.839
make sure that I can just I can just

00:11:41.839 --> 00:11:43.680
work from my mobile phone is what I'm

00:11:43.680 --> 00:11:47.519
trying to say. Yes. I think I think

00:11:47.519 --> 00:11:50.399
still I mean Orca can turn your your

00:11:50.399 --> 00:11:54.720
personal laptop into a harness for your

00:11:54.720 --> 00:11:57.600
like leveraging those Claude Code or

00:11:57.600 --> 00:12:00.640
codecs or PI whatever harness you choose

00:12:00.640 --> 00:12:03.760
via terminal but I think there's a

00:12:03.760 --> 00:12:05.279
couple of tools that go a level above

00:12:05.279 --> 00:12:09.600
that that turn a device into kind of an

00:12:09.600 --> 00:12:13.680
assistant that has its own harness on it

00:12:13.680 --> 00:12:16.000
and that you talk to like you the

00:12:16.000 --> 00:12:17.680
original

00:12:17.680 --> 00:12:19.839
claw Kiro Crew.

00:12:19.839 --> 00:12:21.839
>> No, that you talk to like it it's not

00:12:21.839 --> 00:12:23.920
really a terminal. It's more like you

00:12:23.920 --> 00:12:25.920
give it instructions and it gives you

00:12:25.920 --> 00:12:28.320
overviews of the plans. I don't know if

00:12:28.320 --> 00:12:32.079
like if if the if the layer should be at

00:12:32.079 --> 00:12:33.680
the terminal or it should be at like a

00:12:33.680 --> 00:12:35.200
higher level assistant like when you

00:12:35.200 --> 00:12:37.920
work with Hermes or with open claw or

00:12:37.920 --> 00:12:41.200
with Kira crew um the agent has full

00:12:41.200 --> 00:12:44.000
authority over its machine. You can give

00:12:44.000 --> 00:12:46.800
it a VM and uh and then from there it

00:12:46.800 --> 00:12:49.839
can install any binaries. It can set up

00:12:49.839 --> 00:12:52.240
any project workspace. It can the same

00:12:52.240 --> 00:12:53.680
thing that you give your Claude Code the

00:12:53.680 --> 00:12:55.440
ability to manage like you know work

00:12:55.440 --> 00:12:57.200
trees within your repositories on your

00:12:57.200 --> 00:12:59.120
machine. Uh and it acts with your

00:12:59.120 --> 00:13:00.800
authority. I guess that's the

00:13:00.800 --> 00:13:02.880
difference. I guess it's it's a it's a

00:13:02.880 --> 00:13:04.639
tool that runs on a machine that's like

00:13:04.639 --> 00:13:06.399
set up for you to work with that you

00:13:06.399 --> 00:13:08.959
were acts under your authority versus an

00:13:08.959 --> 00:13:11.920
agent that has its own authority and

00:13:11.920 --> 00:13:16.320
that you uh you know interact with and

00:13:16.320 --> 00:13:18.240
you know you give it tasks so it can run

00:13:18.240 --> 00:13:20.480
on its own it can have a chron to pick

00:13:20.480 --> 00:13:22.959
up work you know like the is radar and

00:13:22.959 --> 00:13:23.440
this type

00:13:23.440 --> 00:13:26.720
>> I see I see yeah this is like level

00:13:26.720 --> 00:13:30.320
yeah okay yeah that's yeah like I think

00:13:30.320 --> 00:13:31.600
I think I know you're saying, Vincent,

00:13:31.600 --> 00:13:33.360
you're saying that like you you want a

00:13:33.360 --> 00:13:35.519
host, you want infrastructure so that

00:13:35.519 --> 00:13:37.680
you can have like a whole pipeline of

00:13:37.680 --> 00:13:39.920
agents working while you sleep. That's

00:13:39.920 --> 00:13:41.360
that's the huge problem I have. It's

00:13:41.360 --> 00:13:43.839
like every every time I go to bed, I

00:13:43.839 --> 00:13:46.560
basically shut down my laptop and

00:13:46.560 --> 00:13:49.600
everything stops. And uh and it's the

00:13:49.600 --> 00:13:51.920
same way for work. Like for example, you

00:13:51.920 --> 00:13:54.000
work 9 to5, you have eight billable

00:13:54.000 --> 00:13:57.199
hours. Do you want your agents running

00:13:57.199 --> 00:13:59.839
in the middle of the night so that you

00:13:59.839 --> 00:14:02.000
maintain them while you brush your teeth

00:14:02.000 --> 00:14:03.920
as you go to bed? That sounds kind of

00:14:03.920 --> 00:14:05.279
crazy, right?

00:14:05.279 --> 00:14:06.880
>> For me also, when I have to move from

00:14:06.880 --> 00:14:09.360
office to home and I I spend 30 minutes,

00:14:09.360 --> 00:14:11.600
I have to close my laptop and I I

00:14:11.600 --> 00:14:13.519
usually drive and if there's dynamic

00:14:13.519 --> 00:14:15.760
workflows running on my machine, if

00:14:15.760 --> 00:14:17.839
there's network connection issues or the

00:14:17.839 --> 00:14:19.920
machine goes to sleep, then it

00:14:19.920 --> 00:14:22.880
completely messes up the thing. So

00:14:22.880 --> 00:14:25.440
having all of that run remotely is kind

00:14:25.440 --> 00:14:27.360
of the dream, right? Um

00:14:27.360 --> 00:14:29.360
>> yeah, that's I I see what you're saying.

00:14:29.360 --> 00:14:31.600
Yeah, I I I I've yet to solve that

00:14:31.600 --> 00:14:34.560
problem myself really. The the other

00:14:34.560 --> 00:14:36.800
problem it's this is more to do with my

00:14:36.800 --> 00:14:40.320
work and my clients is that now that I'm

00:14:40.320 --> 00:14:43.279
using Orca, I see the power of switching

00:14:43.279 --> 00:14:47.440
between uh Codex and and and Claude and

00:14:47.440 --> 00:14:49.680
I'm actually blown away how good Codex

00:14:49.680 --> 00:14:52.480
is. I'm blown away. It's so much faster

00:14:52.480 --> 00:14:53.920
than Claude.

00:14:53.920 --> 00:14:56.160
>> Is it still open source? I thought the

00:14:56.160 --> 00:14:57.199
open

00:14:57.199 --> 00:14:59.519
>> No, it's not open source. It's the least

00:14:59.519 --> 00:15:00.720
>> used to be right.

00:15:00.720 --> 00:15:01.760
>> Is it?

00:15:01.760 --> 00:15:03.839
>> It used to be open source. It used to be

00:15:03.839 --> 00:15:05.839
like Claude Code is s closed source and

00:15:05.839 --> 00:15:07.920
codex is open source.

00:15:07.920 --> 00:15:11.040
>> I think at some point well yeah it's a

00:15:11.040 --> 00:15:13.199
patch 2 license. Interesting.

00:15:13.199 --> 00:15:15.040
>> It's kind of maybe that's why people are

00:15:15.040 --> 00:15:16.880
like more people are able to contribute

00:15:16.880 --> 00:15:20.160
and focus on performance. Well,

00:15:20.160 --> 00:15:21.920
nice.

00:15:21.920 --> 00:15:24.320
>> Oh, GitHub checks are always just messy,

00:15:24.320 --> 00:15:25.600
aren't they? The

00:15:25.600 --> 00:15:27.519
>> you know, Hero Crew, you can't get

00:15:27.519 --> 00:15:29.120
anything merged without

00:15:29.120 --> 00:15:31.440
>> Okay. Okay. Going back to my point here

00:15:31.440 --> 00:15:33.600
is that most

00:15:33.600 --> 00:15:37.040
like like my current client, they they

00:15:37.040 --> 00:15:39.120
give every employee a clawed

00:15:39.120 --> 00:15:42.079
subscription, which is nice,

00:15:42.079 --> 00:15:44.880
but you don't get Codex.

00:15:44.880 --> 00:15:47.760
So basically I feel like my world, you

00:15:47.760 --> 00:15:49.440
know, personally speaking, I have Claude

00:15:49.440 --> 00:15:52.160
and Codex and I've got other ones too.

00:15:52.160 --> 00:15:54.560
My I feel like my mind is expanding. I

00:15:54.560 --> 00:15:56.560
feel like my internet has just got

00:15:56.560 --> 00:15:59.360
higher bandwidth because it's really

00:15:59.360 --> 00:16:01.759
interesting to see what different models

00:16:01.759 --> 00:16:03.839
are doing with the same problem.

00:16:03.839 --> 00:16:05.759
>> And it's very easy to do in Orca. And

00:16:05.759 --> 00:16:07.360
I'm like I'm actually blown away with it

00:16:07.360 --> 00:16:11.199
with open AI. I feel like Theo,

00:16:11.199 --> 00:16:15.040
what's his name? Theo GG3 T3 GG

00:16:15.040 --> 00:16:18.480
>> and I'm like I'm like I'm dissing Claude

00:16:18.480 --> 00:16:21.360
now. But it's it's just interesting. So

00:16:21.360 --> 00:16:23.600
>> I think it I think it's never good to

00:16:23.600 --> 00:16:26.639
just make an absolute blanket statement.

00:16:26.639 --> 00:16:28.959
It we we've covered this that they're

00:16:28.959 --> 00:16:30.959
constantly trading blows. One will be

00:16:30.959 --> 00:16:31.839
ahead. Yeah.

00:16:31.839 --> 00:16:34.959
>> It's very uh problem specific. It's very

00:16:34.959 --> 00:16:37.920
time time sensitive. It's very

00:16:37.920 --> 00:16:41.199
>> um you know personal feeling anecdotes.

00:16:41.199 --> 00:16:42.399
Some people will have a really good

00:16:42.399 --> 00:16:44.560
experience with Opus. Other people will

00:16:44.560 --> 00:16:46.639
say Sona is really a lot better

00:16:46.639 --> 00:16:48.639
depending on the on on the context

00:16:48.639 --> 00:16:50.720
depending on and this is I think a hard

00:16:50.720 --> 00:16:53.519
bit as you work with Claude Code you it

00:16:53.519 --> 00:16:55.920
creates a lot of memories on outside of

00:16:55.920 --> 00:16:59.920
your repository. So even if I was I was

00:16:59.920 --> 00:17:04.559
asking someone you know I

00:17:04.559 --> 00:17:06.559
I I asked it to make a lot of memory. So

00:17:06.559 --> 00:17:09.439
I want to see I I I I was on a call and

00:17:09.439 --> 00:17:11.520
because they asked me how do we do this

00:17:11.520 --> 00:17:14.160
and I said look I took the question that

00:17:14.160 --> 00:17:16.240
they put to me I put it straight into

00:17:16.240 --> 00:17:19.439
the Claude Code while sharing screen and

00:17:19.439 --> 00:17:21.760
I said uh it created the PR it

00:17:21.760 --> 00:17:23.360
automatically figured out a bunch of

00:17:23.360 --> 00:17:25.600
stuff and I said you know I assume it's

00:17:25.600 --> 00:17:28.400
because I built the cloud MD properly

00:17:28.400 --> 00:17:30.160
but I'm not sure if it's because of

00:17:30.160 --> 00:17:32.240
memories on my machine. So I would like

00:17:32.240 --> 00:17:35.440
you to repeat the same thing. Yeah,

00:17:35.440 --> 00:17:38.000
>> do not like let it go like maybe I don't

00:17:38.000 --> 00:17:40.000
push the branch. I already opened the

00:17:40.000 --> 00:17:42.400
PR, but do not let it like see the the

00:17:42.400 --> 00:17:44.720
result of of of work that it did from my

00:17:44.720 --> 00:17:47.440
machine. Uh and and and ask the same

00:17:47.440 --> 00:17:50.320
question and let's see if this is like

00:17:50.320 --> 00:17:52.480
because of memories on my machine versus

00:17:52.480 --> 00:17:53.679
>> Yeah. Yeah. I've noticed this memory

00:17:53.679 --> 00:17:56.160
thing like I I think when I saw memory

00:17:56.160 --> 00:17:58.080
in Claude, I turned it off, but now it's

00:17:58.080 --> 00:18:00.240
somehow back again. And to be honest,

00:18:00.240 --> 00:18:00.960
>> oh yes,

00:18:00.960 --> 00:18:03.600
>> with with with Hermes, I've noticed that

00:18:03.600 --> 00:18:06.000
you just get it by default and that's

00:18:06.000 --> 00:18:07.600
that, right? So,

00:18:07.600 --> 00:18:09.120
>> what what's Hermes have to do with this?

00:18:09.120 --> 00:18:10.880
Because Hermes is an agent that works on

00:18:10.880 --> 00:18:13.120
its own. It's very different, right?

00:18:13.120 --> 00:18:14.720
>> Well, it's not the same.

00:18:14.720 --> 00:18:16.400
>> It's it's not the same, but I've noticed

00:18:16.400 --> 00:18:18.799
that uh that when when I've noticed with

00:18:18.799 --> 00:18:22.160
Hermes that it knows about me, it

00:18:22.160 --> 00:18:24.720
>> agents agents, they are not just a

00:18:24.720 --> 00:18:26.400
harness, right? They are a harness plus

00:18:26.400 --> 00:18:29.600
a memory system plus a gateway plus a

00:18:29.600 --> 00:18:31.280
completely isolated like some people run

00:18:31.280 --> 00:18:34.480
an agent on their machine which I don't

00:18:34.480 --> 00:18:36.240
but like I believe an agent should be

00:18:36.240 --> 00:18:38.559
able should have its own identity it

00:18:38.559 --> 00:18:40.559
should have its memory system it should

00:18:40.559 --> 00:18:43.360
have a gateway connectivity and a

00:18:43.360 --> 00:18:45.760
dashboard like once you add a dashboard

00:18:45.760 --> 00:18:47.840
on top of an agent system it's pretty

00:18:47.840 --> 00:18:49.679
cool like what what Kiro Crew did there

00:18:49.679 --> 00:18:51.280
is really nice I haven't been able to

00:18:51.280 --> 00:18:54.480
>> what do you mean by a dashboard like a

00:18:54.480 --> 00:18:55.840
>> yeah I I think that's what I missed from

00:18:55.840 --> 00:18:58.000
Hermes because like I was using it only

00:18:58.000 --> 00:19:00.799
via gateway like I was only using it via

00:19:00.799 --> 00:19:03.280
Discord or you know you're using it via

00:19:03.280 --> 00:19:05.520
WhatsApp but if you use the the

00:19:05.520 --> 00:19:08.080
dashboard and you you remember the Ko

00:19:08.080 --> 00:19:09.600
dashboard that I showed you I I can show

00:19:09.600 --> 00:19:11.760
it to you again what I mean by a

00:19:11.760 --> 00:19:12.960
dashboard.

00:19:12.960 --> 00:19:13.360
>> Okay,

00:19:13.360 --> 00:19:15.039
>> I know that I have a lot of tabs open

00:19:15.039 --> 00:19:17.120
but this is Discord, right? I have a

00:19:17.120 --> 00:19:19.039
bunch of servers and then I have my

00:19:19.039 --> 00:19:20.960
Hermes agent server

00:19:20.960 --> 00:19:22.320
>> different channels.

00:19:22.320 --> 00:19:23.679
>> I have the channels.

00:19:23.679 --> 00:19:25.760
>> Yeah. So I have it's a little it's it's

00:19:25.760 --> 00:19:27.840
clunky right I have a general channel I

00:19:27.840 --> 00:19:30.240
can you know mention the bot and then it

00:19:30.240 --> 00:19:32.640
automatically creates a thread and then

00:19:32.640 --> 00:19:36.320
in there it shows that the process and

00:19:36.320 --> 00:19:39.360
sometimes shows an interactive card for

00:19:39.360 --> 00:19:43.039
permissions like um here then I need to

00:19:43.039 --> 00:19:46.080
approve and if I don't approve it within

00:19:46.080 --> 00:19:50.000
the timeout then it just uh proceeds and

00:19:50.000 --> 00:19:51.760
and the agent then tries to work around

00:19:51.760 --> 00:19:53.760
it or something silly like

00:19:53.760 --> 00:19:56.799
So this is what a dashboard looks like,

00:19:56.799 --> 00:20:00.080
right? In this case, I'm connected to

00:20:00.080 --> 00:20:04.400
the via the jump host in So here are the

00:20:04.400 --> 00:20:08.480
the the gateways, right? The um remote

00:20:08.480 --> 00:20:10.799
crew. So this one is trying to connect

00:20:10.799 --> 00:20:12.720
over local LAN and this one is

00:20:12.720 --> 00:20:15.760
connecting via a wire guard through a

00:20:15.760 --> 00:20:20.000
jump host. So it's an SSH tunnel to and

00:20:20.000 --> 00:20:22.480
then into the the desktop. So desktop is

00:20:22.480 --> 00:20:24.720
connected to the you know wire guard

00:20:24.720 --> 00:20:27.039
network and then I SSH into the remote

00:20:27.039 --> 00:20:29.360
like the the hub and then the rest is

00:20:29.360 --> 00:20:31.919
the the other nodes are are spoke right

00:20:31.919 --> 00:20:34.320
hub and spoke setup. So, so this is a

00:20:34.320 --> 00:20:36.640
what a dashboard looks like. Right. I

00:20:36.640 --> 00:20:39.440
have sessions. It's very similar to what

00:20:39.440 --> 00:20:41.760
you saw in in the um in the Discord

00:20:41.760 --> 00:20:43.840
channel. So, you can have your threads,

00:20:43.840 --> 00:20:47.120
right? But you get more interactive

00:20:47.120 --> 00:20:50.559
things like um you when I generate an

00:20:50.559 --> 00:20:52.480
artifact, where is it?

00:20:52.480 --> 00:20:54.880
>> Yeah, I guess Ok doesn't have these

00:20:54.880 --> 00:20:56.720
features when I think about it.

00:20:56.720 --> 00:21:00.480
>> Yeah. So, here was a little markdown and

00:21:00.480 --> 00:21:02.640
markdown is not interesting. At least it

00:21:02.640 --> 00:21:04.159
gets rendered, but then when I publish

00:21:04.159 --> 00:21:07.360
it, so I did publish it, uh, it goes

00:21:07.360 --> 00:21:09.600
into CloudFront, but the markdown

00:21:09.600 --> 00:21:11.200
doesn't get rendered into HTML. So

00:21:11.200 --> 00:21:13.440
that's kind of a bug. U, so more

00:21:13.440 --> 00:21:15.840
interesting is if you ask it to to build

00:21:15.840 --> 00:21:19.200
like H is there an artifact here? Create

00:21:19.200 --> 00:21:22.320
an artifact for Yeah, the I'm just

00:21:22.320 --> 00:21:25.679
thinking aloud here that like Orca is

00:21:25.679 --> 00:21:29.120
definitely focused on development. It

00:21:29.120 --> 00:21:31.200
even has this camb board that I didn't

00:21:31.200 --> 00:21:33.919
really show you. Uh, and then when it

00:21:33.919 --> 00:21:35.760
comes to chats, I basically just jump

00:21:35.760 --> 00:21:39.120
into into Gemini because I find it fast

00:21:39.120 --> 00:21:44.080
and free or or I jump into claw.ai. I I

00:21:44.080 --> 00:21:45.679
basically jump into a web browser for

00:21:45.679 --> 00:21:49.600
chats. But I can see here that your your

00:21:49.600 --> 00:21:52.159
chat your chats are probably a bit a

00:21:52.159 --> 00:21:54.480
little bit better organized in Kira.

00:21:54.480 --> 00:21:56.159
>> Yeah, you can you can see active

00:21:56.159 --> 00:21:58.480
sessions. you can search, you can

00:21:58.480 --> 00:22:01.600
probably uh clean up like I haven't used

00:22:01.600 --> 00:22:03.120
a lot of capabilities. Like I said, I

00:22:03.120 --> 00:22:05.280
used it heavily until the beginning of

00:22:05.280 --> 00:22:07.440
last week and then I was in heavy

00:22:07.440 --> 00:22:08.799
planning session for the rest of the

00:22:08.799 --> 00:22:11.360
week and then even the weekend and then

00:22:11.360 --> 00:22:12.960
I haven't been able to use it. I still

00:22:12.960 --> 00:22:15.120
have to contribute my PR that uses

00:22:15.120 --> 00:22:18.480
Codex instead of um Kiro. So I'm using

00:22:18.480 --> 00:22:20.320
my chbpt subscription here.

00:22:20.320 --> 00:22:21.840
>> I need to contribute that so and get

00:22:21.840 --> 00:22:23.039
that merged.

00:22:23.039 --> 00:22:25.600
One thing at work, I'm just thinking

00:22:25.600 --> 00:22:27.679
with my with my the multiplayer team

00:22:27.679 --> 00:22:31.039
aspect is that is that uh at work at the

00:22:31.039 --> 00:22:33.760
client, we're building a lot of tools to

00:22:33.760 --> 00:22:36.559
essentially do what you're doing here

00:22:36.559 --> 00:22:38.640
>> and have the agents set up for for the

00:22:38.640 --> 00:22:41.039
organizational uh context, right? Have

00:22:41.039 --> 00:22:44.320
the organ have have skills set up, have

00:22:44.320 --> 00:22:47.679
uh different connectors set up. And

00:22:47.679 --> 00:22:48.880
that's probably that's probably the

00:22:48.880 --> 00:22:50.880
future. We're probably going to see a

00:22:50.880 --> 00:22:54.000
lot of development in this sort of like

00:22:54.000 --> 00:22:56.400
guys like Kira like Orca where where

00:22:56.400 --> 00:22:58.080
where

00:22:58.080 --> 00:23:02.080
essentially it boots up at the at the at

00:23:02.080 --> 00:23:05.360
said organization and it's it's it's

00:23:05.360 --> 00:23:07.919
just basically got the organizational

00:23:07.919 --> 00:23:10.159
guardrails, the organizational context

00:23:10.159 --> 00:23:13.440
ready for you to go to do your job.

00:23:13.440 --> 00:23:16.080
And and at the moment we're we're we're

00:23:16.080 --> 00:23:17.600
basically just trying to figure it out,

00:23:17.600 --> 00:23:21.039
you and I. uh trying different tools,

00:23:21.039 --> 00:23:23.679
different approaches. We're basically

00:23:23.679 --> 00:23:24.880
>> Isn't this really cool?

00:23:24.880 --> 00:23:26.159
>> Pioneers.

00:23:26.159 --> 00:23:30.400
>> You get the artifact in line right here.

00:23:30.400 --> 00:23:33.360
>> The I'm a big fan of artifacts, the HTML

00:23:33.360 --> 00:23:34.880
artifact thing.

00:23:34.880 --> 00:23:36.480
>> But this is like even cooler than the

00:23:36.480 --> 00:23:39.039
terminal experience, right? Yeah, it is.

00:23:39.039 --> 00:23:40.799
>> It right here.

00:23:40.799 --> 00:23:42.320
>> And and this is why I think you kind of

00:23:42.320 --> 00:23:44.880
need Orca or some electron. I I mean I'm

00:23:44.880 --> 00:23:45.760
no I'm no fan of

00:23:45.760 --> 00:23:47.440
>> But this is what Hashimoto is working

00:23:47.440 --> 00:23:49.200
on, right?

00:23:49.200 --> 00:23:50.880
>> Well, it's going to render HTML in the

00:23:50.880 --> 00:23:51.679
terminal.

00:23:51.679 --> 00:23:55.760
>> He's building a new terminal multiplexer

00:23:55.760 --> 00:23:58.640
that leverages ghost ty the TTY

00:23:58.640 --> 00:24:01.600
libraries with a a different protocol

00:24:01.600 --> 00:24:04.240
like a network protocol that doesn't

00:24:04.240 --> 00:24:07.039
depend on SSH anymore. So they are

00:24:07.039 --> 00:24:08.799
rendering back to the terminal and he

00:24:08.799 --> 00:24:11.360
did write ghosty to to render like 3D

00:24:11.360 --> 00:24:13.360
graphics in the terminal like he has

00:24:13.360 --> 00:24:15.120
these animations right with super high

00:24:15.120 --> 00:24:18.000
FPS. I I wonder if that will you know be

00:24:18.000 --> 00:24:19.600
able to render this type of of of

00:24:19.600 --> 00:24:21.200
>> interaction. I imagine it being a really

00:24:21.200 --> 00:24:23.760
good multiplexer so that you can connect

00:24:23.760 --> 00:24:25.679
back to your session but I don't I doubt

00:24:25.679 --> 00:24:27.279
it's going to be like the electron. I

00:24:27.279 --> 00:24:30.000
think what I'm seeing with Orca is

00:24:30.000 --> 00:24:31.919
already well it doesn't have all the

00:24:31.919 --> 00:24:35.279
features like like rendering stuff but

00:24:35.279 --> 00:24:36.960
but it's it's definitely on the right

00:24:36.960 --> 00:24:40.000
path. Yeah. I I see a a mismatch a

00:24:40.000 --> 00:24:42.159
mismatch a m a mix match is that the

00:24:42.159 --> 00:24:45.919
right term of of kirao features and orca

00:24:45.919 --> 00:24:47.679
and we'll get pretty close to what we

00:24:47.679 --> 00:24:50.640
need in most companies in most

00:24:50.640 --> 00:24:52.400
development environments I'm starting to

00:24:52.400 --> 00:24:54.960
think. So this is the artifact that was

00:24:54.960 --> 00:24:57.360
created from a chat session and when I

00:24:57.360 --> 00:25:00.480
click publish it goes into a public URL

00:25:00.480 --> 00:25:03.440
on my AWS. So it's using CloudFront. Um,

00:25:03.440 --> 00:25:04.880
so this is actually running on my

00:25:04.880 --> 00:25:09.440
desktop at home and you just have to

00:25:09.440 --> 00:25:12.080
>> and can you comment on the

00:25:12.080 --> 00:25:15.360
>> Yeah, you can you you can you I think

00:25:15.360 --> 00:25:16.720
normally you can

00:25:16.720 --> 00:25:18.880
>> but not on the on the publish one can

00:25:18.880 --> 00:25:20.159
you

00:25:20.159 --> 00:25:22.720
>> I can leave comments here but it's a bit

00:25:22.720 --> 00:25:24.799
janky if you create like an interactive

00:25:24.799 --> 00:25:25.679
slide.

00:25:25.679 --> 00:25:28.240
>> Yeah, exactly. Can imagine this if it

00:25:28.240 --> 00:25:30.960
has if it has state. Yeah, because no

00:25:30.960 --> 00:25:32.480
that and you can edit, you can review,

00:25:32.480 --> 00:25:35.279
you can res Oh yeah, you can actually uh

00:25:35.279 --> 00:25:37.600
stage several comments into one new

00:25:37.600 --> 00:25:39.600
session to to address them and and and

00:25:39.600 --> 00:25:42.400
build uh you know build on on the

00:25:42.400 --> 00:25:44.400
comments that are coming in. So but on

00:25:44.400 --> 00:25:47.600
the cloud front, I don't think you can

00:25:47.600 --> 00:25:50.640
comment. I mean it's it's publishing.

00:25:50.640 --> 00:25:52.960
Let's look at the artifacts.

00:25:52.960 --> 00:25:54.960
>> Do you know would be really cool if that

00:25:54.960 --> 00:25:57.120
you could publish an artifact. It could

00:25:57.120 --> 00:25:59.600
be interactive and then you can record

00:25:59.600 --> 00:26:01.440
decisions from your teammates just then

00:26:01.440 --> 00:26:04.799
and there. That would be cool. Like, you

00:26:04.799 --> 00:26:06.880
know, have like a trade-off slider and

00:26:06.880 --> 00:26:10.960
then vote with your colleagues like, oh

00:26:10.960 --> 00:26:12.559
yeah, we should go here or we should go

00:26:12.559 --> 00:26:15.520
there and get and and get agreement and

00:26:15.520 --> 00:26:16.960
and then

00:26:16.960 --> 00:26:19.120
>> get buy in and just move on to the next

00:26:19.120 --> 00:26:20.400
thing.

00:26:20.400 --> 00:26:20.960
>> So here,

00:26:20.960 --> 00:26:23.039
>> this is so challenging at work actually.

00:26:23.039 --> 00:26:24.960
So challenging. This this is currently

00:26:24.960 --> 00:26:27.279
deploying every single artifact under a

00:26:27.279 --> 00:26:29.679
new cloudfront distribution.

00:26:29.679 --> 00:26:33.279
>> So, um I don't think that's cheap if I

00:26:33.279 --> 00:26:35.440
keep them running. Okay. So, it's

00:26:35.440 --> 00:26:37.039
>> Yeah, that's that's my concern about

00:26:37.039 --> 00:26:40.559
using any AWS thing with Bedrock is like

00:26:40.559 --> 00:26:44.559
there's no easy way to budget. Oh, by

00:26:44.559 --> 00:26:46.880
the way, I tried open router. Very nice

00:26:46.880 --> 00:26:49.919
product. Budgeting works really well.

00:26:49.919 --> 00:26:52.320
budgeting on bedrock

00:26:52.320 --> 00:26:55.039
doesn't work well. So, so I'm I'm just

00:26:55.039 --> 00:26:58.240
too nervous to try AWS features because

00:26:58.240 --> 00:27:01.520
there's no proper guard rails about

00:27:01.520 --> 00:27:03.039
expenditure and things like this. You

00:27:03.039 --> 00:27:05.840
know, dude, I I'm using my my credit

00:27:05.840 --> 00:27:07.919
card and my wife is going to see I

00:27:07.919 --> 00:27:10.960
really spend 4050 uh US dollars a month

00:27:10.960 --> 00:27:13.279
on AWS.

00:27:13.279 --> 00:27:15.200
I can't try I can't afford to try

00:27:15.200 --> 00:27:17.919
Bedrock. I haven't I haven't used

00:27:17.919 --> 00:27:20.480
anyway. Bedrock is indeed not not uh

00:27:20.480 --> 00:27:22.080
cheap. What's the difference between

00:27:22.080 --> 00:27:25.279
open router and light LLM? I've never

00:27:25.279 --> 00:27:29.600
tried light LLM, but open but I but try

00:27:29.600 --> 00:27:31.760
open router. It's amazing. It's just the

00:27:31.760 --> 00:27:34.960
the on boarding flow so polished like

00:27:34.960 --> 00:27:36.880
like the way it asks you for your credit

00:27:36.880 --> 00:27:38.880
card but like it says like you know I'm

00:27:38.880 --> 00:27:40.799
just testing for

00:27:40.799 --> 00:27:42.720
>> at least they do it nicely. That's a

00:27:42.720 --> 00:27:44.559
nice way to say it's so smooth. That's

00:27:44.559 --> 00:27:46.640
why they got how much money did they did

00:27:46.640 --> 00:27:47.840
were they acquired for

00:27:47.840 --> 00:27:49.760
>> the product is so good like the other

00:27:49.760 --> 00:27:52.880
thing that really blew my mind was that

00:27:52.880 --> 00:27:56.320
uh it had an MCP that you authenticate

00:27:56.320 --> 00:27:58.880
very smoothly in Clawude so you don't

00:27:58.880 --> 00:28:00.640
even have to put any credit like my my

00:28:00.640 --> 00:28:03.760
balance in open router is zero right now

00:28:03.760 --> 00:28:06.559
but but you can use the MCP and you can

00:28:06.559 --> 00:28:08.640
and with the MCP you can ask questions

00:28:08.640 --> 00:28:11.440
like what is the best model for image

00:28:11.440 --> 00:28:13.600
generation at the moment and what's your

00:28:13.600 --> 00:28:16.159
best price. You can ask questions like

00:28:16.159 --> 00:28:19.600
this to the open router MCP working out

00:28:19.600 --> 00:28:22.000
which model, which price to do a certain

00:28:22.000 --> 00:28:24.159
task and it will tell you and the

00:28:24.159 --> 00:28:26.880
information is like totally up to date

00:28:26.880 --> 00:28:29.279
and it it's not like hugging phase where

00:28:29.279 --> 00:28:31.679
where these rate limits all over the

00:28:31.679 --> 00:28:33.600
place and it's very difficult to work

00:28:33.600 --> 00:28:36.799
out which model you should use. So,

00:28:36.799 --> 00:28:39.360
open router very polished. It doesn't

00:28:39.360 --> 00:28:41.760
surprise me that it would got it went

00:28:41.760 --> 00:28:44.240
from zero to hero. I mean, I don't know.

00:28:44.240 --> 00:28:46.000
I don't know who's using it, but like

00:28:46.000 --> 00:28:48.240
whatever. The the product you can tell

00:28:48.240 --> 00:28:50.559
is just very smooth,

00:28:50.559 --> 00:28:52.880
>> very tasteful, just like Walker.

00:28:52.880 --> 00:28:56.480
>> Tasteful. I I recognize good taste

00:28:56.480 --> 00:28:57.919
>> when I see it.

00:28:57.919 --> 00:28:59.120
>> H another

00:28:59.120 --> 00:29:01.840
>> strange. The the publish didn't really

00:29:01.840 --> 00:29:03.840
work well. It shows, but without any of

00:29:03.840 --> 00:29:06.480
the CSS. So, interesting to know because

00:29:06.480 --> 00:29:08.320
this is like a widget. I don't know

00:29:08.320 --> 00:29:09.919
what's the difference between widget and

00:29:09.919 --> 00:29:12.080
other things. Anyway, uh we were talking

00:29:12.080 --> 00:29:15.679
about Orca and then um the terminal

00:29:15.679 --> 00:29:18.240
experience. You're talking about um open

00:29:18.240 --> 00:29:20.960
router and I I think it I don't know

00:29:20.960 --> 00:29:23.440
actually the difference with lightm LLM

00:29:23.440 --> 00:29:25.520
being something to control budgets

00:29:25.520 --> 00:29:27.279
around

00:29:27.279 --> 00:29:30.159
LLM calls which within the organization

00:29:30.159 --> 00:29:32.320
I work we're also looking at light LLM

00:29:32.320 --> 00:29:35.279
to control uh budgets per team and and

00:29:35.279 --> 00:29:35.919
model

00:29:35.919 --> 00:29:37.840
>> say anything about it. Oh, can I just

00:29:37.840 --> 00:29:41.120
steal the the screen share? I I showed I

00:29:41.120 --> 00:29:43.360
showed the killer crew again.

00:29:43.360 --> 00:29:45.279
>> Well, one thing that I I think really

00:29:45.279 --> 00:29:47.919
that Oh my god. One thing that I think

00:29:47.919 --> 00:29:50.720
that is really good about the orca is

00:29:50.720 --> 00:29:53.039
the documentation is quite well done.

00:29:53.039 --> 00:29:55.760
It's like for example these recipes it

00:29:55.760 --> 00:29:59.200
it's just like only five of them but but

00:29:59.200 --> 00:30:01.760
they basically

00:30:01.760 --> 00:30:05.600
really hit hit what do you call it? You

00:30:05.600 --> 00:30:08.640
know they they hit hit the spot for me.

00:30:08.640 --> 00:30:10.399
Like for example, brace three agents on

00:30:10.399 --> 00:30:13.520
the same task. I did that my I did that

00:30:13.520 --> 00:30:15.279
and my mind was blown.

00:30:15.279 --> 00:30:16.799
>> But why?

00:30:16.799 --> 00:30:19.440
>> Review AI diff. Once I discovered the

00:30:19.440 --> 00:30:20.880
diff view, I was like, "Yes, this is

00:30:20.880 --> 00:30:22.159
good."

00:30:22.159 --> 00:30:24.399
>> Jump between work trees. So, command J.

00:30:24.399 --> 00:30:26.880
Really, really nice. I haven't tried

00:30:26.880 --> 00:30:29.840
this design mode. But then last but not

00:30:29.840 --> 00:30:33.600
least, SSH to a remote host and working

00:30:33.600 --> 00:30:36.640
from that remote host. Yay. I loved it.

00:30:36.640 --> 00:30:38.480
Well, that's exactly what I just showed

00:30:38.480 --> 00:30:41.360
a beefier dev box. So, this is what I I

00:30:41.360 --> 00:30:44.399
use C crew for, uh, which is it it's a

00:30:44.399 --> 00:30:47.039
dev box, but managed by an agent and it

00:30:47.039 --> 00:30:49.440
runs its own scheduled tasks. It's a bit

00:30:49.440 --> 00:30:52.480
similar, but it seems the the focus here

00:30:52.480 --> 00:30:54.320
is different in in that you are the one

00:30:54.320 --> 00:30:57.200
doing the work assisted by a harness

00:30:57.200 --> 00:30:59.679
with with Orca and with Kiro Crew,

00:30:59.679 --> 00:31:01.760
you're the one fully delegating work to

00:31:01.760 --> 00:31:03.919
an agent that you trust to work on its

00:31:03.919 --> 00:31:06.960
own, find new work uh and and and and

00:31:06.960 --> 00:31:09.919
submit work. You're not really like I

00:31:09.919 --> 00:31:12.000
think when when you show me in Orca that

00:31:12.000 --> 00:31:13.919
you are able to look at the kit locally

00:31:13.919 --> 00:31:16.399
and review it, it's it's like you and

00:31:16.399 --> 00:31:19.279
and and the the the hardness are working

00:31:19.279 --> 00:31:21.840
as one like you one is a pilot, the

00:31:21.840 --> 00:31:24.399
other one is a driver or how what you

00:31:24.399 --> 00:31:25.679
say.

00:31:25.679 --> 00:31:26.720
>> Yeah, I think I know

00:31:26.720 --> 00:31:29.840
>> you you submit in your name. So you're

00:31:29.840 --> 00:31:31.360
the one reviewing it locally before you

00:31:31.360 --> 00:31:33.440
PR it under your name. Whereas the

00:31:33.440 --> 00:31:36.559
agent, you delegate it and it submits a

00:31:36.559 --> 00:31:38.960
PR for you to review. So you're not

00:31:38.960 --> 00:31:40.480
actually reviewing it before it submits

00:31:40.480 --> 00:31:42.480
the PR. It it's working on its own. Then

00:31:42.480 --> 00:31:44.240
you delegate tasks like hey go and

00:31:44.240 --> 00:31:46.880
update this uh do this differently and

00:31:46.880 --> 00:31:47.279
things like that.

00:31:47.279 --> 00:31:49.440
>> I guess it all depends what what you're

00:31:49.440 --> 00:31:51.279
comfortable with and what what what

00:31:51.279 --> 00:31:53.360
level of maturity you're on. I think

00:31:53.360 --> 00:31:57.360
Orca is the right is the right uh level

00:31:57.360 --> 00:31:58.880
for me at this point.

00:31:58.880 --> 00:32:00.640
>> I think it depends on the tasks and the

00:32:00.640 --> 00:32:02.799
work. uh some things you can delegate

00:32:02.799 --> 00:32:05.200
some things you have to be hands-on and

00:32:05.200 --> 00:32:08.000
this is kind of also like uh some of the

00:32:08.000 --> 00:32:10.799
difficulty that I had so I've been now

00:32:10.799 --> 00:32:12.799
one month working on one month and a

00:32:12.799 --> 00:32:14.480
half working on almost two months a

00:32:14.480 --> 00:32:17.600
project and a lot of it was like p

00:32:17.600 --> 00:32:19.679
showing that it works a lot of it was

00:32:19.679 --> 00:32:23.679
like dynamic workflows to prove that um

00:32:23.679 --> 00:32:27.200
that it would work and and literally

00:32:27.200 --> 00:32:30.799
just me alone building highly complex

00:32:30.799 --> 00:32:34.799
proof of concepts of a of like an an

00:32:34.799 --> 00:32:37.760
agentic durable workflow solution that

00:32:37.760 --> 00:32:41.760
can be triggered off uh Slack or uh

00:32:41.760 --> 00:32:44.320
version control events. So it's like

00:32:44.320 --> 00:32:47.279
event source uh and also multi-tenant.

00:32:47.279 --> 00:32:50.080
So we we have to be able to onboard uh

00:32:50.080 --> 00:32:53.120
tenants to to basically route based on

00:32:53.120 --> 00:32:56.480
their version control repository. um

00:32:56.480 --> 00:32:59.600
what is the uh you know scope to the

00:32:59.600 --> 00:33:01.360
token then it's scoped down to their

00:33:01.360 --> 00:33:03.200
permissions and and things like that

00:33:03.200 --> 00:33:05.760
very very interesting very exciting um

00:33:05.760 --> 00:33:10.000
but now it's moving from P towards

00:33:10.000 --> 00:33:13.760
actual product and

00:33:13.760 --> 00:33:15.600
>> it needs to be a lot more hands-on it

00:33:15.600 --> 00:33:17.440
needs to be a lot a lot less like the

00:33:17.440 --> 00:33:19.120
agents can build and prove this but it

00:33:19.120 --> 00:33:20.000
needs to be now we need

00:33:20.000 --> 00:33:22.640
>> so it needs to go from from your high

00:33:22.640 --> 00:33:24.880
level where you delegate to agents back

00:33:24.880 --> 00:33:27.760
to my level which is I mean you it

00:33:27.760 --> 00:33:29.440
sounds like you went to level seven and

00:33:29.440 --> 00:33:32.480
then I went but you need as you as

00:33:32.480 --> 00:33:34.399
you're trying to explain I think that

00:33:34.399 --> 00:33:36.240
you go you need to go back to level five

00:33:36.240 --> 00:33:38.159
to sort of productionize it or something

00:33:38.159 --> 00:33:40.960
like that. Yeah, you need to be more um

00:33:40.960 --> 00:33:42.720
hands-on or you need to take back

00:33:42.720 --> 00:33:45.039
control. And I like I let go of a lot of

00:33:45.039 --> 00:33:47.760
the upfront

00:33:47.760 --> 00:33:51.360
like spec driven uh workflows where we

00:33:51.360 --> 00:33:54.880
are upfront identifying the the user

00:33:54.880 --> 00:33:56.880
user stories and and and breaking down

00:33:56.880 --> 00:33:59.200
the phases and the MVP deliverables and

00:33:59.200 --> 00:34:01.760
and phasing the work so that we can you

00:34:01.760 --> 00:34:05.200
know build a larger amount of work in

00:34:05.200 --> 00:34:07.600
into smaller slices that are uh

00:34:07.600 --> 00:34:10.320
controlled and reviewed and grown in a

00:34:10.320 --> 00:34:12.800
in a maintainable way versus you're kick

00:34:12.800 --> 00:34:14.399
taking off a couple of agents to go and

00:34:14.399 --> 00:34:17.760
build a PC and yes you can it it works

00:34:17.760 --> 00:34:22.079
but it's a immense mess to to maintain I

00:34:22.079 --> 00:34:25.280
know and one thing is you get journals u

00:34:25.280 --> 00:34:27.359
inside every comment it just keeps

00:34:27.359 --> 00:34:30.240
growing

00:34:30.240 --> 00:34:32.399
>> hey maybe that that that is a segue into

00:34:32.399 --> 00:34:34.399
another one where uh you know remember

00:34:34.399 --> 00:34:35.919
that I was working on the on the

00:34:35.919 --> 00:34:36.960
benchmarking

00:34:36.960 --> 00:34:38.639
>> yeah I I wanted to show you my

00:34:38.639 --> 00:34:41.440
benchmarking journey uh because I I was

00:34:41.440 --> 00:34:43.839
using AWS Bench

00:34:43.839 --> 00:34:45.520
Oh, so you played with AWS Bench.

00:34:45.520 --> 00:34:46.159
Interesting.

00:34:46.159 --> 00:34:47.760
>> And

00:34:47.760 --> 00:34:49.040
>> what is it that you were trying to

00:34:49.040 --> 00:34:50.639
benchmark first? Start with that.

00:34:50.639 --> 00:34:52.879
>> Well, let me just let me just show you

00:34:52.879 --> 00:34:55.839
and then I feel it will be uh

00:34:55.839 --> 00:34:57.920
>> Can you describe it before you show it?

00:34:57.920 --> 00:34:59.680
>> I think it's let me just show you the

00:34:59.680 --> 00:35:01.040
demo and then and then I think it's

00:35:01.040 --> 00:35:03.680
easier to talk about it. I'm a picture

00:35:03.680 --> 00:35:06.079
and then talk kind of guy.

00:35:06.079 --> 00:35:10.240
>> Oh, so you were trying to use the local.

00:35:10.240 --> 00:35:13.359
>> Yeah. So, to be honest, it was it was

00:35:13.359 --> 00:35:18.160
non-trivial to set up and I had to

00:35:18.160 --> 00:35:20.160
get the agents to patch the hell out of

00:35:20.160 --> 00:35:21.839
Flocky. But,

00:35:21.839 --> 00:35:25.760
>> and did you did you mine the other

00:35:25.760 --> 00:35:28.240
pre-existing work that was using Flocky?

00:35:28.240 --> 00:35:32.079
>> I was trying to point the agents to it.

00:35:32.079 --> 00:35:34.240
I didn't really understand what he did.

00:35:34.240 --> 00:35:34.960
He did.

00:35:34.960 --> 00:35:37.599
>> No, nobody does it. But like the the

00:35:37.599 --> 00:35:40.640
thing like for example so essentially

00:35:40.640 --> 00:35:42.400
like

00:35:42.400 --> 00:35:44.160
I think I have it set up here. It's it's

00:35:44.160 --> 00:35:47.200
passing but then I set I set it up so

00:35:47.200 --> 00:35:50.400
that it's broken right and then I I I

00:35:50.400 --> 00:35:52.560
verify that it's that it's broken. It's

00:35:52.560 --> 00:35:55.200
failing. And then and then I have the

00:35:55.200 --> 00:35:58.400
task here which I sort of like you know

00:35:58.400 --> 00:36:01.040
I I tell Claude to fix or something like

00:36:01.040 --> 00:36:04.079
that you know pipe it to Claude whatever

00:36:04.079 --> 00:36:04.880
I can just run

00:36:04.880 --> 00:36:07.119
>> do the task first

00:36:07.119 --> 00:36:09.440
>> and and then Claude is able to do it and

00:36:09.440 --> 00:36:12.240
and and run verify and things like that.

00:36:12.240 --> 00:36:14.880
So what's my point here? So I I was

00:36:14.880 --> 00:36:17.440
actually um surprised by this

00:36:17.440 --> 00:36:17.760
>> because

00:36:17.760 --> 00:36:19.520
>> that's not a benchmark though. So it's

00:36:19.520 --> 00:36:21.920
it's a single task in a single scenario

00:36:21.920 --> 00:36:22.880
against a local

00:36:22.880 --> 00:36:24.800
>> Yeah. and and and and that's as far as I

00:36:24.800 --> 00:36:27.839
can tell how it it it it works for the

00:36:27.839 --> 00:36:29.839
for the most part. It I think it's

00:36:29.839 --> 00:36:32.000
hopefully fixed it.

00:36:32.000 --> 00:36:33.839
>> Yeah, it's passing. So

00:36:33.839 --> 00:36:34.240
>> Mhm.

00:36:34.240 --> 00:36:36.400
>> This is what this is what kind of

00:36:36.400 --> 00:36:39.920
surprised me with AWS Bench because

00:36:39.920 --> 00:36:41.680
when you look around it's it's like a

00:36:41.680 --> 00:36:44.560
lot of broken things and it's testing. I

00:36:44.560 --> 00:36:47.359
mean, we talked about it and

00:36:47.359 --> 00:36:49.680
there should be no surprises, but like I

00:36:49.680 --> 00:36:50.800
understand that it has all these

00:36:50.800 --> 00:36:53.200
scenarios for for debugging things, but

00:36:53.200 --> 00:36:55.119
to be honest, I was thinking that it

00:36:55.119 --> 00:36:57.920
would have more scenarios for like uh

00:36:57.920 --> 00:36:59.040
>> generating or building.

00:36:59.040 --> 00:37:01.040
>> Yeah, bu generating building things. And

00:37:01.040 --> 00:37:02.560
as far as I can tell, there's not many

00:37:02.560 --> 00:37:05.040
of those cases. There's lots of cases

00:37:05.040 --> 00:37:08.079
for for debugging broken setups.

00:37:08.079 --> 00:37:10.240
>> Yeah. I think what AWS really wants to

00:37:10.240 --> 00:37:13.920
know is what are the tools they provide

00:37:13.920 --> 00:37:16.160
because ultimately AWS as a service

00:37:16.160 --> 00:37:20.160
provider uh wants to know how easily and

00:37:20.160 --> 00:37:22.400
how correct and accurately people can

00:37:22.400 --> 00:37:26.079
use coding harnesses to um work with the

00:37:26.079 --> 00:37:28.320
cloud right so they're providing AWS doc

00:37:28.320 --> 00:37:29.920
MCPS they're providing skill

00:37:29.920 --> 00:37:31.680
repositories and they want to know and

00:37:31.680 --> 00:37:33.680
evaluate if those things actually work

00:37:33.680 --> 00:37:34.079
>> but

00:37:34.079 --> 00:37:35.280
>> and that's exactly why you need that

00:37:35.280 --> 00:37:36.880
>> but the trouble is the trouble is it's

00:37:36.880 --> 00:37:39.599
like a it's It's like curing the disease

00:37:39.599 --> 00:37:41.680
instead of preventing the the disease.

00:37:41.680 --> 00:37:43.920
>> No, no, it's a different target like

00:37:43.920 --> 00:37:45.920
when what they are trying to solve for

00:37:45.920 --> 00:37:47.839
is for for people that are running on

00:37:47.839 --> 00:37:50.000
AWS to make their experience easier and

00:37:50.000 --> 00:37:51.520
to make sure that the tools that they're

00:37:51.520 --> 00:37:53.920
using can troubleshoot and help them.

00:37:53.920 --> 00:37:55.920
>> You know, when you talk to like I think

00:37:55.920 --> 00:37:58.320
that's one thing that focuses on, right?

00:37:58.320 --> 00:38:00.079
>> It's legit like

00:38:00.079 --> 00:38:02.560
>> they want people not just they want to

00:38:02.560 --> 00:38:04.560
increase your usage of their services.

00:38:04.560 --> 00:38:06.079
They're trying to help you save money so

00:38:06.079 --> 00:38:07.839
you can, you know, use more of their

00:38:07.839 --> 00:38:09.920
services for a longer time. They want

00:38:09.920 --> 00:38:11.520
success stories for the humans.

00:38:11.520 --> 00:38:14.000
>> Well, that's the trouble with AWS Bench.

00:38:14.000 --> 00:38:17.119
I don't think it has any of that uh

00:38:17.119 --> 00:38:19.440
value. Like I don't think I mean I

00:38:19.440 --> 00:38:21.280
haven't looked at every scenario, but I

00:38:21.280 --> 00:38:24.240
don't think it's it's saving you money.

00:38:24.240 --> 00:38:26.320
>> No, it's not it's not about that, right?

00:38:26.320 --> 00:38:29.680
I mean, it's about making the

00:38:29.680 --> 00:38:32.160
customer experience as good as possible,

00:38:32.160 --> 00:38:34.240
right? Whether that is by saving you

00:38:34.240 --> 00:38:36.400
money or by making sure that if you use

00:38:36.400 --> 00:38:39.680
Claude Code uh that that if that it can

00:38:39.680 --> 00:38:41.599
really quickly help you troubleshoot any

00:38:41.599 --> 00:38:42.960
issues with your website if you're

00:38:42.960 --> 00:38:44.640
running or or your web services on top

00:38:44.640 --> 00:38:45.280
of them. Right.

00:38:45.280 --> 00:38:48.000
>> Yeah. Okay. Fair fair. But I'm

00:38:48.000 --> 00:38:50.160
definitely more in the whole best

00:38:50.160 --> 00:38:51.760
practice

00:38:51.760 --> 00:38:54.000
guard rails

00:38:54.000 --> 00:38:56.880
mindset of course but you know I get it.

00:38:56.880 --> 00:38:59.680
So, so what I did use AWS bench for is

00:38:59.680 --> 00:39:03.280
to test how good

00:39:03.280 --> 00:39:06.480
uh how easy different infrastructure as

00:39:06.480 --> 00:39:08.400
code frameworks

00:39:08.400 --> 00:39:10.480
can be used and how token efficient they

00:39:10.480 --> 00:39:13.760
are uh compared like I'm only testing

00:39:13.760 --> 00:39:17.359
Claude Code as a harness I'm testing son

00:39:17.359 --> 00:39:20.720
uh and I am giving it tasks to generate

00:39:20.720 --> 00:39:24.079
code right um you need to build an API

00:39:24.079 --> 00:39:27.440
gateway with lambda here is the paths

00:39:27.440 --> 00:39:30.000
and the the you know here's there's a

00:39:30.000 --> 00:39:31.680
couple of lambda like function bodies

00:39:31.680 --> 00:39:32.880
and you need to hook them up for

00:39:32.880 --> 00:39:35.440
implementing this rest API um and so it

00:39:35.440 --> 00:39:37.440
go and generates terapform configuration

00:39:37.440 --> 00:39:40.400
so on one I I have three arms right arm

00:39:40.400 --> 00:39:43.520
number one it's terafform

00:39:43.520 --> 00:39:46.000
I want to have two versions of it one is

00:39:46.000 --> 00:39:48.720
raw no modules the other one is with

00:39:48.720 --> 00:39:51.839
modules and ideally with also skills so

00:39:51.839 --> 00:39:53.920
that maybe Anton Babeno's terraform

00:39:53.920 --> 00:39:55.920
module skills so that that actually uses

00:39:55.920 --> 00:39:56.480
those.

00:39:56.480 --> 00:39:56.880
>> Okay.

00:39:56.880 --> 00:39:57.680
>> Right.

00:39:57.680 --> 00:40:00.240
>> Uh so I I currently

00:40:00.240 --> 00:40:04.160
the second arm is AWS CDK and the third

00:40:04.160 --> 00:40:06.480
arm is my Terara constructs library

00:40:06.480 --> 00:40:07.599
which is CDK.

00:40:07.599 --> 00:40:09.359
>> If you can compare them that that would

00:40:09.359 --> 00:40:10.800
be amazing.

00:40:10.800 --> 00:40:12.880
>> Yeah. So so this been already three

00:40:12.880 --> 00:40:14.720
weeks right and every time I have a bit

00:40:14.720 --> 00:40:17.040
of spare time I I go and prompt it. So

00:40:17.040 --> 00:40:19.760
I'm not actually directing much. I I

00:40:19.760 --> 00:40:22.079
sorry I'm not actually reviewing very

00:40:22.079 --> 00:40:24.400
carefully just reading what happened and

00:40:24.400 --> 00:40:26.640
what is the next phase of scenario

00:40:26.640 --> 00:40:28.720
writing and things like that. So I can

00:40:28.720 --> 00:40:30.560
tell you some of my observations so far

00:40:30.560 --> 00:40:32.160
which is really really cool and really

00:40:32.160 --> 00:40:34.640
interesting and completely against my

00:40:34.640 --> 00:40:36.320
expectations.

00:40:36.320 --> 00:40:38.240
>> This sounds good. This sounds good. Hold

00:40:38.240 --> 00:40:39.760
on. Let me just double check. I'm

00:40:39.760 --> 00:40:41.760
expecting someone to come around at some

00:40:41.760 --> 00:40:44.480
point. Okay. Go. You have my undivided

00:40:44.480 --> 00:40:45.680
attention.

00:40:45.680 --> 00:40:46.079
>> Okay.

00:40:46.079 --> 00:40:49.359
>> Surprise me. So of course I had to came

00:40:49.359 --> 00:40:52.240
up with some uh scenarios actually in

00:40:52.240 --> 00:40:55.200
AWS bench they the scenario is what what

00:40:55.200 --> 00:40:58.000
this how you um build the AWS account

00:40:58.000 --> 00:40:59.680
right you have one scenario that sets up

00:40:59.680 --> 00:41:02.640
a broken um load balancer and all that

00:41:02.640 --> 00:41:04.560
um scenarios in my case don't really

00:41:04.560 --> 00:41:06.640
apply because I didn't have um you know

00:41:06.640 --> 00:41:08.480
pre-existing infra tests although that I

00:41:08.480 --> 00:41:10.480
added them now it's called brown field

00:41:10.480 --> 00:41:12.560
so initially it was green field right

00:41:12.560 --> 00:41:14.160
nothing is in the account it's a brand

00:41:14.160 --> 00:41:16.400
brand new account here's your task so

00:41:16.400 --> 00:41:18.000
there was called one scenario called

00:41:18.000 --> 00:41:19.920
anchor which had nothing in it like it's

00:41:19.920 --> 00:41:22.720
an empty account. The second uh

00:41:22.720 --> 00:41:24.960
component within AWS bench is a task

00:41:24.960 --> 00:41:26.480
right within a scenario you have couple

00:41:26.480 --> 00:41:27.040
of tasks.

00:41:27.040 --> 00:41:27.520
>> Exactly.

00:41:27.520 --> 00:41:31.440
>> So so what what um Fable built was a

00:41:31.440 --> 00:41:34.240
task generator that would take one

00:41:34.240 --> 00:41:37.680
template task build API gateway uh and

00:41:37.680 --> 00:41:40.079
then that would duplicate it with some

00:41:40.079 --> 00:41:42.480
injected additional information like

00:41:42.480 --> 00:41:45.680
build AWS gateway with Terraform raw.

00:41:45.680 --> 00:41:48.000
And here is your environment build AWS

00:41:48.000 --> 00:41:50.560
gateway with AWS CDK and here's your

00:41:50.560 --> 00:41:52.720
environment for that. So each one would

00:41:52.720 --> 00:41:54.720
be basically it generates all the tasks.

00:41:54.720 --> 00:41:57.920
So if I have if I have three tasks it

00:41:57.920 --> 00:42:01.119
would generate three * 3 nine tasks

00:42:01.119 --> 00:42:02.000
right

00:42:02.000 --> 00:42:04.000
>> then it needs to run the task

00:42:04.000 --> 00:42:05.520
>> but

00:42:05.520 --> 00:42:07.680
generating the task I mean the the tasks

00:42:07.680 --> 00:42:09.200
should be

00:42:09.200 --> 00:42:11.520
>> using the template it generates nine

00:42:11.520 --> 00:42:12.160
tasks.

00:42:12.160 --> 00:42:15.440
>> Okay. one for Terraform raw, one for AWS

00:42:15.440 --> 00:42:17.599
CDK and one for Terra Constructs. Okay.

00:42:17.599 --> 00:42:19.280
>> And then it runs the task which is

00:42:19.280 --> 00:42:21.760
called a trial. So when it runs a trial,

00:42:21.760 --> 00:42:24.720
it provisions the account. It runs uh

00:42:24.720 --> 00:42:27.839
Claude Code and it it tells it like exe

00:42:27.839 --> 00:42:30.240
it pro it invokes Claude Code with dashp

00:42:30.240 --> 00:42:30.640
to

00:42:30.640 --> 00:42:32.960
>> so provisioning the account is is it all

00:42:32.960 --> 00:42:35.280
isolated somehow?

00:42:35.280 --> 00:42:37.040
>> Yeah. So

00:42:37.040 --> 00:42:39.119
>> when branch runs

00:42:39.119 --> 00:42:41.680
>> it it it runs against my AWS

00:42:41.680 --> 00:42:43.839
organization like I I I have an orc

00:42:43.839 --> 00:42:46.000
management account and then it creates

00:42:46.000 --> 00:42:50.319
like under an OU it can um leverage AWS

00:42:50.319 --> 00:42:53.200
accounts. It has one anchor um scenario

00:42:53.200 --> 00:42:55.920
account that it reuses for the tasks. So

00:42:55.920 --> 00:42:57.839
it does have like a wipe mechanism. It

00:42:57.839 --> 00:43:00.000
does determine if it's like if it's read

00:43:00.000 --> 00:43:01.760
only like for example it writes the

00:43:01.760 --> 00:43:03.280
configuration and it runs the plan but

00:43:03.280 --> 00:43:04.800
it doesn't deploy which is not what I

00:43:04.800 --> 00:43:06.720
want. I wanted to deploy but if it's

00:43:06.720 --> 00:43:09.280
read only then it will it will run

00:43:09.280 --> 00:43:11.200
multiple in parallel because not nothing

00:43:11.200 --> 00:43:12.480
can conflict with each other right

00:43:12.480 --> 00:43:15.599
because uh anyway so so it created this

00:43:15.599 --> 00:43:17.680
really complicated verification which is

00:43:17.680 --> 00:43:20.480
called oracles so once the the cloud

00:43:20.480 --> 00:43:22.480
code reads the and generates

00:43:22.480 --> 00:43:25.200
configuration it then runs OPA policies

00:43:25.200 --> 00:43:27.440
over that to validate that it it match

00:43:27.440 --> 00:43:30.079
matches certain criteria so so it has a

00:43:30.079 --> 00:43:32.319
task and then it has criteria right you

00:43:32.319 --> 00:43:34.079
need to build an API gateway these are

00:43:34.079 --> 00:43:35.599
the routes that need to be in there and

00:43:35.599 --> 00:43:37.760
then it runs the OPA policy as the

00:43:37.760 --> 00:43:40.240
oracle to say like yes uh it's kind of

00:43:40.240 --> 00:43:41.040
like a judge

00:43:41.040 --> 00:43:43.119
>> you use OPA that sounds very

00:43:43.119 --> 00:43:45.280
overengineered but like all right I mean

00:43:45.280 --> 00:43:47.680
why do you use OPA is it is that

00:43:47.680 --> 00:43:49.599
something use at work

00:43:49.599 --> 00:43:52.079
>> right so so anyway that's the setup

00:43:52.079 --> 00:43:55.359
right so let's talk about what it is

00:43:55.359 --> 00:43:57.119
observed so far which is very very

00:43:57.119 --> 00:43:59.760
unintuitive I would expect that if you

00:43:59.760 --> 00:44:02.720
give it a very powerful uh L2 construct

00:44:02.720 --> 00:44:04.319
library like Terra constructs it would

00:44:04.319 --> 00:44:06.400
work better than raw, right? Actually,

00:44:06.400 --> 00:44:09.599
raw terraform on a green field task is

00:44:09.599 --> 00:44:12.560
way more token efficient. It generates a

00:44:12.560 --> 00:44:14.720
ter. So and and the reason why it's

00:44:14.720 --> 00:44:18.319
interesting the the model Sonnet has been

00:44:18.319 --> 00:44:21.760
trained on so much like you know data

00:44:21.760 --> 00:44:23.920
from the internet that within the the

00:44:23.920 --> 00:44:26.240
weights of the model that is massive um

00:44:26.240 --> 00:44:27.599
it can generate terapform very

00:44:27.599 --> 00:44:30.560
comfortably very confident you know

00:44:30.560 --> 00:44:33.040
green feed is easy right it it just from

00:44:33.040 --> 00:44:35.200
nothing so it just starts writing almost

00:44:35.200 --> 00:44:36.400
instantly you give it a task it

00:44:36.400 --> 00:44:38.400
generates the code it validates it boom

00:44:38.400 --> 00:44:43.359
it passes right you give it AWS CDK It's

00:44:43.359 --> 00:44:45.040
spending a little bit more time setting

00:44:45.040 --> 00:44:46.800
up TypeScript and all that doing the

00:44:46.800 --> 00:44:48.160
synthesis

00:44:48.160 --> 00:44:50.960
>> and then it generates the code very very

00:44:50.960 --> 00:44:53.920
fast. Now give it Terra constructs which

00:44:53.920 --> 00:44:56.400
is a niche library that is not in a lot

00:44:56.400 --> 00:44:58.400
of the you know it's not used across the

00:44:58.400 --> 00:44:59.760
internet. There's not a lot of like

00:44:59.760 --> 00:45:01.119
adoption of it.

00:45:01.119 --> 00:45:04.720
>> And guess what happens? The model spends

00:45:04.720 --> 00:45:08.480
out of 52 turns to complete the task 29

00:45:08.480 --> 00:45:10.800
reading the terracon library reading the

00:45:10.800 --> 00:45:12.240
typescript interface reading and

00:45:12.240 --> 00:45:13.839
understanding how things fit together.

00:45:13.839 --> 00:45:15.599
So most of the time the model spends

00:45:15.599 --> 00:45:17.280
reading the abstraction. So it's

00:45:17.280 --> 00:45:20.560
actually um the my hypothesis was that

00:45:20.560 --> 00:45:22.160
it's more efficient to have a li a

00:45:22.160 --> 00:45:24.000
higher level abstraction as long as an

00:45:24.000 --> 00:45:25.920
abstraction is you know able to cater

00:45:25.920 --> 00:45:28.079
for all the use cases because of course

00:45:28.079 --> 00:45:29.920
the the quality of the abstraction like

00:45:29.920 --> 00:45:32.400
a module versus an actual uh object that

00:45:32.400 --> 00:45:34.960
can be you know that that can bind and

00:45:34.960 --> 00:45:37.280
dynamically uh you know generate more

00:45:37.280 --> 00:45:38.800
scenarios than a than a very fixed

00:45:38.800 --> 00:45:41.040
terapform module. My hypothesis is that

00:45:41.040 --> 00:45:43.280
the modules are not good and and the

00:45:43.280 --> 00:45:46.160
object uh oriented approach is better.

00:45:46.160 --> 00:45:47.680
I'm not there because I haven't tested

00:45:47.680 --> 00:45:49.839
modules against I'm testing raw against

00:45:49.839 --> 00:45:52.000
modules which is kind of not fair. Turns

00:45:52.000 --> 00:45:54.960
out the raw is better. Um at green

00:45:54.960 --> 00:45:57.760
field, right? So so that was not what I

00:45:57.760 --> 00:45:59.280
wanted to see obviously.

00:45:59.280 --> 00:46:01.839
>> Yeah, of course. How come you call it

00:46:01.839 --> 00:46:03.760
ter constructs? Don't you call it CDK

00:46:03.760 --> 00:46:05.040
terrain?

00:46:05.040 --> 00:46:08.480
>> CDK terrain is the L1 uh binding

00:46:08.480 --> 00:46:11.680
generating library and L2s are basically

00:46:11.680 --> 00:46:15.040
working integration patterns. So if you

00:46:15.040 --> 00:46:16.800
write terraform

00:46:16.800 --> 00:46:19.440
you if maybe apples to apples would be

00:46:19.440 --> 00:46:21.839
comparing raw terraform against CDK

00:46:21.839 --> 00:46:23.599
terrain because the raw terraform is the

00:46:23.599 --> 00:46:26.560
L1 resources within the provider AWS. So

00:46:26.560 --> 00:46:28.480
Terraform is using the raw resources and

00:46:28.480 --> 00:46:30.319
generating all the config and CDK

00:46:30.319 --> 00:46:32.880
terrain provider AWS bindings would give

00:46:32.880 --> 00:46:34.960
you the raw resources. I'm actually

00:46:34.960 --> 00:46:36.960
asking it to use terra constructs which

00:46:36.960 --> 00:46:39.359
is a a library of like integration

00:46:39.359 --> 00:46:41.520
patterns. So if you are trying to build

00:46:41.520 --> 00:46:44.319
an API gateway that invokes a lambda

00:46:44.319 --> 00:46:45.839
then you need to also set up IM

00:46:45.839 --> 00:46:48.880
policies. You need to give permissions.

00:46:48.880 --> 00:46:51.119
So you need to do a lot of glue of

00:46:51.119 --> 00:46:54.079
different resources and and and the L2

00:46:54.079 --> 00:46:54.960
take care of that.

00:46:54.960 --> 00:46:57.839
>> So there's two two thoughts here like I

00:46:57.839 --> 00:47:00.560
think with AI has shown itself to be

00:47:00.560 --> 00:47:03.440
kind of bespoke software easy to make

00:47:03.440 --> 00:47:07.920
your own software. So like in a way you

00:47:07.920 --> 00:47:09.760
don't need it you don't need modules so

00:47:09.760 --> 00:47:11.599
much because you can just generate it

00:47:11.599 --> 00:47:14.160
from scratch with your exacting

00:47:14.160 --> 00:47:16.960
requirements. You can get to this was an

00:47:16.960 --> 00:47:18.160
interesting discussion on LinkedIn

00:47:18.160 --> 00:47:20.079
yesterday which is you can get to day

00:47:20.079 --> 00:47:22.000
one which is like day zero is you have

00:47:22.000 --> 00:47:23.520
nothing. Day one you have built your

00:47:23.520 --> 00:47:25.599
first uh you know deployment. You can

00:47:25.599 --> 00:47:28.800
get to day one with raw terraform sorry

00:47:28.800 --> 00:47:32.160
like having an AI generate the the the

00:47:32.160 --> 00:47:34.160
code the config you can get to day one

00:47:34.160 --> 00:47:36.720
very quickly. But the day two which is

00:47:36.720 --> 00:47:38.319
where you need to modify an existing

00:47:38.319 --> 00:47:40.640
deployment and you need to adapt to

00:47:40.640 --> 00:47:43.119
changes like coming in requirements um

00:47:43.119 --> 00:47:44.880
you know compliance web application

00:47:44.880 --> 00:47:46.400
firewalls need to be added on top of

00:47:46.400 --> 00:47:47.119
things type right.

00:47:47.119 --> 00:47:48.640
>> This is when you want to lean on modules

00:47:48.640 --> 00:47:50.079
and libraries don't you?

00:47:50.079 --> 00:47:51.680
>> Exactly. Because modules and libraries

00:47:51.680 --> 00:47:53.920
have been built with with like um common

00:47:53.920 --> 00:47:56.079
use cases in in mind and have been

00:47:56.079 --> 00:47:58.079
optimized for operational type of

00:47:58.079 --> 00:48:00.079
activities like handling change and

00:48:00.079 --> 00:48:01.920
that's why you build abstractions right

00:48:01.920 --> 00:48:02.800
ultimately

00:48:02.800 --> 00:48:04.480
>> you want to encode best practices but

00:48:04.480 --> 00:48:06.079
not just best practices you want to

00:48:06.079 --> 00:48:09.119
encode um maybe sometimes you will set

00:48:09.119 --> 00:48:10.480
create before destroy because you want

00:48:10.480 --> 00:48:11.040
to make

00:48:11.040 --> 00:48:13.920
>> guardrails you want to create

00:48:13.920 --> 00:48:14.160
uh

00:48:14.160 --> 00:48:16.079
>> there's guard you can add guardrails on

00:48:16.079 --> 00:48:18.880
top of that but u but a lot of times

00:48:18.880 --> 00:48:20.560
it's also because you want to encode

00:48:20.560 --> 00:48:23.280
patterns, usage patterns of how these

00:48:23.280 --> 00:48:24.960
things com, you know, bind together to

00:48:24.960 --> 00:48:27.280
to to build your cloud.

00:48:27.280 --> 00:48:29.440
>> Something this is it's so nuanced, isn't

00:48:29.440 --> 00:48:30.079
it?

00:48:30.079 --> 00:48:30.559
>> I think.

00:48:30.559 --> 00:48:34.240
>> Yeah. So, so so so basically the first

00:48:34.240 --> 00:48:37.200
results were surprising. Uh so I decided

00:48:37.200 --> 00:48:38.720
like actually this is not really what I

00:48:38.720 --> 00:48:39.920
want to measure, right? I want to

00:48:39.920 --> 00:48:42.240
measure how maintainable something is

00:48:42.240 --> 00:48:43.440
that you've built. I don't want to

00:48:43.440 --> 00:48:46.240
measure how it how quick it is to to to

00:48:46.240 --> 00:48:47.760
build the first version of it. I want to

00:48:47.760 --> 00:48:50.559
measure how does it deal with change. So

00:48:50.559 --> 00:48:53.920
AWS bench uses harbor under the hood

00:48:53.920 --> 00:48:55.599
which is a terminal bench like basically

00:48:55.599 --> 00:48:57.839
the whole concept of scenarios, tasks,

00:48:57.839 --> 00:49:00.079
oracles that's all built into harbor

00:49:00.079 --> 00:49:02.240
which is the framework that AWS bench

00:49:02.240 --> 00:49:04.319
builds upon and it only uses one

00:49:04.319 --> 00:49:06.559
capability which is called a singlestep

00:49:06.559 --> 00:49:08.559
task. So that means your task has one

00:49:08.559 --> 00:49:10.160
step generate this thing solve this

00:49:10.160 --> 00:49:13.520
problem and Harbor supports multistep

00:49:13.520 --> 00:49:15.440
that means maybe step one build the op

00:49:15.440 --> 00:49:18.800
API gateway step two we want to modify

00:49:18.800 --> 00:49:21.920
the function body and trigger a redeploy

00:49:21.920 --> 00:49:23.680
and this is a very tricky question

00:49:23.680 --> 00:49:26.400
because I know that if you write naive

00:49:26.400 --> 00:49:29.280
terraform and you then modify the

00:49:29.280 --> 00:49:31.520
function body it doesn't trigger a

00:49:31.520 --> 00:49:34.079
redeploy like you must set up a trigger

00:49:34.079 --> 00:49:36.319
on your resource and you must use some

00:49:36.319 --> 00:49:38.319
type of input like a check sum on the

00:49:38.319 --> 00:49:40.319
function body to make sure that when you

00:49:40.319 --> 00:49:42.160
change the function body there's a new

00:49:42.160 --> 00:49:44.240
deployment being generated and that type

00:49:44.240 --> 00:49:47.040
of of of knowledge operational knowledge

00:49:47.040 --> 00:49:49.680
is encoded within AWS CDK so if you use

00:49:49.680 --> 00:49:52.319
AWS CDK to build an API gateway and you

00:49:52.319 --> 00:49:55.280
use the lambda function L2 construct

00:49:55.280 --> 00:49:57.680
when you bind them together it will take

00:49:57.680 --> 00:49:59.599
a checksum of the body of the of the

00:49:59.599 --> 00:50:01.520
lambda that means if you change it it

00:50:01.520 --> 00:50:03.119
will automatically trigger a roll out

00:50:03.119 --> 00:50:05.440
and redeployment of your API gateway

00:50:05.440 --> 00:50:06.960
something that you don't get if you

00:50:06.960 --> 00:50:08.640
write if you're not aware and you write

00:50:08.640 --> 00:50:10.240
raw terraform.

00:50:10.240 --> 00:50:11.520
>> Exactly. Exactly.

00:50:11.520 --> 00:50:12.559
>> That's the type of thing I want to

00:50:12.559 --> 00:50:13.839
measure. Right.

00:50:13.839 --> 00:50:17.839
>> Yeah. Those those sort of like gotchas

00:50:17.839 --> 00:50:20.640
that just come with experience.

00:50:20.640 --> 00:50:22.880
>> Yeah. And and and that are encoded in

00:50:22.880 --> 00:50:25.440
these abstractions. So these these uh so

00:50:25.440 --> 00:50:27.680
what I then did was first off I I

00:50:27.680 --> 00:50:30.720
embedded uh the multi-step trials. So

00:50:30.720 --> 00:50:32.079
not just single step trial but

00:50:32.079 --> 00:50:34.720
multi-step. I removed because uh also

00:50:34.720 --> 00:50:36.880
notice that when Opus wrote the task it

00:50:36.880 --> 00:50:38.640
wrote right you're going to build an API

00:50:38.640 --> 00:50:40.480
gateway and you must make sure that when

00:50:40.480 --> 00:50:42.160
we deploy a new lambda like when we

00:50:42.160 --> 00:50:44.000
change the lambda that it redeploys uh

00:50:44.000 --> 00:50:46.319
correctly uh so that that you know it it

00:50:46.319 --> 00:50:49.200
kind of like led on you know some

00:50:49.200 --> 00:50:50.960
knowledge that sometimes you don't know

00:50:50.960 --> 00:50:52.319
right if you give it to someone they

00:50:52.319 --> 00:50:55.280
don't know that so I I I I make it

00:50:55.280 --> 00:50:56.880
multi-step so the first step is hey

00:50:56.880 --> 00:50:58.880
built this API gateway then the second

00:50:58.880 --> 00:51:00.960
step is here's your the thing that was

00:51:00.960 --> 00:51:02.640
built before so it's kind of like brown

00:51:02.640 --> 00:51:05.520
field uh some of it's already deployed I

00:51:05.520 --> 00:51:07.680
want you to change the lambda function

00:51:07.680 --> 00:51:09.680
and then redeploy it and then we do the

00:51:09.680 --> 00:51:11.599
oracle verification did it really serve

00:51:11.599 --> 00:51:13.599
the new lambda bundy like is it properly

00:51:13.599 --> 00:51:15.440
done and and so they will verify and

00:51:15.440 --> 00:51:16.960
then realize it's not and then it may

00:51:16.960 --> 00:51:19.280
take extra turns to to encode that that

00:51:19.280 --> 00:51:19.760
things

00:51:19.760 --> 00:51:21.359
>> yeah I guess what you're trying to

00:51:21.359 --> 00:51:25.359
measure here the operational nuances I'm

00:51:25.359 --> 00:51:27.440
sorry you got my head in a spin there so

00:51:27.440 --> 00:51:30.079
I'm just thinking of other cases

00:51:30.079 --> 00:51:32.240
>> that's what I need I need other cases

00:51:32.240 --> 00:51:33.920
other well I'm just thinking of other

00:51:33.920 --> 00:51:37.680
cases I mean I've had I mean I'm like a

00:51:37.680 --> 00:51:39.119
48 year old

00:51:39.119 --> 00:51:40.640
>> so do you know what I did to get other

00:51:40.640 --> 00:51:41.359
cases

00:51:41.359 --> 00:51:42.960
>> that's been working infrastructure for

00:51:42.960 --> 00:51:44.960
decades and this there's a ton in a way

00:51:44.960 --> 00:51:45.680
but like

00:51:45.680 --> 00:51:47.280
>> yeah so you know what I did to get other

00:51:47.280 --> 00:51:47.760
cases

00:51:47.760 --> 00:51:49.920
>> okay what did you do

00:51:49.920 --> 00:51:52.559
>> so I I spun up through uh two sona

00:51:52.559 --> 00:51:54.960
agents to uh to go across terapform

00:51:54.960 --> 00:51:58.960
provider AWS issues and look for u open

00:51:58.960 --> 00:52:01.920
issues or maybe issues with a lot of

00:52:01.920 --> 00:52:04.400
reactions uh basically from a users

00:52:04.400 --> 00:52:06.880
usage perspective right what are some of

00:52:06.880 --> 00:52:09.280
the common pitfalls so I asked you to go

00:52:09.280 --> 00:52:11.760
through the Hashi Corp community forums

00:52:11.760 --> 00:52:13.520
through the Terraform provider AWS

00:52:13.520 --> 00:52:16.160
GitHub repository issues and of course

00:52:16.160 --> 00:52:18.800
to be fair to look at the AWS CDK issues

00:52:18.800 --> 00:52:21.119
as well cuz maybe there's some

00:52:21.119 --> 00:52:22.720
abstractions that don't really work the

00:52:22.720 --> 00:52:24.960
way people expect and and it came up

00:52:24.960 --> 00:52:27.760
with 225 scenarios which then I had to

00:52:27.760 --> 00:52:29.040
go through.

00:52:29.040 --> 00:52:32.319
>> Yeah. But I I'm sure a lot of many of

00:52:32.319 --> 00:52:34.400
the scenarios are are probably outdated

00:52:34.400 --> 00:52:37.040
because of

00:52:37.040 --> 00:52:38.720
fixing things and

00:52:38.720 --> 00:52:40.400
>> yeah so but no the funny thing is that

00:52:40.400 --> 00:52:42.240
sometimes terapform provider AWS

00:52:42.240 --> 00:52:44.559
maintainers will say works as expected

00:52:44.559 --> 00:52:46.800
and they close the the GitHub issue

00:52:46.800 --> 00:52:48.800
right or they tell you you need to put

00:52:48.800 --> 00:52:50.240
in a trigger with a check summ on the

00:52:50.240 --> 00:52:51.680
function body that's what I was looking

00:52:51.680 --> 00:52:54.160
for like where is this things that you

00:52:54.160 --> 00:52:55.280
find out

00:52:55.280 --> 00:52:56.880
>> like when you've already built your

00:52:56.880 --> 00:52:58.400
infrastructure and then you go and

00:52:58.400 --> 00:53:01.280
search on the Google and then find where

00:53:01.280 --> 00:53:02.559
you find this thing. So I was like,

00:53:02.559 --> 00:53:04.160
where are those things? Where is this

00:53:04.160 --> 00:53:05.520
operational knowledge? Well, it's in the

00:53:05.520 --> 00:53:07.280
GitHub issues, isn't it? So that's where

00:53:07.280 --> 00:53:10.000
I I mined them and then I sorted them.

00:53:10.000 --> 00:53:11.200
>> So deep.

00:53:11.200 --> 00:53:13.359
>> Yeah, I sorted them by reactions like

00:53:13.359 --> 00:53:14.880
how many comments are there because some

00:53:14.880 --> 00:53:16.800
of them are like, you know, like, hey

00:53:16.800 --> 00:53:19.200
guys, we really need this and then they

00:53:19.200 --> 00:53:21.200
close it anyway. And and then also when

00:53:21.200 --> 00:53:22.559
was the latest activity because

00:53:22.559 --> 00:53:24.319
sometimes it's a capability that was

00:53:24.319 --> 00:53:25.920
added and then merged and released in

00:53:25.920 --> 00:53:27.599
version five of the Terraform ABF

00:53:27.599 --> 00:53:29.599
provider for example. Uh so there is a

00:53:29.599 --> 00:53:31.119
way to do it. For example, there's

00:53:31.119 --> 00:53:33.280
default tags. Apparently, there was a

00:53:33.280 --> 00:53:34.960
bug with the way that the defa the

00:53:34.960 --> 00:53:37.440
provider default tags uh were not

00:53:37.440 --> 00:53:39.440
merging correctly like in version five

00:53:39.440 --> 00:53:40.800
and I think they fixed it in version

00:53:40.800 --> 00:53:43.440
six. So so finding that scenario is kind

00:53:43.440 --> 00:53:44.800
of like useless, right? Because I know

00:53:44.800 --> 00:53:47.200
it's fixed in version six unless the

00:53:47.200 --> 00:53:49.520
unless cloud uses Terraform version 5

00:53:49.520 --> 00:53:51.920
pin somehow and then it still, you know,

00:53:51.920 --> 00:53:54.880
gets that issue. So So I got 225

00:53:54.880 --> 00:53:57.119
scenarios. I think I went and picked 30

00:53:57.119 --> 00:53:59.520
of them sorted by last activity and

00:53:59.520 --> 00:54:01.760
number of reactions like signal. It's

00:54:01.760 --> 00:54:03.119
funny because when you talk to fable it

00:54:03.119 --> 00:54:04.640
comes up with these words, right? Oh,

00:54:04.640 --> 00:54:06.319
this one has high signal because a lot

00:54:06.319 --> 00:54:08.640
of comments and then when was the last

00:54:08.640 --> 00:54:08.880
one?

00:54:08.880 --> 00:54:10.240
>> This is awesome.

00:54:10.240 --> 00:54:12.319
>> Yeah, it's really awesome because how do

00:54:12.319 --> 00:54:15.520
you go through 20 225 scenarios?

00:54:15.520 --> 00:54:17.680
Markdown isn't it, right? So then I

00:54:17.680 --> 00:54:18.720
said, "Hey,

00:54:18.720 --> 00:54:20.240
>> and I think I sent you that message,

00:54:20.240 --> 00:54:22.559
right? turn this into an artifact, an

00:54:22.559 --> 00:54:25.839
HTML that I can see the scenario, I can

00:54:25.839 --> 00:54:28.000
leave some comments, and then I can

00:54:28.000 --> 00:54:29.520
click a button and get all of the

00:54:29.520 --> 00:54:31.520
comments and then feed that back into

00:54:31.520 --> 00:54:33.200
Fable like, "Hey, actually, I got 30

00:54:33.200 --> 00:54:35.119
scenarios right there that I want you to

00:54:35.119 --> 00:54:36.880
have a a deeper look at."

00:54:36.880 --> 00:54:37.200
>> Yeah.

00:54:37.200 --> 00:54:38.319
>> Um,

00:54:38.319 --> 00:54:38.880
>> yeah.

00:54:38.880 --> 00:54:40.640
>> I'm just I'm just thinking aloud. Is it

00:54:40.640 --> 00:54:42.079
is it actually worth doing all this

00:54:42.079 --> 00:54:44.480
stuff? Because there's a part of me that

00:54:44.480 --> 00:54:47.359
that thinks that like that

00:54:47.359 --> 00:54:49.520
>> it it depends on what you

00:54:49.520 --> 00:54:51.680
>> some anthropic engineer is just going to

00:54:51.680 --> 00:54:55.040
train the LLM on all these

00:54:55.040 --> 00:54:57.040
>> these things. Oh, wait. I I think I need

00:54:57.040 --> 00:54:58.640
to go. There's someone.

00:54:58.640 --> 00:55:00.480
>> Yeah. So, so why do I do this is because

00:55:00.480 --> 00:55:02.240
I have a library that I want to I want

00:55:02.240 --> 00:55:04.480
to put some actual data behind behind my

00:55:04.480 --> 00:55:07.200
words, right? So, yeah, more to more to

00:55:07.200 --> 00:55:09.359
follow. definitely will share when I

00:55:09.359 --> 00:55:11.119
actually have. But the first finding was

00:55:11.119 --> 00:55:12.800
already super interesting.

00:55:12.800 --> 00:55:13.920
>> Just unexpected.

00:55:13.920 --> 00:55:16.160
>> I'm glad you're looking. Well, in

00:55:16.160 --> 00:55:18.319
hindsight, you could argue not, but

00:55:18.319 --> 00:55:18.800
anyway.

00:55:18.800 --> 00:55:20.400
>> Oh, yeah. Of course. Of course you knew

00:55:20.400 --> 00:55:21.760
it.

00:55:21.760 --> 00:55:24.079
>> I I knew it. Yeah. Anyway, really

00:55:24.079 --> 00:55:25.760
interesting stuff, Vincent, as always.

00:55:25.760 --> 00:55:27.839
Thanks again for having a chit chat. See

00:55:27.839 --> 00:55:28.160
you.

00:55:28.160 --> 00:55:31.040
>> Yeah. Yeah. See you.