Episode 13: Why do we need the Cloud Development Kit?

Welcome to episode 12 of the AI
Infrastructure Podcast with me, Kai
Hendry in the UK and Vincent Smith over
in Vietnam. In this episode, we talk
about CDKTF.
And long story short, we try to
rationalize where CDKTF fits in in light
of the recent deprecation by Hashi Corp/
IBM.
And
we we do justify CDK. Uh well, let me
give you a summary why we think CDK is
important.
It's a high level abstraction
and it allows complicated cloud
infrastructure to be reasoned about.
It allows you to get the most out of the
cloud to differentiate.
And if you just think conversely perhaps
if you got if you vibe coded your
infrastructure using I don't know AWS
CLI or something you just couldn't get
uh
a solid reproducible deterministic
infrastructure that way you need higher
level constructs uh that the AWS CDK
sort of provide you. So
you ideally need to work with it. So in
light of the deprecation of of IBM that
we need to find a place for for CDK a
programming language in order to do the
next level stuff with the cloud and
infrastructure. So enjoy the episode.
Please uh
if if CDKTF doesn't uh tickle your
fancy, check out the other stuff in the
playlist and have a merry Christmas. See
you. Goodbye.
Yeah, this guy. Which which guy?
>> So, since CDKTF was sunset by IBM and I
created a fork under a project that I
built a bunch of CDKF
L2 constructs. There's one guy who's
been contributing three PRs over
well so far he's contributed I think
seven PRs already and I merged six and
he keeps adding cool stuff. It's
amazing.
>> Yeah. So you just forked it and and
>> No, no, no. He contributes.
[clears throat]
So the one that I forked, I put it on
pause, right? We're waiting for a go-
ahead. Um we're collaborating with the
Open Construct Foundation which um hosts
several of the
you know CDK related projects and we're
trying to find a final and good name. Um
hopefully we don't have to rename but I
sincere I seriously doubt that Hashi
Corp is going to allow us to use the
original CDKTF package names. So we'll
probably have to rename it. And I
renamed the TCON which is Terra
Constructs. It's a four-letter acronym
and people obviously don't like that.
TCON sounds familiar now. [snorts]
>> Yeah. Terra Constructs is the library
that I built.
>> Oh okay.
>> Yeah. So tons is is what I you know it's
a scope that I have on npmgs. So I use
it for a lot of stuff. So I was like,
"Yeah, I'm just going to reuse I just
took immediate action, right? Fork it
and rename it to my stuff and then
[snorts]
>> then um
>> yeah, so it it pushed my project in the
foreground and we're holding on the fork
because we don't have the final name and
I agree maybe we don't call it TCON, but
at the same time somebody else took
notice of my other project and has
contributed about eight PRs to it and
they are all high quality.
>> Can you show me? Can you get give me a
link? I just I'm just curious to see
what what what what you what what this
contribution is. [snorts]
>> Yeah. Uh
it's on my
here.
I have to get used to this um thing
jumping around. Hold on.
>> But you mean your Arch Linux C?
>> Yeah, I use I use Arch by the way.
It's never going to get old. Going to
keep saying it. Okay. I send it on Zoom.
>> Yeah. Like Primagin says a lot. Does he?
>> Does he just We're just following other
YouTubers. So sad.
[snorts] Uh
>> no. But then then I have to start
whining like Teo.
>> Shots fired.
>> I'm already doing a job whining. Right.
One complaint that I get from my YouTube
channel is that like, Kai, yeah, I like
your content, but you you you complain
too much, so I can't watch you. It's
like, what do I am I that guy?
Uh,
okay. Refactor, use AWS CDK. Oh, hold
on. Let me see if I can sh my screen.
I've been digging into that PR because
it actually enables really cool stuff.
So basically a little bit of background.
AWS CDK has this regional information
library
that keeps it track of like which
capabilities are available across the
regions. They download a fact table like
uh lambda runtime support for NodeJS
latest is version 22 in China or
government region for example but it's
version 24 for other regions where
NodeJS version 24 has already been
deployed right it takes time for this
stuff to deploy
>> so [clears throat]
other stuff is like uh S3 bucket
endpoints or like the domain suffix of
an S3 uh original endpoint point, right?
The URL suffix that um they changed the
way that that S3 works over the years.
So new regions, they use a different
suffix.
>> So the you know, I think I noticed that.
>> Yeah. So the AWS CDK has logic in it
that uses a fact table about the regions
to figure out how to build a a bucket or
like if you give it a a bucket URL or
just a bucket name and and it's it has a
it has a regional like your AWS CDK
stack has a region. So then it uses that
information to find out yeah what's
going to be the domain URL for that
bucket depending on the region. So I had
copied that over. I had copied over the
fact table into my code because I didn't
know it was a separate package. And he
says, "Well, you know, we don't have to
copy over the details. We can just reuse
the AWS CDK package on top of CDKTF."
>> Cool. This actually looks like a pretty
good contribution. Holy moly. I'm I'm
>> Oh yeah, dude. All of his contributions
have been absolute fire. The only thing
is he doesn't write integration tests.
So every single one of them I I check
out and I um I I build an integration
test and I validate that you can
actually terraform deploy it into AWS
and doesn't get any errors. Yeah. But
he's been he's been aligning Terra
constructs closely to AWS CDK.
>> Cool.
So I got a couple of maybe noob
questions for you but like I guess this
might serve the audience.
So
CDK for Terraform
basically allow people to write in that
familiar CDK style which was
which was like which came out of
>> AWS
>> AWS uh what's his name? Ben Ben Israel
right that guy
>> not Ben Israel
>> um who was it again?
Is it worth mentioning her name?
Who? What? Sorry.
>> Who who who was the guy that was in the
original team that he left and he he did
did Wing Wing Lang?
>> Yeah. Elot Ben Israel. Uh Elot Ben
Israel.
>> Oh, I I wasn't wrong with the name.
Okay. Sorry.
>> Yeah. Yeah.
>> I thought I was making up the his name
or something. So, CDK
cool beans. I think we went through
yesterday that it was like the the thing
that really makes it different is that
is that it's operationally
uh friendly. You can refactor this the
stuff
and it's a programming language un
unlike maybe Terraform and things like
that. Now the thing that I don't
understand is how it links up with
Terraform. A and B
what did actually CDK for Terraform
actually provide? Did they provide
everything that that uh Terraform
providers did or was it a subset?
Okay. So
the cloud development toolkit CDK from
AWS com is built upon and they've split
this out like some composible
um ideas. So one of them is the concept
of constructs. Constructs are basically
objects that
know how to synthesize or write themsel
out into like a small little snippet of
configuration,
>> right? If you use if you use a con and
they are able to be composed into trees
of constructs so you can nest constructs
into other constructs and so
>> yeah don't people sometimes call them
>> oh is my webcam just stopped don't don't
some people call them like stages or
something like that
>> no stages is something else I think it's
um if you are doing like cloud for
deployment across different
environments. I think you can define
stages,
>> okay,
>> to like duplicate stacks. I I mean I I
know that CDK pipelines uses stages. I
know that there's a concept of stages
within AWS CDK, but the thing is
first AWS CDK was built within AWS.
Right.
>> Right.
>> They had a couple of crucial innovations
in terms of infrastructure as code. The
first one is a tree,
a composition of configuration that
starts at the root note which is usually
your app, your entry point and from
there on you tie like a tree of
constructs and when you and each one of
them have a little bit of configuration
like information so they know how to
synthesize out into a configuration
target language. So if you use AWS CDK,
the constructs that you need are the
app, the AWS CDK app,
>> that synthesizes itself down to a cloud
for stack,
>> right?
>> And then each construct that you put
inside those apps
um will be a cloud for resource, right?
Because within a stack you use cloud
form resources, right?
>> Right.
>> And now if you apply those ideas on top
of Terraform,
>> right? So when when they port over the
the app, it's basically something that
represents a single terraform
uh root module, right? If you talk about
terraform modu um nomenclature or or
concepts
>> terminologies.
>> Yeah. Yeah. If you if you basically have
a directory and you drop in a config
file and then you run Terraform in it,
it will consider that directory as the
root module, right? And then if you
create another directory and you put in
some config there and then you include
that directory as a module that will be
like a sub module right. So in terapform
constructs are kind of like directories
of configuration files and you you um
source them into like module source and
then you can nest those modules as well.
But the one that is the actual um
you know app entry point is the root
module of your directory. Right? When
terraform runs, it will find the
terapform configurations in your
directly sibling files and aggregate
everything into a single config in
memory, right? Both HCL, which is a
separate set of JSON and some JSON if
you want. And then it will create in
memory your Terraform configuration.
And then within that root module, you
may refer to other modules that are
coming from either
>> it does it write it out as normal
Terraform. So you can inspect the
Terraform.
>> Yeah. So the the whole greatness of CDK
for Terraform is that ultimately what
you get once you synthesize or serialize
that construct tree down to the target
language you get pure and raw Terraform
>> and is is it is it readable or is it
like
>> it is
>> nonsense?
>> It's very readable. Uh there's a couple
of things that may you know make people
upset because in the construct tree
um the identity of a resource like if
you write terraform you're going to say
um AWS
instance and then you give it a name
like uh web and then you put in the
config attributes such as VPC ID
instance type imi like what's the image
that needs to be provided right so the
name right AWS instance And then you
have the name web in when you serialize
from from TypeScript down into that
config.
>> Yeah.
>> Kind of flattens everything. So it it
does some type of unique ID generation.
>> That's a good thing though. I thought we
we we discussed that.
>> No, it it's it creates an unique logical
ID. So that resource it's not your
physical ID because in in any case when
you create an instance in AWS your
physical ID is you cannot determine it
either, right? Not all AWS resources
allow you to determine your physical ID.
The instance ID is I- blah blah blah
blah. Your your load balancer ID is I is
LB- something. But if you do a bucket
and you say I want the bucket name to be
FU, it will be exactly FU and globally
unique. You will not get it even if
somebody else created a bucket fu.
Right? [snorts]
>> Um so so some resources they allow you
to determine or to define your physical
ID and some [snorts] resources they
don't allow you and they automatically
get an ID. But here I'm talking about
the logical ID, right? Even though the
logical ID will be something that you
write in your configuration file. So if
you ask me like does it serialize to to
readable terraform, it is just the logic
ids have these weird suffixes that you
can actually disable if you want. You
can actually go and say don't give me
these hash suffixes, but they're used to
make things unique. They're used to make
sure that the logic IDs don't have any
conflicts. So, [snorts] so that's the
only thing that I think people when they
first look at it kind of get annoyed by
is like why are these names so like have
this weird checksum like like you know
this hash at the end of the name. Uh and
that's purely because they don't want to
have a conflict or an error when you
have like nested parts that all get
serialized down into the root module.
>> Um so CDKTF also supports modules
Terraform modules by the way you can
refer to an existing Terapform module.
So it's very good for an existing team
that is already using terapform modules
extensively. With CDKTF, you can take
that terapform module and convert it
into a TypeScript interface. So it reads
the full module definition, looks at all
of the variables and their types
>> and then creates an equivalent type.
>> In my in my current client, we we we
switched to Terra. They they were
they're a CDK shop, but we but when I
joined, we we basically set up a new
workflow with Terraform because of
Snowflake.
But it sounds like I could have used
CDKTF to bring in the Snowflake
Terraform provider into into CDK world,
right? Or to Am I misunderstanding?
the CDKTF provider would allow you to if
you so so that's one thing right you can
create bindings for modules and it will
look at the variables that the module
take and then generate a typescript
interface or you can create bindings to
terapform providers like the so
snowflake provider and yes you will get
a TypeScript interface but no it will
not allow you to to use it with AWS CDK
right because AWS CDK only works with
cloud form and the CDKTF generated one
will generate terraform.
>> Yeah. Yes. So, but it Yeah.
I mean, to be honest, I'm not a huge fan
of all the I mean, this this this magic
converting different bindings of
different languages into CDK kind of
makes me nervous.
>> I I want to clarify one thing, right? in
itself
creating a terapform provider I mean
reading in the terraform provider
resources and generating TypeScript
bindings or Python orn net or whatever
on its own has very little value
honestly um you just get these like L1
constructs and you now can call that
constructor and maybe the one thing that
you get is you're able to use like um
lazy evaluations basically you And for
some attributes, you can provide a
function instead of an actual value. And
so when it synthesizes, it's going to
execute the function to basically decide
what the value for that property needs
to be. And that is very powerful
already, right? Because it means that
depending on how you call it, a function
can execute and then determine what the
value for this particular attribute
could be. Um, for example, if you are
where is this like really useful? Yeah,
maybe in policies, right? Let's say that
you you are you are building some type
of infrastructure and you're using an IM
policy and you're using a bunch of other
resources and you want the policy to be
generated at the very end after the user
has already defined the full program and
all of the configuration that he wants
to be
>> generated. Yeah, there's a lease
privileges type thing. M
>> yeah the functions can be very smart and
identify exactly what are the actions
that need to be added into the policy
depending on what the user um you know
connected. So it's kind of like it's
kind of a mixture of like an imperative
language to create a declarative
definition of infrastructure as code and
like people are like oh you know why
would you do that you know is all about
declarative and you you cannot do mix it
with imperative you start doing that
people will no longer like know exactly
what's going on uh but if you look at
the language itself since uh terraform
0.12
you know before it became uh stable they
added um list comprehensions they added
dynamic values dynamic you know for each
and there's a very clear reason why they
did that because you can't manage AWS
resources with pure static definitions
you're always before that you have free
processors people wrote template
languages to generate terapform
configuration
>> yeah yeah I feel I feel this is a trend
in infrastructures co in infrastructure
game is like
the days have gone where you can just
declaratively put thing is in YAML,
right? You you need
>> you need a programming language and then
>> you're going to keep adding layers and
layers of abstraction to work around
this this core missing feature that you
are not working with a proper language
programming language and then you start
adding um config language.
>> Yeah. So generators
>> I guess in a in a very
uh succinct way CDK gives you the binds
to a proper programming language and
that's that's really what what it does
right [clears throat]
>> and but still I mean and then and then
it gives you the ability to do lazy
evaluations so late binding um so that
you can evaluate the full full program
and the user can program uh imperatively
build up the the configuration so he can
take some object that somebody else
wrote and then invoke some methods on it
to mutate this, you know, the the the
object to their liking.
>> Like for example, in in CDK for
Kubernetes, um when you create a
deployment, it's going to define a pot
template and then a replica uh count
that how many times you want that
template to be created um to create the
pots, right? And and then you want maybe
to set some type of anti-affffinity. So
you can do all of that declaratively in
in Kubernetes and it's not a simple
configuration system because to define a
pot template with anti- affinity rules
um to be fully flexible and this is also
why people like compose right compose is
super simple but like Kubernetes has all
the little dials and knobs.
>> Sorry what's compose oh with a K. You
mean
>> docker compose um you know before that
or ECS people like
>> oh docker compose oh okay
>> yeah I love docker compose because it
gives you like this very simple thing
but if you want to do something earlier
you were complaining um yesterday you
were complaining you were saying
verscell is great but if you want to do
like um custom type of database I can't
do it you know I need the dials and
knobs well that's where it breaks right
these abstractions so the same way way
with compose and and and that's why
kubernetes as like everything under the
sun, right? You can define exactly what
resources on the node need to be
>> for days and crossplane.
>> Yes. And and and and so the actual
configuration you have to write becomes
super complicated. Kubernetes is the
same thing. You end up with like
>> what is the Yeah. So I got CDK is the
programming language answer to maybe uh
Terraform. But what is what is the
programming language answer to to
you know Kubernetes?
So AWS built CDK for Kubernetes. There's
one guy Ilia, he did a lot of like he's
one of the main contributors.
Yeah, I loved it. Like initially it was
a bit frustrating because you
>> Yeah, because one of the things you
learn as a Kubernetes administrator and
if you go for certificates, which I
never do because I don't think they're
worth it. Um,
>> oh dear God, don't go there. That's a
rabbit hole.
>> Yeah. So
>> is this another Ben Israel thing? Um,
no, but it comes from within AWS, but it
applies the same idea of of creating a
tree of constructs and and and the same
idea of building like the core
Kubernetes API objects which are pod
deployment, replica set, um, service and
all of these, right? And now he's doing
robotics.
>> Anyway, so Ilia is one of the guys that
like I see contribute and react and
still maintain this project mostly. I'm
sure there's other people working on it.
But the greatest thing here is
>> the moment you have built those core API
objects, you can still write this, you
know, create your Kubernetes
configuration, but it's honestly worse
worse than YAML because you have all
these additional curly braces and very
nested like complicated configuration.
So pure L1 constructs, this pure
language binding are not very useful.
But what's immensely useful when you
work with CDK for Kubernetes is that
they built these higher level ideas of
like we're going to grab this low-level
API object that's called a deployment
and we're going to add a scheduling
namespace and we're going to add the
idea of spreading pods on the scheduling
namespace and suddenly you are defining
like I want to spread this across nodes
and there you basically you go like my
deployment.spread spread dot uh sorry my
deployment.sccheduling.spread
spread as a method. Your intention is to
spread these pots. That's your
intention, right? And that gets
translated into a declaration, an actual
rule.
>> Yeah. So,
>> the affinity
>> these convenient Yeah, that sounds like
a good useful abstraction.
>> Plus,
>> so what what abstractions?
>> Oh, plus, sorry. Yeah, plus is is the
the the the real power because that's
where they they create these high level
like deployment and chron job and and
then you and they make service and and
then they have like reasonable defaults.
So with very few lines of code it's kind
of like Helm, right? But Helm is
declarative like YAML config files with
no schema. So you don't get
IntelliSense. you don't know exactly
like where do I need to put this
configuration for it to end up where it
needs to be because YAML helm doesn't
have any schemas and even in later
versions they try to add schemas but it
doesn't really work.
>> So have you have you used this
production or you just checked it out?
>> No I I I I
was at an organization that had
everything under the sun. They had helm
they had JSONet they had customized they
had
>> Oh my god.
>> Yeah. They had everything.
>> And did you introduce something new
while you were there? Of course, I I I
built a new standard, but I [laughter]
did more than that. I did not just add a
new standard. I actually built a bunch
of migration tools for everything else.
So, I I had like a very clear path like
if this is a service that you're working
on and we we were adopting Argo CD and
the only way to adopt Argo CD was to
adopt CDK for Kubernetes. So, I made it
so that you can't go and manage your
service with Argo CD without
>> I don't think we've ever really talked
about GitOps. Are you a GitOps? I mean,
do you have uh any opinions there
quickly?
>> Yes, I I
>> maybe save that for another video.
I [snorts] ran Argo CD since 2019
and very quickly realized that a lot of
people like to do magic on the server
side and they like to minimize the
configuration that goes in and then they
like to Argo CD to like expand
everything using Helm uh rendering like
last minute.
>> Oh no no no. You want Argo CD just to be
a UI is is or
>> Yeah. Yeah. Exactly. You want Argo CD to
take whatever is in git and fire it
straight at Kubernetes. Exactly. And
then get back to that's that's one thing
I I use Argo CD and ang
>> crazy like why mix it up.
>> Yeah. So some so they build all these
amazing customized plugins and Helm
plugins to run on the Argo City server
and that's the first thing I throw out
because you want
>> you and I
pattern
>> sorry
>> you and I are the are a little bit too
similar
>> but but you know why this is relevant
because this is called the rendered
manifest pattern and the co and the core
creator of Kubernetes like one of the
co-founders Brian Grant and one of the
co-founder founders of of of the idea of
githops uh for flux with weave and you
know there's flux cd and there's argo cd
>> yeah I don't yeah flux used to be a
contender right
>> I think ar flux is still a contender
they broke it down into microser they
make it it's way more performant but it
doesn't have a fancy UI so everyone
loves Argo CD but nobody like flux is
still evolving but again like I haven't
ran communities for 2 years but what I
wanted to say is that the guys that are
very, you know, smart people, [laughter]
Brian Grant and um Richardson, uh from
Weave, they created the logical
extension of this rendered manifest
pattern. So basically the idea of like
you you make sure that everything gets
rendered down into manifest into git. So
Argo CD can purely track a raw manifest
and just fire it off after Kubernetes
and then you you are able to validate
immediately what is the error message
that that Kubernetes sends back because
it just relates directly to whatever you
have in git. So you can see very
clearly. So they evolved that pattern
and they call it config hub. So instead
of just like having a very dumb git
database, they built a specialized
database for for for uh storing these
configurations. And then the idea of
like um making a change, you can go
straight into config hub and make a a
change where it needs to be. So you
don't need to go through the git
pipeline. Um and then you can also call
functions if you need to make a change
across
>> I thought we thought I thought we
established that changing things outside
the the pipeline is a bad idea.
So, so they turned it upside down and
they they provided an a config hub that
helps you make changes in a controlled
way and in a repeatable way because it's
kind of like functional programming. You
can register endpoints on top of your
config data and then you can apply
functions. So, mutate your config data
across your whole environment and then
you can do it with an approval step uh
workflow. So you you just select what
you want to affect, you apply the
function, you can see a preview, and
then you can approve and then boom, it's
GitOps. It gets ops in a in a in a like
reasonable way.
>> Have you used this product? Have you
used this yourself?
>> Uh no, but I um I understand the idea
because a lot of what what he built
originally with customize, you know, the
idea of applying JSON patches, they kind
of like extended it as those get JSON
patches now run as part of like ant
hooks. haven't used it. They just made
it like
>> Oh, yeah. We were talking about hooks
yesterday. What was using hooks again?
Oh my god.
>> Oh, can do can do
>> Yeah. Yeah. The hooks thing is
>> in the CDK as well.
>> This kind of a little bit different.
Hooks.
>> Yeah. But this one in my mind
>> I don't know if config hub can can
manage any other configuration except
for Kubernetes. This this is like firmly
settled in the idea of like a
reconciliation loop and and having a
control plane that runs that constantly
runs these reconciliation loops and
automatically adjust your like with
eventual consistency adjust your your
cloud towards it. Right?
>> So which is one thing that Terraform
does not have right terraform you run an
apply and it doesn't touch it until the
next time you run an apply where
Kubernetes is constantly re-evaluating
your infrastructure and making sure that
it is aligned. I the the one thing
uh
[snorts]
was so like let me start with the
problem at what we're using terraform
uh actually this is a problem with CDK
too the whole CDK deploy thing is a bit
slow but okay never mind that
>> that's cloudformation issues
>> that's that's infrastructures yeah
that's that's cloudformation issues but
um stop me if I talked about this before
but one one issue that we have right now
is that we're using uh database we're
using Terraform to to manage Snowflake
and we don't want to we don't want to
delete the resources because that could
lead to data loss and all
>> stuff and we don't want to recreate
resources. So, so I have I have like
some crappy
>> worried about constantly re like
>> Yeah, I have some crappy what do you
call it? You know, uh GitHub workflow
where I basically inspect the plan to
make sure to make sure it's not deleting
anything. But I thought to myself, come
on, there's got to be a better way
because because right now we have a we
have a separate plan and an apply stage.
Why? Because we want people to do a plan
before they know what they're doing
before they delete things.
Unfortunately, no one really reads the
plan in all honesty. So, there's that
problem. So, so hence I have some crappy
code to check for for some a deletion or
a a recreate. And I just think to
myself, is this the way people do it? Is
there a better way?
I mean, that that sort of life cycle
type thing kind of sucks.
That's my current.
>> Yeah. And that's also why a lot of
people when we talk about crossplane
there people very um
pessimistic around it because they put
so much um official or orchestration
around this you know plan and apply.
they have a a big like ceremony around
the process and so something going
behind the scene and constantly
reconciling kind of feels wrong to them,
right? Uh and I think it really depends
on the maturity. If you're Google and
you're deploying a thousand times a day,
your services are reconciling and this
is how you build your architecture. I
guess if you I guess the there's an
there's an adage with when it comes to I
mean this is probably one of the first
things I learned when I got into
infrastructure is like don't manage your
database in infrastructure is code
you know because because then you won't
have these problems like separate the
data
>> so that was definitely s uh like true
for the longest time because stateful
kubernetes cluster
>> like
maintaining database in and I think
there was a Kubernetes uh controller or
something where you could spin up a a
database and everyone said no this is a
terrible idea
>> and I and quickly caught on to why
>> if I talk about 2016 then stateful sets
are like just introduced.
>> Yeah.
And and so at the time the problem with
Helm is that everyone can just pull a
chart and go like, "Oh, uh, Helm
installed MySQL. There I go. I have
MySQL now." Yes. But you know,
>> set up a huge cluster.
>> Yeah. What what where where is the data
stores? Oh, so it's provisioning an EBS
volume. How how is the ownership of that
EBS volume tracked? Like what if you do
an upgrade? How do you manage? And then
people they built this is like you know
pets versus cattle where you're building
yes you're creating a a cattle of nodes
but you're building pat kubernetes
clusters. So so what is the the point of
pat kubernetes clusters then you need to
start bu using arcane or whatever it's
called to to create backups of your
clusters and your control plane. I never
like that idea. I always made my cluster
completely stateless so that I can
rebuild it
>> a thermal I don't know.
>> Yeah. So all of all of the state lives
outside right that's how you want to
build your microservices if you can
depending on you know what is the SLAs
and and what is your your architecture
that you have to support um but if you
can and you can perfectly scale
horizontally and you keep all of your
data on a different tier outside of your
cluster then that makes life so much
easier because you can just blow away a
cluster and rebuild it. Yeah.
>> You can just put them side by side and
they can share the state outside of the
cluster. Um and then once you start
building stateful clusters, your SLOs's
are completely different. Your your
operations around those clusters are
completely different.
>> Yeah. But but don't you think that like
like just going going back to CDK like I
mean a lot of tooling does bring state
into it, doesn't it? A little bit.
Sorry about CDK and and like uh sorry to
I'm just trying to bring it back to
to CDK like like like doesn't CDK I
think we talked about it yesterday but
like doesn't CDK have like a strong
concept of context or something like
that and that in itself is a state isn't
it I mean
so on one side in these traditional
infrastructure as code frameworks you
have the concept of a state like cloud
form keeps track of the resources it
created And Terraform has a state file
that keeps track of the resources it
created. So that it knows, you know, it
ties back those logical ids in the state
file to the actual physical ideas. And
if you remove a logic ID in your
configuration, it will make sure that
that resource is deleted. And if you
modify the configuration,
>> yeah,
>> it will make sure that the actual
physical ID uh resource is updated.
Right. So
>> it uses a state file.
>> Yeah. So basically I'm just
uh wiring these in my brain. I mean I'm
crosswiring these in my brain. Like if
we have infrastructure and we want to
have a goal of keeping separate state
I guess maybe it should be scoped to
application state or something but but
it's it's um it's not I mean if you're
religious about separate state it's not
that clear to me separate cleanly
>> you wouldn't run a terraform controller
in your kubernetes because that means
you're adding state to it or the
terraform controller must have access to
an external state bucket uh within the
>> Yeah, it's got to it's got to be it's
got to be a very clear division, right?
>> Yeah. So, that's a problem with
crossplane, right? Because where does
crossplane store that state tracking? It
stores it inside the Kubernetes cluster
control plane.
>> So, now you've created a stateful
control like control plane. You've
created a stateful cluster.
>> So, so that's where where like you know
you're using crossplane to bootstrap
Kubernetes and that becomes a stateful
cluster. So you need like one Uber
cluster that controls all the other
clusters and that becomes your stateful
cluster that you need to do very careful
upgrades around operations are are
different. And of course it makes sense
for a company like Upbound which
sponsors crossplane to say hey you know
we are your your Uber cluster we manage
that for you. We we provide you a
stateful cluster. We guarantee that your
your control plane is backed up and can
be res restored and we're we're going to
take care of that. And that's the you
know the the managed service offering
that they provide. Um and that's why
they they they create something like
crossplane right. Um and I think Google
cloud went the same way. They have like
service connect or config connect. They
have a service in GCP that's very
strongly based on of off crossplane that
basically you you you you define and and
to be honest Kubernetes is already
taking care of a lot of your a your
cloud resources right because when you
say I have a service and I need to make
make it externally exposed it goes and
creates a load balancer right so you
already have a cloud controller inside
Kubernetes creating um AWS resources you
have a controller that creates nodes to
add capacity to the cluster you have a
controller that that um creates um
routing rules if you are depending on
what type of uh networking that you set
up so that your your uh routing tables
can route the traffic between your pots
across the nodes. So, Kubernetes control
plane already modifies your cloud and
creates resources in the cloud. So, it
makes sense to extend that and also
create databases and buckets and
>> Yeah. Yeah. This is this is where things
get look crazy to me.
>> Yeah. I mean I I [sighs] see the point.
I I think if you're going all in and
you're a big Kubernetes shop, it makes
sense. The only biggest issue I have
with Kubernetes is that you have these
declarative YAML that becomes super hard
to to manage and then solutions like
config hub make sense because now you
can define your load balancers, your
buckets, your databases, everything in
config hub and let the reconciliation
loop constantly. tell okay maybe let's
turn the conversation back to AI like if
you were to if you were if you were to
like start on a green field project and
use Claude to to set up um
I don't know we kind of do need to scope
this to some sort of example let's say
that we need to run um I don't know
WordPress
at scale you know WordPress.com
Uh, so PHP, MySQL, blah blah blah.
[snorts] the
>> be careful. Would you do would you do
that in Kub would you do that in
Kubernetes and and just let Claude
spin up some
uh I don't know CDK uh what did we call
it CDK Kubernetes like what what would
give the best
iteration you know what would give the
best AI friendliness slash
iteration loop
is what I'm thinking
Oh. Um, so this are you thinking about
what type of system would be able to
leverage AI in the me like the best way?
Yeah. Would be intuitive to the AI,
right?
>> Exactly. So, so the so that kind of
assumes some the AI is sort of maybe
kind of familiar to the workflow. I
mean, you don't want to spend too much
time teaching it something new, I
suppose, is what I'm trying to lose.
>> So, that's where everyone jumped on
creating MCP servers or or finding ways
to to provide the LLM, the generic LLM
that has been trained of lots of data to
basically give it the right context for
the work that it needs to do. So, if
you're take a generic
>> MCP, MCP is more for doing tool calls
than giving context.
Some people seem to mix that up a bit.
>> I think it's No, it's the difference is
tool calls allow you to pull in context
on demand. The LLM sees a tool that says
you can call me to get information about
the weather in in London. Um, you can
call me.
>> So, it's tool calls
>> give the LLMs the arms and legs to reach
out into the world to find information
that it doesn't have.
>> So, it can do it on its own. If you if
you don't use tool calls, you as an
operator of the LLM have to kind of
figure out everything that that LLM
might need to solve the problem that
you're trying to do, right?
>> On the subject of tools, you know,
Kubernetes tools, cubectl is pretty
pretty awesome, isn't it?
>> Um, so that's where things have evolved,
right? If you if you use Claude Code
skills and you give it a high level
overview what cubectl can be used for
and then it goes and finds out more
information about like the user has a
problem uh with autoscaling and I need
to figure out if there's enough capacity
in the cluster and then it goes off and
uh finds the cubectl commands that you
know tells it how much like the top like
node stop for example to see how much
CPU is being used across the cluster.
So, so skills are this like gradual
discovery of information without
overloading the context with every
possible
uh tool that the QTL exposes, right?
because ultimately cubectl can be used
in such a way that you can overload um
with like hundreds of tools but you you
need to slice it and dice it down into
like you know if you're working on this
particular problem then you might want
to look at this set of tools uh and if
you want to work on that type of problem
you want to look at this set of tools so
then cloud knows like hey I'm looking at
this type of problem oh can can we go
look at those set of tools and then
loads them in and figures out that it
needs to like
>> that's what what what the skills changed
compared to tools.
>> Yeah. Yeah. But like yeah, sorry.
>> But which which infrastructures code
technology has the best tooling in that
regard?
Kubernetes does come into my mind
or do you think CDK is uh I mean are you
managing infrastructure with AI and CDK
is my next question.
>> Actually funny I can give you an
example. Um because when I built Terra
constructs I copied the AWS CDK
interfaces. So my company they adopted
GitHub copilot into PR review process
and my colleague created a pull request
with Terra constructs that basically
creates a load balancer sets up
listeners and things like that but the
code is TypeScript and it looks like AWS
CDK. So copilot started to to comment on
it and say like hey um your load
balancer listener here uh is not using
TLS or whatever. So even though um you
know it hasn't been trained on terra
constructs and
>> and it had no it had no like uh what do
you call this type hinting or
>> no nothing I I I think you know big part
of the corpus I don't think we can
imagine how much of of of you know these
weights actually encapsulate it's like
billions of of internet data you can ask
it something silly like um you ask about
cats in in in in uh in Dermund Belgium
and they will tell you about a tradition
back in the Middle Ages where they throw
cats off the walls and it's like who
would know that, right? And and it's not
made up like it's true.
>> Why did they throw cats onto walls?
>> After the winter um they they they kept
cats to kill to prevent the mice to eat
the grains or something or to bite the
rats or the mice were were destroying
the all of the storage like the linen
and so on. So, so they have cats to
hunt, but then when the summer comes
around, they have too many cats, so they
threw them off the off the off the
towers. Uh, tradition, Belgian
tradition.
>> So dark. [laughter]
>> So, this is one thing,
>> but with Belgian, it's always like some
dark stories like
>> Yeah. So, if it knows this horrible
detail that we Belgians have been trying
to hide, how of course it's going to
know how to write AWS CDK, you know.
[laughter]
>> Interesting. So, so you're basically
saying that it doesn't matter what you
use, AI is going to be flexible enough
to get you there.
>> And what I'm saying is for me and the
reason why I spend all this time on what
I'm doing for me AI is is pure
intuition, right? All of this
hallucination which we sometimes we
don't like, but sometimes it is pure
like out of the box thinking and we
embrace it. All of this hallucination is
intuition. it's built into the model to
like feel like that's the way I need to
go. So whatever you know framework you
work with should be an intuitive
framework. So I feel AWS CDK is
beautiful because it's all about
intentoriented methods. It's not like
this massive declarative get every
single attribute correct or it's not
going to work. It's intuitive. It's like
I want to deploy I want to deploy my
service. I want to spread it one per
node. Um and and I want the spot
disruption budget. That's a good
intuition.
>> Yeah. So that's where I feel like it's
been trained heavily on Typescript and
JavaScript because that's like still in
the top one languages of of Stack
Overflow every year. So it has a massive
amount of JavaScript samples and
TypeScript samples. It has I don't know
probably more JavaScript and it has the
ideas and hallucinate and hallucinates
constantly. It's actually not Lams is
all hallucination like every single
token it outputs is just hallucination
and int intuition right so the fact that
it's correct these hallucinations are is
is mindboggling right uh but the fact
that it can generate AWS CDK correctly
and in Typescript to me is proof that I
would rather um focus on a highle
language like that
>> yeah but don't you think it's strange
don't don't I mean surely
going back to what I was asking earlier
like so you don't give a type hinting
like surely if you if you if you let uh
Claude I mean VS code must somehow do
this but like if you open up a project
with with uh with with type checking
then
and you let AI make some you let Claude
make some changes surely it it can
recognize that it's done a bad change
when the uh LSP starts That's this is
the agent's shell, right? This is how
the shell surfaces the information
to the LLM. It captures a whole bunch of
information. It also gives it guidelines
like I made a change. Like if you look
at cloud, it will say I made a change to
this file. Let me make sure I don't know
it says low bandwidth. Maybe my
connection is bad.
>> Yeah. Yeah. You just dropped out a bit.
>> Sometimes cloud says,
>> yeah. So, so cloud will say I made a
change to this file. Let me run the
tests or let me let me run this. This is
like agent shell instructions, right?
This is like part of Claude Code's magic
sauce. It just it just says like you do
this then you do that. You run this
tool. You connect to the IDE and look
for the problem. If you go to Visual
Studio Code, you can see problems and
say file. Exactly.
>> Yeah. Language servers integrations and
so on. So this is agent cell. This is ID
context. This is what makes a coding
agent good at what it does by by
surfacing the information in small bits
right when it's needed. Um,
>> yeah, it's it's it's it's such an
incredible technology, isn't it? It's so
incredible.
>> And and so what I wanted to give you an
example of where I think it's obvious
that cloud will not be good at doing
things with Terraform compared to AWS
CDK.
>> Okay, juicy. I had the requirement
a theory to set up swap files for nodes
because we wanted to allow bursting. You
know, you can overshare resources on a
node and we wanted sometimes resources
to be overs subscribed and some services
to basically burst out and take more
memory than the node has. So we wanted
to alleviate some of the pressure by
creating a swap file on disk. So I asked
cloud to help me prepare the launch
templates for the autoscaling group. And
this was pure terapform, no AWS CDK. And
to do the swap file configuration, you
have to define the swappiness on the
task definition. But on Terraform, the
task definition is a massive JSON blob,
right?
>> It's just a massive JSON definition. And
cloud can do like the first level of of
of um of attributes of this JSON
definition. it it can do it decently,
but it when I mention about swappiness,
it also just put the attribute
swappiness straight on like the the top
level as an attribute as a field on a
JSON document and it didn't work. Uh or
it looked like it worked. It didn't give
me any errors, right? But when I looked
into it, it actually didn't work. And I
had to look really deep into AWS docs to
figure out that swappiness was not a top
level field, but was nested within
something called kernel attributes or
something like that. So on the ECS task
definition, you had to create a field
called kernel attributes. I'm not sure
I'm just making that up, but it was
something like that.
>> And then there was a field under there
called swappiness that you had to set to
like a certain value to actually take
effect. Now both the AWS API did not
throw an error because it just ignored
any unknown fields and and cloud did not
know what was wrong because it's just
it's just a JSON blob. But if I look at
AWS CDK, they actually exploded this
this this into a construct. They created
a task definition construct and every
possible field um is properly structured
in a TypeScript interface. So if you put
swappiness of the top level, you get a
TypeScript error that doesn't compile.
>> Yeah, but that but but you're assuming
that someone's done the work in the CDK
band. And that's the craziness about the
AWS CDK is that somebody has been doing
the work for 10 years and it's likeund
contributors and it's spread across
>> but that could work against you. They
might not have been well maintained and
you can't set
>> well that's why a framework like that
needs to be backed by the cloud
provider. This only works on AWS CDK
because AWS invests resources into
building this library. And that's why
AWS CDK is much more powerful than
Terraform because it's backed by AWS and
fine-tuned for its cloud.
>> And
that's what they use internally is what
we
>> exactly because when I look at the pull
requests, they're like, I'm at a service
team of AWS and we need this, you know,
this particular EC2 or task definition
field needs to be made available. On the
topic of CDK, um, let me just share my,
uh, thing again. I had a question for
you on operational flexibility
or operational features.
Okay. So, so here's an example. Uh I'm
I'm maintaining u let's just call it an
airflow an AWS ma MWAA airflow instance
at work and it's getting overloaded. I
need to scale it up quickly. So in
terraform
if I was man if I was managing airflow
in terraform I would set the CPU type
from
I don't know T1 to T5 or something M1 to
M5 or something right and then I do then
I do a plan and then I do apply and I
don't know half an hour later the
instance of airflow would have been
updated
in AWS CDK land
Um, again, I don't think it's any
faster, right? You you would still have
to go into your CDK code, change of
change something, and then do a CDK
deploy, and it's probably the the same
amount of time as the Terraform plan and
apply. Right?
What I'm trying to say here is that is
that is there
um what's the quickest way to make a
quick change in of these technologies
besides don't say click ops I mean yeah
I I think click ops is probably the
fastest you know what I mean the speed
to make a change
>> I don't think we captured it very well
>> um
yeah I
It's a trade-off, right, that you you I
it's all or nothing in these frameworks,
right? You either go all in and you make
single source of truth, Terraform, and
there's no way of making a change
outside of Terraform. Um or you
completely ignore that attribute in
terapform and then people can change it
and Terraform will not complain or try
to revert it. You know, you can say life
cycle ignore this attribute so try to
reset it. Um that's where other
frameworks recently try to improve
because they say by nature the cloud is
birectional. It's not one way. You know
forcing your infrastructure as code to
be single direction single sort of truth
in practice doesn't work. It slows teams
down and that's where config that all of
what I just said is basically from a
block um series from Brian Grant, right?
the guy who who then announced uh Config
Hub. So, their idea is that you must be
able to go into your config store and
change a value and have it applied
quickly without going through a massive
um and slow uh process. But that's a
trade-off, right? I mean their point is
that if you do it through our system I
guess there's more tracing and more
control over it than if somebody goes in
click ops at least as the config field
is stored inside the data and you can
have an audit trails there. I don't know
I do feel you get you can you know have
your cloud trail
events siphoned off to just note what
infrastructure changes they are.
>> Yeah. Um so system initiative they also
have this idea of we are a digital twin
and we synchronize whatever you do in
AWS back
>> I need I need to give assistive
system initiative a trial before I meet
up go meet up with
>> they have great ideas I mean the people
behind it have a lot of experience with
chef and puppet and there some of the
you know original creators of some of
the ideas of how chef originally worked
and they carried over after working with
Terafhone for a long time and and they
they they just built a new idea and the
way I understood from his podcast that I
I listen was that system system
initiative is a programmable graph. So
you create nodes and every node in the
graph has hooks like create, read,
update, delete and those notes those
hooks are all programmable. So you can
decide this node
It doesn't. Yeah. This node if it gets
created, this is how the function runs.
And this node if it needs to be updated,
this is what function it runs. And you
can therefore build your own nodes. It's
kind of like terapform provider
resources, right? But all split out into
individual node because a terapform
provider resource has create, read,
update, delete hooks, right? And the
telephone provider just looks at the
graph, determines that note needs to be
updated and then calls the update hook
for that resource and gives it the um
you know the actual values and and the
the new values and then that resource as
part of the provider will apply those
changes by making the correct AWS API
calls. Right? So they they kind of take
this terapform provider but they split
it out into individual programmable
nodes and then they they reimplemented
each of those nodes. What I don't like
as far as I understand is that under the
hood every single function of those
nodes is like using AWS CLI commands. So
it doesn't like it
>> and and
I don't know because honestly if you I
think they spent four years rebuilding
Terraform provider AWS by defining every
resource type as a node and giving it
create read update scripts you've only
done the very basic foundation because
what really makes infrastructure as code
powerful is like what I just told you 10
years of AWS CDK and hundreds of
contributors splitting out every
possible field and building integration
pattern.
Right?
Imagine you spend four years
re-implementing Terraform providers and
offering it as a solution and you
haven't even gotten to the level of like
now I have a lambda and a gateway and I
need to integrate them and that means I
need to call all of these resources and
I need to call all of these hooks in
sequence. You're you've you
>> Yeah. But the
but in a in an in a in a in an ideal
world the AWS CLI should be exposing
these things should be exposing even
these high level obstructions in some
ways the AWS CLI
um I mean I'm just playing devil's
advocate here
>> the CLI should be should be actually the
interface
should be enough is what I'm trying to
say.
>> Yeah. So, I haven't played enough with
system initiative, but I can imagine
that based on what he says, if they
started from scratch, it would take an
immense amount of work to reach anything
like AWS CDK level of UX
>> or DevX developer experience. And sure
you can show a demo which he says we you
know we have a demo where you give us
your AWS account and we point system
initiative and it at it and it will very
quickly create your AWS account
resources as a digital twin in our graph
our programmable graph and you can
immediately start modifying them through
our interface with our digital twin
technology but that doesn't talk about I
am a devop developer and I need to
deploy nextgs
And I don't care how to set up API
gateway and every single lambda so that
nextgs has a server side and can execute
right because that is what a developer
wants to do. This is the actual
functionality like I started Terraform
sorry I started my career you know
understanding how the control plane
works how reconciliation loops works but
if you work for an organization
nobody cares right we need to deploy the
feature and what is the best
architecture for us to deploy it I don't
care how kubernetes you know atcd needs
to be set up and backup procedures and
all that you want to write code
>> yeah I I that is the business value
unless you're you're a hosted kubernetes
this cluster provider as a DevOps
engineer you're coming in and only
knowing how to set up clusters you don't
bring any value unless you can do these
high level integration patterns and
after you know it was very interesting
for me but after doing it for five or
six years I wanted to actually be
valuable [laughter] you know actually
focus on these hard
level integration patterns as you'll as
you
are only really possible with CDK right
Mhm.
>> Yeah. So I CDK is great but there
obviously I think other approaches like
Pulumi they I was very surprised because
CDK has this as as I just explained it
has like these amazing deep level
integrations within the AWS services. So
if you take a TypeScript object uh for
that represents an API gateway, it gives
you a method to to like bind it to a
lambda and it knows exactly what to do
under the hood. Um Palumi is kind of
like starting from scratch and then
build little like helper methods on top.
For example, the bucket object that you
get within Palumi has the ID of like an
onevent
um hook and and you can put the function
handler in that onevent.
Uh it's on the bucket object. You should
search for bucket.onvent.
And apparently that's not in their like
L1s but also in in their like higher
level on event. Yeah, I see it's in Pumi
crosswalks. So, so AWS CDK is like the
the L2, right? And Crosswalks is like
their version of L2. It's like you take
the low-level bucket API object and then
you add ideas on top of like do onevent
and and that's a really neat little
thing. But again it required somebody at
pulley to implement the idea of a bucket
a cloudatch rule an SNS topic have this
idea of invoking a function and does
they rep they they created a repeatable
pattern that on event you pack in you
pass in the function handler and it will
automatically create the bucket create
the lambda create the bucket
notification that calls the lambda set
up the lambda permissions everything for
you right and then the only event is
like this this um this pattern that they
expose it's an interface because it's a
programming language. So you can
reimplement it. If you have something of
your own that you want to implement the
on event then you can implement exactly
what are the grant permissions that need
to be added to the policy. So you can
implement your own on event. So that's
the the the benefit of using a
programming language. You can create an
interface. You can define a contract.
You can't do that with terapform
modules. Can you try to think about
doing that with terapform modules?
Well, and I'm and going back to your IM
example
to achieve these these priv privileges
without some sort of dynamism is going
to be hard.
>> Yeah, that's exactly because of you have
this like synthesis time, lazy
evaluation, late binding, setting
properties after the program has
executed.
>> But I can't think this cuts both ways. I
mean
[sighs and gasps]
there surely there's an argu there's an
argument
>> policy that you want to to attach
>> there's there's an argument to be made
is like as soon as someone starts
writing a whole bunch of code with with
like onvent uh logic it's going to get
pretty probably quite difficult to
follow right the beauty of opening up a
lot of terraform projects is that you
kind of know what's what's happening and
it's kind of it's constrained by by the
by the language right if if like just
like any code in the world if people can
do things they will and it will just get
complex without any
>> uh yes without any uh abandon right
>> I just got the question to the today
from someone he says like do you think
that it makes sense to target an
intermediate DSL and and when you're
saying what you're saying just now it's
Like with the terra constructs, if you
do this complicated coupling uh of of
creating a lambda in an API gateway
ultimately when you run synth you get a
flat config file which is the DSL that
is terraform and you can run your
whatever um you know llinters and
problems but it's probably going to be
fishly complex but yeah okay
>> yeah because the cloud is complex dude
>> this was I Um the trade-off is you're
getting
capabilities,
elasticity
and uptime for services to move fast in
terms of a business value and does it
end up being complex? What's the
alternative? You're going to spin up an
EC2 instance and build your own and then
you know yeah on it there's different
way there's different ways of looking at
it like
you know, the more I need to give system
initiative a bit of a whirl to be
honest,
>> but I'm I'm start I'm but like I'm I'm
my I'm working backwards from AI
and I feel with AI it's about uh I mean
it's also it's also about the same thing
for humans really it's about iteration
speed and uh
>> and a to and a and a tool full chain
like I'm just thinking aloud that like
maybe calling AWS CLI from from Claude
uh you know something where system
initiative does behind the behind the
scenes is perhaps not a terrible way of
uh bootstrapping an account right to
what it needs to do for for for
a you know a typical a typical
organization and let's take a step back
what you just said about iteration and
achieving results quickly. Um, it's kind
of like how how I was understanding what
you're saying. I think this is also
related to people like originally people
were like, "Oh, Kubernetes is going to
give me this full cloud independence.
I'm going to be able to do multicloud,
you know, as a resilience strategy and
and and everything, right?" Very quickly
that breaks down because the advantage
of being in a cloud is to maximize its
capabilities. Like if you are an AI team
and you need to set up like this complex
um data ETL pipeline, but you need to be
completely cloud independent and you
need to do everything yourself versus oh
I'm going to use some step functions or
durable lambdas now and I'm going to
leverage all of the serverless
capabilities and I've got something
working you know because you have like
DBOS you just define your your
programming and then under the hood
whenever a function is called, it can be
suspended and it can be persisted and I
don't need to worry about exactly how
all of that works. I can just focus pure
on my business flow and it's a neat
clear data pipeline and under the hood
it's it's automatically creating lambdas
and they are being suspended when
they're waiting for events and they're
resumable and it's taking care of of of
um making sure that the event is
propagated and no notifications are
being missed and and all of this like
fall torren and and and re repeat and
that that letter Q setup and all of this
complexity gets handled by a cloud layer
and that gives you the ability to move
forward and to deliver a feature that's
between in in an organization
that's between being the market layer or
being leader of being out of out of the
market and closing down.
>> I worked in many organizations.
>> Well, okay. Just to replay my
understanding what you just said is that
like if you don't uh
get fine if if you don't
get
>> realize
>> get down in the weeds then you're not
really innovating type is what I is what
I heard.
>> It's more about if you what is what
business are you delivering? What is
your core business right? your core
business is not creating a perfect ECS
cluster and and taking care exactly of
like setting up um pipeline that that
does exactly one delivery of all the
messages and and does does all of this
core like engineering most organizations
just need to deliver business value or
they're or they they die
>> they're out of business. Yeah. And then
you
>> I think you made the argument that like
you can't do this in the CLI. You need
to you need to get down into the the
details. I mean sorry if I My link back
to to what you were saying earlier about
um
my link back was basically
maximizing highle services of cloud
without
um
worrying about it not being reproducible
or um yeah I don't know I I I had a
point
>> okay we we I sorry I I my kids uh need
to be taken to school so I ran out of I
sound like I have a point just now. I
need to rewatch this to remember what my
point is.
>> Okay,
>> I'll put it in the comments.
>> We'll we'll get to the point. Thanks for
watching if you're watching this. Bye.