Episode 3: Infrastructure as Code & Platform teams

to uh the value add on Terraform.
As by means of introduction, Vincent and
I are seasoned infrastructure engineers
and the following is a conversation
where we talk about Terraform CDK
infrastructurees code and I think it's
the best conversation I've ever had
about platform teams. It's a bit of a
slow burn. I hope you enjoy it. like the
video. Um, and I recommend listening
this to this in in a car. You don't need
to look at the screen. I I did capture
an angle of me from the side, which I
think is a bit unflattering. Uh, so you
don't need to watch this. We we do try
share the the screen and browse to the
things that we talk about, but
essentially I feel this is a podcast
format. You don't need to watch the
video, but the video helps. Do please
enjoy and comment below.
>> See you.
>> Why did I rebuild it? I probably didn't
even need to rebuild it. Okay, so the
value on top of Terapform. Um last time
we talked about a lot of drama around
the cloud providers and the way that you
manage Terraform state and the resource
under um under management pricing that's
exuberant and then some drama around
some open source projects and I'm very
excited about them because I have my own
open source project which is focused on
strongly typed terraform right um
originally terraform
>> I mean just to interrupt you I mean like
terraform is pretty typed I mean you
have your strings, you have your default
values, you have your
>> strings,
>> strings for enum values.
>> Uh, okay. You have strings, strings,
strings. No, you have um I'm sure you
have integers. I'm I'm sure.
>> No, no, but the problem is
>> you have reg x you have uh you have
stuff, right?
>> Yeah. So, Terraform basically
is built on the concept of plugins or
providers for a for clouds and cloud or
for pro for SAS or whatever services
cloud services that give you an API
where you can create resources such as
AWS creates gives you an EC2 API where
you can create an EC2 uh resource,
right? And that was very good and well
um you know 10 years ago where you just
needed to create a VPC couple of subnets
an instance or an autoscaling group and
the number of knobs and dials that you
could configure were not that much but
then you start to having like life cycle
hoots on autoscaling groups that works
with lambdas and you have SNS topics and
you have a lot of different services and
different integration patterns between
them right the cloud became way more
complex uh a lot more serverless is now
being part of a lot of the tech stack
And Terraform is originally just a
config file and a lot of programmatic
features were completely omitted. So
they tried to keep it dumb. They tried
to keep it as simple as possible to
avoid people shooting themselves in the
foot. Most of the time when you talk to
people about using actual programming
actual programming language for for um
cloud configuration, they say like that
is a mistake. Like um what are you doing
that requires a programming language?
you should stick to the the proven path
>> like a more declarative style of uh
>> yeah declarative way of of doing things.
But then they introduce for each. Then
they introduce list comprehensions. Then
they introduce local blocks. And then
you look at the Terraform AWS community
module that has like 45 different
variables for input and and like five of
them need to be set in a specific way to
that trickles down through different
local blocks with different tertiary
conditionals that then need to value
like that go down ultimately in one
resource configuration and then you
don't even know how those variables
interact. Speaking speaking of
complexity, I mean I always thought
serverless was a godsend because it's
going to uh simplify
uh deployment of um of uh of application
code, simplify the you know the life
cycle, simplify
uh everything to do with with that sort
of stuff. For some reason um in
Terraform at least provisioning a lambda
is just a nightmarish uh I wonder why
like why is that? How come Terraform
never managed to make lambda versioning
simple
>> because Terraform's
provider is built on the bare minimum
resources to cover the API. So you go
into very granular level to you know to
to set up a lambda you must configure
the lambda permission resource you must
configure the IM role you must couple it
and bind it to the lambda
>> so there was no good uh there was no
good abstraction or something um
>> so the the abstraction in terraform is a
terapform module which is very limited
in the way that you can configure it. It
provides a fat set of variable inputs
and it provides some outputs and there's
no guarantee around the type system of
the output at all. Like the outputs are
any they can be maps, they can be lists,
they can be strings, they change if you
depending on the internal of the module.
If you decide to put a count inside the
resource and then you use those outputs
uh or those attributes in an output,
then suddenly those outputs become a
list and you know for some reason
because under the hood something
changed, right? Yeah, that's inherent to
how terrible modules are developed.
>> Actually, I mean my usage of of of
serverless is generally quite
>> what what do you call it? uh
standardized
>> you know I usually just run a go binary
but I guess that's the trouble with
serverless is that you can do you can
you can do a lot with serverless you can
integrate it in lots of different ways
right
>> well so the great thing about serverless
is that you you're able to configure
very expressive business logic across
cloud resources but that's also
immediately why it becomes so complex
right because now your business logic is
spread across your serverless components
and if you work with something like
Terraform, those components are very
low-level resources that all need to be
configured exactly right for your actual
business logic to work.
>> I mean I mean
>> think about like a simple lambda
chaining setup where you have lambda.
>> Yeah.
>> Yeah. I I want to definitely get away
from that because my my usage of of of
serverless and is to is to essentially
run my monolithic application.
Like for me serverless is like a way
better uh ECS
But of course that doesn't work. I guess
that is the crux of the problem with
with lambda because yeah you can get
chaining, you can get uh distributed
systems, you can get microservices,
you can get all sorts of things with
lambda and then of course the complexity
explodes.
>> That's right. And the complexity
explodes not only just because of how
your business logic gets like trickled
down across all of your cloud resources
um but also because of how these cloud
resources are spread across different
services of the cloud provider. For
example in AWS you have a cloudatch
service you have a uh where you define
your logging groups and then your
permissions of your lambda must be
configured to write to those logging
groups. So that's an integration between
two different services. you have your S3
bucket where you have to have your
lambda notification like settings done
or you use event bridge where you have
to have like event bridge rules that
then need to be capturing the right
event fields that you want to propagate
towards the lambdas. So the main problem
is the integration across the different
services and you know how to bind them
together and that's where AWS does like
solves it really well
>> but lambda and cloudatch should be a uh
what do you call it a fixed pattern or
something like that
>> it changes a lot and and and it's funny
because sometimes
it could be very simple like you have a
situation where you want a lambda to be
triggered on an event bridge rule and
you can simply go to the lambda and say
async config and then say on success and
then it will you know trigger another
lambda or something else right and if
you do a very simple like lambda
chaining and you can do that directly on
the lambda resource it works until you
have a use case and that's a very fixed
pattern but then you have a use case
where now when the lambda success
happens you want to extract some field
from the um success event like you need
to get the actual um I don't know result
field from the ext JSON of that success
event and you want to get the extract
the payload like extract a certain
payload then suddenly you can't do that
with async evoke that's not possible
with the way that the the lambda API
works and you must use event bridge
rules so you're very use like very
simple pattern of like integrating two
things together which should always be
the same thing no longer works because
now you have an event bridge rules that
like match messes up your whole like
situation because you need to parse the
payload of the event and get out one
field. And that's an example that I
learned because I wrote a library to to
implement um these integration patterns
from AWS CDK on top of Terraform. And so
I saw that like this is very this is an
integration test actually that they have
on their Lambda function integration. Um
>> okay. Okay.
>> Yeah. So
>> I can show you the the actual running
code for that.
>> Okay. So so you're just showing some
Lambda REST API. That's actually quite
interesting because uh we we we use that
a lot at work.
>> Yeah. So so that's where the reality of
the cloud today is that people are using
these type of patterns more and more and
and that's also because you want to you
know leverage the cloud to accelerate
your business feature delivery. So you
can't be like completely decoupled from
the cloud. You start to be really
ingrained and integrated with the cloud
features. Even though a lot of original
cloud adoption was like we want to be
cloud agnostic, Kubernetes is great. The
reality is even if you use Kubernetes,
you're going to have a whole bunch of
custom resources just for AWS um to use
to to to integrate with the cloud that
you run. at at work we we love to use
the lambda rest API because because it
integrates with IM and then IM is the
way that we uh uh mediate govern uh
different APIs between different uh
teams and stuff and a lot of the things
in AWS is like the benefits is that
tight integration between services right
because like you just said IM is is a is
a really nice way to control permissions
across all of across all of for
resources. Um, and so yeah, but I I
pulled this up because this is kind of a
nice example and this is a great example
from originally when they released the
AWS CDK workshop um in how it simplifies
like try to imagine how you would do
this with terapform modules. You're
creating a function,
>> right? And and you're specifying here is
something very special already which is
the from asset. So this is actually
going to
>> this is a whole asset pipeline built in
right in there. It's going to take care
of bundling the code, you know, putting
it in S3, making sure that the lambda
can pick it up or you can, you know,
directly put the code in line. That's
something that you cannot imagine how to
build this with Terraform modules,
right? I mean, I know that there's an
attempt serverless.tf
to do this, but I mean, it's just it's
just a mistake and I'm sorry because I
know the people behind this uh and I try
to to to explain that, but yeah, they
have, you know, this their projects at
PET. So uh so anyway yeah so this whole
this one line already is an asset
pipeline and then this one right here
the lambda rest API is a special rest
API so this is object-oriented
programming and it's it's like an
extension it's just not a rest API it's
a lambda rest API that understands the
concept of a handler that is backed by a
function so this handler must implement
the function interface and then it knows
how to bind and automatically create the
the API gateway integration and back end
lender call right
>> yeah So, so, so just these lines
already. Um,
>> and you get all the LSP, you know, what
do you call it? Checking to make sure
that everything's there.
>> So, because this is an actual
programming language with enums and and
TypeScript is specifically built to add
typing on top of a non-type language
like JavaScript. So, it has a lot of
functionality to to inspect the type
schema and to to really be strong around
your typing. So you do get like indeed
like the from asset then it will the
handler with the I function and it's
strongly typed right.
>> Okay. So I I'm I think I know where
you're going with this. So you're not
talking just about primitive typing.
You're talking about like fun function
typing and such and so forth. I'm not
sure if that's the right terminology.
>> I'm also not sure about the terminology.
But what I'm trying to say is that if
you have the telephone provider you're
going to have a string. You say it's
very strongly typed, but yeah,
>> string has only like three possible
values. And when I'm building my
library, I'm actually have to go to the
EC2 or the Alumba um you know API docs
to figure out what are the valid values
because that's nowhere in Terraform,
>> right? And and they they they can't
really because they they need to cover
so many AWS services. It's manually
written as well. And I'm still relying
on the Terapform provider AWS except I'm
building the CDK library on top of it.
So, so this is the example from AWS CDK,
but now it's running on top of
terraform, right? So, I have the same
things. I have the code, the lambda
function, the lambda rest API. Um, for
some reason, I didn't call it function.
I called it lambda function. And that's
because I put everything under under one
name space. I put it all under AWS
compute, which maybe was a mistake, but
okay. Uh, and then I have the AWS asset
like the stack, the AWS stack, which is
a um extension of the classic Terraform
stack. This is a stack that understands
about AWS asset pipelines. So it knows
about ECR. It knows about S3. So it
knows that if you use and that's
necessary because when you use AWS CDK
it's built on top of AWS. It's built on
top of cloud form. So it has the ID of
asset pipeline and things like that. So
if I want to do this code from asset
here, I actually am building if I
synthesize this
>> I I actually have the S3 bucket. If I
don't pass in the bucket to the the
stack, it will create a bucket for me
and it will automatically upload my uh
lambda source code bundled up and put it
straight into my S3 bucket. Now some
people when you are doing terraform
they'd be like no your your code
delivery must be decoupled right on one
side terraform stand up the the skeleton
and on the other side you have your
actual code that needs to be built and
and deployed. But when you're in the
serverless world the resources and your
actual business logic are very tightly
coupled. That's so true. That's so true.
So I think the next question in my brain
was that okay you create a bucket for
the for the sourcing your your
application code but like what happens
when you destroy that with the bucket go
>> uh or yes or do
>> so this is your terraform under the hood
right um there is yeah the the terraform
call because the bucket is the object is
also managed by terraform so terapform
will first delete the object then if the
buck then it will delete the bucket AWS
will prevent that
>> if there's still some some objects
inside then it will not allow it.
>> Okay. So, so you're saying that
Terraform will completely handle all the
objects inside there I suppose.
>> Yeah. So, so this is purely translating
the AWS CDK on a deep deep level
directly to Terraform resources and not
just any resources but the actual
Terraform provider AWS resources. That
means that all of your existing like
terraform linting um you know code code
scanning everything still works. Cool.
And then and then another question
because uh C CDK I believe has this
bootstrap uh functionality where it does
create its own bucket for the stack up
>> right.
>> Yeah. So I don't I don't do that. Um no
my AWS stack has like a it's called like
a singleton pattern. It will create an
S3 bucket if it's necessary. You can
pass it in directly to the AWS stack as
u stack as well. So when you invoke this
like my CDK workshops tag when I call
the constructor if I in the constructor
properties pass in my own bucket then it
will take that one right
>> this is again
>> the power of actual you know I mean a
lot of people hate on object-oriented
programming but the ability to define
like to use the basic object of a
terapform stack and then extend it with
functionality just reuse the constructor
and have this capabilities built in
makes it so much nicer like you can't
really do that with terapform modules or
like bundling terapform models on top of
each other. Right.
>> Right. Right. But like Okay. So if you
had the option to use the CDK bootstrap,
you didn't that's obviously just not a
default here like by default.
>> Yeah. No, the CDK bootstrap does a lot.
And the actual asset pipeline from AWS
CDK is built into the CDK CLI. So it's
not a cloud for thing. The AWS CDK CLI
has the idea of uh bundling code and um
putting it into a staging directory and
then preparing a manifest of what is
inside the staging directory that needs
to go into S3, what needs to go into
ECR. So that's what the CLI does.
Terraform kind of like worked around
that by providing special provider
functions. So it doesn't like Terraform
doesn't understand or doesn't care about
AWS asset pipeline, right? But you do
have the ability to create like use a
terapform provider to create an S3
bucket and upload an S3 object. So I'm
kind of offloading that asset pipeline
to native Terapform provider
capabilities which I'm not sure if it's
the best way but it does allow me to
like hand off the life cycle completely
to Terraform
>> the way that it does.
>> Yeah.
>> Okay.
>> Those are all conscious decisions as
well.
>> There's a there's a lot of details in
here too. So so say a company's using
CDK. Um,
>> but let's be honest, I think we talked
about this before. CDK got there um is
great, but it it's only 80% of the way
there. Like it and and correct me if I'm
wrong, like this the support from CDK on
the AWS side seems a bit weak right at
the moment or do you think they'll come
back into it or like what?
>> So that's interesting to hear. Um
>> I'm trying to I'm trying to like map the
future of CDK
>> for me is a pivotal tool for or
developer tooling for AWS to hear that
they I know that they've withdrawn some
of the stuff um but they are still
actively developing it like they they
added the C the CDK refactoring for
refactoring stacks across was integrated
directly I mean the cloud formation
refactoring feature was was integrated
completely into the CDK. You can
refactor stacks now through the CDK CLI.
They added an API or like a programmatic
way to call the AWS CDK as well. So they
are developing and adding more features.
>> Okay. So what does what is sorry what
does refactoring actually mean in in the
in CDK?
>> Yeah. So in the in the world of
Terraform or in the world of
infrastructure as code, the moment you
that you start treating your AWS
resources as code, um you must be able
to do things that are you know very
common with code. And I can give you an
example for you may start with one
project and you create a network and you
create a database and you create a
cluster or you create something like
that and then you deploy your web
application. You're running a whole
Kubernetes cluster for your blog, right?
I'm just kidding. H but then you decide
that you know you need to start a second
project and you don't want project two
to completely depend on all the all of
the resources that belong to project
one, right? because one day maybe you
want to deprecate project one and you
want to um you know get rid of it and
then project two depends on half of the
resources. So at that point you must
move those shared resources maybe to a
shared layer or a shared ownership. So
maybe there becomes a platform team to
maintain the the network and the
kubernetes cluster and then you have the
different product teams that deploy on
top of it and integrate within it. So
you asked me what re what refactoring
was, right? And I just said that I think
refactoring is a very central part of
infrastructure.
>> Okay, I I got that. So okay, I for some
reason I had I think you we were talking
about it a couple of weeks ago that like
you know the CDK the original AWS CDK
team um I can't remember the the name of
the guy who who Israel. Yeah.
>> Yeah. He went on to to do wing lang but
that didn't that didn't fly
>> reasons about it. A lot of the comments
I saw was it's too early. Um people are
not ready for this.
>> I I never really got into it. Um um
though, yeah. So in my mind, I thought
to myself, okay, so CDK doesn't have its
uh you know, BNFL.
Is that the right word? I don't know. It
doesn't have its uh have a direction
now. Uh so um and
>> I know that a lot of the original core
members have moved on but there's still
a very large community and I think most
of the projects I know that AWS uses it
heavily internally like half the time
when I look at like progen or
bootstrapping or for example the PDK um
kit
>> okay so they're using it um
>> when I look at the features and the
issues the GitHub issues I see people I
work at this service team at AWS and we
have this problem and this is how this
would help.
>> AWS project to make it. What is that?
>> Oh, this is an a really cool um
what I like about this is that it
includes like highle graph generation
and which is another thing that I I
believe is something that Terraform is
horrible at and also because of the way
that Terraform modules work.
>> You can generate diagrams from CDK.
>> Yeah. So there is a project called CDK
DIA and and the thing is because you
know how these objects these constructs
they they they contain like a lot of
highle uh like intent and and meaning
right so they actually capture what they
represent. So you can pres represent
those in into logos nicely in here.
>> Wow I can decorate your constructs.
>> I need this at work. Wow. uh but but
>> so there's an I inspectable interface
that you need you can implement to to
basically say how you want your
construct to be shown on the diagram
which you can't even imagine to do
something like that with terraform
>> this is fascinating so so so basically
this PDK is is some
>> not just that it it has thread like
plug-in thread composer it's integrated
with cloudscape react ts website so you
can you know build a their design system
setup up cloud cloudfront um you know do
your CDN um
>> what the hell does cloudscape react
supposed to mean
>> it's a system design um sorry not system
design it's a design system from AWS
that's how they build their dashboards
and their components
>> oh interesting
>> as far as I understood cloudscape
>> okay so okay so CDK still has a lot of
buying because I mean sorry like
>> I'm telling you it's a whole world and
and
>> because I I I mean yeah to be honest at
work clients are for example this client
I'm working with
>> has been using CDK and to be honest
>> it's been fine I mean I I can get behind
it there's some quality of things that I
hate about CDK like it doesn't seem to
be able to plan
>> so that's why Terra constructs would be
great
>> yeah uh so um but besides that um one of
the one of the technologies that they
bought bought into is a is a database
and the the only official uh
>> provider is
>> provider is terraform. So they're
basically
>> those are all cloud form limitations.
>> So basically they're stuck because they
have to they have to start using
terraform. So at the moment their
pipeline like looks like you know CDK
deploy and then do your Terraform um
applying cool beans but but of course
the question in my mind is like okay we
can't just keep on using CDK and keep uh
and then and then use Terraform as a
baseline. We have to think about like
>> aligning things.
>> Yeah, aligning things. So, so I and
yeah, we talked about it before, I'm
sure, but like I think Terraform is the
the de facto winner of the day when it
comes to IA probably for the next god
knows how many years
>> mainly due to adoption.
>> Yeah. Um
>> yeah so
>> I say that and yeah so so where does CDK
fit in with this the fact that Terraform
is is definitely not going away
>> and it's got that sort of mind share uh
you know
>> so at some organizations I worked
>> the expectation was that the team
decides what they want to use if they
want to use CDK they can the platform
team had three years of platform team
stuff so we were bootstrapping VPCs EKS
clusters everything with with Terraform
And so we just had to provide
integration points that is kind of
painful for um because you now have to
deal with like different execution
engines you know cloud formation and so
on. It can work if the team takes full
ownership of their CDK stack. Um but the
moment yeah I've also had experiences
where a team has full ownership on their
stack and then half the team leaves and
one guy is left to maintain it and he
was supposed to like work on really cool
data engineering stuff and now he has to
maintain an airflow cluster and he hates
it. This sounds very accurate like too
many details like where does that come
from?
>> Yeah. Uh I know what you mean with
airflow. I mean the whole MWA airflow
3.0 I don't know if you know about that.
That's amaz
>> I haven't used it for the last five
years.
>> Okay. Well, there's there's basically no
upgrade path between two and three,
which makes life fun and interesting.
Uh,
>> LLM upgrade path.
>> Yeah. So, LLM upgrade. Um,
yeah. Okay. So, I guess I guess what
you're saying is that CDK and Terraform
can live side by side. Then sorry where
does Ter I thought when when you
introduced to me terror constructs to me
I actually thought in the back of my
mind since because of that problem that
we have at work we have CDK we have
Terraform I thought you were like almost
gifting me the the perfect solution to
my problem uh to align everything to
Terraform
>> right so what terror constructs does at
this point um it's my effort to
translate the AWS CDK developer
experience on top of the core terraform
resources and provider model. Right? So
there are some projects um Hashi Cororp
created Terraform CDK to build the
foundation to automatically create
TypeScript classes that wrap the
original Terraform provider. So you can
configure your Terapform resources using
TypeScript or Golang or Python, right?
But Hashior never built the actual
system like service integration
patterns. it never built the IM um the
idea of of you know connecting IM. For
example, in this case when I when I bind
the lambda the handler on top of the
REST API, it automatically creates the
service role and gives the Lambda
service principle the ability to assume
that role like under the hood and it
also
>> creates
>> uh hold on. It also creates the Lambda
permission for the gateway. Yeah,
>> for the actual execute API gateway to
call that lambda and so on. So all of
that is is is built on top of the
service integration pattern between
lambda lambda API gateway and IM. So so
that those are patterns that exist in
AWS CDK.
>> So Terraform CDK never did
>> did that. But how but that that that to
me you're saying that you can't imple
you can't deploy a lambda with terraform
CDK then it's going to work
>> you you just have to write your your
very very nice and and small um Hashi
Cororp language config file now you have
to write a complex TypeScript program to
generate that config file which of
course a lot of people will say that why
would I use Terapform CDK. The idea is
that now that you have that TypeScript
object, you can build your own
abstractions and you can build your own
higher level constructs around it. But
there's no interest for Hashi Corp to
build that, right? They're not af Okay.
So, so they just got you bootstrapped
bootstrapped.
>> Yes, they got you bootstrapped.
>> So, surely the whole community must have
cropped up.
>> This there's a few like
attempts. So one of them is uh CDKTF
plus which was from the original people
behind CDKTF. So they tried to build
like a level two um library of of or
layer 2 construct library which is like
3 years ago but this is pretty much
archived now. Um and they also built
something else which is called the AWS
CDK adapter where they automatically
they basically let AWS CDK do its job to
generate cloud for resources and then
map those cloudformation resources on
top of terraform provider resources.
Unfortunately that doesn't capture half
of the of the functionality that is
hidden between these objects interacting
with each other. like the way that you
generate resources on for cloud form is
different from the way that you generate
them for um for terform providers and
they try to use a common generated
autogenerated provider through the AWS
cloud control API. So the cloud control
API is an attempt to make available the
the engine behind cloud form. So you can
actually autogenerate the terform
provider from the cloud controller API
but it's like it's gong ga I think this
year uh or maybe was it last year I'm
losing track of the years been working
on this for a year.
>> Oh what do you mean gongji? What did
that word made?
>> So it was under beta for like two or
three years and then it went generally
available. So the project to the AWSCC
provider
>> because yeah because yeah like now that
you mention it cloud form to terraform
makes sense here because ultimately CDK
is producing cloudformation.
>> Yes. So they have an adapter but again
it doesn't capture some of these
patterns. So if you align the providers
then you should get the same
functionality but there's still a whole
bunch of like resources and patterns
that are not supported by this Terraform
provider. this autogenerated provider
and to use it like nobody's really using
it because when you look at these
resources you basically get like no the
good thing about Terraform is that when
you talk to an API object you can expand
a JSON schema into like very strongly
type like an expanded um schema in
terapform language right but in this
case the schema is not expanded like
let's go to the documentation and look
at some of these resources maybe take um
let's look at the AP API gateway
>> uh or maybe something simpler SNS or
something I don't
>> no no let's look at the API gateway
because that is a real resour
>> we talking
>> so here we we we have the um you have
the open API spec that you pass in
>> and then you have the deployment the
stage but what's missing is the
integration right like if you hit here
it's just doing a response
um but it's like it's an X X Amazon.
Anyway, this is this is not strongly
typed. This is just JSON. You end up
just shoving JSON all over the place.
>> Yeah. So, so this is the Wait a minute.
I
>> understand it though.
>> This project takes cloudformation and
generates it those resources. Is that's
what you're saying, right?
>> No, this is the cloud control generated
Terraform provider. So, there's two
providers for AWS. One is manually
written and maintained by people. Every
time AWS releases a new feature, adds a
new field, adds a new whatever to any
service, somebody gets, you know, a pull
request or somebody issue is created and
then it's added in
>> um manually and manually mapped. So a
lot of like decisions have been made to
do things a certain way which is like
completely different from the way cloud
form does it right.
>> Interesting. Okay.
>> Yeah. So the terraform provide AWS is
written by humans. That also means it's
kind of like a created experience
because it kind of tries to make sense
of where it expands things uh or how
does it expose certain configuration
fields. For example, the way that you
define um I think it's like on the again
because I did the API gateway conversion
recently there was uh the way that you
define the method responses on on an API
gateway. The way that the the actual API
endpoint works is with like a JSON patch
like you have one endpoint and you have
to like JSON patch in the differences
and in the provider they made a
different resource for each one of them
and when under the hood the provider
will build the JSON patch and send it to
that one endpoint to achieve what you're
trying to do. So this is the type of
stuff that they can do because of the
API endpoints that API gateway exposes
is very complex for a human to you know
build a JSON patch. uh and then cloud
for does a similar thing. It kind of
like like abstracts away this complexity
of that the way that API is built and
allows you to define it um in in one
resource. But in terapform provider AWS,
they made like many different resources
and the Golang code takes care of it
like building a nice little patch under
the hood. So the way that you write your
config is really nice. Um so that's the
advantage of a manual written uh
provider right compared to like an AWSCC
autogenerated provider. Um, but that
also means that most of the tooling like
uh code scanning and everything is
around this provider because everyone is
the one using it. Like all of the
Terraform linting, all of the Terraform
security scanning, all of the Terapform
cost calculation tooling is around this
provider because everyone's using this
provider.
>> Yeah, I mean that's top provider by like
a way a long way I'm sure.
>> Yes. I'm not sure about like I know some
people sometimes say like I we're using
AWS CCC because the feature has never
has not been built on top of the
manually maintained one and the CC one
has a chance to adopt features faster
because it's autogenerated.
>> And do you know how AWSCC works? I mean
they they must have a pipeline that
kicks off with uh
>> yeah they have a they have a lightweight
prox API to discover, provision and
manage cloud resources. So they they
autogenerate it. Um I guess it
>> the source of truth for um AWS if I
remember is like like a Mason schema or
something like that that describes the
APIs.
>> Yeah, there's that's part of the of the
PDK here as well. They they have the
type save API. Smitty smithy
>> Smitty.
>> Why did I say Mason?
>> Smith because Mason Smith's same thing.
>> Ah,
it must be Yes, it must be. Yes, you're
an LLM after all.
>> Cuz like 10 years ago, I was working on
uh in a video startup in Singapore. I'm
sure I don't know if you remember.
That's probably when we met. And
interestingly,
um the the service hadn't launched and
and AWS gave us like this JSON file that
was generated with Mason if I remember
correctly. I don't know why Mason, it's
probably the wrong word, but they gave
us a JSON file and then we just copied
it under the AWS CLI directory and then
all of a sudden we had all these
commands available to us, which I
thought was really cool.
>> Oh, I haven't heard about that. That's
sounds really cool. Um, but yeah, as far
as I know, Smithy is the the language
they use for API specifications or
definition.
>> Yeah. So, so that's a cool thing about
the like the AWS de project development
kit again because they have this like
REST API generation from Smithy. Um, and
this is how you build your your product.
Um, so it's kind of like uh build on top
of the AWS CDK to do that. But yeah, the
whole thing I was saying is that you
have different providers. Uh, one of
them is very popular but manually
maintained. The other one is
autogenerated but doesn't have the full
coverage. And I hear from people that
there's still quite a few errors or
problems that they're facing. But but
that almost doesn't that almost doesn't
make sense. Surely something that's
autogenerated has full coverage.
>> No, it didn't have full No, because the
cloud control API doesn't have full
coverage.
>> The thing that backs this Yes, it's the
control. If you go to the website for
the AWS control uh cloud control API, it
will tell you how much of their actual
services they do cover on this.
>> But that but that's I mean didn't Jeff
Bezel
>> because every team developed it's like a
two-piece pizza team in AWS, right? They
build their own services. Nobody did a
centralized cloud control API, so they
started building a centralized cloud
control API.
>> To me, it kind of makes sense.
>> Well, wait a minute. Like the the folk
law goes that Jeff Bezos mandated that
every service team uh publishes their
API. It sounds like they don't if if if
they can't if it can't be syndicated by
the cloud control API.
>> Let's look at a year old LLM summary.
Uh
ah see see for example the the
certificate uh doesn't support update.
Maybe that's simply because you can't do
that. But here's a little table for you
of the coverage. Maybe this is also
maybe this relates directly to the
coverage. Looks like it has quite a lot.
But where are the services not covered
though? This is anything that is
covered. Um I'm wondering if cl if local
stack is using this because they have a
very similar table. So local stack might
be using this to to generate their um
their local stack.
>> So just to reiterate what I think you're
saying to me, you're saying that the
Terraform uh AWS provider has more
coverage than something that was
autogenerated.
>> When they G8, the AWSCC, they put out a
number on the number of resources
covered. and it looked like it was the
same number, but when you looked into
it, it was actually a bunch of like a
split between some of the resources. Or
maybe I'm getting this wrong because the
Terraform provider AWS has special
resources that don't really have a
backing API. Um, but I I need to look
into it. They do have like that's
definitely something maybe we can add in
later as a clarification on how much is
the coverage actually of the FCC
provider. But there is a difference.
>> How did we even get here? Oh my god. We
talked about ter we talked about ter
constructs. We talked about why
>> we started with spec kit. We started we
started with spec kit and how it like
formalizes or gives you workflow to do
your LLM work like um development. We
talked about some evolution in the cloud
code budget thing. We talked about you
then asked me more about um like more
about to towards Terraform and why I
built something like Terra constructs. I
think so. I gave some examples about the
evolution of the cloud, the fact that it
changed and became so complex and that
the integration patterns between the
services are complex and therefore a
simple config file or terform module
doesn't do it anymore. So that's where
um I also give you examples of why the
AWS CDK is so great. We talked a little
bit about like it not really being
maintained but there's still being a lot
of new features being added. But but
with Terra Constructs, I thought I I saw
it as a way as as as migrating it. Now
that I'm speaking with you at length, it
sounds like you are not thinking like
that. You're thinking that you're going
to still leverage CDK, but your output
is going to be Terraform.
>> No. So, so there isn't an a project from
Hashi Cororp that puts uh a host service
a host um pro object around AWS CDK and
then encapsulates the process like when
AWS CDK generates something it then
generates the Terraform. I'm not doing
that. That's the adapter project and I
don't believe it can achieve the actual
functionality required. Okay. And we
talked and that's why we ended up
talking about AWS CC because it uses
that AWS CC provider that not so many
people use that you know doesn't have
all of this ecosystem tooling around it
and and that doesn't like allow you I I
believe like I talked to some people
behind the project and they told me that
it's really just for early stage
companies like quickly want to you know
prototype how moving towards Terapform
would help them maybe get them halfway
there like you know have existing AWS
CDK programs and put like a crud cr
grudge around it. So, so you understand
what I'm saying? It's called the AWS
adapter and um but it's not like a
long-term solution for um
for your
Yeah, it's called the AWS adapter for
CDK. And as far as I understand, it's
not something like if you start
Greenfield, you should start, oh, I'm
going to write a CDK and I'm going to
use the adapter. I'm going to be using
Terraform. That's not what it's for.
It's meant to be like we have a lot of
CDK stacks, we want to like quickly
convert them over to Terapform and this
can give you like a quick uh bridge
towards Terraform. That's how I
understand it. Okay. So I threw that
away and I started on originally not so
detailed. I started just I want to be
able to programmatically build terapform
configurations and the Terraform CDK
seemed like a good solution to me but
then I realized it's missing a lot of
stuff. So I started building high level
but then I keep going lower and
ultimately ended up reimplementing all
of the core features like asset pipeline
bundling pipeline IM pipeline like the
actual IM modules the the concept of the
you know the interfaces and the patterns
the way it works under the hood and
these these are called level two
constructs right
>> yes okay
>> correct uh asset pipeline is more like
like CLI tooling around it but it's part
of it's part of you could say it's part
of the L2 stack construct because it's
like a higher level. It's using the
low-level stack cloud for stack under
the hood, but it's adding a whole bunch
of functionality around it.
>> And just for my vocabulary, L1 is the
low level. L2 is like the asset
pipeline. Is it is there an L3? No.
So there are some L3 tries which are
created by professional services at AWS
where they have like some um more
advanced cloud patterns maybe for banks
that are building like a subscriber um
like producer type of pattern in the
cloud. So they will use several L2s and
combine them into a cloud pattern so
that it can be used by businesses that
would be an L3 or an L3 might be um
maybe
the real product that that I'm building
at my company um that is actually
deploying the front end deploy deploying
the back end.
>> The whole product that makes sense so
and then and then the other terminology
I'm just mapping into into these L1 L2
L3 is like construct and and pattern. So
a construct would maybe would be level
one and then a pattern would be level
two like how do you or it's not a very
good mapping
>> they they they this they this they this
they this they they apply the L label to
all constructs. So constructs are either
of L1 or L2 or L3. So the L1 constructs
are autogenerated from the b basic
resources that are available in cloud
form or if you're talking about
Terraform the L1 resources are the ones
that are the Terapform provider
resources directly. Okay, that makes
sense cuz I think I've seen I got
confused by that. So, and then where
does the the word pattern fit in here?
It doesn't really fit in.
>> Yeah, I think it's more like when they
talk about L3, there is like the service
team. I know that there is like a
website from AWS service team that says
that these are L3 patterns. These are a
bucket with a in with a with a lambda
invocation triggers for example. That's
a pattern that you may want which is a
very common pattern that you build in
terraform as well right um you have a
module that creates I have a bucket and
then I have a handler for a function
that whenever an object get pushed in
maybe have some type of filtering logic
to invoke the lambda that's like a
module that you would build right um and
in in Terraform you would build that
module from the low-level L1 um you know
lambda resource and and the bucket and
the service permission the IM rule
resource and all that in in AWS CDK you
will build that L3 using the L2
construct which is encapsulating the
lambda and knows how to bind itself uh
like it it it's backed by an I principle
that means that the lambda has a IM role
behind it right so so it it it exposes
itself as a as a interface so it's I am
a principle and a principle you can
grant things right so when you have a
lum when you have a bucket and then you
need to give a permission to the lambda
you are basically you you're using the
grant method method that accept an I
principle. It can the I principle can be
a lambda in this case but it can also be
an EC2 instance. Isn't Isn't the better
terminology would be to call it like a
service principle because it's a it's
it's the service is
>> it's I think it's closely tied to the IM
uh terminology which like in an IM
policy you have the principle the
resource the action
>> I forgot the rest so they they use the
the same idea the I principle the only
new yeah
>> the only new one is the grant
>> but but there can be different types of
principles right you can have a user
principle but you can als you can have a
human all of behind I principle in the I
module. Yeah.
>> Okay. Okay. Let's we're probably just
splitting hands at this point. Okay. So,
so terra constructs
um
so I still actually don't quite
understand. So where is it going to fit
in in a in a company just just where
where people
>> So the way that we use it at my
organization is we have two three years
of Terraform landing zone and so on and
then we had the product teams that
wanted to build with CDKTF uh because
they prefer Typescript they write uh
their lambdas in Typescript. they write
their nextgs front end, back end, BFF,
whatever, and they like TypeScript and
they want a CDKFF and I I had experience
with AWS CDK and I was like, yeah,
that's I I encourage it. I mean, um I I
would rather you not use AWS CDK because
of cloud formation than I have, like you
said, I have to build uh CI/CD pipeline
to do the plan to post the the preview
in my PR. I've got all of that already.
I've got Terraform, I've got Atlantis,
I've got, you know, coordination,
collaboration. So, we need
>> So, basically, how does the pipeline
work? The product has its repo. it
writes its uh their CDK and then you
have a you have like a GitHub action or
something that basically turns their CDK
into Terraform and runs it or do you or
do you like get a snapshot and then you
commit it to your own Terraform uh repo
or something?
>> Yeah. So I just did this presentation at
the um AWS community day in Koala Lumpur
Malaysia.
>> Oh yeah. How was that by the way? Was
that was that at the AWS offices right
or
>> Yeah, it was no it was at the university
uh in in Koala Lumpur. Oh
>> um but I do cover exactly how does the
workflow work between the team. So how
do I
>> Okay, we need to link this link link in
the description.
>> Yeah. So this is an example of an L2. Um
it's actually more like an L3 because
this is like a load balanced Fargate
service. So this uses under the hood it
use you see it uses something called
patterns. So under the hood it actually
set up a load balancer. It set up an ECS
far task definition.
>> So that's that's something that comes
out of AWS CDK right?
>> Yes. There's a separate library of
patterns and an application load
balanced fargate service is a
composition of several of the L2
constructs to to expose this.
>> Okay. Okay. So pattern does exist. Okay.
I wasn't going crazy. Okay.
I think patterns is like the service
team um built L trees. I think
>> I I think when when you talk about L
trees, it's like you can build L trees
and we build a couple of L tries and we
we have we expose them as a pattern
library. Yeah, I also forgot about that
it exist that that's the name of the
library. Yeah. So our workflow at at my
organization with CDKTF is that the
product team lives in their own
repository, right? And we're trying to
reduce the number of repositories. So if
a product team has front end and back
end uh and and an infrastructure, we're
trying to use more of a monor repos now
because we became good good at them. We
use like a many monor repos. So it's
kind of like
>> so the so one product one one repo,
right? That's what you're aiming for.
Okay, that make sense.
>> Yes. Kind of like you know the idea of
of is it Conway's law where you organize
your services based on your team or your
organization layout. So if you have one
team working on one product
>> so so we have the product mono
>> and they are defining their product in
Typescript lambdas whatever tightly
integrated with their infra resources
that they require for their product.
some of the resources like in this case
the the the cluster the VPC they will be
injected when they integrate into the
lending zone right but then but they
must be able to define those and they do
that in a very strongly typed language
TypeScript and they use SEO constructs
originally they use CDKTF but that means
they have to build their own L2s instead
we are building the L2s publicly make
them available on Apache 2 and then we
are using them internally within our
organization okay
>> nice nice so
>> so then yeah
>> okay So
I mean like I'm just thinking of um my
current client
they use CDK
and then they also use Terraform. So So
why what I mean surely didn't didn't you
think about uh allowing the product team
to run their CDK?
I was lucky they didn't ask they didn't
push for it. I did I think about it. I
was I was thinking actually about
talking pointing them towards Palumi SST
because that's kind of a bit of a
similar idea but then we would need to
adopt Palumi because Pulumi is
crosscloud but then I wasn't sure about
like the self-hosted Palumi story and at
the time I didn't know that you could
like self-host Palumi. I only heard that
it was expensive and that you had to pay
for the cloud and so on.
>> But I'm not too sure why you're talking
about Palumian because you can't just
you could have a
>> Yeah, I could have let them use cloud
formation, right? And then it could
publish the cloud.
>> You didn't even you didn't want to go
there or something. What were you
>> No, I have experience with that in the
past. Um we had one team heavily in
cloud formation and then we had the
whole rest of the company data team
everyone else using terraform and I was
put as part of the team that was using
AWS CDK and I was completely convinced
that AWCK was way better. I was I was
put in there to convert them to
Terraform and they converted me to you
know the this the the CDK story and then
I spent a year trying to build um
integration patterns between Terraform
and cloud formation. So you must consume
like integration between two different
products and then you must expose them.
So I was using SSM parameter store as an
integration registry which is Keith
Morris um you know infrastructure as
code.
>> Keith Morris is my line manager. I don't
know if
>> exactly right you told me that. So, so I
followed his advice uh about his
infrastructure as code third edition
whatever uh book and follow the
integration pattern of um integration
registry using SSM parameter store. Now
the
>> tell him I sent him a message on
LinkedIn and he never replied.
>> Okay, I'll I'll uh I'll let Ke know. Ke,
if you're watching respond. Uh okay, so
chapter what 17? You you're oddly
specific here, Vincent, but carry on.
Yeah, because I did a presentation about
it. Um,
I called it integrate this. I did it
like at the time when I had just
finished building the integration
patterns and I thought uh this is great.
Um,
>> and that's the latest book, right?
Because he he had a
>> yes I I I have already uh subscription
so I get access early early access and I
read it. Um so these are the actual
diagrams from his book and he's talking
about organizing your infrastructure and
then the different in integration
patterns. You have one provider, you
have one consumer and then you can do
resource matching which is you look up
the resources based on some tags or you
can do um stack data lookup. So if you
have one stack in cloud form and then it
has some outputs. So you need to read
those cloud form outputs and then um
integrate them within another cloud for
stack for example. But you know need to
know what the team is using. And then
the other one is the integration p uh
registry lookup pattern where everyone
writes to a different key convention and
then you can look up through the
integration registry. Right.
>> Okay. Okay. I mean this is quite common
actually. So Keith basically wrote it
up. I mean I've seen this everywhere.
>> SSM SSM is the registry.
>> Yeah. So when we adopted it we didn't
have strong conventions about the the
format of the keys. What are the like
the different paths that you need to
capture. We didn't have a clear
guideline of how to write your
dependencies out and how to read them
back because if you put some rules
around that then you can you know
determine that this will be like a
network will have a a list of certain
ideas and so you read it out the right
way the right way because now you're in
type languages like CDK and Terraform
maybe you can read it properly.
>> So the key go specify the actual typing.
>> No I don't think that's part of it. uh
it just mentioned that this is a pattern
you can follow. It's just my personal
experience when I I started to use it.
We had problems like that like teams
didn't have a clear way or convention on
how to write to it. Uh another problem
is that this becomes an a layer of
indirection and becomes stale like how
do you you know keep it up to date? How
do you know that the value has changed?
How do you notify also because it's a
layer of interaction how do you have
direct connectivity between provider and
consumer? How do you know everyone that
consumes that key if you don't have like
a an actual like point po point po point
po point po point po point po point po
point po point po point po point pointer
counting like if you do this in memory
you will you keep track of how many um
actual variables point to the stack
right so you have some counter and then
you can know if anyone is still reading
it and so none of that right and so when
you start it sounds very simple and I
was very confident I did this
presentation I say this solves all of
our problems but at that time I already
realized that like having some type of
generics around it so that we and type
when we read it back out is actually
really nice. Um, but that never really
happened, right? Our teams had written a
whole bunch of stuff into SSM parameter
store and then we couldn't really keep
track of like who was consuming it. I
also thought that this was great that
this was a a read indirection layer
because now it meant that I could move
things around and then just update the
registry from different places. Big
mistake because you have no idea where
it comes from. So, so this pattern um is
why I did not like my teams to use cloud
formation and then adopt some type of
indirection layer to communicate with
them, right? So, I actually tried to
roll this out and I had my landing zone
publish all of its like integration
points into the SSM parameter store. Uh
and then you get other problems like how
do you control the consumer permissions
or who can write to these keys? How do
you, you know, do you cross-ost like do
you do you have a central one or do you
have this one provider right to
different accounts where they have to
find it? So, there's a lot of of of
problems there and I decided I'm just
going to get rid of this whole problem
by just using the native built-in
Terapform stack feature which is, you
know, Terraform outputs. Right.
>> Right. I mean, I'm just thinking I'm
just mapping this to my own experience
about a registry. I think the one at my
workplace that we ended up just doing is
is essentially we keep a huge a like a a
source of truth a JSON file in GitHub
and we use that to sort of coordinate a
lot of stuff because like hey you want
to know where this is which account this
goes into what what secret you need.
>> It's horrible isn't it?
>> I I like it. I like
>> it's horrible. I like a single uh large
JSON file. I mean because you know
>> JSON even even worse if it was YAML
you wouldn't have all these like
trailing commas or next line changes
when you know these git tips would look
a little bit better.
>> In my opinion the JSON can is is just an
output uh here but but it's good. It's
it's it's a better paradigm than
sprinkling stuff across an SSM paths for
Christ's sake.
>> Yeah. Yeah. Agree. Um so so very common
like you said is the TFRs in terapform
world where people you know copy paste
TFRS files around for this environment
we need those TFRs from that environment
we need this TFRS it's a bit similar
>> but like but this but this is like even
bigger this is across environments
uh
>> with Terrant you also have the ability
to like define dependencies and it
automatically create the JSON files and
caches them under the hood and then it
keeps track of them to like more quickly
run like consumer provider interactions
and so on. But Teragrant again also very
quickly becomes like sprawls out of
control, right? So, so you're asking me
like why did I not just allow them to
write their infrastructure in AWS CDK
and I I'm telling you that my biggest
problem was the integration patterns.
You're saying one solution could have
been to create more like a central file
so that they would need to read from
there um and things like that. But when
you say central file, it sounds very
similar to SSM parameter store.
>> Yeah, but but SSM parameter store gets
out of hand because you could you could
use arbitrary paths. You can use
>> what about the JSON file? What are the
arbitrary like fields paths that you use
in JSON files?
>> They're just they're just keys and you
know,
>> but what's the structure of the keys and
are they nested keys?
>> Well, you have the same problem.
>> Well, they are they are nested. Yeah. Uh
>> yeah. how
that's where you then get similar
problems.
>> I think I think majority of them are
just are are just keys. They're not like
lists if that makes sense.
>> That's I think one thing I learned about
Terraform because a lot of people when
they write a module they go like well
this resource that I just created should
really be accessible and they just you
know output the all of the attributes of
the resource to the module and then you
know you're basically have zero
encapsulation. The idea of modules is a
bit encapsulation so that you don't
create these deeply nested attributes
dictionary key values um but you rather
create individual outputs like if you
want the gateway uh endpoint there's a
variable there's one key for the gateway
endpoint if you want the subnet ID uh
maybe um you know maybe that's a list
because subnet ID is usually like three
whatever but but not deeply nested
objects basically detailed attributes as
individual outputs
>> yeah I think That's I think we're
aligned there. Yeah. But like
>> it it works well. Um um and the the best
thing of all cuz I don't think SSM can
really do this very well except you can
look back in the versions. But like the
brilliant thing about central registry
is that you have the really nice change
log.
You have logs on on SSM parameter store.
But I think I think the idea is this.
either you put it in git or you have
something wrapping more as a service
around SSM parameter store but just like
no control over parameter store
>> is definitely a recipe for disaster.
>> Yeah. A config file in git is the sweet
spot for me. I know it works well.
>> Okay.
>> We're getting a little bit distracted. I
mean, so you you were saying you you
were pushing people you you didn't want
people to to deploy to run CDK deploy
just because of coordination issues, I
guess, is what I'm hearing.
>> Yes. Um, one of the main issues was to
avoid stale values and copy value across
the source of truth of the integration
because this again is not in a single
repo, right? It's across many
repositories and
because of that we originally had tried
to have the product monor repo run the
terraform apply and consume across um
you know SSM parameter store but that
didn't really work so instead we still
keep the dev infra directly applied from
the product manual repo. So for the dev
environment they can run terraform apply
every time they make a change without
going through any integration
repository. But when they want to go
towards
um so when they run apply here on the
dev infra folder basically the package
gets consumed in the dev infra
application uh or environment and when
the developer runs apply it creates the
environment the the the resources in
pink here purple here and then the
integration with like the dev ingress
ALB and the dev VPC and so on right so
so we are copying over the VPC ID the
ALB reference as values into the dev F
monor repo but it we're not copying
staging and prod and all the other
environments just one only dev okay
>> okay okay environment they're completely
they have full full control over the dev
environments which they should they
should be able to do what they need
right and then when it comes to actually
promoting this out of dev um they've
done their QA themselves and they want
to put this into like staging or prepro
that's where they are um versioning and
releasing
a an actual artifact. So they're
publishing their infra package which no
longer is a module. We don't need
special terapform registry or anything
like that. We can just use their
existing u pnpm workflow or npm
packaging workflow or python if they
want because cdk can be python but we
don't do that. So so so you don't need
like special artifact management for
infrastructure anymore. Infrastructure
is managed the same way as your code.
You're deploying, you're publishing and
minting, versioning, change lock,
everything. All of your tools can just
be reused and we p push it into code at
effect.
>> So, okay,
>> I'm still a little bit puzzled here. You
call it a package, but is surely it's
it's it's not a whole bunch of code.
Surely it's just some
>> Oh, it is a bunch of code. It is a bunch
of code. It is like
>> So basic so basically the code and dev
and the code and
>> it's if you are familiar with
Kubernetes, we don't use Kubernetes. So
it's like a Kubernetes Helm chart. Oh,
okay. Okay. Sorry. I I read product as
uh as production. Sorry. Sorry. I think
that's where I was confused.
>> It's good you asked this because this is
the most common question I get about
Terra constructs. Like sounds great, but
how do you actually use it? And that's
what I try to focus on on this
presentation.
>> So yeah, you're saying that this is the
most common questions uh why of the
thing.
>> Yeah. So after I talk about all of the
great things that you can, you know,
that you get because now you get these
really nice developer patterns or
service integration patterns.
>> Now, now that you mentioned Atlantis,
yeah, Atlantis is uh is great. I
actually need to work out how to
introduce that into my into my uh
company. M so I'm I'm I have a
customuilt EMI and I have custom CDKTF
workflows and I want to for Terra
constructs and I want to publish my AMI
as like the bootstrap for Terra
Construct. So if you want to start with
Terra Constructs in your organization
here is your construct to bootstrapped
Atlantis. So it deploys Atlantis with
all of the settings for running CDKTF
synthesis to Terapform running Terapform
apply integration with your pull request
and all that.
>> That's my I mean if you yeah like if if
you
>> but I don't have if your elevator pitch
is basically how to use Atlantis with
CDK I think I get it actually. My my
pitch is you want better devx for your
developers and most people are like
uh it's complicated why do I need an
additional synthesis step how do I
integrate it into my Atlantis um I can't
control my developers and my engineers
they are very creative they create
really convoluted
infrastructure if I let them write them
in typescript uh how do you control that
well that's where I think part of
product like platform team is that you
surface is the pattern so that you can
you know with bootstraps so that you can
>> so that you extract the convoluted
patterns into reusable patterns I don't
know
>> to keep the things align I mean the same
thing with modules right people are
going to write crazy modules
>> it sounds like you are doing a a great
thing in a sense like allowing the
product team to use a code that's
familiar to them but ultimately uh
keeping Terraformers
>> the first class infrastructure uh uh
DevX I think that makes a lot of sense
actually
>> I'm trying to bridge the platform team
and the product team because the
platform team lives in breeds terapform
they have three years of you know code
scanning they have their whatever maybe
they're not using Atlantis but they have
their own GitHub workflows to run and
apply um their Terraform with pull
request like PR reviews uh previews
sorry yeah
>> I I love what you just said there uh
building a bridge between platform and
and product team though like I'm just
also mapping this to my current uh gig
right now. I mean I think if I showed
them this at at my workplace they would
be terrified because we don't want
products to develop to write
infrastructure as code. We want them to
config like we want a very very tight
configuration for them like all all
>> find this funny because there's a saying
is it from
we want products to be able to do what
they need to do without being blocked by
platform right platform must provide
golden paths must provide easy adoption
for the um for the cloud but they cannot
be the bottleneck they cannot be the
ones that are preventing
the team. Um,
>> well, that's I I agree. I totally agree
with that. But like but in a in a
>> uh Yeah, but in a in a but in a big
company it it's not scalable.
I I can see it making a thousand.
>> No, that's why platform teams exist,
right? Platform teams are supposed to
provide the tools so that you can keep
it under control so that the product
teams can still evolve. So you in like
big banks you will have like a
declarative YAML config file that under
the hood sets up 10 different terraform
modules in just the right way so that
they get deploy uh what they need or
maybe use custom resources because
you're on Kubernetes or whatever right
so this is from platform con 20 uh 2024
talk from Gregor Hop and he talks like
nobody wants 80% of Dynamob like your
module is not just like exposing sane
defaults as a better Dynamob under under
the hood, they still need the features
of Dynamob and if your module doesn't
expose it, your module is not better,
right? It's just it's just restricting
access to it. But it's
>> that's a very that's very interesting.
Uh
did you ever meet Gregor? Because he
used he lived in Singapore a few fair
few years.
>> I was at a few conferences where he was
as well, but I never really interacted
much with him. Yeah, he he spoke at the
AWS meetup, too. That this is very
interesting because like I think I mean
I feel like I've been drinking through a
straw somehow because every every gig
I'm on, we make a module to give give,
you know, 80% to the developer. And now
you're sort of you're sort of flipping
the table over here by saying that no
you're uh
your platform team should be allowing
them to do
>> one of my previous CTOs at uh previous
company he was ex AWS as well I don't
think he was in a technical role there
but he um basically had a lot of
experience on how they did things at AWS
and he says the platform team's
responsibility is to put in guard rails
they're not the bottleneck they just
need to make sure that teams can do what
they need to do without shooting
themselves in the foot. So your job
really is put in place, you know, the
the scanners, the llinters and the
dashboards around it. So if we do have a
misconfiguration, we can catch it as
most of the lift as possible.
>> I mean, I'm I I'm fully behind that.
Like so basically the platform team
isn't uh producing modules, they're more
like producing confest type stuff,
right? Or linting type stuff. But like
I've just never seen
>> but at the same time like the goal of
the platform team is to observe teams
usage of the cloud and extract golden PS
so that we can keep teams aligned so
that you know teams don't do completely
different things so that if a new team
wants to start a new project they have a
tech radar and they can figure out today
the recommended way of doing that is
using this golden path and if I follow
this golden path
>> that definitely doesn't happen in my
current workplace like The idea is that
is that uh we're laying the tracks
before the product teams get there and
uh hopefully it's the tracks that they
want to take which is something that
I've always heard as a complaint around
platform teams and actually one company
I worked at was very resistant and very
against towards moving towards a
platform team they had embedded SRE
there was a huge um compliance
initiative and they completely refused
because they were like you guys are
going to be a bottleneck. You're going
to prevent us to do what we want to do.
>> And and to just to be clear, uh I as an
infrastructure engineer, I I hate
platform teams. The whole idea for me is
just terrible. I'm more in favor of a
cross functional embedded uh so uh
embedded infrastructure engineer
pattern,
>> but I always end up as the
infrastructure engineer in a platform
team essentially. Uh and the funny thing
is I've been at a lot of organization
that try to push me towards in embedded
SR role and I always end up completely
withdrawing myself from product and
focusing 100% on the cloud and the CI/CD
and how to make a solution that
stretches across all the teams and then
complaining if people are not using it.
So I I have like the the opposite like
habits of view
>> but I I try to really focus now like I
want to be part of the of the product
team. I want to be part of the road map
but it's very hard for me.
>> There's some very interesting approaches
here. I mean can you I mean I can't
claim to I can't claim the current
working uh approach in my client is is
successful. I can't claim that at all
but it's very uh conservative and very u
established way of doing things. Can you
honestly say to me, Vincent, that you've
you've worked with product teams and
you've given them a golden what did you
say? A golden
>> golden path.
>> A golden path and they loved it. I mean,
that sounds also kind of Let's be
honest.
>> I did I say that I give them a golden
path and they loved it.
>> Well, I mean,
>> I I did work at one organization.
>> That's what you're alluding to, right?
So we did I did in in that same
organization where they went full in on
AWS CDK um where the platform team had a
lot of Terraform. We had a very heavy
Kubernetes footprint and we built a
Golang CLI that bootstrapped deployments
on top of Kubernetes and
we had the funny thing is we had like 40
brands and but it was behind it was
always this two teams and so it was very
easy to roll it out across the teams and
we were able to like align all of the
teams but we did get the feedback from
the CTO like I I didn't mean to create
like an isolated team that built the
platform and then another isolated and
then the product team. We want I wanted
DevOps. I wanted you know embedded
SRRES. I wanted you guys to to build on
you know for the product and I mean
I don't know um we rolled it out we used
it. It was we used Argo CD so it was
generating all of the uh manifests for
our GitOps and it was able to centralize
the Helm charts. So we had like the
ability to roll out and ch make changes.
I think the biggest problem right when
people when teams move away from like
embedded SRS towards platform teams is
compliance. When you start doing audits
and sock 2 and all that then suddenly
now you have to integrate the zero trust
all over the company. Now you certainly
have to have the you know bill of uh
software bill of material asbomb and you
have to track and you know provide audit
reports that you are addressing uh
vulnerabilities um on on time. You have
to provide your full uh network
infrastructure and your firewall rules
as a compliance audit report quarterly.
You have to have great glassass
functionality. When all of that becomes
imperative to the product teams,
suddenly you need to have a more
centralized approach towards managing
your infrastructure. And that's where
organizations tend to then switch more
towards platform teams. That's also I
think one of Hashikcom's um
presentations from like evolution of
like maturity. So all of the
organizations start with this um
no that's not what I wanted. I just
wanted
>> center view. Who who does that?
>> Uh I'm clearly not a a dude that's used
to make presentations. Anyway, so so
stage one like singular teams, ad hoc
approach, everyone does whatever they
want. Sing to standardizing, centralized
deploys, policy enforcement, team
collaborations, and then platform team
self-service.
>> Yeah. I'm I'm usually in that stage
three uh uh what do you call it journey?
Yeah.
>> Yeah. So so and and that's also funny
because like most Most startups or
organizations don't hire DevOps until
they start hitting these centralization
problems, right? And then they're hiring
DevOps because they have reliability
issues. They have a disparity in
technology stacks across different
teams. They have no way of like rolling
out compliance. Uh but the good thing is
they have money, they have revenue and
so they can hire DevOps team which is
DevOps. the whole idea of like being
embedded in like breaking down the walls
uh being doing agile now becomes no we
actually are hiring a system
administration team to help us adopt the
cloud and tame it and
>> are you suggesting standardizing is uh
is dev is is the embedded approach
because
>> no no no no standardizing is the
platform approach.
Yeah, because I thought
>> a lot of organizations they will have um
>> so so so so stage one so stage one is
the is the uh cross functional team
approach right? Yeah. And then and then
already in stage two we have to have
>> So what do you mean with cross
functional? I think stage one is when
every team has like ah cross functional
meaning every team has a guy doing
DevOps or the CI/CD.
>> Yeah. Exactly.
>> More like embedded. Right. Yeah.
>> Yeah. It's interchangeable here to me.
So stage one is embedded because that's
because we because at my my employ my my
consultancy employer we we always
advocate uh a cross functional embedded
uh infrastructure engineer but it never
really happens for one reason or another
as I mentioned. So and that's already
stage one. I feel that's that's kind of
frightening because there's a very
interesting book that one of a candidate
I interviewed pointed me out. I think he
worked at works as well. He told me that
to read team topologies which is there's
an evolution of the teams and and how
you like there's there's an in between
there's like fully embedded everyone
does its own there's like platform at
complete opposite and then there's like
a cross like a lot of like bandwidth
between um how you may have
>> because because this diagram doesn't
seem to even it's it's either like
>> yeah no this is a sales pitch from Hashi
Corp which sells a platform solution
right
>> yeah I get it I get Uh yeah, so this was
a original slideshow that I made for
before I started Terra Constructs which
was the idea was to make something
called the environment toolkit and I
don't know why I cannot zoom in uh zoom
out anymore. Um but basically
the problem is that we have two personas
in a lot of organizations. We have
product teams, we have platform teams.
Well, there's a few others, but like uh
and the idea was like the app team, they
are the ones that need the like a very
nice interface. They should
>> if you just give them YAML without any
schema, then they may shoot them, they
may not really know how it works. Um and
and it's not a very nice experience for
them, but that's commonly what's done
today, which is like you give them a
YAML spec file that they have to fill in
and then under the hood, the platform
team builds the actual modules. like you
say they set the path they they
>> the things right um
>> why do you say why do you say it's not
very nice
>> cuz well
>> I think YL is not very nice
>> well I mean it's I mean it's a lot
easier than them bloody writing code CDK
let's be honest
>> why is the file read only this is my
file
>> and maybe maybe a separate comp
>> sorry did you say again about what's not
very nice.
>> The the YAML driven
uh product team. What's wrong with that?
>> Um the problem with YAML is that
if you miss a space, you get an error.
If you you don't know exactly where
>> use pickle, guys.
>> No, you should use a proper programming
language. Stop inventing all these
additional like Q and um what's the
other ones? JSON net and like there's a
thousands of them there's a hundreds of
them and all of them have the same
problem
>> but they become a programming language
ultimately in the end all of them become
programming languages
>> during that conf
I I I met some Apple engineers and they
were they pitched pickle and I was
convinced that pickle is the is is a
really good way of uh
>> yeah so so that that's what I'm trying
to say with this thing here um there's a
whole bunch of like you just told me the
IDE experience. Does Spickle have a nice
IDE in terms?
>> It does. Unfortunately, it seems to
>> Wrong example.
>> They they seem to favor Jet Brains, but
uh you know
>> Oh, so anyway, does Pickle have a proper
package manager and registry?
>> Yes. Yes.
>> It's so good. It's so good.
>> This is is pickle touring complete?
>> I think so. Yes.
>> Wrong. Copic language shouldn't be too
incomplete.
>> Then why are you not writing it in
Golang?
>> Uh you can't expect people to pick up
Golang.
>> Yeah. So So
>> Pickle's designed to
>> for one task in mind. Uh is explained to
me by by the um the Apple engineer. He
says like uh you know you need it's the
alarm has gone up gone off. It's 2:00
a.m. and you need to make that change
safely.
So, so essentially you're logging into
that m to that machine at 2 a.m. and
making a config change that you could
literally cannot mess up.
>> Okay. Well, I don't know why I'm getting
these results here, but it says the
learning curve is really difficult. It
has a GVM depend because I entered why
you should not use pickle.
>> That's actually a very good answer. U
yeah that that's that's very good
answer. like but yeah you're right it
does require JVM but uh you know I can
look past that
and it it does have a learning curve but
like oh my god I mean
>> it's enhance safety and maintainability
it's reduced duplication it's
abstraction of modality to me pickle
sounds like JSON net like why why should
I use pickle and not JSON net
>> JSON is horrible when I last looked at
it
>> yeah so agree like I never really got
JSON at properly. Um, so
>> also very very both like pickle's very
tight actually. It looks looks and reads
very nicely.
>> Uh,
>> I may have to play with it. But I have
my my reason to get into Terra
constructs and Typescript was
specifically because TypeScript is
everywhere. Uh, it's that's one of my my
my
tenets in life
>> but still but still
>> the last five years
>> I I I don't think I can get behind. I
mean, I can maybe you're working with
startups, but I still think
>> product teams maybe I've been working
for too many big companies and stuff.
>> Product teams shouldn't be writing uh
CDK. It's going to get crazy real quick.
>> That's always like when I talk about
this is a standard reply. Startups
shouldn't be writing CD like product
engineers shouldn't be touching infra
code and
>> I mean I mean startups and midsize
companies I think it's actually pretty
quite good but if you're like a large
heavily regulated industry that's never
going to fly
>> and I I think at that point you don't
ask the product engineer to write it but
you give the platform engineer the right
tools to build better extractions for
the pro for the product engineer. there
because again like this is what we
talked about last time as well. There's
this deck CDK
which gives you the ability to build a a
strict types like strongly typed schema
YAML um interface or front end which
Pulumi also has which is like
>> it reminds me of the the serverless
application.
>> Oh yes like SLA SAML
>> uh not SAML sorry SAM. Oh yeah.
Serverless applica. What is the end
server?
>> I've been doing too much authorization
and authentication.
>> Yeah. So, so but but this is a bit
funny, right? Because it kind of like
gives you the L2 con L2 or L3 construct
as a YAML uh composition thing.
>> Actually, I I love it. I love it. It
works for me like uh nine out of 10
times.
>> And and I think that's kind of the
experience that we can build on top of
Terra constructs or CDKF. basically a
front end question right under the hood
like docker file is a front end they
completely reworked uh docker as well
like docker file does is not the
standard like you can have different
front ends you can have a yaml front end
you can have an other front end because
now you're building actual proper
programming objects that can you know
bind themselves and evolve and can be
configured differently you're not stuck
with static terraform modules that you
then have to bunch put a whole bunch of
go templating around because you're
asking you're giving your engineers um a
YAML interface to basically configure
modules under the hood. That's what I
hear banks do, right? And and it's the
same with with Kubernetes. That's
exactly what Helm is. You're just
putting Go templating around bunch of
YAML, which is Kubernetes um resource
definitions or manifests, right? And and
and
we see where that goes. We see a Helm
chart with like a values.yamel where you
have no idea how that gets, you know,
interpolated by the gold templates under
the hood. The gold templates are
extremely complicated with like
>> well
>> too much logic within the template
engine.
>> That's why I showed you that Terraform
project that I was cooking up the other
day because because I use pickle and and
when you run it through pickle you can
see the pickle schema. You can see your
pickle. You can see how it it it becomes
Terraform JSON. But why do I have to
learn a new language when Typescript is
already a TypeScript compiler that gives
me strong typing and capabilities like
Pickle except I don't have to learn
Pickle and I can just run TypeScript.
>> Well, the thing that I like about it is
that it's clear to me
what
what is the entry point into Terraform
when I'm writing in Pickle. It's it's
very it's clear as day. But if you're
writing CDK
uh and using Terra constructs, then then
you you really don't know what the
Terraform resources are going to look
like until the
>> No, that's not the same thing as what
you just started the sentence with. You
said you you you know the entry point
and then you say you don't know what's
going to look like. That is a very
different
I know the entry point when I use CDK. I
know exactly how this is going to get
invoked uh because I have a strongly
typed interface now that is defined
under TypeScript and TypeScript compiles
to JSON schema. So I can expose that
typed interface as a with a JSON schema
on top of a YAML file and then my
engineers can write YAML uh except they
get an error message if they don't put
the field in the right place or if they
have a typo or if they put misplace a a
space. Maybe you get the same with
pickle because you get a strongly typed
pickle file or pickle config. But you
get exactly the same with TypeScript.
You can expose it as a JSON schema and
you can allow people to provide config
files and you can control the entry
points.
>> Okay. Okay. I I might have misspoke here
but what I'm trying to say is that the
the translation is is more is more uh
straightforward in my mind because
>> but then you're not dealing with
complicated terapform configurations.
You're probably I I need to know exactly
what is a terapform configuration.
>> Yeah. I mean, am I rec
I'm just I'm just mapping it into a
module and you're right when you when
you in your example when you're messing
around with API gateway or or lambda
>> which is a realistic cloud because you
just cloud workloads because what you
just said this is what we all use and
terapform modules are horrible at it and
then you're going to have to write
pickle to wrap the horrible terapform
experience which is very limited because
it only has like 40 different variable
inputs the outputs are not even typed
you write pickle again to do that. But
how about we go back to the source? How
about we build a proper L2 construct and
then we they know how to bind each
other. And this example here, these 230
lines of Terraform is not to tell you
like this is what you're going to work
with. This is to show you the complexity
and the ugliness of what's under the
hood.
>> This is what you write.
>> Yeah.
>> And this is a well-known pattern because
this is actually the AWS CDK pattern.
This is literally the CDK workshop
example except it's on my landing page
and now it spits out terraform.
I I think it I think it it definitely
makes sense to empower product teams,
but I I I keep on going back to the to
the thought that in a in a heavy
regulated company, you you would have to
just give them a very locked down uh
>> Yeah. And that's where a platform team
would need to build using these
constructs instead of these modules
actually with a proper programming
language.
>> Um they would be able to build what they
need.
>> Okay. So the platform team uses terror
constructs to create the golden paths
>> for example. Yes. Um like the pattern
here that I showed on the other slide uh
which was
I'm allowing my teams to deploy onto
Fargate and we use an application load
balancer. In our case the application
load balancer is part of the landing
zone. So they must import it using from
application load balancer attributes.
Maybe I can wrap all of that up in as my
sample app. um you know Typescript
interface. So you must provide the
necessary attributes to import the load
balancer, the VPC and the cluster. And
this is a strongly typed interface of
what are my my integration points?
>> Okay. And then how complete is Terra
Constructs? Do you do you support like
everything CDK throws at it or is it
like some
>> No, no, no. So, so I started with like
every single AWS module that is in AWS
CDK and more moving it over into on top
of Terraform and that's been very
manual. Then I also used LLMs to
automate that process. But I still need
to write all of the unit tests. I need
to write all of the integration tests. I
need to validate everything.
>> So just you're just getting started sort
of thing. I mean
>> like a year getting started. Um, so at
this point I I I'm focused on building
the the features that I need at my
organization, right? Um, what what I
want
>> that's a great slide deck actually. I
really like that.
>> Yeah. What what I want
at this point if people are interested
they should reach out and they say like
great we use Fargate a lot. I don't see
Fargate right now. Like this pattern
that I show here it's a little asterisk
at the bottom there. pending the ECS
pattern module public release. So this
is not in Terra construct integrated
yet. I need to make that public.
>> Just just very quickly why why Fargate
instead of uh serverless? It just there
needed to be persistent.
>> We use a lot of serverless at at my
organization and um at some point
there's longunning job like services
that you need to run and better to run
them in containers
>> without the whole Kubernetes
>> um complexity. So we use Terraform a
lot.
>> I'm with you. I'm with you. Kubernetes
is such a nightmare.
>> So it's very funny because when we met,
I was running the Docker and Kubernetes
meetups and you would tell me like why
don't you just use Compose? It's so much
simpler. And I would be like Compose
doesn't have all the dials and knobs
that you need and Kubernetes is like
properly designed and easy.
>> This this must have been years ago.
>> Yeah. 2017 maybe. Yeah. Well, oh, well,
that's back when uh what was that? Core
OS. I love core OS. Don't know what
happened to it.
>> They were acquired by IBM. Now there's a
flat car Linux. It was fully integrated.
No, they were acquired by Redhead and
Redhead was acquired by IBM, right?
>> Yeah.
>> Yeah. So, so Coras was acquired by
Redhead. All of the core OS projects
such as tectonic were integrated within
open shift and then they replaced like
the CentOS or whatever the Redhead
container Linux became was rebranded
Coros really really so all of the
Cororez like um systemd integrations
which ultimately systemd is a redhead
project as well right um core was 100%
built on top of systemd
>> so the so so then some of the original
cororos people that were not part of the
acquisition or some people involved with
the project created flat car Linux. I
don't know where that went, but um today
um Talos is super interesting because
they're they're taking this whole like
exposing the the the Linux
administration
surface completely through the API,
which was another OS also had this idea.
I don't know if it was Talos, but there
was another one doing the same thing.
>> Not sure what what Talos is. Sounds like
a security product.
>> Uh Talos. Talos. Talos.
But anyway, just to to defend my Oh, you
know what it is. Yeah, and I think I
know what it is. Yeah.
>> Yeah, they just have a new landing zone
uh landing page. Sorry.
>> They expose
>> But going going back to my going back I
I don't remember talking to you about
Docker Compose, but I think actually
Docker Compose is one of those
technologies that I I still regularly
use and love.
>> Oh yes, absolutely. It's just when when
Docker tried to take it and replace it
with Swarm, right? They they like,
>> "Oh that was a show."
>> No, nobody liked Swarm. Um, so so yeah,
doctor time with it.
>> Okay.
>> Okay. I think we need to somehow end
this uh because I'm uh but I
>> getting tired running out of time.
>> I did really really enjoy this. I think
we covered a lot of things. I think
maybe I'm in the right. Okay, one more
thing.