Episode 38: Fable maxxing

What about yourself, Vincent? What's
news with you?
>> Well, Fable is back, so that was
exciting.
>> So, you're smashing it.
>> Yeah. So, not like
I just last time Fable was here, I asked
him to create a proposal on how we
should implement newer Terraform
provider features in in a way that's
compatible both with Terraform and Open
Tofu, given they have diverged in the
protocol they implement. And Fable
created a beautiful proposal of how we
should implement this.
And I was like
you know, at some point I'll implement
[clears throat] it.
Uh but now Fable came back. So, I was
like, "Okay, Fable.
Um I tried this theory.
Um so, I I I thought that if you give
Fable as the orchestrator the
instruction to use Sonnet 5, which is
almost Opus quality at a much higher
speed um and just basically create
dynamic workflows. So, I for the first
time in my life used the /goal,
which is a Claude Code feature to set a
goal. And the model will keep going
until it reaches that goal. So, I said
So, I thought this was a perfect um
perfect scenario, right? I I I used
/goal. I pointed to its proposal
it created a month ago. And I told it
>> like a a page a one-pager, cuz I thought
goal was supposed to be like one prompt
or something.
>> No, I said the goal is to land this
proposal. So, it could explore the
proposal.
>> I see.
>> And I made sure that it was on the
latest head. So, the CDK terrain
project, which is the, you know,
community fork of CDK for Terraform
after HashiCorp deprecated it, has moved
on significantly. We have removed a
whole bunch of like deprecated
dependencies, reduced the binary size,
addressed several CVEs, and landed like
functional support like function
support. So, newer Terraform core and
OpenTofu, they have a whole bunch of
like additional functions like cider,
subnet calculations, and things like
that. All of that has been now
integrated within CDK terrain.
>> of improvements.
>> A lot. And another thing, somebody's
>> rewriting the whole CLI to not longer
use or depend on react for the UI. So
initially the project creators they used
react library so that you can render
components react components in in the
terminal and it's going it's being
completely replaced. That one isn't I'm
I'm not think I don't think we're going
to cut that right now because we have a
release PR ready ready to go but we keep
landing more features and and replacing
the the user interface like that we is
quite complicated. So
>> Wait.
>> Yeah. So oh he has no access. Okay.
Anyway, so yeah that that it's been it's
very interesting.
>> So you're living you're you're I'm
assuming you you're doing all these
development on using Fable now and
>> So today Fable became available like
it's supposed to be from July 1st but in
Vietnam yesterday even at 10:00 p.m.
before I sleep I was checking cuz I
already had the direct goal in mind and
you only get 7 days.
>> Yeah.
>> I know that my limit was resetting this
morning. So I had like 12 hours for my
weekly limit. I thought I could like
maximize because then I know my weekly
limit resets.
>> Yeah.
>> But they do track Fable separately so
I'm not sure how they're tracking it. So
I had a very clear like goal of what to
build with Fable the moment it came
available.
>> Those features that you mentioned all
those improvements you've done to city
cater rain like
this is only really possible with Claude
right? This is what I was going to
allude to.
>> of it comes from community contributors.
I know that some of them are using AI. I
think the the the most difficult bit is
like having a reliable reviewers.
There's one person John Steinek who has
worked on the project a long time ago.
He's also a Java user so he has a lot of
requirements related to how the Java
part of it works. He's been reviewing
and I really appreciate his reviews
because they have deep technical
references like we used to do it this
way. We decided not to do this. This
could be a problem. He he goes in a very
good detail. And I feel a little bit
guilty if I send a PR and it's all AI
and honestly, I I just ask AI to
validate it, which is bad, right?
Because I come up with a whole bunch of
scenarios and when it all passes, I do
like cursory reviews. Um, I do look at
the the code a little, but not not in
great detail. So, I feel a bit guilty
there because I don't want to be the guy
keeps contributing AI slop. And and and
and that's like the biggest problem.
>> Yeah.
>> One thing that's I feel one of my
tenants that I've had
at
that I've had throughout my life is that
if I don't know what something is, I do
try to learn.
>> Of course.
>> And but of course with with AI it makes
it easy just to ship. Like it's kind of
working. Yeah, go go go. But like
if there's something in the in the
programming language like I don't really
understand like what's the difference
between
I don't know this read byte thing and
this raw thing. I actually usually just
I try take the time to and and this is
where again where AI is damn useful.
Like, can you explain to me the
difference between this, you know, byte
array and this raw representation of a
string? Uh
And then it and then it tells me and
then I know and then I can basically not
understand
or keep understanding what I'm doing.
And I feel I mean there must be people
who have just uh
Oh yeah, there's been blogs written
about this cognitive uh
>> offload your brain
>> term?
At At the osmotic cognitive
>> serenade
>> serenade or something. When you just
basically like, okay, now the AI knows
how to do this. I don't need to know
this. No, no, you know how to you need
to know how it works. Oh, yeah. Oh, I I
wanted to share that um I tried a new
feature yesterday. Have you heard of
Hold on. I have to find it. This might
take me some time. Give me a second.
>> Yeah, I need just needed to kick off
SonarQube 5 to do some validation. So so
one of the main things that I wanted to
validate
was explain able to orchestrate
SonarQube five like I told you right.
>> Let me share something with you. I think
I'm I'm going to make a video about
this. Okay, so you know I'm a number one
AWS fan for many years and I've been
doing serverless stuff for a while. But
when you use
>> Yeah, I've read this.
This is nonsense. I mean, they're
basically getting on par with Terraform.
>> Well, yeah, yeah, there there are tricks
to update the binary and this I guess
this is what they're doing right?
They're using some sort of some sort of
I don't know exactly I don't I don't
think they go into detail about how
Express works. But I'm I'm assuming it
does what Terraform
could do.
>> No, the the main thing that Express does
is to avoid the validation phase, right?
I I read one blog post about
CloudFormation. They added the
deployment validation. I thought this
was Express.
>> Is it that simple? They just removed the
bloody validation step. They don't
mention that in the blog. They don't
mention that but like yeah, I was I'm
not showing my whole screen. I tried it
out and uh
it is much faster to deploy.
>> Yeah, it's about the resource
stabilization. So basically
CloudFormation is famous for like it's
very famous it does something Terraform
doesn't do which is it does
validates your deployment and if it
doesn't pass the validation it rolls
everything back which is a massive
feature of CloudFormation that Terraform
misses and to be honest because I was
building a startup with my friend to
automate and he was using CloudFormation
and then we were looking at like uh
Azure and the biggest issue was how do
we ensure that the Terraform deployment
is correct? It's hard. CloudFormation
does that out of the box but that's also
the main gripe people have against
CloudFormation because it goes into this
crash loop back up not sorry that's
it goes into these like rollback revert
failures where you know it deployed like
an S3 bucket or a DynamoDB table and
that is flagged as like something
stateful and then in the rollback it
refuses to remove those. Those are all
issues that were like very common in the
past. I'm not sure if they introduced
more like fixes for that. But the whole
CloudFormation validation step was the
main issue like when people say
CloudFormation is slow, it's like
Terraform is so much faster and easier,
it's because Terraform doesn't do any
validation. And Terraform also you have
to build your own runtime around it. Or
you have to adopt a cloud for it. Like
CloudFormation is a runtime provided by
AWS for you. Gives you a whole bunch of
like um validation mechanisms. The
runtime is cross-regional, so you don't
have to go and boost everything. It's
all there.
>> For you, I like I'm just try Sorry, I'm
trying to summarize and make sense of
this. Do you find the validation useful?
To be To be honest, I don't find it
useful. But like am I an idiot?
>> Well, if you are having to guarantee
that something It's like a database
transaction, right? Is a database
transaction useful? You can you start
transaction and then you commit
transaction. It's all or nothing, right?
That's very hard to guarantee in
Terraform. Terraform can go go and go
off and apply a bunch of things and
leave it like all of these resources
were updated, but then it failed because
of the the API endpoint returned an
error
and it's blocked. You can't roll back.
>> like many many times I've deployed
something and and the rollback feature
caused more problems than
it it should have solved. As you were
saying, right?
>> I mean, it's it's really up to how you
like your tolerance for and how you
approach the problem, right? In some
scenarios, a full validation is what you
want. So it's all like it depends. And
to be honest, if you try to build a full
validation mechanism, the failure modes
like you I think this is probably also
why CloudFormation fails. The failure
modes in Terraform are very hard to
recover from sometimes because it goes
and creates like half of the
configuration as it should be. And then
sometimes
the other parts are not there. It's not
simple like take back the old config and
reapply it to go back to your working
state. It could be completely broken and
requires you to manually go and figure
out what actually worked, how do I
recover from this? Like, is it roll
forward? Is there a way? Cuz cuz my
friend had these ideas, like he was
using full cloud formation, it was very
reliable, and he was like, "How do I do
this with Terraform?" He was trying to
like, "I'm going to snapshot every
Terraform configuration before we do an
apply, and then if it fails, we do a
rollback by just reapplying the old
one." And I told him, "That's not going
to work. Terraform doesn't work like
that. Terraform providers don't work
like that, and the the the APIs that
they talk to don't work like that."
>> Yeah.
>> And I think that's the main thing that
people say when they say uh you can't
treat infrastructure as code, it's in
configuration, it's not the same
deployment pipeline as you do with like,
you know, it's like database migrations,
right? You can't treat it the same.
Like, you can't auto approve. That's
also why people say you can't have like
a cross plane controller reconciling
your infrastructure all the time because
all of these scenarios where the cloud
goes and upgrades a database engine uh
behind the scene, and then your
reconciler tries to downgrade, which is
not allowed by the API. There's a whole
bunch of like edge cases that that that
these like that doesn't fit with ice
like infrastructure. But
>> Yeah. Well, I I mean, I know you said it
depends, but like the majority of the
time I feel like the Terraform flow and
the fixed forward approach has been good
to me. Like, I I don't think I
>> I wouldn't I wouldn't make a blanket
statement like you. I would I would
never. Because
>> It's all about I I think it's all about
the
>> database migrations are also not useful.
I think you will find if you claim to be
a developer, a lot of developers upset
if you say all database migrations are
not useful. I find that I should just
always fail forward and never use a
database migration. Like, no, it
depends.
>> True, true, true. I I guess I'm just
trying to debate with you
that uh at least the majority of the
cases don't need that level of support,
but yeah. I think your point is is
better made than mine.
>> Hello and welcome to the AI and podcast.
In this podcast we like to discuss our
experience using AI as infrastructure
engineers and try to hit on some of the
implications towards how it will impact
software delivery life cycle and
security at our job. I think the main
topic or the
one of the main highlights of today is
that Fable is back as you can see on the
shared screen with lots of tabs open
which is another
>> because it requires to turn off
zero data retention which is not allowed
in the enterprise environment for fear
of our our amazing source code being
leaked to
>> Yeah, it makes total sense. These models
are being trained on our usage, right?
So I am using it for an open source
project mainly. Some things that I
really wanted to land into this project
and I think regular listeners will be
familiar with the fact with Terraform
CDK and that I keep talking about how it
was deprecated and how I forked it with
a couple of people. And in this
particular case Fable helped me plan out
some features. And I was actually before
we go into that rabbit hole asked a a
question to Kai earlier which was is he
actually still using cloud Claude Code
plan mode which is like one of the
famous things that drew people into
cloud coming from chat GPT experiencing
um you know, an unstructured chat mode
where you're very hard to set targets
and getting snippets and bits back or
evolving into cursor where the agent
goes and modifies your files and you
look at typing and re-rewriting the
whole file originally and then cloud
code coming up with this plan mode
laying out cleanly what it is going to
do and and then people like TV Egge
coming up with task trackers to do cross
session content like re-reloading the
session.
>> I don't use plan mode
uh as much anymore. I like I I I do find
I like yesterday I put it into plan mode
as a as a as a safety mechanism not for
it to start editing my code. Like I just
had
That's what I use plan mode for. Like
don't edit my code little idea here, but
then whatever.
>> That's right.
>> I don't use it very much to
>> I actually see also in open code, it's
kind of like plan versus build. And I
see when I work with other people, they
very heavily use plan mode to put it in
a read only mode.
To me it was always like it's going to
explore and find some things. I think it
was like you say a guardrail.
Even if the model decides to to write
something down, plan mode often blocks
it saying like you know, you can't. The
only allowed path for you to write is in
the plan uh file uh path and using the
planning tool, I guess. But plan mode
was great until you had to plan
something complicated that involved
doing research on certain libraries or
even getting buying with other people.
So then something that became quite
famous is spectrum in development. And I
would I adopted it back in September and
I found it much more effective. I think
one of the big differences between then
and now is that we now have 1 million
context uh tokens and also the agent
like interactions are much better. The
main models tend to create complicated
workflows with sub-agents or even just
branch out to agents to keep the context
focused and allow you to go a lot
further without having to resort to
these elaborate plans.
>> I always thought spectrum in development
to me
more to do with team alignment. Have you
Have you used a slash team onboarding
shortcut or whatever command?
>> I tried it a long time ago and I didn't
like it. I don't know. But But like I've
also got some experience like Originally
the way I saw uh spectrum in development
was to create like a a breakdown of why
I I built something. Sometimes first to
get buying for other people to comment
on. So there are very nice tools like
crit uh and others that allow you to
collect feedback from people and I think
that's a lot of these tools are still
>> You like crit? Okay. I thought you were
>> I
>> see very like nice tools like crit. Crit
became absolute slop. Sorry. Like I
looked at it and then in just in a span
of two weeks, the whole root of the
repository was filled with markdown and
>> I think crit I mean Thomas is still
working on it and he's still on X and
he's very approachable. So I'm going to
give let's give him the benefit of the
doubt and say it there is tools like
crit, but it might have been
a bad experience.
>> and I think some of them do it a lot
better. There's Anyway, I don't want to
go too much into that details because I
think what we concluded after you
answered the question and I answered the
question was that with the state of the
tools, I don't see the point of it that
much anymore. Definitely not for like I
think we've gone back to Yes, I do read
the source code and we need to break
things down in small PR's like what you
were saying like you you spend a lot of
time focused on how do I make sure that
the PR is small enough and reviewable,
right?
>> My sense has come in like I I used it in
my project that I mentioned yesterday to
break up the commit. So now
can I share the screen now?
>> Just just no. Because breaking up
commits, I I don't see the point and
also actually Fable is doing this really
>> makes it really good not not just for
PR's, but for me.
>> So
if I walk you through this, I don't
think
Okay. So I didn't see the point of
breaking down individual commits, but I
I do see Fable doing this really well.
Like when you do a plan like when in
this case, okay, let let's just walk
away from all of this history of
spectrum development goal. It's not
really goal. It's like building
elaborate documents for planning like
data models and so on. I think they have
absolutely have a purpose in certain
scenarios, but if you're doing open
source work and you're depending on
other people's time to do the review and
you need to land small little PR's that
work. But they don't care about going
through massive amount of markdown. And
and actually when the Vogel he posted
this uh this famous AWS working back
from from the customer and user
experience.
>> that.
>> I love that. He He posted, "How does
this evolve in the age of AI when you
can build a prototype? Like, when it
used to take months to build software,
now you can build a prototype in days."
And so, they don't just write the
document any anymore from just a press
release and FAQ, but they actually also
build a POC, a prototype. And there were
two articles, [clears throat] Werner
Vogels shared one article, and then he
he pointed to uh I think the team lead
of Kiro who who who who did that with
with with her team, how they developed
uh Kiro. Those are very interesting to
read. What does it like what what does
this like working back from the customer
requirements? But, what I really wanted
to to dive into right now without going
too much in any other details there is
that I tried a Fable 5 with this new
thing called goal.
>> Stashka, could you just control plus
whatever, command plus?
>> Okay. So, so I opened Cloud, and I do
have Fable active. And there's a lot of
people complaining about the the usage
that you get. You only get it for 7
days, and it's very hard limited. But,
the way I saw it is like Fable shouldn't
be the one doing most of the work,
right? It should just be the expert like
overhead orchestrator. And then, I
thought because Suno AI 5 came out as
well, and it's like almost Opus level at
a fraction of the cost and much faster
than Opus 4.8, they say. Therefore, I
thought, "Hey, I have an RFC here that
was built by Fable a month ago when it
when it was available to
Yeah, for It was available I I used it
for two two or three days, I think. And
basically, I asked it to to do a full
sweep of all of the Terraform provider
capabilities across OpenTofu and
Terraform because each of them land the
support different at different versions
at different releases. So, we want to
make sure that if somebody uses our meta
language, which is in TypeScript, to
configure your Terraform, basically gets
like a lot of niceties like if you use
this feature, you got a little synth
warning like the configuration you're
generating right now, it will work with
OpenTofu 1.10. You are currently on on
1.5. It doesn't It won't work or
something like that. They started at 1.6
actually, I think. So, yeah, that that's
kind of like the things that that Fable
actually landed a month ago, the ability
to and we went through some iterations.
I I let it do completely organically.
Fable, just tell me how you would do it.
Initially, Fable quickly highlighted
like, "Hey, we're doing a config
generation and we're we're calling out
to a binary, which is kind of bad
because the binary callout could be on a
synthesis machine that doesn't have
access to the right binary, right? The
binary only gets called when you do the
apply." So, we we we flipped it around
like Fable kind of highlighted it that
it would make more sense to let the the
people writing the configuration declare
what they support and then when you call
those configurations, you get this
validation happening when the binary is
actually being executed when you do the
plan and the apply and then it will
highlight, "Okay, you know, there's a
couple of functions in here that that
are not going to work uh because it's
not supported in Terraform, it's only
OpenTofu available, for example." So,
anyway, yeah, so that that Fable was
able to land before it was taken away
and then it also did a full sweep of all
of the provider schemas. So, looking at
at the plugin protocol, so if you're
familiar with Go lang and Terraform, Go
lang doesn't really support like plugins
well unless you do some type of GRPC or
HTTP
um
in like interaction between the main Go
lang process and then other processes,
which are the plugins. So, this protocol
is is is defined.
>> Okay, like like binaries type stuff,
right?
>> Yes.
>> Okay.
>> Go lang is not good with like in-process
plugins system.
>> that's the benefit of Go is everything's
statically built, but yeah, okay, I get
where you're going at. Yeah.
>> Yeah, so so so the Terraform um
works through protocols like the plugin
protocol and Fable did the full matrix
built of what are the the different
versions that the port which plugin
protocols
between
both of them. And then I just kept it
very simple. Like I didn't even pointed
at the proposal. There's an RFS it's
like an RFC. There's a proposal in there
but there's like a lot of like a static
schema everything. I said look, our goal
is to land this proposal and we want to
land it into this into this library. And
I I did want it to like not do bulk of
the work and use iterative dynamic
workflows with Sonar 5 agents because I
thought that's going to save me cost,
right? Fable is going to do the like the
overall project status planning and then
Sonar is going to do bulk of the bulk of
the
>> I got deja vu. This was the case with
Opus and Sonar months back, right? But
okay.
>> Right.
>> Here we go.
>> So
So this and then actually funny because
I had to leave the house. So I really
wanted to kick this off this morning
because Fable was available. You only
have 7 days. I have to go out of the
house for like 2 3 hours.
>> hell for you, man.
>> Yeah, so
there's auto mode on and goal was
acknowledged and it just went off,
right? It started doing commits and this
is like it makes very small commits,
very detailed and it even when it
finishes it gives you an overview and
we'll see that. So it it it generates
like the the mappers, blah blah blah,
whole bunch of stuff. I didn't even read
it. And I think this is another pattern
that I started to follow is that I'm no
longer following the stream of what the
model is doing. Like I I usually am not
at the at the at the chair and I'm
scrolling back and trying to figure out
what happened is kind of annoying. So
what I I I tend to do now is if I have a
question if my ADHD pulled my attention
away before I was able to read the whole
summary that that that Fable provided, I
just go by the way, was there anything
unusual or was there something wrong in
the proposal that that was that you
needed to work around, things like that
that I wanted to highlight.
>> Isn't [clears throat] there isn't there
like a slash recap command, too?
>> There is like it always it
shows up, but it's like just one line.
It doesn't answer my questions
>> Yeah, true. I I
I'm just like you, Vincent. I It does
some stuff, and I'm like, I'm switch
context, and then I'm back there like,
"What did it do what I wanted it to do?"
>> I think this is normal. I We're at the
point where we're no longer driving the
shell. We're not at the seat. We're We
may have five or six running at the same
time, and we're just switching between
them, and we need to very quickly pick
up what happened.
>> Yeah.
>> posted there is definitely a feature
that's exactly aimed at that. Like,
you're coming back into this What Where
are we? What did we just do? What So,
it's fully landed, it's told me, but
that didn't give me much. So, then it
also told me like all of the things it
did. Um
>> What landed?
>> This was the This is review. Hold on.
We're not yet at the review. Here, this
is This is where it goes. So, done. It's
landed, and there's seven commits. And
here are each one of them, and what is
your focus for the review? If you review
these commits, this is what you need to
focus on.
>> Following conventional commits, is that
Is that out of the box, or did you have
that in your claw.md?
>> I can't I don't know because does it
does it look at the git log? I deployed
release please from Google, which
requires conventional commits. Maybe
that drove it. So, I can't say if this
is a
>> Should Should ask it why How do you know
you're
>> I'm I'm 100% sure it's actually
following the convention because there's
a contributing.md from the original
Terraform CDK, which which clearly says
you need to put the the convention with
the library or the area of effect. This
is how contributing guideline tells it
to do. So, it it followed that, yeah.
So, it did four iterative workflows, and
I sent you a screenshot of how much it
used of my 5-hour session, right? It
used about, I would say, 60% of my
5-hour session in just 1 hour. Yeah,
this is the cool thing here. Shown Shown
for 1 hour and 14 minutes. Now, this
isn't a very big I Actually, I think
it's 4,000 line diff or something, so it
is not small. Uh but it's not like
people that used to show Fable building
like whole 3D games in one shot, right?
>> Yeah.
>> So, I don't think it's
>> I don't quite understand your point. I
mean, it's still you
>> said it kept running for like a day or
so or like 12 hours. For me, I was I was
happy to go away for 2 hours and come
back to it having run 1 hour and 14
minutes.
>> get that point. That for it to to to
work away for an hour and get some great
results is fine by me, too. Yeah.
>> Yeah. And then, because I I also want to
use the models I efficiently and I had
to also go eat and stuff. So, I I then
asked my Hermes agent while I was on the
go, uh which is using GPT-5.5,
to basically build a demo repository.
So, this was the PR it created. It's
quite massive. So, then I was like, now
I have to read all of this, you know?
And I was like, I don't
What do you mean? I don't really want to
read it.
>> did you get Codex to pick up the the
code session? I don't quite follow.
>> So, no, no, no. This this is a draft PR
open on on there. And then I just asked,
um where is it?
>> Well, you use spaces like a fiend. I
should I should maybe use spaces,
actually.
Do you do you use your trackpad or do
you use the keyboard to switch?
>> I I use my my uh MacBook uh laptop
trackpad. Well, you know, with the three
fingers, like you can do so much more.
Um so, yeah, I
I basically asked Hermes to
when I was at the keyboard still, I
should go to the top of the thread.
Yeah, I say, "Hey, can you review this
draft PR?" How do I make this?
>> Sorry, and this was the PR created by
Sona.
>> Uh by yeah, Fable. So, this is
>> Fable.
>> This is the draft PR.
>> Okay, so it's
>> And I have some tooling called review
polar. So, the review polar is a tool
that runs on a cron and always prepares
a work tree for certain repositories
that I'm reviewing. I set this like it
did a full review. Oh, it actually also
read contributing, everything. So, these
ones pop up, like you like you
mentioned, this is annoying. I think
this one I missed. So, this one didn't
execute. So, Codex is like GPT is trying
to do a review, but half of the things
it's trying to see are are like timing
out because I'm not paying attention to
my phone and I didn't approve it. So, it
actually found a few like concerns. I
wasn't sure if they were very relevant.
But, what I really wanted this reviewer
to do is
is to build a demo, you know? So,
basically, where is my I wanted it to
build a a demo of this new feature. So,
this is not yet landed in into the
binary and I wanted to build examples
using this feature before it landed. Of
course, I do think we have full
integration tests inside this as well,
but I wanted something very simple that
I could easily review. So, this actually
showed like it's importing the the
Terraform provider AWS functions and
it's using the ARN parse, it's using the
ARN build, it's using trim IAM role
path. So, it's using all of these
functions that the Terraform provider
AWS now ships with it so that you can
use in your Terraform configuration. So,
it's actually came up with a couple of
like basic examples of how to use it.
And then it did the whole Terraform plan
against my AWS account. That's what the
Hermes agent did. So, my Hermes agent
had a Of course, Claude Code on my laptop
has a lot of capabilities as well, but
my Hermes agent was accessible while I
was outside with Discord and able to
>> Yeah, that seems pretty cool. Like
getting your getting your Hermes agent
to to review Claude output. So, long
story short, was a bit But, I mean, what
I often think is lacking here is is the
back and forth. Like, is there some back
and forth? Is there some iterations
going on?
>> So, after the Hermes agent did a bunch
of tests and executed, one of the things
I asked it like, "Hey, tell me how this
feature felt from a UX." Cuz the LLM is
making assumptions on how it works and
it writes an example, then it runs and
it fails. And so so so so Hermes said,
"I don't know. I saw that it tried to do
something and it failed." And I said,
"Can And write down everything that you
assumed was going to work but failed.
Because that's important UX as well.
>> All right. That's interesting.
>> It did actually say here, UX integration
notes from the demo. I don't know if
they're that useful, but it did say
like, "Hey, I tried to import from a
barrel import and Node.js
went ran ran out of memory. So, this
this Hermes agent had 8 GB of memory and
um
and it doing a barrel import didn't
work. So, I think that's an like very
relevant UX
feedback. And another one was like, I
tried to do things while the rest is,
you know, fallout. Another thing is
like, ephemeral as null had to be used
with sensitive true. So, this is like
very Terraform specific. Didn't really
care too much about that. But, one of
the things that that that made me
annoyed was like, I removed an example
of user agent. I was like, why? Why do
you need to remove an example? What what
went wrong? Why why didn't it work? And
so, I I spent like a lot of time trying
to figure out what is this user agent
function for? And even asked at ChatGPT,
what are the common use cases and how do
you use this? So, I went a bit down a
rabbit hole of this one particular thing
that the Terraform provider AWS
provides. But, it was interesting
because it showed, for example, that you
can use this feature in your in your
module so that CloudTrail shows which
Terraform module is creating these
resources. So, you can attribute things.
So, where is the actual usage of it?
Here. So, in in the database version.tf
>> small screen here.
>> Trying to zoom in. So, here is the
Terraform provider AWS module and they
insert metadata to highlight that that
it's the AWS modules creating this. And
and if you go back to the reason that
feature is there is because CloudTrail
shows you generic Terraform execution,
but this augments your logs showing you
which module is creating those things.
>> That's cool.
>> Yeah, it's it's really cool and it's
very useful also if you build
complex constructs, right? If you have
like
L2 constructs.
>> I'm now I'm now curious what the log
looks like. Does it also put a version
number in, I wonder?
>> Yeah, I need to find back to the
>> Cuz version numbers are so key in my
opinion. Like every software needs to
show me the version number. And and
>> By default, yeah. The annoying thing
about on about Jet GPT is when I ask it
to give examples, it goes from all of
the memories it knows about me. Like
this is a tool I bit built. And it goes
like, "Oh, this is how you could use it.
You could
>> You could use your your binary show the
the binary and the version.
>> memory for these very reasons. Like it
just picks up something random. Like,
"Oh, I know you have two kids and this
is why I've done it." Like, "No, I mean,
you shouldn't
>> This is the Yeah, this was the one why
they added They added I think the main
function is like provider meta support
and and it shows here like can show
example module version, example comment.
>> Okay, so you went down some rabbit hole
because of your adversarial review. So,
okay, so you you went down that rabbit
hole. That then what? I'm still curious
if your I mean, your your PR review
thing with with Codex or looks really
cool, but is it going back and forth?
This is the key that I feel is missing,
but I guess being able to review on the
go from Discord is is pretty cool.
>> Yeah.
Um so, it was able to like build all of
this and then I told it, "But it doesn't
comment until I tell it to." So, then I
had I had I was on my phone, so I have
to like copy these messages and you
can't select things in Discord, which is
very annoying. So, I have to like right
click, copy text.
>> Oh my god.
That kills me. I hate Discord. I hate
it. Okay, but
>> Yeah, so anyway,
>> I'm I'm going to
>> back and forth that happened is that I
asked Hermes to
comment this and then I went back to
Fable and I said, "Do you think some of
this should be in this PR? Is it out of
scope? Should we create follow-up
tickets?"
>> So, you just basically copied and pasted
some feedback in there, right?
>> No, I told Hermes that these particular
things it found, I found that those were
relevant and asked it to to make the
comment. So, it actually didn't even
make the comment because I think I I
made it very like don't post anything
because if it posts something stupid
>> and then Fable is able to suck that
down.
>> So, the thing is with Hermes, I don't
want it to post comments at willy-nilly.
>> Yeah.
>> Because sometimes I'm like conversing
with it and it and it says I
misunderstands me and it goes and posts
a comment. As discussed, blah blah blah.
And it makes no sense because nobody has
that comment thread. So, I'm like very I
told it several times don't comment
until I tell you. And so, it actually
even created like a a draft review
comment and and and tells me exactly
what it's going to post and then I said,
"Okay, go ahead. Like looks good."
>> That that that sounds like a good thing.
>> Yeah, it's it's because you will lose
trust. Like people filter out.
>> Yeah.
>> If it's
>> I
I already on my on my work thing there's
there's some agent that's running
reviewing things and it creates so much
noise. I'm like and the trouble is is
that like I reply to the bloody comment
and the agent doesn't pick it up. So,
anyway.
>> Yeah.
So, the back and forth is is really
plain. You can ask either on Claude Code
in the web or in here, you can ask it to
monitor the PR and look for comments and
wake up.
>> Interesting.
>> In this case, yeah, you can say
>> How did you get it to propose it?
>> It even proposes it. So, if you use
Claude Code in the web, it will say,
"I've created a PR. Do you want me to
watch it?" And then you can say, "Yes,
go watch it."
>> Holy I mean,
that's amazing.
Though,
can I debate with you here because you
you have shown me a cool workflow for
dealing with a 4,000 whatever line
change.
>> Well, I still need to read it. Like I
can't dump this on other people.
>> I still
I would I wouldn't do that. I I I'm
still in camp tiny change, tiny change.
>> I I 100% expect somebody to come back
and say, "Can you break this up?" And
I'm this close of asking Fable to do the
same, to say, "Hey, can we break this up
into smaller PRs?"
>> sensor because the brilliant thing is
like I was vibing yesterday on that on
that tennis membership thing. And it
made 1,000 2,000 whatever li- lines of
thing. And then right And then when I
told it to commit, only then when it
committed, then it was like working out
how to sequence it. And then it built a
really good good history. All because
all because of my pre-commit hook
limiting it to like 100 lines of change
all the time. It was It was fantastic.
>> Yeah, but the thing is you get a lot of
overhead in that, right? I mean, it
again, it's all depends if how many
people you're working with.
If I'm working alone on something and I
don't have to get a PR review, I will
100% do just get it through. Definitely
if I'm vibe coding. But but like to be
honest, in this project where there's a
lot of other people and they have a lot
of opinions of how it should be, that's
where it that's where I'm forced to
break it things down.
>> I'm This is like the software
development life cycle is about if I
mean, this is the agile approach and PRs
are always in the bloody way of this.
Like, I hate Like, I'm I'm working with
with with a guy on a project right now
and it's just typical PR ping pong where
like, "Oh, I made a change." And then
And then like, you know, a day goes by,
I review it and I say, "No, it's wrong."
Then a day goes by and he fixes it. I
mean, it it doesn't work. I feel it's
really ineffective to do this in a in a
in a business. I I know it makes sense
in open source because it's a low trust
environment, but but in business it's
just it's pissing me off.
>> a whole lot of discussion because
there's quite a few discussions on
LinkedIn people say like PRs are broken
and it took agents for us to realize it,
how broken they are. But, um but I just
wanted to highlight here, and then I'll
stop sharing my screen.
>> how did agents help us realize that PRs
are broken? Because you To me, you
showed me agents loving the whole PR
process because it gives you an
opportunity to work your Hermes PR
review agent, you know.
It doesn't really show It doesn't show
show off PRs being broken to me.
>> I think I think the whole industry is
questioning Git as a versioning
mechanism. There's at least three, you
know, well-funded startups that are
reinventing Git or reinventing version
control,
um that are more that is more agent
native.
>> You mean pull requests?
>> top of that, there was also LinkedIn
post about trunk-based development and
the way that PRs like work. I need to
find it. Let me try and find it. That
explains the problem. I think it was
mainly on the review bottleneck that
they were trying to highlight. And And
it was also focused on trunk-based,
yeah. Let me try and find that.
>> Maybe let's take a little break cuz I
need a pee.
>> Actually, yeah. Maybe this is something
I want to share is while I found the
Hermes agent really good at finding
things that I read on LinkedIn. For
example, I asked ChatGPT about a post I
read a day ago, and it couldn't find it.
Like, it came up with generic Google
results.
And then I went back to
Hermes on on Discord, and the agent has
capabilities to scrape the web on demand
more than than ChatGPT. So, I don't
know. So, this was So, this was Yeah, so
you're back. Okay, so I want What I was
saying while you were gone is that to
find back something I read on LinkedIn,
I found ChatGPT very bad, very poor. I
tried to find like, "Hey, I read this
thing. It was about something." And
ChatGPT can't find it, or it comes up
with generic Google results or web
search results. And this is an example
where I just read the LinkedIn post of a
end-to-hand hug end-to-hand hugging face
model provisioning. So, it was a post
that talked about you can pick any model
card from hugging face and it will look
at the GPU requirements. It will then go
and find the properly sized GPU to rent
it from like an
like an a GPU
inference provider and it would deploy
the model on top of it and it would land
you with Claude Code router configuration
so that you could immediately launch
your Claude Code using that model on
rented GPU so that you can basically try
an open weights model and before you buy
all the GPU and hardware for it, you can
experience what is it tokens per second
is it is it like
>> With Claude Code not using pie or open
AI.
>> Yeah, so so so the post was very cool,
right? I just wanted to find it back and
when I asked her GPT it did an absolute
horrible job about how to take hugging
face models run GPU and deploy them and
here I ran out the gateway actually was
was crashing and then Sony told me how
to fix it. But then I came back and said
hey
>> You're using fire crawl, aren't you?
>> Yeah,
and and and the funny thing is it
actually found him like it found GA
Ahama who posted this this this thing
and then when I asked it to use web
search to to scrape and you whatever
scraping skills you have to find it and
then it very quickly found the exact
post which was then posted a day ago and
found the actual AI on demand GitHub
repository and then we went ahead and we
built we wrote a little blog post about
it.
>> fire crawl use your LinkedIn credentials
to do the scraping?
>> I think I
not fire crawl but you can install
custom tools. I I tried for a week to
use XAI with an API key and I put $10 in
it and it was it was it was empty within
like two or four like within five days.
So XAI is extremely expensive
to use AI like API tools to to to scrape
things. So don't use that.
>> I bust through like a $10 budget within
a a week and I was like I can't afford
this even though XAI web search seems
awesome.
>> So so I wanted to I wanted to highlight
actually if I want to find that LinkedIn
post that we just discussed, it's better
for me to ask the Hermes agent, sorry. I
think Firecrawl with the web search is
better than ChatGPT.
>> It's like a a personal chat.
>> Yeah, but but still I can't have a thing
you need that most sites are getting
good at at blocking it. Robot, so you
have to use your logged-in creden- you
have to use your your you have to like
offer Chrome remote debugging port to
your Firecrawl instance just to get
anywhere.
>> Yeah, maybe. I I think one thing I I I
want to do is I know that somebody built
a LinkedIn CLI tool for AI agents, so I
could try to use that, I think.
From Ion Murdock, he posted it. And
there's also
>> That sounds like familiar. It's a guy
from Singapore, right?
>> Yeah, yeah.
>> That's our friend. He used to attend the
meetings.
I just had the same experience that you
did about uh you know, Claude asking me
a whole bunch of questions, and then it
sort of no response. This is really
interesting development.
>> Yeah, but also I also had a few times, I
think mostly in Claude Code web uh or
Claude web, where I say use the ask user
tool. No, even in in Claude Code on on
the on the on the CLI, I had a quite a
few times where it gives me a proposal,
and I I I can add notes into it, and I
add some notes, and then it come and I
press enter and it goes like, "No
response. Okay, I'll just assume." And
I'm like, "I did respond." Like, "What"
And then I I escape like, "I told you."
Maybe I should double escape and go
back.
>> Do you have Claude on your on your
mobile phone? Because I can't help but
think it would be cool if it just
notified you while you're having your
lunch like, "Okay, these three things,
yeah.
Yeah, the second one."
>> So, what what what I did see happen is
if you have the Claude app on your on
your laptop, and you're doing something
in Claude Code on the web or even on my
phone, and then that that Claude Code
instance finishes, you get a
notification on your laptop saying like
that Claude Code instance finished, you
can go it's like it's waiting for your
response. So, it does from the app go
back from your like from the web. I'm
maybe on the phone as well, but maybe I
don't have notifications. But, there
were quite a few interesting posts about
the Claude Code web app and also Cloud
Code sorry
about the Cloud
>> I'm I'm starting to think I need my own
Max plan. I got I got serious FOMO
because I can't get Fable because of my
enterprise stuff.
>> But, then you have to like find a way to
like toggle accounts. Like you said you
were you were telling me that you were
doing all of that with Docker sandboxes
to set up like individual sessions with
different authentications.
>> Yep. I I I I think I'm I'll have to do
it I'll have to do it I'll have to do
it. I got my FOMO's too real. I need
Fable.
Okay, I think it's lunchtime here. I
think we have enough content for for a
for a podcast. It was a bit haphazard
cuz I was getting interrupted, but if
you Yeah, we should maybe plan out what
we're going to talk about a bit better,
okay? Cuz I did want to talk about the
charity majors thing, too. The the
>> What? The tennis? For the charity No, I
definitely wanted to talk about that,
but have you read it? Have you all read
it?
>> I I just read the headline.
>> Okay, so I don't think we can talk about
it much.
>> Okay.
>> I send you the link and I'm like I want
to read it, but
>> Okay, we should have we should we should
have an agenda where we have have things
and then have we read it and then only
when when we both have click tick, we
can get back to it. Okay, I'm going to
sign off now. Hopefully, Riverside too
can make sense of this recording and if
you like the podcast, like it. In fact,
if you didn't like it, like it. Comment
why you didn't like it. Yeah, that's a
good idea. Geez.
>> You're a genius. Okay, see you. I'm
going to take some lunch. Bye.
>> Okay, bye.