Episode 8: AI acceleration with Anthropic and Beads

Cool, man. That's I'm just trying to
think of what we spoke last week so we
don't sound like broken records
repeating ourselves about
>> last week. Gemini just came out. Um
>> I just rewatched it before this call
>> and we talked. Um
>> yeah, but no, I forgot everything. Yeah,
I I mentioned that I got some good time
in on anti-gravity, the new
IDE that Google launched.
>> I haven't I haven't tried that.
Oh man, I I'm going to
>> Yeah, I guess I guess we were I I think
we're we're both anthropic Claude fans,
aren't we?
>> And guess who else since 20 hours ago
said he is a fan who is a massive
antiropic dude. quite famous
>> andropic who
>> T3 G3
>> also Theo is like every time he posts a
video for a while on Reddit people would
be like he's obviously an open AI shell
he gets private access to some of their
um models and um and and and basically
people he will never say anything
positive about Antropic he hates
Antropic um but he did a video was
Uh,
it was
>> Let maybe share your screen.
>> Okay.
>> I do watch a fair few Theo videos, but
to be honest, I don't subscribe cuz he's
one of those. It's like like Primogen
and Theo. Their content is good, but I
do find it like a bit too much actually.
For some reason, I I feel like only
dipping in when I want to.
>> You see it? So
we talked about AI for infrastructure
engineers.
>> Oh yeah,
>> that's subscribe.
>> But there was um I am subscribed. No,
>> I don't know.
>> And there was um
I was going to I was hoping that the
title would ring me a few more bells,
but I also rewatched what we discussed
about based infrastructure a little bit.
But basically this was posted.
Um
I tried to open it in a
new window.
>> Not what I wanted to do.
>> Oh my god. You don't have premium.
>> Uh no. Why would I pay?
>> You're an animal.
>> Where's the
>> It's not this one. This one is about
opening eye. This one.
>> I have a new favorite model.
>> How many hours ago? 22 hours ago. So
little.
>> Oh, he's he is
he
>> is this some sort of ironic?
>> No, no, no, no, no, no, no. He he goes
into great detail. Surprisingly has a
lot of benchmarks. He does a very
interesting demonstrations comparing the
output of different models. He says Opus
4.5 got massive boost in UI generation,
that the entropic models generally were
not good at that originally, um, but
that it's really much better at it right
now. So he has this um snitchbench
repository and he uses cursor with git
work trees and he says in this video
five times that get work trees are not
working in cursor and that they're
working on it but it makes his life a
living hell and every time he talks
about his workflow he mentions work
trees and how they're broken. It's
really um anyway
>> yeah I mean there's some things about
git that I've never really enjoyed like
subm modules and things like that. Oh,
some modules stay away from, right? But
git works sound like very interesting in
the age of AI where you can just spin
off a branch in a different subdirectory
and have like a PC quickly, you know,
developed by
>> you need it, don't you? You need it.
Yeah, even speckit idea is also to like
define the specification and spin off in
implementation
um and so that you can do different PC's
like maybe you have the user stories and
functional requirements but then you
implement it in like five different ways
and different technologies different.
>> So wait a minute I thought you were
saying that Theo was team open AI and
now he's anthropic.
He's yeah he he gets access and early
access to AI open AI and he does not get
any access and then when he talks about
it in this video he mentions like
there's a few other um
AI channels that are benchmarking models
that are also not getting early access
to the entropic models and he's happy to
be in the same boat as them. something
like Antropic is horrible. Um, but I've
got to give it to them. This opens 4.5
is insanely good. And
>> okay, sorry.
>> He als he he also ranks here. He talks
about the the output the UI output of
Opus and then he he gives his ranking
here of output ceiling versus
consistency and um he hasn't finished it
there. So, but to him he still says
yeah. What does output ceiling mean?
>> The best you can get out of it. Like
like sometimes you give it a prompt and
it gives like a really amazing output.
He says a couple of times GPT 5.1 Pro
really beat Gemini 3 um by a mile. And
then
the problem is the consistency
um that you get from these models is
nothing compared to open opus like oppus
and son they are very consistent. So
he's reranking them on the right saying
uh or left however is this mirrored? No
it's not mirrored. Um yeah [snorts]
anyway um but Gemini tree
I think last time we talked I mentioned
that I had very limited access to Gemini
tree because uh I used Google Claude Code
assist uh licenses. I have a Google
cloud account. It's a personal one but
you can subscribe to um Gemini in GCP.
So you can run code assist and you can
then use it and you get access to like
the the Google cloud models but the way
that they release it to different
subscription plans is a little bit
different. So you can subscribe to
Gemini AI Pro or um and you can even get
like a massive promotion for 3 months or
so at half the price or whatever it was
to to use it to to that that Gemini
Gemini Gemini AI Pro subscription. But
for some reason to me it made more sense
given I already have my my billing
accounts in GCP to just use the the
seats there. I basically
>> was trying to see if I could use it.
Clicked for one month subscription
before I knew it. I realized that that
was probably not the best way to do it.
Uh figured out how to assign a seat
because they have users within Google
account Google cloud that you then have
to allocate to licenses. You can do that
automatically. So when you sign into
>> GCP is like project. So there's there's
a project concept and then and then then
there's a seed concept or something.
>> Yeah. So on one one side it's really
cool that you have so many ways to
authenticate and you can use an API key,
you can use the Gemini AI, you can use
this and that on the other side is they
are not consistent in the way that
they're giving the different
subscriptions access to the new models.
And so the only way I had access was
through the anti-gravity
uh IDE which is apparently just Windsor
rebranded because if you search for
strings they still have the original
Windsor strings in there apparently. But
anyway, so um so that's and there it's
free access so it's nerfed right because
um it's against my accounts but the the
availability of the model is constantly
like um to make sure my experience was
not unique and to have a bigger you know
background around benchmarking I watched
three or four videos as you can see from
my uh
>> from my my history around these latest
model announcements.
>> Right. Right.
>> So,
>> so my experience was horrible. Uh after
the first
>> for Gemini, you're saying, right?
>> Yeah. Yeah. After the first
>> Sorry, who who owns who owns Wind Surf
now? The earlier comment you made.
>> Okay. So there was a stream
>> there was a stream of like um also a
live stream just 20 hours ago from the
Mastra people which building AI
workflows and they went a little bit
into the drama of windsurf because
windsurf was like the startup um and I
think the CEO or founder was poached
away they were like negotiating a
contract with Google uh but then Google
didn't manage to acquire them afraid for
antitrust lawsuits or thing. So they
just took the CEO. He left his whole
team behind and all the IP.
So he joined Google within one one or
two months.
I don't know why. On the right my my
suggestions.
>> So basically when Surf went from like a
huge valuation to to the CEO being
poached. I guess that's kind of smart.
>> I know. horrible for the the early like
early team because all of these guys
probably gave up a good amount of their
life working full stop on this startup
and then the CEO just walks away with
all the IP and the company is still
there. They're still employed but it's
worth nothing or something like that.
>> Yeah. So everybody hates that CEO. I did
watch I did watch a like a hacker news Y
cominator interview with I think the CEO
of Windsurf and I did find it
interesting how the company started by
like I think they were doing like GPU
type of 3D gaming cloud stuff and then
they pivoted into uh to Windsurf.
>> Yeah. Um, so, so basically
they were saying that they got like 10
seconds of good use out of it. They
kicked off a plan and they immediately
got like 10 seconds after 10 seconds is
Gemini Tree just said, "No, I'm
overloaded." Um, or uh you reach your
quota. Um, so for me, I feel like I got
a lot more out of Gemini tree initially.
Um but what really made my impression
bad over the last week is one that the
way for me to get access to it is
impossible. Um I have to go on a waiting
list but I haven't like received an
approval on the form to all go on the
waiting list. They have a question which
is like are you contributing to Gemini
CLI? I'm like no. So guess what you're
not going to get it. [laughter] I don't
know
>> why questions.
>> Why ask right? When I see that question,
I'm already like, "Yeah, okay."
>> Okay,
>> I guess not then.
>> But then on the opposite, Opus released
was it yesterday for me? Like for me it
was yesterday. Was it for you yesterday?
Opus the announcement of Antropic was
yesterday, right?
>> I don't know. It was fading days of
fading. Maybe the day before yesterday,
but yeah,
>> it was like Monday, right? Today's
Wednesday. Yeah, it was like Monday
during the day and I woke up on Tuesday
and [snorts] I and it was released and
andropic
litm I mean made it so simple right in
Claude Code you can go extra usage I mean
you can yeah you can go extra usage
and it will immediately throw you into
the into your account management your
billing [snorts] uh page and you can
just use your credit card because I'm
already a subscriber it already has my
billing um method there, right? So, I
can just click, okay, top up wallet and
I immediately have like I don't know, I
put in $30 uh equal to like a month
subscription, $13 and um and right away
I have access to Opus straight up. I can
just slash Oopus. There you go. 4.5.
Even though I pay $20 a month or $30 a
month in Singapore dollars
>> uh for just the pro subscription, I can
use OPUS, but it's API uh charges. That
means it's not like on your monthly uh
budget, you get charged by the call. And
I thought like it's supposedly much more
token efficient and therefore they say
in terms of cost, it's almost similar to
SA 4.5. That's what you get when you are
a Max Pro user. They say when you're Max
Pro, you use Opus. They slashed it by
like a third the price. So that's one of
the things T3 mentioned and Tropic
sliced the price like crazy. So first
they made Opus 4.1 completely useless
and forced everyone to use Sony 4.5 and
then
everyone saying they were sitting on
this model until Gemini came out and
then the next week they released 4.5.
>> Oh, nice. Yeah. So they can move
quickly. Hey, can I just can I just
share my screen here?
>> Yeah, sure. Sure. Um,
>> so I'm a little bit puzzled by the way
things work. Like I I got access to
Claude Opus through Gemini. I mean,
sorry, through co-pilot
pretty quickly. You can see my screen,
right?
>> You use Yeah, Visual Studio. No, I don't
think I see your screen. Sorry.
>> You don't? Oh, god. I
>> [clears throat]
>> Um,
you don't see it at all, do you?
Okay. May maybe let me just try share
again.
>> Did you click the button?
>> Yeah, it says I'm screen sharing.
>> Now I see your link shared inside the
chat. You're trying to share Excalibraw.
>> Yeah.
>> Oh, now I can see it.
>> Okay. So,
so like, okay, let's let's we can
annotate this with some. So, so Gemini
3. Oh, no. That's too that's too big.
So, Gemini 3 came out.
I got that. They charge 1x and then of
course uh Opus got released like a
couple of days later.
I thought Opus was the was the, you
know, the the the clever big thinking
one. How come?
>> So, basically, Haiku is now,
if I understand that, cheaper, right? Or
I think that is it.
>> Haiku. Haiku was super cheap. Yeah, I
think this is promotional because and
it's preview because I don't think I I
don't think Opus would be the same as
Sony 4.5. It's still more than so son
4.5. Haiku is the cheap you know like in
pros haiku is a very short um body of
text. Son is longer and then opus is
your like your big life.
>> Yeah.
>> So uh I just I just think it's a bit
curious the way that they've presented
it here. Like I'm I mean credit to VS
Code for even showing these clawed
options very well. Like I know I know
that the GPT
you know open AI is owned by Microsoft
VS code is Microsoft they do show their
their own mod models first for some
reason they throw in gro I I don't know
why I don't even know know what Raptor
is
>> do you know what that is
>> no
preview
it's just a curious state of affairs my
VS code manage models uh dialogue.
So, did have you tried Opus? Like, do
you agree with what Theo said? Is it all
hunky dory? There's no controversy.
[gasps]
>> You know, my opinion about like I don't
have a proper benchmark, but my
>> just go go on vibes.
>> Yes. On vibes,
I think Opus is amazing. I mean, I'm I'm
I'm 100% entropic right now. So, um like
I have the
>> I'm really considering like but I don't
have much else to cancel, but I want to
cancel everything and just go in like on
the max plan, you know, $100 on Tropic.
There you go.
>> Oh my goodness. [laughter]
>> Yeah, because I mean I think this month
I spent about 100 anyway. I spent
>> speaking speaking of spending money. So,
so basically I got an email from my
employer
uh and they said that I'm limited to
like a 50 a soft limit of $50 with
Anthropic, but for some reason the the
whole Anthropic UI is messed up. I can't
see how much my spend is
and looking at dashboard.
>> Um, let me just double check.
Last time I logged in, I I just couldn't
see it. It's all in under this company
account and I couldn't get any
visibility.
>> Okay.
>> No.
>> Yeah. It's a bit hard to know
>> what you can do if you if they tell give
you an abstract number and they and
there's no way for you to check.
>> Exactly.
But like for example, what I like I went
I tried um here's another thing I wanted
to share with you.
I went nuts with um this agents feature
because I just learned about it. I
didn't know about it. the picture that
you sent about the dependable thing.
>> Yeah. So,
>> it's interesting.
>> Yeah. So, I basically
set up an agent
>> to basically merge all the dependable
bot PRs on like 300 repos.
>> Smart idea. [laughter]
>> But
>> wait, you just pulled in shalloot
on every ripple. [laughter]
[snorts]
>> What?
You know the the worm that the
npm worm that
>> Oh, no. Hopefully not.
>> Hopefully not.
>> What's his name? Shad Halute.
>> Well, that's I don't know, right?
>> Okay. So, let me just run you through
this uh prompt. So, use the GitHub CLI,
which is the name of my agent, by the
way. I'll show you the agent in a bit.
to spawn agents in parallel to look at
the issues to each of my repos
and look to merge dependabot and then
I'm I just thought it was prude to do
something like you know touch repo name
once done
to ensure the repo isn't done again
then I went bananas going going through
I think it launched 50 agents in
parallel so this was like three or four
it was like doing three or four or five
tool calls each one for some reason.
>> Is this haiku or son?
>> Uh I was using haiku.
>> Yeah, it's really fast.
>> And so I when it was running, you can
just imagine this is like 3045. It also
showed the token here. Now that's gone
cuz I'm just I launched up to the resume
thing right now.
>> Don't you have it on your screenshot? I
think the screenshot you sent me has it.
>> Yeah, may maybe.
>> Yeah. Yeah.
>> So it it went it did the job I think as
best as it it could. I mean, obviously
it runs into conflicts and some things
got messed up, but all in all, it's
quite a safe process.
About like, I don't know, 10 minutes
into the whole process, this thing kind
of blew me away. So, you can see that
it's processed 45
um maybe another 100 or something like
that.
And then like
sort of like uh halfway through the
process, it says, "Let me create a
script
to process all remaining repositories
more efficiently."
>> That's good.
>> And I was like, "Holy shit."
>> That's what a human would do. Right.
>> Exactly. I was like, "Holy [ __ ] I
didn't even
tell it to do that, you know."
>> No, but that's exactly what you and I
would do, right? We would start to do it
manually. We would always get like
similar problems and kind of get a good
feel about like what works, what are the
problems we encounter and then we make a
script for it. And it's kind of like
what it did.
>> Yeah. I'm like I'm like you know my
I was getting kind of hairs on the back
of my head type of thing. Um and then
and then it b then it kick uh then it
kicks off the uh the shell script and
runs it in the background and then it
gives you and then it checks again
>> and then it does a bit of a summary
>> but okay what I wanted to show you this
is that I I just don't have a how many
it used a bunch of tokens but I don't
actually know how many is there some
clever what is the way of seeing the
context window it's just like context
isn't it
>> yeah but like the context window is not
going to tell you anything. It's usually
the slash usage thing, but that's so
limited now. It's just two bars. And
it's so funny they went from showing how
much you have used uh from how much you
have left back to how much you have
used. It's Did you have Did you not
watch that whole situation with the bar?
You I think you were asking me or I you
sent me.
>> Yeah. Like there's a lot of controversy
around that status line and everything
like that. I guess that they're I mean
it doesn't inspire confidence when they
do [ __ ] like that, do it, does it?
They're trying to hide. Funny thing is
what I saw on Reddit, somebody posted
like how come when I use Claude Code on
my Mac um I get the the how much I have
left and when I use it on my um like my
Linux box my DBN um it shows me how much
I have used and some comment was like
isn't it funny that like the DBN package
administrator refused to integrate this
[laughter]
change and he didn't like it and now
they're back um like in the change log
they said like reverted the usage bar to
the original. But that's [snorts] one of
the comments that Tio said like um you
did a great job on Tropic. Um I'm open
to talk. Yeah. As if they need you. But
anyway, and then
>> yeah, talk to me. Um
>> I I want to move from open AI.
>> Yeah, he needs the red carpet. He's he
was also constantly showing how much he
spent like, oh, I've you know, spent one
hour like, oh, it's $150 now. It's a lot
of my pocket, but I'm happy to show you
guys the benchmark. I mean, credit
though. It's an impressive benchmark and
it the way that he's very quickly
spinning up all these models with the
same tasks and showing them.
>> So, how does he how does he know how
much he's spending and I have absolutely
no clue because because of the usage of
>> because you're in an org account it
looks like like on mine. So, a while ago
we were thinking how can we track our
usage properly and I was like yeah I
don't really have an idea. I just wait
until it's done and I look at my usage.
But now I found a decent way which is on
my account in my billing page I can see
exactly how much like pretty much live
um as it is running I can see little by
little my week my hourly budget filling
up and my weekly budget filling up and
then my extra usage like the wallets
that I I put in there also filling up
with open
>> maybe can you give me like a a rule of
thumb like for example this context
window here I mean it's half full.
>> There is a package called
>> is is is a is a half full context maybe
you know two bucks and a full context
about four you know four bucks. Can I is
there some rule of thumb I can use?
>> I think is really not the way to go for
this because um there's even new like
with Opus 4.5 they added new
capabilities into the Claude Code shell
things like the ability to progressively
search and expose tools to the model. Um
the ability for the model to you know
this whole blog post that I did about
MCPS.
>> Yeah. Yeah, but this is expressed here,
right? This is expressed in the system
prompt, isn't it?
>> So basically this whole shell even has a
new functionality to automatically
compact the context. So you cannot go
off like this context status uh diagram
to determine how much you've currently
used because they are changing how all
of that work and this is not meant to
display your your your usage. There's a
there's a tab for usage. Well, let Oh, I
forgot to show you the um the actual
agent.
So, this is what a agent looks like.
It's it's in the docu agents directory.
And then it's kind of like a a skill,
but you're supposed to specify tools and
model. I I mean, maybe it's an optional
thing to be honest. So, here, as I
mentioned, I I specified Haiku. to be
honest, I grabbed this from um some guy
I've been email email conversing with,
guy named James.
But yeah, so this ba this agent
basically
equips Claude to use the GitHub CLI,
which is, you know, the GitHub CLI is
one of the best things about GitHub,
right? You can go nuts with it.
>> You also wrote a skill, right? You wrote
the GitHub action skill for work.
>> Yeah, I did. I did that. Yeah. So, um,
that's also something
I'm I'm a little bit confused between
agents and skills and all the rest of
it. I'm a little bit like and also my
Claude when I run Claude. Um, I I need
to uninstall beads because every time I
go slash command, it's like 99% bloody
beads commands.
>> Absolutely. This get rid of beats MCP.
It's a complete disaster because they
don't have Isn't that I think it's the
plugin that sets all this [ __ ] up.
>> Oh, yeah. I didn't do that. All right, I
just installed the BD CLI. I tried the
MCP and I already uninstalled the
>> manage. Okay. Okay, this is going. How
do you uninstall? Delete to uninstall.
Okay, it's gone. And hopefully Oh man,
still
>> so I've got a story about a about skills
cuz I I've [clears throat] used actually
Claude Code the shell
uses has built-in agents right since a
few releases ago. So when you go into
plan mode often times when you prompt it
to start planning launches agents.
>> Oh my goodness.
>> Oh, you have costs. I haven't seen that.
There you go.
>> They have a aside from the usage now you
have an because I was every single CLI
actually has a really nice summary.
Gemini CLI.
>> But how come that how come that was so
Wait a minute.
How come this I guess it doesn't have
the my previous run does it?
>> Probably not. No, this is probably your
cost from this shell session. Even if
you restore a previous
>> See, this is why I don't want my
commands to be polluted. That's the
first time I just noticed this stuff.
>> Yeah, you beats and the the same second
you see the command that you needed.
>> Yeah. [laughter]
>> Okay, so plugins pros and cons. Um same
with MCPS, right? But but we were just
talking about agents and and I was about
to like share with you my experience.
You you also don't know what's the
difference between skills and agents.
>> So basically agents actually are part of
your shell. They come come with it no
matter if you build them or not.
>> Okay. So what's what agents are there
already then? Are you you suggesting
that
>> the planning agents the explore agents?
If you go into agents without building
any uh slash agents, you will see that
builtin there's two agents at least.
Yeah. And I think they're adding more.
>> No, no. Well,
>> oh, built in. Okay. Sorry. [ __ ] I
[snorts] just killed the share.
>> Let me let me share my whole screen.
Hopefully I don't share anything insane.
>> Um,
yeah. So in agents, I've got my GitHub
CLI agent there. And then then it's
built
>> in agents general purpose
status line setup.
>> I ran that one.
>> Oh, okay. Okay. And then it sets up the
status line. Okay. What is it? That
status line is kind of like by default
it agents copies your bash shell status
and then it shows your bash shell status
line at the bottom.
Uh I think you just
>> Oh, I supposed to
>> Yeah.
>> run.
>> I think after a while would have come
back and and tried to run it.
>> You're auto compact. I have set that
off.
[snorts]
>> Yeah, that's
>> okay. So that's what did I just run
there? you wanted to do status line but
let not do that. Um so basically some
agents are in there the explore agent
from uses haiku to quickly go through
all of the paths and make a summary of
what's inside your repository. Um that's
what sometimes I guess that's what it
uses when you do your initial in it. It
will use the explore agent to quickly
find the data so not to waste money on
Sunday or things like that, right? And
then it has the um planning agent which
when you ask it to go into plan mode, it
will maybe spawn an explorer and then it
will spawn a plan agent to build um look
it gives it the context and it says like
I have to do this thing. It's very
interesting because when you when you do
control O you see the actual prompt that
it sends to the SP planning agent. So
it's interesting to see I've been trying
to leverage agents and spend them spawn
them the way that you've done but like
what you've done with the GitHub CLI
agent is really nice. I mean that's a
really good good use and you I'm happy
for you worked out because when I tried
it it was a disaster. [gasps]
Um and then skills. So I have a very
very very funny experience. um with
skills.
>> Okay. I'm I'm I'm trying to describe
just to capture. Yeah, you're welcome to
add screenshots and other things to this
excal.
>> I'm sure if I can, but basically what
happened was this. I um I spent like
close to $15
pretty much my whole monthly budget on
Opus to generate a really extensive plan
um of a feature that I'm working on. And
then I was like, "Okay, I I can't use
more than that, like because I'm I'm
over budget."
>> I I still don't understand how you can
spend $50 like that. Okay, carry on.
>> 15.
>> Oh, okay.
>> On API calls on Opus 4.5. And so I went
to to say to to anti-gravity and asked
it like okay here's a very extensive
plan and here's the prompt that speckit
provides. It's a very detailed prompt
like do this, do this, read the
research, read the plan and start
generating tasks. Right? So I told you
that I I switched all of the prompts to
use uh beads to to create uh tasks
instead of generating a long markdown
because then it gets lost when it needs
to do the work and it basically dies as
soon as it tries to read the number of
tasks it need to do and try to figure
out what it has to complete. So beats
does help me there. I do think it really
helps me to to create very focused
sessions. Um, so I asked Gemini to with
the prompt that works really well with
Sonnet and it did an absolutely horrible
job. It created every single issue with
just a title and a oneline description.
Now the idea is that you can just read
that one task and you can just go off
and do what you need to do. So that task
needs to have information about which
file to change, what to change, why to
change it, acceptance criteria,
references to the plan and the research.
I mean surely you could change to happen
>> instructed it to keep to some sort of
template.
>> Yeah. So that's the prompt, right?
That's the prompt I gave it. I gave both
Son and Okay. So basically you told it
to do it, but it it ended up just doing
one line
>> and it did one line and I was like, how
is it possible? How come Son 4.5 is so
much better than Gemini 3, which is
supposed to be like the premier model
right now from Google? And of course,
we're talking about skills, you can
guess, I have a beat skill that I
completely forgot about. So even though
my prompt was identical and it only
showed like use beats, create a task
like this.
>> Oh, so that you're saying that the beads
the beads prompt messed up the the
process or something or what? What I'm
saying is that son has an agent skill
capability where it like progressively
learns how to use beats and my prompt
was identical but behind the scenes
within the shell environment or however
the Claude Code is built because it's
closed source somehow son creates very
detailed issues and maybe I can share my
screen actually.
>> Yeah, sure. I'm curious.
>> Okay, so basically Claude came up
Claude won because of of beads and
knowing the stuff. Okay. But that still
doesn't explain why Gemini did these
oneline things, but anyway.
>> Okay, I can I can explain it. I actually
sent you a detailed video about
>> Oh, yeah. Actually, I haven't read your
chat very closely. Let me just read it
now.
>> Uh,
phone a farmer.
That's uh apparently a Belgian dude that
works for Austra that um is one of the
like original Gatsby developers and he's
one of the early
he's Belgian and he he he's like the
first far farmer founder
members of of Mastra and Master is a
really interesting um TypeScript only
framework to build agent workflows. So,
it's a really cool uh framework that I
that I did a conference talk about uh
back in March. So, you can
>> Oh, by the way, are you going to be in
Belgium soon because there's a there's a
good conference at the end of the year
or the new year.
>> I I I was telling so so I went to
Singapore three times and I went to
Koala Lumpur and I was telling my wife,
let me up for conferences and I was
telling I'm going to Belgium in in
January or um and then she was look the
look she gave me I I knew there was no
way. [laughter]
>> [ __ ] Okay. When was the last time you
were in Belgium then?
>> Almost. I don't know. Um I can here.
This is Beats. [laughter]
>> Okay. Let's
>> distracting me. Okay. So So this
>> Yeah. So this is a a cheap Beats UI.
It's pretty cool actually.
>> Was this your UI cuz you one?
>> I started coding.
>> Um this one. So there's a discussion on
Beats where they talked about different
UIs. It's out of scope for the project
because they don't want to pollute it.
It's a beats is meant to be a CLI text
focused tool for agents to use. UI is
out of scope and it's hard to maintain
and puts with maintenance. So they
completely separate the beats UIs.
Therefore, there's quite a few of them.
>> This one is actually quite actively
maintained. I've
>> Yeah. Anyway, um so it keeps track of
all of the issues. It works directly
with the beat socket, so it's pretty
fast. Um the way I've instructed it to
use beats in my task generation is to
create epics for my big features and
then to create sub like features for the
user stories. So I get these nice flows
of like these are my two features I need
that I that I was working on and and
here I have like the different phases.
So this is what opus generated.
>> No, this is what Gemini generated. I
mean, if we're going to compare that to
this, which is way more in line with the
actual prompt, which is like user story
one, user story 2, and and when you look
at them, you have the graph and
dependencies.
>> Oh, man. This makes me this is so much
faster than Jira. I could cry.
>> Yeah. Yeah. [laughter]
>> I mean, Jira is so bad.
>> Yes. So, so this is one of the use the
the story that um the the user story
captured by Gemini, right? oneline
description automatically generate and
then in the dependencies are all the
tasks that belong to this user story. So
one of these like for example this one
was generated by Gemini which uh in the
overview we can see it's just like oh
add this um Golang packet library to to
automatically infer this the JSON schema
from JSON data sample
>> uh add this into a new infer service
into internal. It doesn't even say like
it's part of the grid API because this
repository has many different things.
So, so this is absolutely disgusting. If
I want to give this to Haiku, it's going
to completely it's going to go bonkers
and and completely mess up my code,
right? Um,
>> so this is one of created and then this
I show you the other one now created by
um
by like this one for example is created
by Son. Oh no, this one is
>> you need you need to you need to add a
label that is created by um
No.
>> Gemini or so many or um where is it?
>> Yeah. Can you can you alter your
integration skill?
>> Make sure you label which which model
put this thing in.
>> That's actually not a bad idea because
there's the author part here and I can
see sometimes cloud puts itself as the
assigne and that actually keeps it like
nice to see. Um but integrate inference
into state upload update proto
definitions update goal schema is that
okay this is so in the video I sent you
I cut out all of the me searching which
one was created by which model uh and
it's really neat like the difference but
here you can see the difference right
this is how I want an issue to look like
which is update the go SDK schema uh
source field um so that it lives here
right um this is the description then
this is another flag that beat says to
design right this is the changes you
need to make there's some notes um which
hasn't had any notes and then there's
the acceptance criteria output key um is
like this this is the the final result
and this is kind of like what I want
like the big planner model opus
generates this type of design and then I
want to give it to like a very fast and
dumb model and if they are not if they
are not going outside and too creative
then hiku normally would end up with
exactly to the spec and then I would be
able to like everything would perfectly
Cool. Yeah, that makes a lot of sense.
That's a really good optimization. So,
just to replay my understanding, you you
get Opus to plan everything, put
everything in beads as as task and then
you then you spin out haiku agents to
implement it.
>> Yeah. So, beads beats allows models to
focus on the tasks and then you can kill
a session and you can just spin up a new
one, tell it to read this is your work.
You need this is your scope right here
and then it goes off and does that
thing, right? Right. And it keeps
>> cool. So, how do you
>> So, how come Gemini was so horrible at
it, right?
>> Yeah.
>> Um, so in anti-gravity, this was
actually the output from Gemini. Um,
>> it's just anti-gravity right here.
>> Yeah, this is anti-gravity.
>> So, it's just exactly like VS Code, I
guess.
>> Yeah, they're all VS Code clones. They
>> um
>> I your animal I can't believe you have
mini map.
>> That That's how I judge people.
>> Mini Map is great. I use it all the time
to scroll. Whenever I see an error, I
just click on it and then I'm right
there. I see it. It's really nice the
color coding of it. Yeah, I really like
it.
>> Okay. Um I don't like mini.
>> So this is the speckit. This is the
specket task um that you you need to
give in anti-gravity. Again, you see how
many stupid folders I have. I have
Gemini, I have codeex, I have cloud, and
now I have to add another one for
agent/workflows because that's what
anti-gravity uses. And they're all the
same. They're all exactly the same. Now
the problem is here when I say like this
is how you use CLI calls. This is how
you create a feature an epic right? This
is how you create a user story. Um and
you add the specs like this and this is
the face.
>> Okay. So you're templating a little bit.
>> Yeah. But as you can see the only thing
I say is you give it a title, you give
it a description and nothing else. And I
was like huh. So maybe this is my
problem. How did I have a difference
with what I give son? And I go to son
and it's exactly the same. You just give
it a description and there's no
information about giving it design notes
or acceptance criteria, right? So,
>> okay. So, it must have pulled out some
context from your from your user profile
or something.
>> Yeah. So, so you put
you put this uh under my home directory.
I don't know if you can see it in the
terminal.
>> Yeah. Yeah. Claude. Yeah. I'm I'm always
on on I'd like to keep everything on a
project level. I hate to put things in
my home directory. I absolutely agree
with you because I want to check this
into to git so if anyone pulls it down
they can get the exact same experience
right uh but anyway this is how
apparently when I installed this is how
skills worked I don't know if right now
you can do it at the project level for
skills but issue creation is here
>> nice
>> and issue creation is very detailed
>> design fields acceptance criteria
>> yeah this is so good
>> oh my god I did I completely forgot
about it right and and and So I I didn't
even know like that.
>> So mystery solved, right? So be so
Gemini is not that bad.
>> You can't be blamed.
>> So at that point I think Gemini my first
intuition was like Gemini is really good
at following the instructions because
another thing that's different between
the two is that here they are perfectly
like this. Use a story colon and then
the name. But I also just noticed now
when we're talking about it here it's
like phase 2 phase B to B. Why is it so
different?
um why are they not like following the
same? Because if I look at at um any
other issues in the um epics in the
back. So this is some problem with this
this uh this this UI. I can't go back to
the old epics to show you how they
looked. Once they're closed, they're not
here anymore. Um also I can't show you
the comments. Um so on a lot of these I
solved it by letting Sony add comments
with missing design documents. Uh but
yeah, so now I just noticed as well that
I actually give it the sample is like
this, right? It's like face setup.
Actually, it is like this story US1.
>> Yeah. Yeah.
>> So, it does put the story on it, but
Yeah.
>> Well, this is cool. I [snorts]
>> Oh, man. I I just love that that I need
to sync my work Jira into Beads or
something. I did see someone developed a
plugin or something. So,
>> I'm I'm like I think 10 versions behind
Beats by now, right? Because this thing
moves so fast. Um, and then when I look
at the change logs, it's like um new
feature, then two two commits later or
new release with fix and revert. And
then I see commits like um
fix uh like corrupt data issue. And I
was like, okay, I don't want to update
[laughter] cuz it's like it works.
It's moving too fast. I don't want to
like spend a day to move to a new
version. And then also I have to make
sure my skills updated. I have to make
sure my templates are updated with the
right. It's such a cool time right now.
So like I wanted to maybe just touch on
um
on I mean we we are early adopters.
Is the rest of I was just curious to ask
if the rest of your company or your your
team is is using the same stuff as you
or or it's not not not yet really.
>> So this is where we talk. So last time
you one thing you mentioned to me was
Steve Jagger's talk at the summit where
he mentioned about people that do not
adopt AI significantly have a
disadvantage and there's a big problem
in open AI where um they don't know how
to do the reviews because they cannot
compare somebody that does use AI
against somebody who does not use AI and
you hate and I watch
>> I don't know if you noticed that that
Claude also published this like paper on
productivity gains
>> you shared it with me as well. Yeah,
>> it was I think it was quite a big number
like speeds up individual tasks by 80%.
I mean that's
>> the thing is one of the first lines in
the in this paper is like we asked Sonnet
to analyze and Sony came up with these
hypothetical [laughter] goals and I'm
like
>> we as we asked Sonnet to analyze the
productivity gains of Sonnet.
>> Yeah. I mean like one of the things I
always get from son is like it's going
to take 3 4 days to build this thing or
one of like two weeks and so on. Um
>> well you asked it to estimate.
>> No I don't ask it every single time you
ask it to plan. May maybe it's in my
spec kit prompts again right I'm using
these tools I don't know
>> okay your context is polluted then more
than mine at least.
>> Well it's it's been effective. Um but
but it's kind of funny because another
thing that happens since we spoke is
the research preview of Claude Code web
ran out and on on Sunday, right? So I
had like we spoke maybe on Tuesday and I
had like a week to and I only I was
literally rushing. So in to maximize the
use. So usually I use specit right I I
build out these big plans. So, I went to
the Claude Code web research preview and
on the mobile app while I'm at, you
know, um a restaurant to watch the F1 uh
Las Vegas. I was on my phone uh specking
out a feature for my project in cloud
code uh web uh or actually on the on the
mobile app and and and and there's no
specit there's no beats in that
environment, right? It's a very sandbox.
So how how the hell were you doing that?
it kind of what I wanted to to say was
like without specit without all of this
like what you just said context polluted
things just give the tool in its like
natural environment and I think that
they were triing opus in this
environment too to be honest um they
they tell everyone here's $250 give it
some task we control the environment and
we'll see how son and you know how they
do trials right so so it gave me really
good results um so Then
I wanted to adopt this one because this
one is quite significant right I want to
basically what I'm working on is on the
terraform state management whenever you
do terapform apply and you have some
outputs I want to capture a schema
around those outputs so it will it will
store a JSON schema so if you're
building some other infrastructure
somewhere else you can pull down that
JSON schema from the other state and you
can code against that because that's
what you can expect that the outputs
will look like and and it will also
change right if if the if the output
schema changes it will it will flag it
it will say like yeah the values change
you need to replan but not only that the
schema changed so you may may have like
complete breakage when you consume these
outputs right so so it's a quite a big
features with a lot of um capabilities
in there so the initial thing that I
asked while I was in in uh watching F1 I
asked chat GPT like I've got this API um
I want to do like I want people to be
able to register the shape of those
outputs. Um, my primary library is in
Typescript and they have these
definition files and TypeScript is a
great language to defining schemas. Um,
however, not everyone is using
TypeScript. Um, and it would be annoying
that you have to define a TypeScript
interface. Um, although I like the the
way of writing TypeScript is way more
nicer than than defining JSON schema in
my opinion. Uh, but I thought maybe to
keep it simple, TypeScript can easily go
to JSON schema. uh maybe there's some
fidelity loss and not exactly the same
but you can go between them. So I
thought like JSON schema seems like the
best option to go across languages cuz
then you can have JSON schema supporting
everything Python um and everything. So
a little bit a little back and forth
between me and open eye which is really
more like a rubber duck at that point
right he just says yeah sure go with
Jason
>> great you are great [laughter]
>> absolutely also shadow conspiracy
anyway um so so so that was the first
step right uh identifying how uh like
what how do we capture the schema um and
this this whole inference thing wasn't a
thing I was just like I want to give
people the ability to just post like an
endpoint to post what the output shape
will be like and and then we can look at
like validating it right so but there's
I was thinking to myself this is so
complicated for 100% there's 100 like
edge cases and scenarios that I'm not
taking into account but I think that's
what's great about Sonnet um and this is
one of the things that I shared in a
video with you as well is that I ask son
to do these things in a web console you
know using my phone with a very basic
description um and it gives me prompting
which kind of works. Um but then what
happened is when I got home I was like
it only implemented half of it. For
example, it didn't do the schema
validation part. It it it give the
endpoint for people to post the schema
but it didn't give an like ability to
when you do Terraform apply it doesn't
track if the schema validation is is is
still valid for those outputs. So I was
like oh that's missing. Um and so I
decided that I wanted to adopt this
branch created by Claude Code with
specit. So I I I wrote a special prompt
like and modify all the scripts which is
like this this branch was created
outside of specit
look at all of the existing work that
has been done and then create a a
feature like description and then also I
asked Opus to review it from like an
infrastructure um you know consultant
engineer uh what are the day two
operations that you know may be issues.
For example, we we designed the feature
to track the JSON schema per state per
output. But I'm like, "Yeah, but the
network goes from defaf to prod. So the
schema should be the same. So why do I
attach the schema to the state output?"
And um and he's like, "Yeah, absolutely.
But I [laughter]
I'm always like the devil's advocate.
I'm going to tell it we should make it
global." And he was like,
>> "Yeah." Okay. So just just to summarize
what you were saying to me is that
>> is that
>> you were you watching F1 you're on
claude code mobile you you you developed
some ideas opus or son shows that it's
much better
>> and then you were able to adopt it in
your your spec kit flow that your b your
mobile session effectively right
>> yeah yeah yeah um I was able to clone
the branch adopt it with the speech
workflow that goes in through detailed
user story and edge case analysis and
then identifies what was done and wasn't
isn't done high priority to
>> I mean I'm trying I'm trying to find
reasons not to use my mobile but yeah
okay awesome awesome
>> yeah um but yeah
>> I wanted to quickly ask since you you're
showing the beads UI here
like
one thing
one thing I don't have good visibility
of which is which is of course why
project managers love to look at Jira or
um Pivotal or whatever the those things
are called. How quickly are you munching
through these tasks? Do you have a
velocity or something like that? Like
how do you measure?
>> I really like beats in the way that it
tries to capture it tries to be a local
Jira. It gives you stats. It gives you
rundown. And it gives you a chart.
There's a whole bunch. Yeah, there is BT
stats. Yes, it it tells you how long is
initial.
>> Can Can you show me some stats?
>> Yeah, but like like I said, I've used
Claude Code web for a while, so Oh, it's
not going to be here. Hold on.
>> Oh, yeah. I think I'd noticed the BD
stats command, but I'm
>> There is a BD stat using it in anger
yet. [snorts]
>> Oh, I I need to get my prompt to do a
carriage return like yours.
>> Yeah, there's the average lead time.
What does what does that mean? That
means it takes uh 60 hours for something
to get done or what does that mean?
>> Yeah. Um don't ask me because I don't
use these. But your question was are
there stats and there are um there are
related to to hours and time because it
keeps track of when an issue was created
and when an issue was closed um and
things like that. Um it has quite a lot
of things like what you would expect.
This is why when I was talking to you
about building a UI for this is like I
kind of was inspired by Trello, right?
Because you have these little cards, you
have the labels, you have the comments,
you have
>> what?
>> Travel.
>> Inspired by
>> Trello. Trello. You mean Trello, right?
>> Trello that was acquired.
>> Yeah. Like a camb board or whatever.
Yeah.
>> Yeah. Which was acquired by Atlassian
unfortunately.
>> Oh yeah.
>> But that that's super cool. Um, I
basically at the back of my mind during
our conversation, I'm I'm I'm just
thinking about the the human aspect
because I feel supercharged by AI. Um,
but I'm just thinking like, you know, in
in a corporate environment,
how does this work with team members?
How do you
>> Yeah.
>> rationalize the productivity? How do you
train people? All these things are just
floating in my mind right now. That's
what you were asking earlier and I
diverted to the talk from Steve Yagi
about um about how
>> Oh yeah, like senior staff engineers. I
love that.
>> And what happened was the craftsman were
doing the same thing our staff engineers
are doing today.
>> No,
that's word for word, right? That's what
they say.
>> I love that.
>> Yeah. Yeah. Yeah. And then he say word
for word.
>> Word for word, by the way. And the whole
room left. But then uh Jean Kim also
talked about you know the Dora reports
and and the work they do there where
they um identified and significant I
think the Dora report did say that there
wasn't like a performance improvement
but one of the things that he
highlighted and I think was very
relevant to the talk that we had last
time which is that AI usage is a skill
that you become more confident in the
more you use it. Yeah,
>> right.
>> When you never used it, you're going to
use an LLM. You're not really familiar
with how to prompt it. It's going to
give you something that looks good, but
when you poke through it, it's actually
garbage and you're immedately going to
use the trust,
>> but you don't know how to validate it.
>> It definitely rewards people who spend
time using it. Like I I was going to say
that like what you were doing by trying
different models,
>> by using work trees to try the things
that that Theo is doing, like the more
you do that, the more you you almost
resurrect an old project and just try
again.
>> I really think it's
>> the more you're going to learn, right?
The more investment in time. It's it's
really changing the workflow of a
software engineer.
>> Um
crazy
>> way more towards review part which I
always thought was a great part of it
though. Um I had this
>> well you say review but I'm I think more
like it's more of like an orchestration
thing. I feel like it's a pipeline like
you got to go back or you
>> it's both of them
>> both you need to know how to prompt and
feed the information and keep the
context st so that it can be focused and
delivers a task. Although that the tools
will more and more be better at that
because right now Opus is doing a lot of
things like automatically compacting
tool finding the tools relevant and
executing them correctly which is less
and less on you and more on the tool
right but on the other side so you need
to orchestrate it but you need to be
able to capture the requirements
fine-tune them and um you know review
basically review what the model is
telling you are the usea user stories
reviewing what the edge cases are
scoping it down making sure that is like
that's acceptable for what I'm trying to
deliver um you know talking to your um
stakeholders to make sure that we are
okay with like it working this way and
not doing that part cuz that's going to
be and that's one thing I really liked
about Opus cuz after I had it like
generate all of the things I I give it a
I should have saved the prompt which is
like okay look look for sentences that
look simple on paper but have a huge
implication in terms of complexity that
they add and I give an example which is
like one of the sentences that the model
loves to loves to put in these
requirements is like make sure that the
user that the error messages are very
detailed about what's going wrong and
that the user knows like how to react
and and work on the error message,
right? It sounds great. Yeah, actually
sounds great. And then you get to the
implementation part or you do a review
and then I use another model to do the
review and it keeps tripping up on this.
It's like this is not clear like what
does it mean? What is what is an
actionable error message? and and it
goes crazy and then you have to like and
and and I basically asked it and in this
case, right, this is this is kind of MVP
and I'm okay with the with the error
message just being a log in the server
and and just being like a red bar basic
uh feedback that's didn't work. Um and
and then Opus went through all of the
specs and immediately highlighted like
five six cases where this is kind of
like overkill for an MVP. this is kind
of like this might be really have big
implications uh in terms of like yeah I
I love that one um you know because
that's my biggest problem like scoping
down these models um so that's the first
part right being able to to to control
the context keep the
>> I still don't actually understand how
you can bugger off for for five you know
like weeks ago I think you maybe
exaggerating like you just get some some
your your tasks together and then you go
to the gym for 5 hours and then you come
back and the work is done like I mean
that doesn't how that doesn't sound
exaggerating uh that was about one
feature which is adding labels to
terraform state and then giving a query
engine to be able to query the labels
right and I spent two days on going
through the specs and the user stories
and selecting the Golang library that
allowed you to build expressions for
those labels and and it's kind of a
small scope compare that to building a
whole authentication system with OIDC
and and and keep
Right. So, so I kind of didn't realize
how how very scoped and narrow that
particular story that I gave you was
because I I felt like
>> I guess
>> I I I got a I get a I got a little bit
obsessed. Well, I I was captivated by by
this whole idea that you can walk away
from the computer and it's doing work
for you while you're asleep or
something. But I think in practice that
doesn't is not really the case because
you know models and agents are getting
faster and faster. It's it's all the
block. It's not it's there's no real
optimization there. The optimization is
purely in your prompting, your workflow,
your review.
>> Yeah. So, so I I I do always tend to
walk away when I see that the model has
kind of kicked off in a loop and is kind
of working on things and I don't want to
sit here and just watch it at that point
when I'm kind of sure that it's it's
okay. So, I I walk away from my machine
a lot more and I decide that I'm going
to go to the gym and I'm gone for an
hour and I come back. I don't know how
long it ran, right? But it pretty much
got to a state where it was like, "Oh,
I'm approaching context limits. Let me
summarize and and see what I've done."
Um, and things like that. So,
>> I mean, you must go also yolo because
when I when I was running that agent
thing, it kept on asking me for prompts,
you know, all throughout the session.
Annoyingly,
>> it permission. uh
>> modals, right?
>> So funny, I never had it delete my home
user directory. I've never had it like
get hard reset anything. I don't give it
access to any databases. Um so I've
haven't had those issues.
>> But I've noticed that the CLI prompts
now like when I go into codeex and you
go to approvals and you go to like give
it full access in codeex before it was
like oh okay you give me full access
great. Now it goes like are you sure it
may completely destroy your machine? Are
you sure you want to do this? I'm like,
whoa, whoa, you're scaring me now.
Before it was just okay, but I guess
>> go going back to Jean Kim, I just
noticed I have a book by by Jean Kim
right next to me. This the Accelerate
book. Accelerate.
>> I don't know if you read that one. I
mean, Jean Kim is a a legend. So, so
just just to repeat what you said that
Jean Kim said, to be honest, I didn't
watch Jean Kim's presentation. So he's
saying that the more time
>> people like us spend spend with working
out working on these things, the more
>> the more they trust it.
>> Yeah. Well, it's not so much trust. It's
more it's the more
>> he says trust
>> comp competent we get too I suppose
>> I guess but it's more also about like
letting like trusting what it's doing.
It's okay knowing where to look for what
you know are like the areas where it
will mess up.
>> Yeah. I guess trust trust is a bit is a
bit wishy-washy, isn't it? It's not it's
a bit
>> I think it's a great word because
because if you don't trust something,
>> you know the the quote trust but verify,
but if you if you don't trust something,
you're going to really go into very deep
details verifying like every single line
of code. Um whereas at this point I'm
like I'm verifying the integration
tests. I have a couple of layering rules
that I define like do not create a
direct dependency between the repository
and the uh HTTP handler. Make sure that
you go through a service layer and the
service layer is used by HTTP because
we're using the services in many areas
not just the handlers and things like
that. So, so more and like those
layering type of things and those scary
refactors to be honest because like I
know that unit tests were working but
then now I suddenly have to completely
break down all the folders and and do
all of the logic all over the place.
>> Oh, sorry. My my wife has just messaged
me. I need to to help her out with the
kids.
>> Okay.
>> I'm sorry. I'm sorry. Emergency.
Emergency.