Episode 15: Scaling AskUserQuestionTool

Published: Saturday, Jan 3, 2026 • Duration: 68 minutes • Season 1

Scaling AskUserQuestionTool

Download MP3 | Watch on YouTube

Vibe coded Chrome extension to show Amazon UK Data First Available: https://github.com/kaihendry/date-first-available

My failed thought experiment to get AI to setup my PI https://github.com/kaihendry/ai-pi

My blog about how I got my Pi setup as a VPS host: https://dabase.com/blog/2026/machinectl-trixie-pi/

Markdown to Confluence POC https://github.com/kaihendry/md-confluence

https://x.com/trq212/status/2005315275026260309 promotes the AskUserQuestionTool idea

Kiro seminar (PAID) https://www.eventbrite.com/e/hands-on-spec-driven-development-workshop-tickets-1812679024759?aff=so%E2%80%A6

https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16dd04

Watch on YouTube

summarize "https://youtu.be/V5Q8_3BuMcs" --timestamps --slides

A long, informal conversation between two engineers about experiments with AI-driven development and lightweight infra. They demo a small Chrome extension and cloud coding workflows, describe a Raspberry Pi used to spin up many tiny VPSes with public IPs, and debate reproducibility, testing and how to fold AI into spec-driven development for teams. Key concrete details: the Pi hosts a subnet of about 64 public IPs (roughly 62 usable), the Pi has ~4 GB of RAM, a guest tool called Gastown can cost three subscriptions at about $200 each ($600/month), and the speakers repeatedly stress the need for integration tests and docs-first validation. Two lines stand out as representative: “Build reliable infrastructure with confidence.” and “you don’t create mock-ups or you don’t come with a design specification with AI. You come with an MVP.” Slide 1

Chrome extension and local AI tooling

They built a tiny Chrome extension to scrape Amazon details and experimented with multiple LLM front-ends (Claude, Gemini, Conductor) and “cloud code” sessions. The extension turned into more generated files than expected — roughly dozens of supporting files for a small JS feature — and the speakers note that AI scaffolding can feel bureaucratic compared with the 100 lines of core code. The session stores screenshots and artifacts (cloud code keeps images in the home directory), and they discovered a non-programmatic shortcut: Amazon’s on-page AI assistant (“Rufus”) can answer product-age questions directly, which reduces the need for the extension for that single task. The speakers also show how AI assistants can hallucinate small CLI aliases (typos like “BD comment ad” vs “BD comment add”) and stress that hallucinations are useful design signals when handled cautiously.

Slide 2

Raspberry Pi as a tiny VPS host

One participant uses a Raspberry Pi to bootstrap many isolated VPS-like containers using systemd-nspawn (a systemd container primitive). He maps an entire public subnet to the Pi so each nspawn instance can “own” its public IP; he reports having 64 total IPs from his ISP (about 62 usable). The Pi is modest (about 4 GB of RAM) but considered “fast enough” for many small experiments; each container usually runs very little (an SSHD and a setup script). The setup flow pulls SSH keys from GitHub to populate authorized_keys, disables password auth, and then the container can run Docker or other services. They discuss socket-activation and service activation patterns (systemd features) so containers can be dormant until traffic arrives, and compare this lightweight approach to serverless and Firecracker-style micro-VM isolation while flagging that container isolation still has security tradeoffs.

Slide 3

Agents, beats, hallucinations and Gastown costs

They discuss “beats” (an agent/task markup workflow) and its integration with cloud sessions; agents sometimes mis-invoke commands due to minor token or alias mismatches, which exposes brittle UX edges in agent tooling. The guest project Gastown is invoked as an example of a powerful but costly platform: the speaker cites running three cloud subscriptions around $200 each, implying roughly $600/month in baseline cost for his Gastown setup. They emphasize that LLMs often fail in human-like ways and that hallucinations should be treated as test cases or cues to improve the spec/UX. The conversation also touches on the idea that when an agent fails repeatedly at a decision point, that signals a design ambiguity that humans likely share.

Slide 4

Docs-first, Markdown, and enterprise friction

A recurring theme: treat the Git repository as the source of truth and start with text/Markdown specs so AI can consume and iterate them. They contrast this with enterprises that still rely on multiple doc silos (Confluence, Google Docs, Word, Notion), which complicates an AI-in-the-loop workflow. The speakers argue for a flow where design artifacts (mockups, specs) are plain text that feed LLM-driven generation, then integrate back into repo docs and tests. They acknowledge cultural hurdles: many stakeholders don’t use Markdown or repositories, so adoption requires bridging tools and automation that sync repo docs into enterprise doc platforms and keep stakeholders in the loop without forcing them into developer tools.

Slide 5

Coordination: “ask user” tool, planning and context limits

They sketch an “ask user” or “ask group” pattern: an automated prompt flow that polls stakeholders with simple choices to speed alignment, akin to planning-poker or majority polling for decisions. Open-source tooling (OpenCode and terminal-sharing tools like Warp) are discussed as integration points; OpenCode maintainers are adding plugin hooks so interactive flows like ask-user can run across different front ends rather than being terminal-only. The speakers reference Human Layers and the “harness engineering” talk about context window management and the productivity cliff: when only ~30–40% of the context window remains free, work becomes non-productive. They emphasize lightweight voting and alignment UX (unanimous vs majority), and the practical need for a loop that surfaces AI output to stakeholders quickly.

Slide 6

MVP-first development, tests and maintainability

Their recommended workflow: identify a minimal viable product (MVP) from the spec, let agents implement it, then validate via integration/end-to-end tests. Playwright (test harness) and integration tests are critical validation artifacts that must run reliably and be separated from a single ephemeral workspace; one speaker caught a design bug where server-side code read files from the same workspace rather than from a separate server workspace, so they refactored tests to ensure server and client run in isolated folders. They argue that good docs + integration tests are the guardrails that make agent-driven changes safe: if integration tests fail, you know the docs/tests disagree with the implementation. They accept that full enterprise systems remain hard to spin up for tests, so automation and environment bootstrapping are prerequisites for safe AI-driven workflows.

Model: openai/gpt-5-mini

Transcript (auto-generated from YouTube captions)
Merry Christmas.
>> Merry Christmas. Happy New Year.
>> Happy New Year.
>> So, the over the Well, okay. I got I got
Claude installed here. The The two
things that I worked on was I might have
worked on this
>> in the top one. It was the Pi.
>> Yeah. But there's also this um I think I
mentioned this to you before. I don't I
sort of vibe coded this not even with
beads I think with Gemini and and
conductor.
>> Oh, nice. Yeah. Yeah. I saw you tried
out conductor.
>> I'm very excited to hear what
>> I don't know. I think
I thought it did a really good job like
all the stuff here. Um
>> have you gone through the whole specket
as well just as an
>> or have you only tried? I think I've
tried it, but like I I felt I felt the
conductor experience was a little bit
more streamlined. Yeah.
>> To be honest, I don't know exactly what
these archives. I think tracks are I
think you mentioned it before.
>> It's it's the it's like the specs
basically.
>> Yeah. So, I think I think if you looked
in here, you can do some archeology
archology. Archology
and you can work out what I did. But I
thought it was quite a smooth process. M
>> of course the I mean is what this thing
is is just a a very simple uh
uh
Chrome extension. It did seem like but I
guess this is maybe normal. You know it
did seem like we're generating 50 files
to do like one tiny little piece of
JavaScript. Um but you know I guess
that's just normal. I mean the I mean
what and God knows how many tokens. It's
like it's basically 100 lines of code
for basically a lot of a lot of uh
bureaucracy in some ways. Do you
>> have Yeah, these systems like conductor
>> it's saving my screenshots. Ah, that's
really cool cuz I was going like ah this
is not working.
>> Oh, it has a Oh, wow. It stores. Okay,
so Claude Code does the same thing but it
shows it stores your images in your home
directory.
H
so
I don't know if it ever yeah see so the
end result here it says age four months
ago but the problem is that is that
Amazon is like sometimes having this
information sometimes isn't but there is
a I did figure out there's a shorefire
way of figuring out the the age and
that's to load that that AI Roffus
>> and then you ask the AI Roffus what how
old is the product and it will tell you
so It's almost like the whole extension
doesn't need to exist. You could
>> cuz you see it right there at the top of
the bar.
>> You could just uh talk to Rufus, but
unfortunately that's not programmatic.
>> Oh, so wait, Rufus is an Amazon thing.
That's always
>> there like an AI thing.
>> Okay. And then the other thing I wanted
to show you was that yeah I so this was
an I think this is actually I told you
the thought experiment would be instead
of using anible to to basically
administer this thing which it was
before uh
it was some anible crap and then I
thought to myself hey I'm not going to
use anible I'm going to use um
uh claude to admin to set this thing up
and it did work. But the trouble is
again is that it's quite a simple setup
and I almost think there's more markdown
than there was there was there's more
markdown now than there was in uh in
anible YAML
[laughter]
and I'm just thinking to myself and then
of course if I had to set this up from
scratch which I'm don't really have the
u the uh the goal the I I don't really
have the u the time to do it to SC to to
set up app from scratch using this
readme. I don't think it will be able to
do it. I mean I'm not confident it will
be able to do it but but but if I used
anible
then I know it would there would be some
you know reproducibility there and I I
know there would be some some tests
there. So so basically I learned a
lesson that that you know things like
anible are definitely not going away.
uh you need the anible to sort of
uh just just to run reproducibly and
then you you use AI you can't use AI to
administer and set up a machine you
because it's just not reproducible. I
mean it's it's obvious in retrospect but
I got I have to remind myself what
people say is like the ELLMs are the new
click ops. They're doing things and they
you know are not reproducible. But I see
you use beats for this particular task.
Did you need more than one session? Did
it require a big context that you needed
to use beats?
>> It did. It did, but I got a bit
sidelined with things. It's It's almost
like being a human with the AI.
Well, I got sidelined because there was
there was a networking issue. Um
>> Yeah, that can really mess up.
>> And the trouble the trouble is the
networking issue wasn't really down to
the the Pi. it was down to my ubiquity
setup and my ubiquity setup is a bit
complicated.
Um, so basically it was it was fighting
it all its assumptions about how the
network should work were not really
working very well because I had a I had
a screw up in my ubiquity thing and and
it was just crazy. So that's kind of
dumb. But I I wanted to show you that my
home setup. I think you might be
marginally impressed by this.
Uh let me just stop sharing. I'm so good
at this. Here we go.
>> Yeah.
>> Okay.
>> I need I need to install Ghosty on my
Mac. I use it on my Arch, by the way,
but I don't use it on my Mac.
>> Okay. So, this is where you might be
impressed.
Have you ever played with uh machinectl
or systemd uh endspawn?
>> Endspawn. All right. Um only through
rocket from core o. So that's a very
long time.
>> I mean I could I could get these I could
get it running a little bit quicker than
I am doing now. But like for example
I have an IP range pointing to my Pi. So
this is why the networking is a bit
confusing.
>> So you make like a flat network. you you
give them you allocate the range to the
pi so that the containers
>> so basically basically I can I can make
this a little bit faster but I'm just
debooting debootstrapping from absolute
scratch as you can see but in the matter
of a few seconds
I can I can basically boot up a VPS on
uh a on a static IP is that the right
expression on on a standalone IP. So
basically this VPS
um
completely owns that IP. And I've got a
I've got 64 of those IPs. Well,
actually, I think 62 are usable, but
whatever.
>> You get the point.
>> So, basically, I have I I one thing that
I was doing with this Raspberry Pi over
the weekend was essentially set up uh a
way of spinning up VPS's just quickly.
So, I don't have to use Digital Ocean. I
can just use my Pi to
>> well to
>> How much CPU and memory does a Pi have
that P?
>> Not not a lot, but it seems fast enough
for what I do. I think it's like 4 GB or
something. [laughter]
>> Okay.
>> Something tiny.
>> But this VM is like um it's like what is
it? KVM or or
>> it's it's using systemd nspawn which is
some sort of systemd thing.
>> Still containers. Yeah.
>> Yeah. So, um I don't know there's
nothing nothing much running inside
except SSHD
the the setup script as you can see here
like um I I I download the keys from
GitHub you know cuz I don't know if you
know this trick this GitHub
>> yeah I do this
>> keys I just I just set up SSH
>> I throw in my my keys into the
authorized keys
>> set up my user disable password
authentication and all that sort of
stuff And then I'm in I'm in there. Um,
and then of course what the next thing I
was going to do is that I was going to
maybe run Docker in each one. That seems
to work fine. So they're all isolated.
So they're all isolated. I can I can run
my little experiments in them. Um, I
mean I could even like create you a VPS
if you wanted one in UK.
I could become a digital ocean, start
charging people five bucks a month.
Yes,
>> but the I think the specs are pretty
bad. Like I don't know how do I see the
memory?
>> Yeah,
I got 4 GB.
>> But it's it's it's fun though. It's fun.
So I'm
>> But this IP range is on your local
network. Now when you want to allocate
public IP addresses or are you using
something like tail scale or or some
>> these IPs are public
>> also? How did you get an RP range? Uh, I
asked my ISP very nicely and they did it
for me.
>> You paid them.
>> Well,
>> it isn't. It's not cheap. Definitely not
IP. Um,
>> it's it's part of my subscription. So,
for example, this this this IP I set up
here. Uh, which one was it? I I think I
What is my host name here?
>> Uh, you want me to open it again? HTTP
8000.
Hello Vincent. Yes, I see that. Do I see
it on the curl
curl? Yes, I see H1. Hello Vincent in
curl.
>> Yeah, so you get the point. Like now I
have I have a whole range of I have a
subnet of public IPs mapped to my Pi.
>> So you can run some game servers.
>> I can do whatever you can do with 4 GB
of RAM.
Now is where you start buying pies.
And then but this is the thing about um
that I saw some people play with is
slicer VM from um Alexis Alice. I think
he he built like open fast or functions
as um
uh functions as a service. So open fast
is using docker under the hood to
basically it bootstraps your container
with an entry point and then when http
request comes in it calls your container
and sends the message
>> like systemd has all these uh things you
can you can set up systemd to activate a
service on socket or something like
that.
>> Yeah, socket activation for your
>> or some other little like knockk
knockock. It can basically spin up a
VPS.
>> So,
>> so what you're saying it can be kind of
like serverless, but not really because
Okay. So, what
>> I mean CISD is is its own set of
features and tools, but it's very
comprehensive. Um, and you can do things
like I showed you. You can you can spin
up isolated VPSs for people.
>> It's very nice. Um, yeah, like you said,
but the thing is it's not really very
strong isolation, I think, still, right?
Uh that's where people are like
>> I'm not I would have to research that. I
mean that's one of the things I find a
big of a a bit of a t isn't a taboo in
our industry but like isolation
>> is one of those things that always comes
up. Docker never really had a a very
strong security stance with their
isolation. Right. Yeah,
>> a lot of and a lot of machines basically
assume workload isolation on on a Docker
host and they're just guessing, right?
>> Um, of course, Amazon, I think, have got
a more complete story with Firecracker
or something like that.
>> Yes. So, that's the thing that's
>> still but still I think there's some
people that are not a thousand%
convinced, right?
>> Mhm. Cool. So, what I did over the
weekend, over the weekend, over the over
the holidays is I
um I did my year in your review blog
post where I had a lot of fun with with
the Claude Code. Did you did you try it?
>> Oh. Oh, like
>> Oh, that one
>> uh wrapped. Okay. Yeah. Yeah. Yeah.
>> No, no, no, no, no. I did not cloud web.
I did uh basically I have my landing
page which is pure JavaScript, JS and
CSS and HTML. No React, nothing because
I wanted maximum like speed.
>> Oh, okay. You should share it. I Or
>> no, that that that landing page I built
a while ago. Let me share the screen.
Um, hold on. I don't know how much time.
>> Yeah, I got to I got to run in like two
minutes though.
>> I think it's this one. Oops. Maybe.
Oh, [snorts] there's a lot of tabs.
Let's do like that. Okay. So, I'm going
to
So, this was my original landing page,
right? And there was no blog. Oh, wait.
I can't see anything. Am I blind?
>> What are you seeing?
>> Build reliable infrastructure with
confidence.
>> Yeah. So, that was the original landing
page without the the block. And this
landing page, I literally asked like
Codex and Gem and Gemini back in August.
I want SEO optimization. I explained
what are my keywords that I want the
ranking because I realized that my
wording was horrible. And this is what
it came up with. Um, and also like it
does really cool stuff like it uses
puppeteer to launch the browser, look at
which CSS elements are above the fault
and it inlines critical CSS. So it puts
only the CSS needed for rendering the
top part of the page and then the rest
gets loaded uh or gets loaded later. So
the top part render super fetch and then
I run all kinds of lighthouse reports
basically. lots of fun. And then I built
this thing which is like the, you know,
the whole tour thing where it highlights
different sections of the code to
explain.
>> Yeah, this I built back in September,
right? But this is all pure JavaScript.
So I I I run the synth and then I it was
really cool to build because these
things are segments. So I define a line
range and then it draws when it when it
does the I asked the codeex when you do
the highlight.js. I'm using highlight JS
here. when it renders all of the color
codes, the HTML elements like the divs
and everything that creates these
different colors for this code, it
>> creates a map of line ranges and then it
it basically creates a big div while
it's rendering everything around this
these HTML elements. So when you click
start and next, it basically this
element is already rendered within
actually look the line range is wrong
here. Um and then it it highlights that
particular div element when you you know
>> Yeah, I really I I do like that tour
idea.
>> Okay, I I need to go to my standup.
I'll have a look at your terror
constructor there. But but that
>> the block thing is on new. So I wanted
to keep this front page fast and this
whole block thing this is all react. So
this is much slower. But I actually have
a voice
So, I have like the whole listening of
this whole
>> I don't know if you hear the audio. Do
you hear the audio
>> when I press play?
>> Oh god, you went crazy, man. You went
crazy.
>> Yeah, you have like table of contents
with like um scroll tracking.
>> Yeah, I'll have a look. I have a look. I
need to run.
>> I had a lot of
>> Let's Let's catch up when I'm in a
um and let's try Yeah,
>> it took 10 days.
>> We We got We got to catch up on Steve
Jger stuff, too.
>> Yeah. I see you. You're working with
Gemini. Is that why?
>> No. No. I'm working with cloud. It has a
skill. It knows how to use beats. Yet
when I ask
>> get it wrong sometimes.
>> Yes. It's still getting it wrong. Like
there's still this
>> there was a comment
>> exhaustion. I guess
>> on one of the previous uploads to the
podcast to say as much actually that uh
that sometimes it didn't work. Let's see
if I can find the comment.
>> Yeah, there's one, right? I don't know
how many people are listening to us, but
if someone comments, that's always a
good sign, isn't it? Someone's actually
listened to us. At least some parts of
it. [laughter]
Yeah.
Um
gosh, community.
>> Yay. Did it right.
Looks better. Okay, perfect.
>> [snorts]
>> So,
did you ever have chance to play with
the Gas Town?
>> Not much. No, I've still I'm still
reading it. You know how I guess the the
real thing is you have to to try it out.
But, um,
Steve says that many warnings, be
careful. This is expensive. He runs
three cloud three Anttopic Pro
subscriptions
to run Gast Town. Each one I assume are
around $200. So it's like $600 a month
to to run Gast Town.
>> It's it's crazy. But but back to the the
thing, right, with with basically we
have a cloud skill that that's there. Um
we tell Beats um we give it information
about how to create beats issues and yet
it still somehow sometimes thinks that
when it needs to add a comment instead
of doing BD comment
add it runs BD comment add without the s
and then there's an alias beats have
both it has beat beats comments add or
an alias for comments ad is BD comment,
but there's no BD comment ad. And and
and actually it's funny because Steve
says that um you know this um picture of
when you look at the park in a city and
the the city planning decides that there
should be roads here and here, but then
actually when people walk to work and to
school and everywhere, they like walk
across the park.
>> We call them car paths. Yeah.
>> Yeah. So basic basically it's a very um
interesting design, right? where that's
>> and he he says you should treat the LLM
hallucinations as such. So if it
delucinates something, you should
implement it. So next time, you know, it
has all the options. [laughter]
>> It's crazy.
>> It one thing I find remarkable about
using AI is that it it does like behave
like a human. Like it does it trips up
in the same way. Absolutely. It's it's
crazy. It's just like
>> so people who've designed some software,
other humans must have tripped up at the
same point. So I find it like an amazing
way of now validating some a design
because if AI can't figure it out, the
humans are not going to figure it out,
right?
>> Yeah. Absolutely. Uh and I thought so
I'm doing this in a context where I
already did a significant amount of of
like research and refactoring because I
already explained some information about
how to create and use beats. Um
reexplained it and redirected it. I
thought, you know what, I don't want to
kill the the session. I want to keep
this one. and I still had like 45%
context uh left. So I thought it's going
to be fine. But maybe that's why it
started to hallucinates these things.
But it's exactly the same as me, right?
I can't remember. Oh, is it BD comments
ad or is it BD? Like I can't remember,
right? I also have to go real quick go
and look at BD help,
>> you know? I mean, right now we're at
like a golden age where like the AI is
probably as smart as a really smart
human
and I don't know few months from now
it's it we're getting to the point where
you know this mythical AGI depending how
you define it but like [snorts]
>> I I I guess my feeling right now is that
AI is going to be smarter than the
smartest human
uh pretty much Now, especially when it
gets more context, right,
>> it and it impacts the way that we work
together so much, right? I I had an off
off recording conversation with you
about how does the you asked me several
times like how do you see this scale
across a team? How do you see how do you
keep other team members that maybe do
not run Claude Code yet and and engage
them into these like processes so that
they
>> Well, it's never too late. It's never
too late. It's never too late. But but
um
I I I saw a comment on on LinkedIn just
a few hours ago which was um about using
AI's agent to work on your issues and
somebody said ah I remember it was
purely about the documentation part
saying like oh you know do you a lot of
people think that the code is the
documentation and and and basically you
should read the code and documentation
is always outdated but this is like um
you know really stupid. Imagine that you
have to work on React and there's no
proper docs, right? And then somebody
who works at Facebook Meta told said
that and you know I found that a lot of
times the documentation was outdated and
that the code actually is more accurate
than the documentation kind of this back
and forth. And I feel that the whole
idea of specri development is that you
completely inverse this, right? you you
inverse the the now you lead with
specification documents and you iterate
on the on the UX user experience through
those documents with like mockups and
all and whatever you can
>> get into a text format yeah that that
can fit into LLM workflows so the humans
you know with stakeholders in real life
you know work through these things and
maybe not just AI like not just other
human stakeholders but definitely it's
like you send an email you you said hey
JPD expand this and then the email goes
out and then somebody hey ex compress
this right so so you're inter
interacting between humans humans and
AIS are in in between everywhere um but
you start with the specifications and
basically the documentation is your
living proof and all of your integration
tests
>> are the validation of the documentation
and if the integration test fails it you
know they're onetoone with the with the
documentation and and he said well
that's like a fantasy because validation
is so hard at the moment, right? Um,
>> but I think we'll get
>> Wait a minute. I thought we I I thought
we started I thought you were talking
about specs in the sense that how do we
get the teams to sort of adopt that sort
of workflow
because like I I'm just speaking
anecdotally from working with the sort
of enterprisey clients that I work with
the thing that that that makes my my
mind panic or I I don't know how to deal
with is that is that like these
enterprises are so slow and AI is so
fast
that the cadence is completely
mismatched. So I like like how do you
how do you have these meeting how do you
coordinate um all the stakeholders in a
typical enterprise to basically iterate
at the speed of AI and spectrum
development. I I just don't even see it
happening.
[laughter]
But [clears throat] maybe I'm just being
pessimistic at this point. I'm also
trying to find because I read somewhere
someone saying basically you know I am
entrenched in what what the same thing
you said I'm entrenched in these
enterprise systems confluence and um you
know Jira and basically you I I don't
live isolated like my cloud needs to be
aware of it I need the bridge across my
>> exactly so one thing I worked on on
working hours yesterday and I don't
think uh people were impressed or they
they didn't understand what I was trying
to do perhaps even though I tried to
explain it to them. I I was just trying
out this uh workflow. So I don't know if
you know about this u there's a couple
of workflows. This Telefonica one seemed
to be the most featureful one and I
started at the Huh?
>> Telefonica.
>> Yeah.
>> I don't know this
>> Telefonica. the the biggest uh mobile
operator in Spain, I think.
>> Telefonica workflow.
>> Well, anyway, they Telefonica is the big
enterprise that that created the action
and and what happens is that you you
have your I don't know if you're looking
at my uh
>> and now I can see it. MD Confluence.
>> Yeah, you have all your your your
markdown
>> and what it does is that it ships it up
into into a particular place in your
Confluence, right? And then it adds this
automation notice. This page is synced
automatically. Changes made.
>> Yo, I was trying to do this like five
years ago. I'm so sick of confluence.
Even the markdown editing confidence
doesn't work.
>> So I was basically telling my colleagues
that like hey I'm doing this so that we
can start using markdown more which I
think I mean it's going to be such an
uphill battle cuz like I I mean I'm not
I'm exaggerating but I'm not
exaggerating in the sense that I think
some of my colleagues don't even know
what markdown is. You know what I mean?
It's like this is going to be a
struggle. [laughter]
>> That's That's pretty dire.
>> Well, actually, my answer is
>> how do they feel about YAML then? Still
XML?
>> Well, it's going to take Yeah, like
um
>> Well, they're ready for us to go full
circle. [laughter]
>> They're they're in the opportune
location for us to go full circle back
to XML.
>> Yeah, but like
>> I mean I'm I'm probably exaggerating
things. I think I think my colleagues
will can start adopting these AI
workflows and we'll see very great
results. It's just that um
>> I found it
>> I'm just I'm just trying to formulate a
plan to sort of get them using markdown,
get them get get them in just writing
down things would be amazing,
>> right? So but this is like an extension
of the the the the code is the latest
version. It's like I feel there's
different tiers because you can start in
Confluence then you can start you can
start you know create a project or
repository out of it and start writing
documents in that repository and have
the code next to those documents and
then as you change things um you can you
can assume that it trickles back up you
know from the code back to the in repo
docs back to the conference.
>> Yeah. Like somehow like I had this
discussion at infinitum at my previous
company where I was like I hated we had
like three four different document
management systems. We had Google Docs.
Well I probably introduced that one.
[laughter] We had Quip. I don't know if
you know that one. We had notion and
then we had inreo markdown.
>> And basically you work with external
people that don't have notion using
Google Docs. And then you have uh quip
where you put like short like the um you
know it's like a shared document space
and then you you um you had the original
founders of the company started with
notion and basically all of the HR
tracking documents and everything was in
notion and and so
>> that's a good way about it
>> because but yeah so in the past it was I
don't know Microsoft word or something
and then it would probably trickle into
um into into Confluence for want of a
better word.
And this is the temp the typical content
management of a of a enterprise. And
then if you're lucky if you're lucky it
would it would be yeah GitHub repose or
something like this. This is this is
like the the the the machine of the
enterprise. And what needs to happen
really to just to accommodate AI is that
well I mean this is my my current
thinking is that the GitHub repo is is
the source of truth
and then we then we go from there to
>> to to Confluence so that other people
can read the docs.
>> But then how does it get to Microsoft
Word? Um,
yeah. This is where things get a little
bit weird, don't they?
>> You still I mean, I thought you were
talking about the 70s when you No,
sorry. About the the the the '9s when
you mentioned Microsoft Word. What are
you talking about? Who is using
Microsoft Word? I thought everyone
already
>> enterprises are enterprises are like
there's people who, you know, don't code
and
>> yeah, I mean like how do you get
>> I don't understand what you're saying.
[laughter]
But this is this is the problem I have.
How do how do we get uh
to this flow essentially?
>> Yeah. So so I make a joke about this,
but this was exactly the problem I had
with my previous company cuz I kept
saying Markdown is a source of truth and
he was saying we have other people that
don't code. I guess I just don't get
that. Um and that that don't you know
access um markdown. He says like there's
a a design step before you go to the
repo.
>> Something that is
>> across multiple repos that's
so the design step is is now with AI
essentially talking to the AI and and
and the artifact is markdown or some or
plain text and that's that's what we're
familiar with.
I think those people that are still
using Microsoft Word are also those
people that are still uh going into a
chat GPT session and then ask to and
then manually copy paste out the code
that it proposes. [laughter]
>> Yeah.
>> Or like or like do a bunch of deep
research. I mean I I'm jo I shouldn't be
too
>> Yeah. Yeah. Yeah. I mean I did that
>> the people in this in this Microsoft
Word domain are are using JBT in their
web browser for sure.
>> Exactly. Right.
>> Um Yeah. like how how to in how to
include people here in the in the AI
iterative iterative process
>> iteratorative
iterative
yeah this this is this is my thought
here so I send you a link in the chat um
because for me I I think we I just I
think spec driven design and you just
turned this upside down right but I can
imagine to introduce that to enterprises
is very hard. Uh but this guy's um appar
this is this post looks like my workflow
blah blah blah but actually he's he's
one of the like it's a promotion for a
platform called core. So at the bottom
you see yes I work for core which is
like red planet HQ core. Um but core is
seem to be like a drop in to integrate
multiple platforms. He says I have to
work on tasks that are spread across
GitHub issues uh inside linear which is
I guess the modern Jira inside Slack
threads and I even get emails. So I want
to coordinate my tasks and collate my
work items, you know, determine my
priorities
>> between all these different I the
>> to me is like kill the other platforms,
>> you know, but realistic from
>> all these I mean this confluence inter
this this confluence integration in my
opinion holds very little value really
because because uh it becomes uh it
becomes read only doesn't
Yeah,
it I mean but to to be honest um a pull
request flow to make adjustments to a
document is not the same as like a
Google Docs shared session and co
collaboratively editing and proposing
changes. Like if you use Google Docs or
Quip, you can highlight a section, add a
comment, or you can go into proposal
mode and just delete a se a paragraph,
rephrase it, and then the author or the
document owner. Even if it's yourself,
but you want to keep track of the
changes that you're proposing can then
accept or can discuss, which try and do
that on the pull request. Try
>> Yeah. Yeah. Yeah. Yeah. I mean, this
GitHub repos
Yeah. that conjures up nightmares of
pull requests, which is uh
>> so something like I think has a purpose.
Something like a collaborative doc
platform. Confluence is horrible. I
think I love how Microsoft works like
the Google Docs.
>> I like what you said. So, so you can
comment,
edit,
and then that and then that feeds into
um but how do how do we get from
Confluence into uh you know,
>> don't call it Confluence, call it
documentation platform. I'm just
allergic to Atllesian. Ah,
>> okay. Just never mind.
>> Okay. Doc platform. So, how how do we
get Yeah, I don't understand how we get
from there into there.
So, it would basically be listening to
any dock changes
and then and then claw code would do a a
plan like because
Yeah. Like how is this going to work
Vincent? How is this going to work?
I built something but I don't want to
talk about it. [laughter]
>> Okay. Well, I built something that is
doc platform that u allows you to
comment on line ranges and it creates
edit batches uh as and basically it is
up to a human to to
step through an edit batch which is
basic basically
>> but we iterate through the comments. you
select which comments you want to
address and which ones you want to skip.
>> Actually, let's wind it back a bit. I'm
gonna attack the idea of of stakeholders
actually commenting and editing because
in my experience
many stakeholders and big enterprises,
they're never going to edit or comment
on a on a doc platform. I'm you know
this is this is this is like middle
management zone if that
>> I don't know if this is placed
advertisement but I think this is would
be interesting to attend the um packed
publishing workshop on specd driven
development with ko where they talk
about writing specifications
and collaborating on and and basically
it's from an AWS principal engineer and
he talks about how AWS manages these
documents
>> you understand Uh
>> I think I think that's why I would go to
that workshop because it's not just
about specdriven development but also
the aspect of how does AWS approach this
document approach and how does it flow
into a specification that then is uh
worked on in KO.
>> Yeah,
>> that that is the step that is that is
basically this right how do I get from a
doc to to Claude Code to whatever is my
um agent shell. Um,
>> yeah. Yeah, it be interesting though the
I mean I think Amazon is probably
there's definitely some people who are
doing cool things in Amazon, but it
doesn't seem to show, you know what I
mean? Like
>> Yeah. No, but like I also um
Amazon uses a dock platform before it
goes into KO, right? And
maybe these are, you know, pro processes
that have been so long in place that,
you know, you can't just overnight
replace because of AI is now in the mix.
>> And so most of the effort is like, how
do I get AI in the loop on all of these
other processes, you know, we're we're
always thinking about letting AI do the
work and keeping the human in the loop,
but in a lot of these scenarios, and I
read this from a LinkedIn post, um, you
need to look at it from how to get AI in
the loop, right? What's your question is
cloud go there is AI how do I get AI in
the loop
>> I like this loop I think is important
because even if we make integrations
here
the stakeholders really need to be here
to see what's AI is is spitting back at
them otherwise they're going to be out
of the loop right
>> uh
>> like how do how do we create a loop here
there need there needs to be a loop uh
as you as you said
you know what I'm saying like you we
might form this hero or whatever.
We might be able to connect the dots,
but if the if if the loop isn't uh you
know, if if this isn't going if this if
this loop isn't isn't like super fast,
then the whole thing breaks down. This
is why, you know, claw code in a
terminal is such a fantastic experience
is because you you're there in with the
code
and uh you're looping so so so quickly
with each person of the key.
But doesn't that go offline? Like
I I think it has a it has a benefit to
have cloud in at the meeting table
saying like oh by the way guys think
about these different ways that you
could address this. Um but doesn't it go
go from we have a discussion either on a
document either in a in a in a call or
in a meeting room and we based on that
discussion we have highlighted sections
that need to we rework and then somebody
takes that offline basically with cloud
code and and iterates through some of
these proposals and makes decisions and
then those reisions result in in changes
to the document and a new version of the
document is published. Um then overview
it.
>> Yes. And you say that's too slow because
you get a very fast feedback loop with
Claude Code. But I feel it is not
productive to have everyone at the table
for those individual iterations.
>> Okay. So so basically let me just
now I'm really talking about what I've
built. [laughter]
>> So essentially
>> because I spent one week like se five
days on on on this on this idea.
So the so basically it goes back into
the document.
Um
>> I think you you you can just the the
area above arrow above can go back into
the document
>> anyway. No. Yeah.
>> And then it goes
Yeah. I still think this is going to be
too slow.
>> Yeah. [clears throat]
[snorts] or just
I mean there is a there's surely sign
some scientific uh
>> too slow compared to what to reality to
today.
>> Well like
>> because today it's way slower. There's a
there's a law of you know the more
people you add to a project the more
communication overhead you have and I
feel I feel like you know even though I
put one person here in reality there's a
whole team there could be like 10 people
here that we have to like
reach alignment with and the
stakeholders there could be another
another you know three or four
stakeholders or whatever
there could be
the this.
Yeah, just just reaching alignment is
going to be so challenging here.
>> There was a human caterpillar. Oh, what?
The human uh
for a while. [laughter]
Never mind.
>> This is going to be nuts. This is going
to be nuts.
>> Yeah. So, um I'm actually been like
opening threads like crazy to see how
people and I think one of the
interesting organizations working on
this is human layers. They did a a
presentation where they heavily
referenced Steve Jagis beats project as
well and talking about um you know at
what at what stage you have um brain rot
how do they call it? So, so because of
the I think they they say at the 30% 40%
mark, if you only have 40% left of uh
unused context, that's where you are in
the non-productive zone of your context
window. And so he he highlights it's
called um the the talk is called harness
engineering and we can link it um later
but he talks about how to control that
context and common mistakes and um this
relates to this I'm I'm sure because it
talks about how you get the context um
and how you you know how you optimize
your workflow. Basically, human layers
does a lot of effort there. At the end
of of of his talk, he talks about if you
like what what I've talked about, go
check out human layers. Um, in terms of
like how do we, you know, interact and
integrate this um this LM
>> human layers is some software then like
so I just I'm startup.
>> Well, I really like that thing that we
were talking about over over WhatsApp. I
mean everyone's going crazy about that
that that sort of uh I keep on it's got
such an in memorable name that that tool
that which basically prompts you what is
the name of it again?
>> Ask user tool.
>> Yeah, ask ask user tool. How about this?
I'm just riffing here.
But say you're working with AI,
the ask user tool
sends a message to to each of the sort
of stakeholders, the middle managers or
something like that and and ask them to
agree on this certain thing and then all
they need to press is like enter and
then they move on to the next thing and
then enter move on to next thing and if
if they don't so basically it's like
some sort of democracy because you know
stakeholder A and B might answer
something option one but stakeholder uh
uh C might option might say option two
or something. So we need some sort of
like democracy type or you know or get
get into some uh ubi no ubiquitous what
do you call it when unanimous decision
about a certainity thing.
>> Not unanimous I think it's majority.
>> Well yeah that's democracy but in
reality you want unanimous but you want
everyone to agree.
>> You're right. You're right. You're
right. I did.
>> Um,
>> but this is this is the thing that's
missing cuz like we we jump in and and
and and chat with Claude and it's like
some sort of like mindmeld.
>> Okay. So, what exists
>> social paradise but in reality there's
loads of stakeholders.
>> Okay. So, what exists today is you have
tools like warp terminal that do not
have a single Okay. what you're talking
about seems to be like um there's this
tool that you can uh share your shell
and then you can have other people on
the same shell. You know this one?
>> Yeah. Yeah. Yeah. Like you can do that
with T-Max.
>> Yeah. You can do I think it's it's
basically a wraparound T-max that that
allows people to connect and have
multiple cursors and even
collaboratively edit in a shell. Um
so
and there's a lot of really cool stuff
using T-Max around agent that uh around
agents also but anyway today warp
terminal has the concept of an
individual session that you have with
Claude Code that you can then after you
submit the code share that session right
um so you you get the link of that
session um so you have the full context
of how that code ended up to be what it
was But and another thing is open code
also has this feature. So open code they
have a server site that you can um I
think it's private. You can as an
enterprise subscribe but open code by
default. Open code is basically a cloud
open source Claude Code version.
>> Yeah. I I tried it the other day. I was
>> I love it.
>> I've been playing on it a lot now.
>> But does does it have an ask user tool?
So I I linked to you the the proposal.
There's like two or three people do the
PR to add it in, but they're adding in
it into the core. So the reason why I
bring up open code and warp terminal and
sharing sessions is because what you're
proposing is like an ask group tool
which is like sending you know maybe
with with some type of team uh shell
sharing like um you know asking everyone
in the group to to respond.
>> I mean I'm not sure team sharing is
going to work because the way T-Max
works you share the same window, right?
>> Right. So, so everyone would need to
answer the same questions. But what I
want to say is open code has like three
or four PRs open to ask to add the ask
user tool into open code because it's
like one of the highly requested
features. But instead of adding it to
the core because apparently open code um
also has like a web app um environment
like the the terminal user interface is
just one of the interfaces that you can
use with open code. So they don't want
the ask user tool to be tightly coupled
to the core and tied to this terminal
user interface interaction because then
it doesn't really work well across all
the other interfaces that they expose
which to me things there could be a
collaborative document type of interface
on top of of of open code as well right
but to solve the problem they're adding
a new kind of user interaction model
into the core of open code that means
anyone can then implement ask user tool
as a plug-in so the core supports
your plugin to define like rich model
interactions like open a tapped
interface provide options and and with
with up and down. So I think that's
amazing. Um and that's only because it's
open source.
>> Yeah.
>> Yeah. So we can build something like
this, right? But like but like let's go
into the nitty nitty diddy details. Do
you think it should be unanimous? Do you
think this like say there's three
stakeholders in a project? Do you think
do you think they they can come to
unanimous decision or they should
>> Sounds like
[snorts]
>> everyone raise their hand at the same
time and if you don't agree then the
person that don't agree with the
majority has to explain why and then you
have to all raise your hands again
on the
>> Yeah, that sounds like a really good
agile exercise, isn't it?
>> I have no idea what agile is.
>> Planning poker. I I've never heard of
it.
>> Yes. You never you don't know planning
poker.
>> Where does it come?
>> You need to use Fibonacci numbers.
That's all I remember. Or t-shirt size
from poker.
>> Yeah. So the idea is that that you
discuss about uh a feature and then you
have to estimate the size story points
basically.
>> Oh yeah. But that's not I mean
estimating that that's the trouble with
um with AI like does it make sense to to
estimate a story anymore? No, we're not
talking about that. We're talking about
the the process of planning poker,
right? So, the process of planning poker
is you're in a group, there's a feature
and people need to agree, need to align.
And your problem is
>> Yeah. Yeah. Okay. So, it's an alignment
tool.
>> Yeah. Planning ping poker in the process
of planning poker there's an alignment
proc there's an aligning concept because
everybody the way is like you raise up
your hands. uh you're not supposed to
like look at each other. You have to
make a decision and and and you know
>> Yeah. Some people use t-shirt sizes,
some people use numbers, but if you use
numbers, it's like Fibonacci Fibonacci
sequence to make sure that it's not too
like if you have a a gradual number
range, people always go for the middle.
But if you have Fibonacci numbers, it's
more like um I read a lot about this a
long time ago. I don't remember.
>> Yeah, I' I've played this game. I've
played this game.
>> Yeah. But but I mean if coming back to
your original question about finding a
unonymous or or
>> Yeah, I think it's a good I think I
think it's a good suggestion. I really I
really do actually. Um
>> yeah,
>> there's lots of there's lots of tool I
think there's there's like Jirro plugins
to to to do what you just described. So
we we need we need an ask user tool
planning planning poker
>> ask group tool
>> and we and we need plugin
>> and I need it planning poker I need it
implemented by um Monday. So see you see
you then then Vincent.
>> So first off the open code PRs you could
you could look at open code and take
these PRs and and but then it goes into
core. So the reality is that open code
says this is a highly requested feature
and we're going to build these nice UI
um you know functionality directly into
the core so that plugins can build
really interactive um you know models
then imagine that that plugins but still
I think the plug-in system with open
code will be on a single user interface
it won't be something that goes over
across a server with websockets that
then updates everyone's screen like hey
here are here's the next question here
are five options
Which one do you pick?
>> You build it like beads and then you can
have your your your crazy UI on top of
it, right?
>> Yeah. So, for me, I the way I I I solved
it in in the thing that I've been
building, it's um it's more of an
offline flow. It's like one of the
persons goes and evaluates all the
options and then the system keeps track
of what they received.
>> Actually, that brings me sorry to to
slight tangent. beads. Surely beads
could maybe incorporate some sort of
option to to refine the uh the story or
something like that or
>> but it to be honest beads is more of the
the the actual task markup than getting
to the Yeah, you're right. Okay. And I
think it's good to um isolate this
because that's where I love beats
because it fits perfectly in my process
or any process that I choose to use for
me right now individually going through
the you know research plan implement
phases which I do with specit but
there's openspec there's spec cli
there's like a 100 different
implementations of specdriven
development under the sun so beats is
perfect because it doesn't concern
itself with however you get to those
tasks right Okay, so let's just
hypothetically imagine that
this is working. Stakeholders are
agreeing,
beads are created, beads are executed on
and then something is built, right?
Um then what happens then? Is there some
sort of accepting testing? Is there
like how do we get the stakeholder cuz
like this is another sort of
communication issue whereby like you
know you and I we would we would plan
something then we would create the beads
and then we would you know BD ready get
the agent to to implement it but and
then we would have to like uh review the
code and check everything is looking
great
>> and then and then and then and then and
then test the product and then and then
deliver the product and then
>> I think there you're you're you're first
of forgetting some of the agile
principles, right? Which is build a
minimal first to get user feedback
quickly. We all know that whatever the
user agrees to is not really what they
want until they see it.
>> Oh, okay. So then we're going we're
going back to an MVP.
>> Yes. And and this is one of the
takeaways that like people say when they
work with AI is that you don't create
mock-ups or you don't come with a design
specification with AI. You come with an
MVP. You actually implement it and you
let people play with it.
>> Um because it goes so fast. So So we
need when we do these planning poker,
we're talking about like small sprint
type of things and we need the
acceptance criteria aside from all
agreeing that this is what we want to
build. We need to get an MVP. We and
this is the great thing about specit by
the way when when I do a feature
definition it's always going to identify
the MVP of like what is the smallest
type of features that I need to to to
that I can show to someone for the
initial like UX feel of it um and I
think that's great uh in in these tools
um like
let's let's role play here so let's just
say like oh here's this MVP uh check it
out on localhost
uh 8,000.
Um so they they check it out and then
then what happens then there's new
requirements like ah actually this
should say um you should say food
instead of bar and then and then it it
at this point it engages Claude again
because like especially I'm just
thinking aloud for like UI based uh web
based things like Claude probably
doesn't even know what the render looks
like or something like that or you know
like I'm just I'm just thinking there's
going to be there's going to be new
challenges here to engage with the argu
I you mean this this is this is the this
is a planning tool but like but we need
another tool to facilitate the
iterations once the MVP is built or
wouldn't you or maybe I'm just
complicating
>> no I think that's where we need to just
fall back to existing processes because
your session your cloud session of this
planning session is gone, right? That's
that's an ephemeral shortlived session.
You have some output artifacts that you
you either, you know, that you fed into
because I think there's still a very
>> So [clears throat] you I think I know
what you're saying. You're going to you
just start again. You you you throw away
the MVP
and then you you start you start from
scratch or something. Is that what
you're saying?
>> It depends. Um it depends how what what
what you want to do with the MVP. If you
if you're an enterprise and you can
spend $1,000 on on on on cloud tokens, I
think you can throw away and or generate
five different MVPs. Me personally, if
if I spend a weekly budget of 20 dollars
a month on my on my MVP, I don't really
want to throw it away.
Okay, so let's imagine now that we
started from scratch, we used a whole
bunch of tokens and now we have
something that that the three
stakeholders
are happy with.
um isn't there going to be a question of
maintainability like
or uh
>> so this is where I feel the the idea of
maintainability of software is going to
change significantly because and I think
be uh Steve also says this very nicely
when he talks about beats he says I
created in October so what three months
and it has thousands of users right now
I have never looked at the code I've not
looked at the code and if you look at
but I think he's exaggerating
And if you look at Gast Town, he says he
gives like a bunch of caveats. It's
going to cost you a lot of money. It's
going to cost you an arm and a leg. It's
going to be insane if you're not at
stage seven of AI adoption. He he makes
a diagram of like um you know people
that um that use AI.
>> Is that in his in his blog? I didn't
>> Yeah. Yeah. Very nice diagram of like
the level of AI adoption. And then he he
asks are you at the stage where you
learn run at least five different
headless cloud sessions? If you're not,
then this is not for you. Yeah, [snorts]
that one. So, figure one is
um
zero or near zero. So, you don't use
figure two is AI is doing proposals. You
say yes, no, and you adopt Exactly.
>> Okay, we've been talking about this,
>> right? Figure three is you go yolo mode
with the agent completely managing your
code and you are still looking at the
code. Figure four I forgot what's the
figure the difference between three and
four but clearly the code is less
important. Figure five is you're fully
in the terminal with one cloud session.
Figure six is you are in the ter like
multiple cloud sessions and figure seven
is like guest town is if you're at
figure seven or eight you have like a a
significant amount of parallel cloud
sessions running. He explains them in in
uh in the blog uh below. I
>> I uh I haven't got through the blog, but
yeah. So, this makes me think we've been
talking about this at length and but
like at at this point we're only on
stage two and it feels like we're a
little bit behind
>> each and everyone individual right I'm
I'm pretty much as figure five. I'm
having I'm usually running a single
cloud session and I don't look at the
code. This is why and you say it's going
to create a problem with code
maintainability. I believe that docs
first
um integration test or end to end
playright is your validation of the
docs. So these are always a sync. If
they're not then you have a problem and
then your code. You may want to run some
security of defec ops agents to identify
vulnerabilities and things like that and
some well architecture reviews of your
code but personally you will not like
you know you only look at it at a high
level architecture and you put your
observability around it. You look at
your memory usage and you basically
identify bottlenecks and you only
refactor when needed like no pre
premature optimization. Anytime you look
at the code it's premature optimization
like you should just you know trust the
process.
I think so. And and you you trust that
your docs and integration tests catch
any regressions. So you can say, "Hey, I
want to change this whole feature and
then uh or I add a new feature." And
then you can just run the thing and it
goes like, "Yep, all of the existing
user stories still work. You're good to
go." And we have a new one and it's
fully covered by integration tests.
>> Wow, man. This the future that we're
heading in is really crazy. But like
take beads for example. Say
Say Steve decided like I'm not looking
at the code and I wanted to reimplement
again. Um I'm assuming beat has some
sort of test harness or something like
that. And that's where that's where it's
super important like what the whole
thing I said, right? You need to have at
least
a um you know a quick feedback loop for
your AI when it makes a change to know
that like oops I broke you know all of
this other stuff that's suddenly block
the integration tests are not working.
But I'm happy to say that I don't see
cloud going and saying oop this test
isn't doesn't
>> I'm happy to say that cloud doesn't go
like oh this is a pre-existing issue let
me disable the test which it used to do
like a month ago.
>> Well this doesn't in my experience it
will just change the test.
>> No no it doesn't. So that's where I I do
look at the diff on my test
>> because that's a trouble with tests like
the tests can encode a lot of
assumptions and things like that.
>> Yeah. So I just I just did the whole
thing um which is like a dock platform
with LLM prompt for cloud ask user tool.
So that's the thing I built is is
basically tracking content going into an
cloud session with user tool to make
changes to and then check it back in. So
you have the the that's what I built and
I just realized that it assumes the
server processes access to the files on
local disk and I'm like hold on a minute
the do platform doesn't run on the CLI
you know the client side so it doesn't
have access to the disk so so the whole
design I totally missed um the fact that
the client needs to send the content to
the doc platform for to make changes and
so all of my integration tests are
passing because it is running within the
same workspace. So when the server gets
a request, it just reads the file from
the workspace instead of it running in a
separate one. So right now I'm doing a
massive refactor saying like okay create
a whole new set of integration tests
where the server process runs in a
different folder and to make sure
there's no case where um you know so
there's no way that that that the server
can you know modify a file locally
thinking that the client changes have
not been persisted. So, so yes, the you
need to look at those integration tests.
You need to you need to look at the
documentation and you need to look at
the integration test and I also ask
>> code the tests are part of the code,
right? I'm just I'm just thinking aloud
here. Yeah, there's there's no way of
getting around. Yeah. So that's another
comment I got is um some systems are so
complex multi microser and all that and
so this is very hard of a of a workflow
to implement over these complex systems
if you don't if you cannot spin up a
whole integration testing environment.
To me the integration testing is the
only place you or like even the
endto-end testing is the only place you
can actually do um validate that that
things work and then you
existing systems are not built for this
right existing systems heavily rely on
manual verification of QA um and and are
way too complex to to build a whole
integration testing environment but if
you're building something new
>> I just thinking allow just to put just
to put the tests in the code is kind of
dangerous um they kind of need to be
separated just for safety almost.
>> No, I don't think so. I prefer what I
like right now I I I've done four or
five projects where I use specit
development specit driven development
and beats and I go for a full monor repo
everything like side by side and it's
perfect because cloud can go yeah
>> and directly see hey my API is here my
protocol definitions are here my
integration testing are there and and I
need to go full stack like I need to go
and change my my my um my protoraph
>> okay so say we have some in end to- end
tests. I mean, normally in the modern
world, this would look like some,
you know, most most uh pieces of
software end up being a web page or or
an iOS app or something.
>> Mhm.
>> And and let's be honest, doing end
to-end tests on the web is still or or
or on mobile still a pain in the ass.
It's going to be quite manual, isn't it?
No, I I mean for the web app that I
built, I use Playright. I never used
Playright in my life, but Claude Code
wrote the scripts and I I manually
verify it and I know it works. If it
doesn't, I say, "Hey, the playright is
not covering this." And it goes and and
fixes it. It says now the play is
failing.
>> My like it works in trivial cases, but
in like big enterprise, if you ran
playright over in an Atlassian estate,
it would consume a billion tokens and
explode and
>> No, no, no. Not playright MCP. Okay.
Absolutely not Playright MCP. Playright
the actual test harness.
This is what I hate about the Playright
MCP is like this. They seem to assume
that you want the LLM to drive it and
spend a billion tokens to go through the
process.
>> Okay. You need to show me your usage of
playright. But I still I'm still
>> playright. If this is what we're going
to rely on to make sure that we're
delivering something that works, I'm a
bit nervous about this.
>> And that's that's reality today that so
many organizations don't have a proper
first off database seeding mechanism or
ability to even create a full
integration environment for integration
testing and then they don't have fully
automated end to like end toend testing
scripts. But if you write playright end
to end testing scripts, it actually
doesn't need an LLM to drive. Now go
click on this button. Let me take a
screenshot. Oh no, no, no, wait for this
button to appear. And then the LLM is
doing that. That's insane. That's
stupid. Playright has a scripting
engine. You can say in this particular
test case, we're going to, you know,
we're going to click on the login
button. It's going to redirect us to
Keycloak for enterprise login. So we
wait for the key logo to to appear, the
login to appear. uh we then log in with
our click click credentials for our
integration testing environment which is
bootstrapped by the way during the setup
phase. So I have a unique set of
credentials to use during my end to end
integration testing.
>> Doing credentials flow is going to be a
total pain.
>> Yeah. But that's why you need to build
these systems with automation in mind.
So so that that's real. It's true. I I
join many companies and they're like oh
we don't know how to do our things and
we want to have an integration testing
environment. And I'm like great. Yeah, I
can I can do that. And then like okay,
how do you bootstrap user accounts? How
do you set up this and that? And nothing
can be automated. Well, maybe I'm
talkingual failed career, but like I
have implemented a few integration tests
in my life, but they just tend to be so
expensive and hard to maintain and you
you make one or two
>> and then you know how it works, but then
you know you move on to another team or
you
>> But the great thing about LLMs is that
they like I never wrote a playright
script in my life, but the agent knows
and then it just works and then it
actually runs and when you go into the
playright studio, whatever the thing is
called that shows up the UI when you
when you run it in UI mode. It actually
you can see every single step like you
see the screen.
>> Okay, you need to show this to me. But
anyway, I think I think we're
>> but the thing is that you get a full LLM
context. Now this playright UI has a
rightclick button, copy error, copy
prompt, and you can prompt it. It says
we're in the playright test suite. This
is the step we're at. This is the error
that the browser console is showing.
This is what we saw in the previous step
and you don't need an LLM to drive.
>> Okay. In the next video, you need to
show me playright working as you've just
described.
>> You don't believe me?
>> I don't believe you. But but I want to
believe to be honest. Okay, let's let's
wind up.
>> I know that your comment is going to be
it's a trivial system that you're doing
this on. I showed it to you before. I
think I showed you the grit uh you login
flow.
>> Okay, so next is a focus on this. Let me
see if I can highlight it. Okay, cool. I
need to get on with my day. I'm sure you
do, too.
>> You don't believe me. I'm tell I'm I'm
telling you I agree with you that today
the systems that the way that they're
written and the way that humans have to
maintain these things, they tend to run
out of sync. But I feel I have a very
strong feeling that with the LLM and the
capabilities of the LLM, the LLM takes
on ownership on a lot of these
components and the users are just there
to validate. And it's pretty much what
Gasttown is saying as well. The human
role is reduced to an overseer. We just
validate the process.
>> The validate like we I thought we were
just discussing that the validation can
be automated and enter.
>> No, the human validates the validation.
>> All right. I um anyway I wanted to talk
to anyone who's listening to please uh
rate the podcast
comment in the YouTube email
how do people reach you Vincent
>> uh reach me
>> don't say LinkedIn
>> I I watch the comments sometimes so if
there's a comment I will I will respond
>> wow man
>> I don't know you're a man of the people.
[laughter] I I will sometimes look at
what you have to say, but otherwise I
don't care.
>> No, no, I do. I do. I go care. How do
they reach me? You're like, "What?"
>> Do you have any like You can email me at
henry.fe, for example, and tell me that
I'm wrong or you have a better idea.
>> Okay. Uh, you can
God, this is so hard with you, Vincent.
Honestly,
>> can't really reach me. No.
>> [laughter]
>> Okay. Well,
>> you can even and Kai and then Kai can
tell me if somebody says that I was
totally wrong that play right doesn't
work the way I say it does.
>> Cool, man. Anyway, it's uh Oh, we didn't
Yeah. Okay. We couldn't cover
everything. This is crazy. Loving it.
Okay. See you guys. Bye.