Episode 48: Agents built the AWS emulator
Download MP3 | Watch on YouTube
AWS says its new emulator was built by agents, and it doesn’t accept pull requests. Vincent De Smet is using it anyway: his CDK Terrain workshop in Singapore runs entirely against the emulator, so attendees need no AWS account. When the emulator couldn’t have one Lambda invoke another or enforce IAM policies, Fable wrote the patches and rebuilt the binary.
Vincent tours the workshop: AWS’s original CDK workshop ported to Terraform, a box per attendee behind Caddy, and a progress tracker and leaderboard built on Cloudflare Durable Objects. Kai Hendry opens the emulator’s repository in tig, finds agents committing every couple of minutes, and admits to a feeling of dread. Vincent argues that reviewing commits is the wrong metric now, and that control has moved up to the intent layer.
The second half is about skills and prompts: Google’s elephant and goldfish model, Matt Pocock’s grill-me, Lauren Tan’s verification skill on Kai’s Singapore bus app, formal verification with TLA+ and Quint, and tending skills like a garden. Kai wants to prove that a page-long AgentCore prompt is no better than a short one, and they finish on PII in logs, break-glass access and background agents.
The episode opens with Vincent showing the view from his window in Ho Chi Minh City. Recorded on 17 November 2026.
Chapters
- 00:00 A view of Saigon, zoning and Belgium
- 03:49 Config Management Camp and the CDK Terrain community
- 05:45 A CDK Terrain workshop in Singapore
- 08:28 Car-free Ghent and hearing horses
- 12:28 Patching and building the AWS emulator with Fable
- 18:07 Five gigabytes per workshop attendee
- 21:57 Caddy, Traefik and Kong
- 23:28 Inside the workshop box
- 32:32 Commits every two minutes in the AWS emulator repo
- 38:59 Reviewing code in the agentic era
- 40:44 How the clouds are adapting to AI
- 42:13 Google’s elephant and goldfish model
- 46:53 Lauren Tan’s verification skill and Playwright
- 51:18 Formal verification, comments and TODOs
- 55:35 Stale skills, gardening and glossaries
- 59:55 Rethinking Claude Code best practices
- 1:03:21 Does a page-long prompt beat a simple one?
- 1:10:10 PII, scoped tokens and break-glass access
- 1:13:54 Background agents
Mentioned
- The AWS emulator and LocalStack
- CDK Terrain and TerraConstructs
- Caddy, Traefik and Kong
- Cloudflare Durable Objects
- Beads, Matt Pocock’s skills, Spec Kit and Conductor
- Playwright, TLA+ and Quint
- Amazon Bedrock AgentCore and Strands Agents
Watch on YouTube · Browse episodes and subscribe
summarize "https://youtu.be/AHl4atOHnKo" --timestamps --slides
Kai Hendry and Vincent De Smet start with city life in Vietnam, Belgium and conference news, then spend the core of the episode on Vincent’s Singapore workshop, built on AWS’s new agent-built emulator, and the emulator’s own commit history. They finish on agent skills, prompt design and keeping production data away from agents.
00:00:00 Saigon, Belgium and Config Management Camp
Vincent shows the view of Phu My Hung in Ho Chi Minh City, an area that used to be swamp. He says there are no zoning laws there, which he finds both freeing and sad. Kai isn’t a huge fan of zoning laws, though he accepts that building control is needed. Vincent describes Belgian “lintbebouwing”, houses built in a line along every road, and looks forward to visiting Belgium in January for the first time since about 2016.
From 00:03:49, Kai is disappointed that his AI talk proposals for Config Management Camp were rejected without feedback, though he is still going. Vincent’s CDK Terrain talk was accepted; he credits the community and the Open Constructs Foundation, started by Elad Ben-Israel and Sebastian Korfmann. Kai says his colleagues adopted CDKTN independently, and argues that recording conversations, even face-to-face ones, matters more than ever because AI can learn from the recordings later.
00:05:45 A workshop on the AWS emulator
Vincent is running a CDKTN workshop in Singapore on Saturday the 28th, which Michael Cheng will record. Kai asks how the emulator AWS announced the week before compares with LocalStack. It is a single Rust binary, early stage and openly incomplete, and Opus told Vincent it is generated from AWS’s Smithy API contracts. There is no ARM macOS build, so he runs it in Docker. Building it needed a 16-core, 128 GB instance, and a single-threaded link step was the bottleneck until Opus suggested a parallel linker. AWS doesn’t accept pull requests, so Vincent keeps his own patch, which Fable wrote. Kai assumes the emulator is much faster than the cloud; Vincent says that is the point, letting agents verify changes locally.
00:18:07 Inside the workshop box
Vincent estimates about 5 GB of memory per attendee, much of it for the TypeScript language server. Rather than a container service per attendee, he shards attendees across Graviton EC2 instances, about seven to each, behind Caddy. He pays for it from his own account and his AWS Community Builder budget: a few dollars per instance for five hours. From 00:21:57, they trade views on Caddy, Traefik and Kong.
From 00:23:28, Vincent shows what attendees get. The emulator has no console, so steps that used the AWS console became CLI commands. He took the original markdown of AWS’s CDK workshop, which Elad Ben-Israel’s team wrote and AWS has since archived, and ported it to Terraform. A progress tracker built on Cloudflare Durable Objects lets attendees put themselves on a leaderboard, and validation steps check each endpoint they deploy. The names and emails it collects will feed his next talk, on stopping agents from leaking personal data. Kai calls it completely gamified and offers to try it first; Vincent says the whole thing took about two days.
00:32:32 Agents built the emulator
Kai opens the emulator’s repository: about 64,000 Rust files and a great deal of markdown. Vincent says he never even looked at his workshop’s code: Opus stood up the emulator and ran each workshop step against it. The emulator runs Lambda functions on QuickJS, so the workshop uses a handler that works there. It also blocked one Lambda from invoking another and returned success without checking IAM permissions. Fable patched both and rebuilt the emulator image in two to three hours, which Vincent puts down to the problem being well defined. During the call, his spot instance is terminated.
Kai is struck by commits from issue-finder and issue-fixer agents landing about two minutes apart, around the clock, and says it gives him “feelings of dread”. He remembers reviewing the day’s trunk commits with his team. Vincent replies that reviewing code is the wrong metric in the agentic era: you control verification and the intent layer instead. From 00:40:44, they joke about Microsoft’s 24-hour outage, and Vincent says Google never kept the lead Gemini’s million-token context gave it.
00:42:13 Skills, methodologies and gardening
Vincent describes Google’s elephant and goldfish model as a good summary rather than anything new: it echoes Steve Yegge’s Beads and Matt Pocock’s grill-me skill with domain-driven design. A shared language is built up first, and short-lived agents then pick up work in it. He contrasts spec-driven tools such as Conductor and Spec Kit with “get shit done” styles. From 00:46:53, Kai shows Lauren Tan’s verification skill, which Opus 5.5 used to discover the features of his Singapore bus app. Vincent approves of plain Playwright specs over a Playwright MCP. In his opinion Pocock explains things beautifully, while Tan brings experience of large projects.
From 00:51:18, Vincent says he uses Tan’s state-machine ideas with Quint and TLA+ to check a race-condition fix. He agrees with much of her no-comments skill but wants a sanctioned way for agents to mark placeholders. From 00:55:35, Kai says his client’s shared skills repository has gone stale. Vincent calls looking after skills “gardening”, and they agree on shared glossaries, though Vincent warns they take several iterations and Kai says meanings blur across an enterprise’s markets. From 00:59:55, Vincent shares Anthropic’s “Rethinking Claude Code best practices” talk: emphatic instructions that older models needed now distract newer ones, and a Fable orchestrator handing work to Opus agents matches how he works.
01:03:21 Prompt evals, PII and background agents
A colleague of Kai’s wrote a page-long prompt for an AgentCore agent that decides whether a security finding is a false positive. Kai believes a short prompt would do with Opus 5.5 and wants to prove it. Vincent suggests evals and deleting paragraphs at random to see whether the agent still works, a technique he calls prompt “obliteration”. Kai didn’t expect Strands to keep memory and wants it lean. He also finds AgentCore slow to start.
From 01:10:10, Vincent previews a demo that blocks agents from reading personal data, and proposes short-lived tokens scoped to each task. Kai says the problem is human: engineers who break the glass can still paste production logs into Claude. Vincent replies that device management can flag it. Kai calls that a detective control and doesn’t want to go into it, because it is current at his workplace. From 01:13:54, Vincent argues that background agents, which don’t run on the engineer’s laptop, remove the copy-and-paste route; Kai doesn’t see it as a technical problem. Vincent’s Red Hat talk in Singapore on the 27th covers these enterprise adoption patterns.
Model: cli/claude/sonnet. Reviewed for names, attribution and timing. Generated from the approved recording’s timed transcript because YouTube captions were not yet available; slides from the uploaded video.
Transcript (transcribed per speaker from the edited recording, with technical-name corrections)
Okay, just, I just wanted to show the view like at 4, 5 p.m. I don't know if you can see the tower, like the Bitexco Tower, but the sunlight is shining off. Wow, it's a lovely day. Yeah, well, this is the regular day in Vietnam, but you can see how the 4 p.m. sun. Oh, can you angle the camera down a bit so I can see the buildings? Oh, yeah. It looks like Singapore. Yeah, so this is, this is. the Phu My Hung area, which used to be swamps, but now it's full of construction. So all of those villas that were there for like 50 years now are being replaced by restaurants and some of them being removed. You see, they're all kind of the same, right? It was a huge development and people buy them and then they open a restaurant and then you have motorbikes everywhere on the street. They open an office. So there's no zoning laws here, which is kind of sad. In a way, it's kind of, you know, it's nice that everybody just does whatever they want but in a way if you live in a nice area then you suddenly have cars everywhere and and then people get upset because they want um they don't want cars parked on the street so they um they put all kinds of cones uh so there's a restaurant but then the neighbors put cones and then the street is like overloaded i don't know you don't have that problem in the UK I guess because there's strict laws, zoning laws of what you can do in each area, right? Yeah. Yeah. I'm not a huge fan of zoning laws because I guess it's just a sliding scale or something, right? Like you want to be free to do things and you don't want the government to basically weigh in all the time. Yeah. So you do need building control and zoning, but sometimes it's a bit much, I feel. I think the balance can be quite weird. So in Belgium, we were really bad. Well, they are. Oui. We are. I grew up. I don't know. I think at this point in my life, I've lived more outside of Belgium than in Belgium. But we, Belgians, have a lot of problems. And we have a name for it. It's called lintbebouwing. It means that there's like one street. and because there's no regulation on how people build, you literally have like a street and then the houses are all built along the street. And then behind those houses is nothing. It's like a strip with like a lint really means like a line. So if you drive from Belgium into the Netherlands, you immediately see the difference between a nice well-organized country where everything is neatly zoned and then Belgium, which is the roads. Your car starts bumping on the roads at least back in 2006 when I left. Crossing the border is like you hit the road starts bumping. dear The lights on the street are triple the lights that you have in the Netherlands because Belgium even you can see them from space because they put lights everywhere. I'm looking forward to go back in January. so yeah I'm looking forward to the Belgian trip how long has it been since you last were in Belgium again? I think probably since at least 10 years I think the last time I went was in 2016 wow that's going to be a lot i suspect yeah I'm going to see all the cultural enrichment I mean in a good way hashtag controversial the the obviously the the thing that disappoints me about about the belgium thing is that is that my AI related proposals, my talks at Config Management Camp, I guess I didn't get accepted. I even followed up asking for feedback and I didn't get feedback. So I'm a little bit disappointed at the organizers, but I'm still booked in to go. You tell them. You tell them in their face. You didn't recognize me? I go every year. Finally, the first year I propose and you guys don't even give me feedback on what's wrong with my proposal. That's not very encouraging. That's not a very nice community organized event. I guess they're super busy. I don't know. Well, maybe my proposals genuinely sucked, but, well, there's that. Maybe you don't know what they compare to. I mean, I got my talk about CDK Terrain. I guess the only reason is because there might be an audience that actually like, I mean, there might be people that like to use CDK for Terraform after the... I think, as I was saying, my colleagues independently started using CDK Terrain. Like, what you've built is commendable, Vincent. So, oh, yeah, of course. Not just me, right? I mean, it's a community. I couldn't do it alone. yeah Claude, what's his name? John Steinig, Eduardo Codex. Codex. Simon Heather Gabe there's quite a few people that are and Torsten of course the person who offered the Open Construct and basically manages the Open Constructs Foundation that originally was created with Elad Ben-Israel and Sebastian Korfmann you know the foundational people okay behind the framework So it's only possible and also because of, yeah, there is a kind of a foundation there, yeah. And so you were messaging me earlier about a demo that you have with the new AWS emulator. Am I right to assume that you somehow got CDKTN running on the emulator and you've got some value out of it. Yeah, actually, that would be an interesting little demo. Where is that session gone? God darn it. Anyway, did I close it? I don't even know where it is. It doesn't matter. I can ask any new agent to spin it up. I'll spin up the agent so basically what I've built is I'm running a workshop in Singapore on the 28th on Saturday which is with you know, Co Dojo from Michael Cheng who does work for GovTech and talks a lot about, he's very Michael Cheng is awesome. yes, sorry and if Michael yeah he is listening howdy howdy from the UK he's he's the OG behind a lot of the meetups in Singapore, right? And then he started going to meetups with a camera and a sound setup and everything, yeah and then getting volunteers, trying to cover every meetup that's happening across Singapore yeah and setting up cameras and having everything recorded and online, creating speaker profiles, capturing the event and the attendees and what the meetup was about, creating a little website that gives you the ability to review that even today i'm like oh yeah there's a there's a recording of of that like event that's only thanks to michael and his yeah volunteers yeah Yeah, recording this stuff and documenting this stuff is so important. yeah I just, a little aside, with AI, it's so important to record your meeting correctly. Like, it's so important to have that artifact so that as AI improves, that you can sort of throw in those meeting artifacts and summarize and learn from those artifacts. So I don't know if you feel the same way, but I'm pretty keen, even when we're meeting face-to-face in Belgium, Vincent, that we record our conversations. oh yeah that's not a bad idea to set up some time for that i'm there for a whole week we can sit in a nice Belgian café in the Belgian winter have some french fries a beer a nice duvel and listen to the horses in the center of Gent if that's still the case I didn't see horses last time I was in Ghent but Jesus I'm gonna be sad it was when? 1820 were you last in? no so even when I was I go there from high school all the way to uni right So from when I started high school, goddammit, I finished in 2001, so 18 years, sorry, six years before that. So 1997, 98? we must be very close in age i don't think i I'm from 82, you're like three years older, I think. oh all right but i i i went to university in 98 myself maybe because yeah 98. Sorry. Yeah, well, I started in 2001, Uni, in Ghent. So the thing what I was referring to is, at some point, they had a project around the city of Ghent where they diverted all the cars into parking areas around the city. And the center of the city is car-free. cool There's no cars allowed. yeah So you're only having bicycles, trolleybuses, and horses. So, like, I have a story of a friend of mine who had a hearing problem. And his parents were traveling at the time he was receiving his in-ear, the first time they built the in-ear, according to the ear channel. So, he stayed in Belgium and didn't travel with his parents because he had behind-the-ear hearing aids. And he wanted to get the new in-ear ones because they are obviously better and less visible. Yeah. And so, he was very excited. So he stayed with us for a couple of weeks and we took him there to pick up his earring aids, new ones. And right after, well, they asked him, you know, he's a funny guy. So they asked him, like, what do you think about your voice? And he says, good, I guess. It was like, what am I supposed to say about my own voice that I, you know, never really clearly heard? But anyway, well, once we go outside, he was like, horses? Do I hear horses? so yes there are like chariots and horses uh at least back then you know um even in 2000 click click click click click so you never heard them before oh sweet so many like small little details um that he shared with us about like you know differences he had a lot of techniques like whenever somebody would say something he was actually very good at lib reading so he even though he didn't hear or understand you always he would very he would pick Mm-mm. up very quickly uh and then the second thing is he would always like say really is it real because he wants you to repeat it or he he's trying to guess more or get more context like he didn't actually like fully hear what you said but he's he's got pieces so he's just trying to without revealing that he didn't hear it he had a lot of like defensive mechanisms and when you don't know it you like some people did they were like what's wrong with you like are you not That's interesting. hearing me and yeah That's interesting. I just, I'm comparing that with my experience with AI. And if anything, I really wish AI would ask me more questions. I'm always having to add like a suffix, ask me how, or ask me for clarifications. You know what I mean? Like AI doesn't have that sense sometimes, at least the ones I'm using to ask me for clarification sometimes. It's probably just me. So anyway, we got a bit distracted there. Did you want to show the... I guess you don't have the demo ready, right? Of the AWS emulator and CDK Terrain. I just... Yeah, I just asked, I thought Haiku would just be able to find the directory on my laptop and spin up the EC2 instance. and it's reading the Terraform state and it's getting really confused. It's simple. It just needs to flip a toggle and run Terraform apply and then the EC2 instance spins up. Okay, so you've used the AWS emulator in anger. How does it compare with LocalStack No. and floky, flotchy, floky? Yeah, I didn't try Floki in this case. What I really wanted was the... So basically, the CDK workshop is API gateway invoking Lambda, IAM policies, a Lambda invoking another Lambda, like a proxy Lambda, and I guess S3 buckets, not too much. So I thought, I know that the AWS emulator announced last week is very early stage and has a lot of gaps. So they even say it's absolutely not complete. It's a very early release. And when I asked Opus, it says, yeah, looks like most of it is there, but it's all automated built from the AWS API services, like from the Smithy API contracts, right? So... expected something like that. It's cool that they've sanctioned something because there was a few sort of AWS projects I've been following which were promising to be some sort of emulator but didn't really transpire to be one. yeah so what what the so the first thing that happened is i wanted to say it's it's all built like rust a single binary um on rust that spins up really fast uh but it's like 900 megabyte at least the one that i built so to build it it also takes a lot like when i first evaluated it on the relaxed macOS, Opus said like, yeah, it doesn't have a Darwin release. Like it doesn't have an ARM Darwin release. So it spun it all up in Docker to avoid. Like it does have an ARM Linux release. So when you run it in Docker or Colima or whatever, it runs a little Linux VM on your machine. And then you can run the AWS emulator in there without having to rebuild it. Because if you want to rebuild it, the estimates of Opus is you need like a lot of CPU kernels, vCPU and a lot of memory a lot of disk space and because i'm like running agent fleets my disk is actually running out so i just told opus all right spin up a build instance so it's it spun up like this 16 core 128 gigabyte memory instance to build the emulator and then estimated about 40 minutes build time but it got stuck after 30 minutes um so it builds every module like every Rust create independently. That's all in parallel. So you have a lot of cores in use at the same time. But then after that, it needs to link all of those library files with a dynamic linker, like LD, Linux linking. So that needs to like, that actually was a single CPU, like a single core at 100% load that was not benefiting from the multi-core instance. So Opus, funny enough, I don't know, I asked it, isn't there like an instruction in the readme but there is a ldd or like lld there's like a parallel linker that can link up things in parallel and opus suggested it so while that initial one cpu linker was running it started a second build which just took like a few seconds on the original crates because nothing changed and then the linker itself also just took like a few minutes so like The total build time is just, I don't know, 30 minutes or so if you build from scratch. So that's pretty beefy. But if you use the right multi-threaded linking process, it can build really fast. and you're talking about building cdk tn with awc so i didn't quite understand what you're building No, here I'm talking about because the emulator is very early stage and has a lot of gaps, I had to make some changes. oh i see They do not accept PRs. and then yeah i can see it's all rust i was going to mention that the other sort of so I just created my own like get patch so basically Fable came up with the actual patch that I needed to support the workshop and then it rebuilt the whole thing yeah aws thing i noticed was this project which was in local web services and the guy that was working on it uh is an aws employee so i was i was at least i think he was maybe he isn't now i the i was thinking that this would be the the the aws emulator but but this one here yeah you're right Python I think It's not going to fly. This one seems... Floki is in Java I think yeah This one's looking promising. So once you have that Rust binary, it's, I'm assuming, super fast to work with. It's way faster than the cloud, right? yes the goal of this emulator is for local like agents to quickly um you know spin up things against an emulated environment and verify things and spin things down with a lot less memory than the other solutions. But I heard it's still quite beefy. Like for my workshop, it estimates that each candidate, if we all apply at the same time, because that's how workshops usually go, right? Everyone kind of follows along. So everyone runs Terraform apply at the same time. So it estimates that the memory requirements for a single attendee is easily five gigabyte. And you know what actually took up a lot of that memory? No. I'm running a VS Code like Monaco Editor and it runs the language server, the TypeScript language server. Yeah, and the TypeScript language server with VS Code spikes up to like six gigabytes of memory just parsing node modules and providing type information. So it was really fun to spin up a workshop because Opus was like, hey, you know what? We can modify the configuration of the TypeScript language server to eliminate the TypeScript type definition for all these node modules. And then I said, why don't we use TypeScript Go? Why don't we use TypeScript 7? And then it found the TypeScript 7 Visual Studio Code extension, which is not allowed to be used. The licensing does not allow you to use it So why don't you run your demo in the cloud? with non-Visual Studio Code, apparently. Because like, oh, you are. That's exactly what I am telling you that I'm doing. I'm running, I'm holding. I need 5 gigabytes of memory per attendee. but so for each participant you're going to have a cloud instance for them So I'm not doing that. Yes, we could spin up ECS task definitions per attendee, like one container, but I'm actually using a single EC2 instance and it can hold, I think it's R7 Graviton, holds about seven. I think we estimated seven attendees per instance. So I'm going to open source the whole infrastructure for it. But it spins up an instance and shards the attendees. So if you say I have 14 attendees and you say the instance type, the size, it's going to shard the attendees across the instances. So if one instance dies, then not everyone in the workshop stops having access to it. But yes, there is a blast impact. basically I'm saving money by not using ECS I don't have to do like it's just simply having a single EC2 instance with a public endpoint and I put Caddy in front of it it serves the workshop like a dedicated workshop per attendee in Docker containers everything, it's running now let me open the link okay that's how did you get in roped into doing are you getting obviously you're getting paid to run this workshop, right? no no this is no um i don't know where which screen i should share What? Who's paying for the compute to get the stacks running? i'm using one ec2 instance for five hours it's like five dollars or less like three dollars to host seven attendees yeah it's on my account plus i have a community builder um uh Okay. So what is your goal to run budget. this demo just so that people so you teach Because I wanted to do it. who pays for your flight and everything for Christ's sake I wanted to go to Singapore. so you're doing this I pay for my flight. so you're upscaling people Yeah. through the goodness of your heart but it's going to be recorded by Michael Cheng so it should have some distribution right okay so this is the the every attendee will get a key a session name and a key and um and basically when they go to that dns it goes into a reverse proxy that is caddy ZeroSSL with TLS via I don't know if you know Kadi do you know Kadi something's running I'll just give it access I don't know. so Caddy Oh, Caddy. Yeah, yeah. I was one of the first users of Caddy. Kadi oh yeah sure you were Okay. weren't we all aren't we all at least you didn't say you were one of the first users of Traefik Victor how is vector is not traffic no I wasn't one of the yeah I know Traefik I mean I'm more of a Kong kind of guy oh my god that's just Lua and Nginx that's OpenResty with a brand on it yeah I know I know anyway I know but it works really well though Caddy is amazing it's the best I love it the https integration with uh Let's Encrypt was chef kiss chef kiss ZeroSSL Yeah, it's my favorite reverse proxy at the moment for simple projects. and i'm trying to remember the guy the the guy behind it he's from the church of the Saints. Matt Holt or something like that. Okay. All right, so this is what the attendees get. M. Holt. They get a key, and when they join, it also runs a workshop. it actually runs a statically like built off that of the workshop uh which has a couple of modifications because there's no AWS required here um so if you compare that to the the one that's live that is the actual you know uh it has a disclaimer about cost and blah blah blah Mmm. so this one is meant you use your own AWS account um I don't provide you anything this one is actually, it is a static compile of the box mode because it's inside the box. So then it has a couple of different steps. For example, I don't have AWS UI. So if I go to the writing constructs, here there's a whole section about finding the function and looking at the, like going to the monitor in the UI. If you go to the same one here, CloudWatch Logs, it's all CLI commands because we don't have a UI in the emulator. And that's fine. That's fine. So that's the box. Sorry, did you just repeat that? You don't have a what in the emulator? AWS emulator doesn't come with the console. It doesn't come with the UI. There's no AWS UI there for you. Oh, okay. I understand. Yeah. That's interesting, actually. Yeah. Okay. You can combine these two. How do I add tap? How do I split? Split view. Yeah. Oh, man, I'm loving split blue. Do you know you can hold down command, option, and then click, and then it opens on the right all the time? Okay. So here it is. Okay. You can have the workshop on... when i do it in the i i recommend that the workshop will actually be on the on the display and i will walk through it and anyone who wants to do it at their own pace and then ask me questions they can do it at their own pace um Have you done this before? It sounds like you're going to be chucking down a lot of knowledge down, people. I don't know if people are for that. This is going to be a lot of stuff. ready for it yeah no this workshop is really good have you never done the cdk workshop like Yeah. AWS actually runs it for companies, and we ran it in my previous company. oh no I haven't actually Yeah, it's the official... I didn't know about this so so you're basically building on the CDK workshop well that's interesting Yeah, so this CDK workshop is a repository that uses Hugo in the past. It's like seven years old by Elad Ben-Israel and his team. hmm And then they archived that repo, and they made it in the catalog, so it's no longer public. but I could still access the original markdown so I modified it to basically run oh that's really cool man completely on Terraform. So if we look here, hello cdktn, it's exactly the same thing. I just didn't do the Python one, so it's the same except I want it nicer. yeah well it's great that you're doing this i i there's been a lot in the news lately about unemployed youth and upskilling people who don't know about stuff. And I sometimes think to myself, like the videos and the podcasts that I make, that's my attempt to basically share knowledge. But I really like how you're going out on a limb and creating this material, this structured material, and doing a workshop. This is good. do you see this little progress tracker at the top Beautiful. what this does I need to white so this is using Cloudflare durable objects so when this do I have the Cloudflare here dashboard I guess here so this is all Opus built for your question that you asked me about like the Cloudflare MCP and the code mode so I build a little dashboard for data so I have data these are my deployments so this tracker here at the top when you scroll and you complete you can put yourself on the on the board if you want to right so you can put in your name your email so this is the sensitive data that the agents should never see that you want code mode to protect right so that comes to the next part of this this this talk um for you okay your question um you have sensitive data in cloudflare and you don't want the agents to show those emails in the transaction log that's my yeah so so people can Oh, yeah. if they want enter their name and then they will be on poc this dashboard oh shit i don't know the secret what's the secret for so no no no i'm just gonna put up a dashboard of uh the progress of Oh, so not only are you just introducing people to CDK Terrain, everyone during the workshop they don't have to put their name they don't have to put their name oh that's cool but that's gonna give me data for my next presentation at in singapore which is about policies and preventing agents from raising private data such as email addresses and if cool man nobody signs up with their email then i will just generate a bunch of fake data which you will see now Oh my God, you're prolific compared to me. This is really good stuff. I think you're getting your hopes up a little bit much. why I mean, I'm going to be amazed i cannot find a secret if one person in your workshop gets through all this material. so that's what i'm gonna know because i'm gonna see if people complete it. Ha ha ha. There's even a validation step. So when you say use the hit counter, like for example here, when you deploy the API gateway, when you run the Terraform apply, it's going to tell you to put that endpoint in here. And when you click next, oh man you're completely gamified can i run through this as a can i offer to run through it's going to call a verification. this as a guinea pig yeah like i can spin up an instance uh and give you access yes yes it would be good i suggested michael as well to go through it first because i think the the biggest issue with the workshop like that in limited amount of time is usually it'll be a second set up laptop uh you know is the laptop have enough memory and capability to you know have the node cs version and everything setup and and this kind of like takes that away right people just get a box they they sign in hmm they can run the um the cdk in it so i'm i can copy this i added oh what the hell it's offline Okay, let's can I share my screen? I wanted to have a closer look at wait one one one thing where the hell is the the emulator. One last thing. why haiku is so bad oh it found it okay hold on a minute i need to run one comment You're using what's haiku again? You're using the very basic Sonnet, Claude you're using haiku just because it's cheap yeah there it is yeah i just want to see sometimes you just want to know if like the cheapest model can do what it needs to do like does your repository have enough so you see obviously these are fake email addresses that Opus was using when it was trying to verify that everything works as expected. So here's my trial. So you can see wherever it actually verified, it recorded that endpoint. Again, this is, I don't know why it needs to keep it, but maybe that's part of the sensitive data demo, right? That the agents would not see these live endpoints or whatever. That's just a little scenario that I told Opus to come up with. Yeah. And another cool thing, okay well hold on when you click define here when then it goes into like verify it like hit it every curl command that you get here in the workshop is then showing that endpoint that you created back then you see that endpoint is the one that you so you can just copy it and run it oh so the the docs live update and yeah yeah i mean i had a little bit of fun anyway that's the last thing so that's all i did for the workshop that's going to happen in two weeks time and it literally just took like two days to Cool. So if anyone's listening to this who is around Singapore, you should go and meet Vincent and do his demo and learn a lot about how to stand up infrastructure using the cutting edge tools, AI and CDK and all the rest of it. build all of this it's insane And speaking of, can I just replace the share here? now let me stop yeah yes I just wanted to have a closer look at the AWS emulator. So can you see my screen? It's like 64,000 files of Rust. Yeah, I can see the screen. Quite a lot of markdown. I don't know what the hell that markdown is. Steering. Operations. what I'm guessing is that that's context for the agents to build maybe there's the Smithy like let's have a look at docs I opened one of them Lambda or something but that's the coolest thing about my experiment you know building this you know this oh it's like issues and building this web this web box for people to attend. I never even looked at the code. And I just, even Opus just went, okay, step one of the workshop tells me to run Terraform Apply. So it stands up the emulator, runs Terraform Apply, and then does the next step. So it even built like a whole bash script of part one, part two, part three. It runs them in line and then validates if the emulator can do it. um yeah so you have a whole end-to-end verification thing It's insane. Yeah, and then it told me, like, okay, so the emulator does not, like, it runs QuickJS. So when you build a Lambda, a Node.js Lambda, it actually runs a QuickJS engine. It's not the real Node runtime that you run in AWS. So it doesn't support some of the, like, the Lambda handler, for example, that is currently the official one on my website, doesn't work with QuickJS. So the workshop version of the website, the workshop, is using the JavaScript that works with QuickJS. And then the second thing is the QuickJS can invoke, I don't know, it can invoke like many services. So you can invoke from your Lambda, you can invoke DynamoDB to write data into it. You can do this and do that. But you cannot invoke another Lambda. Like that's blocked in the emulator on the version that I had last week on Thursday. But Fable just highlighted part three, step four fails because the emulator doesn't support one Lambda invoking another Lambda. And then it built a little patch. It just patched that one emulator feature and just made it so that it can invoke a Lambda. Another thing that the emulator didn't do is IAM policies. So part of the workshop is to actually create a DynamoDB table to record data. So Lambda has to get access to write into DynamoDB. And this is a very important step of the workshop because it teaches you how to grant permissions across different services in AWS and how easy it is with AWS CDK because you just take your table and then you grant your Lambda to have access to that table to write or read only. And this is a very important part of the workshop. But when the emulator doesn't support IAM policies, it just gets 200 OK on the first step, even when you didn't grant any permissions. So again, Fable built the IAM policy component in such a way that if you don't grant the table permissions to be written to from the Lambda, it will actually fail the workshops and you have to go through that step. Again, small little patches that Fable did, then build a new binary of the emulator, put it into the container, build a container image, spun up and do the whole test. Fable did all of that in just two, three hours to build the whole workshop with the custom emulator and everything. It's insane. That is insane, sorry that's very hard for me to I mean, it's a very, very well-defined problem. It's very easy for Fable to define what is the steps that need to pass. It's very easy for Fable to hand that off to Opus to build and then verify that the result works. So when you have a very well-defined task, You can have it completely autonomously be done by the agents, depending on how many tokens you have. Yeah, I've got to try this emulator. I've got to try. I was just noticed that there is an IAM directory here, but. Well, maybe the version that I tried on Terzid is already outdated, right? Maybe if I try a new version of the emulator, it will already do some of this stuff. I'm also What happened? I wonder how Oh, shit. I wonder how it it just like smithy changes So guess what? and stuff like that You know what happened? what So my whole workshop is on spot instances, which is like 60% discount. The instance got terminated. I guess I'll pay $10 instead of $3. issue ID just seeing the engineering behind this is quite interesting it's like it's all no one's name right it's this particular issue fixer agent So when AWS announced this emulator, they said it's been built by agents running for I don't know how long. They said like thousands of agents have been building it over a certain period of time, and they don't accept any PRs. I think it makes total sense. They have just an agentic process running along, just slowly ticking off and verifying and building everything. Unfortunately, for one of the big projects that I'm working on, but I the gaps are too big. I can't use it for a lot of stuff. Do you see the spaces between the commits? It's kind of insane, right? It's like two minutes between each commit here. i wonder if they if they're still using git or they are using like if they used already another cicd mechanisms and version control It's going 24 hours a day. It almost doesn't make sense. Look at this. It's like two. There's a two-minute gap. I mean, I'm exaggerating, but it's about that. Between every commit here, it's... I can show you some of my repositories with one or two minute commits and 100 commits built by agents. Oh my God, I can't... I've got one of those sort of like feelings of dread almost. Like we're just living in a new fucking era. When shit like this is... When you look at a commit history... Like I remember... You keep telling about all of these tools to control commits, and I keep telling you your metrics are all wrong for this new, like, agentic era. Everyone is moving to the higher layer, the intent layer. You cannot verify or review code. You have to define, like I said, you have to have Yeah, I know that. a very well-defined problem. I know that, Vincent. It's just that I just can't help but think that it wasn't that long ago when I was sitting with my team, going through the commits that landed on Trunk in the last day and making sure that the team was aware of what was happening with the code base, including myself, as a lead. but that you don't relinquish that this is a big problem with the way that a lot of people are adopting AI and vibe coding, they think that you completely do not control anything but you control very much the verification process and everything, right? yeah yeah yeah i mean but it's still come on man i just wanted to acknowledge that like the world has changed like freaking big time when when you see things like this it's insane how what is the extent of the good history here so it's it all kicked off in october It's not even that long. It's a folder. Yeah, I've got to study this. I mean, this is fascinating. I mean, this is a game changer. And is it fair to say that AWS is doing pretty well? I wonder how the other clouds are doing in the face of AI modernization or what do we call it? I think this is just basic everyone. AI-ification? I mean, in terms of when you say cloud providers, I think everyone has to adopt this, right? You see, Microsoft is adopting it very well. sure but like i'm you know since we're a podcast here we're quick Microsoft did a really great job adopting it that's sarcasm i'm sure yeah because Microsoft didn't really go down for like 24 hours at all what is the google gcp story because everyone seems to to bash Google for having crappy models. have you forgotten that Microsoft 365 65 was down for 24 hours. I'm talking about Google now. Everyone's bashing Google. Google. Yes. Why we... Googly gun drops. So everyone likes to hate on Google because they have these... They were TensorFlow. They had these custom hardware built for LLMs. And then Gemini was like blowing everyone out of the water with the 1 million context. But they didn't really recover from that, right? They're still kind of struggling, it seems. Well, yeah, that's the story. They open sourced their Goldfish versus Elephant practice. That was a blog post that became, they made it a framework. You know this one? Gold? No. You don't want to share that? Yeah, Google Goldfish Elephant. So there was this Google engineer that shared months ago. Where is the share? We talked about it on the podcast as well. Because to be honest, he's not introducing anything revolutionary. To me, this is just exactly a repeat of what Steven Yege wrote, beats are for Blobfish. There's a couple of things that I really liked about this. It's a good summary. So Beads are for Blobfish is very clear. I like the comparison that an agent is born with zero context. You fill up the context, and very quickly the agent is useless because the context has filled. So beads is a way to keep agent context tracked. Do you still use beads, by the way, or you don't feel the need? Honestly, I need something like that. With Atlassian MCP, the V2 one, it's better. But anyway, let's get back on points. So then you have this one from Elephant and Goldfish model, which there was an original Medium article. Dave Ranson did that article, I think, in March, or was it? 2026, where he talks about April 2026. and he shared about some of the practices that within the engineering teams within Google were following. And they're basically grill me type of practices, right? You ask the agent to ask questions for you to fine tune onto your... Ask Find the alignment between the user intention and the agent's understanding, right? Because you can say one word and it can be interpreted into many, many different ways. so there was I mean, I like to use a prompt like, ask me how I want it. Ask me how I want it. yeah so he has some really good examples and prompts in here which all honestly like Matt Pocock's grill-me skill with domain driven design you know is the same except it's a little bit more useful because people you know kind of really like the ideas of domain driven design and bounded context and Matt Pocock's skills really help there. So it's very similar. You ask questions, you find alignment, you find a shared language. And that's your, if I understand correctly, this is like your elephant, right? You build your whole, your overall view, no code, right? There's no code. You're defining a shared language. And then you have your goldfish, right? Hmm. They come along and they pick up pieces of work in that shared language Yeah. and they deliver. Okay, thanks. I'm beginning to understand now. Thanks for the explanation. yeah and then they made it they made it in october they made it like in october 22 they said here's the the thing as a as a framework for you to use so So what does the elephant look like? A bunch of intentions or... What is... yeah i'm actually curious because we we actually we use Google Conductor as well right so i think it's like an evolution of of uh of conductor or a different approach which it's like there's these different schools of thoughts, right? You have the spec-driven development, which people say is like waterfall. You have the, you know, Conductor, Spec Kit. All of those are spec-driven development. Then you have the get shit done. You know, I want to work closely with the agent on small little problems and, you know, iteratively develop them instead of like more waterfall type of processes. as conductor was their attempt in this spec- driven development world. And I feel like I never used Get Shit Done. I have friends who really like it. I feel like maybe the elephant goldfish model is similar to that. Again, I'm out of that here. I actually don't know. I haven't tried these because I'm still using a Spec Kit-like variation. So speaking of skills and approaches and techniques, I recently tried P-Stack from Potato on one of my apps I can maybe share. yeah i i use a lot of her skills but like i said i have my own so i i don't i just take what i like from other people Oh, yeah, I used her verification skill. yeah i use the same it was a bit weird because I had my end-to-end tests already but it just seems like I guess now I have a skill I don't know. so it's a little bit easier, it's a little bit more Asian friendly and then the other thing I thought that was really interesting am I sharing my screen? Yes, I can see the verify LTA bus. so what was interesting I thought was was the fact that it managed to discover the features of my Singapore bus app, like the fact that it auto-refreshes, the fact that it shows you the closest stop. yeah what is this opus did this like custom Opus 5.5 and the verification, create verification skill. yeah nice And I thought that was really interesting. And then you can also ask it to create a feature map. are those specs Playwright Sorry? are those spec.js Playwright specs or what are those they are hand to hand drive Yeah, yeah, that Playwright. yeah it's what I told you It's Playwright. how people should use Playwright right no stupid MCP you write proper specs and you run them and the models get you know nice feedback Yeah, yeah. I mean, I was doing that before. like code mode for tests like in fact it didn't seem worryingly it didn't seem to pick up on some of my old specs so whatever uh huh so but but maybe we should talk about the ideas behind these things rather than like but i mean it's it's because honestly Lauren Tan Matt Pocock they both have a lot of overlapping ideas they have some really good presentations um Matt Pocock has a course that my colleague did yeah yeah and um it's interesting but but i really like uh Lauren Tan's presentation where she highlights if you follow her on Twitter you basically saw everything that she presented but in the presentation she really like organizes it in a very nice Yeah, yeah. when I see it I'm like shit I wish I was presenting like that because it ties everything neatly together and how everything is Yeah. Yeah, her AI engineer video was great. connected talk is really good and and yeah like i um yeah i just wanted to quickly show the the artifact where it sort of yeah like you know shows shows the the feature map a little bit better hopefully i'm showing the right the UI. screen okay uh yeah Potato, Pocock they're they're great and it and the fact that they're so popular Yes, it's the Itobus feature map. that means that they're actually filling a huge demand, right? but they have different angles like Matt Pocock is a great explainer he has beautiful courses I mean, yeah. he visualizes things and explains them really well but i feel and this is my personal opinion that he doesn't have the hands-on large project experience that then you get from Lauren Tan. Mmm And Lauren Tan is great at explaining as well, Mmm but she has all of these hands-on skills. yeah She's like, yeah, but it shows that she actually runs 3,000 PRs merged in a month, yeah even if it's a stupid metric, and she admits that. She says, it just visualizes a little bit Vincent your explanation of the differences there what we are doing. was 100% Exactly how you feel, right? exactly so yeah Lauren is Yeah. doing great and you know how it is it's like you and I are bashing away at AI. And when I use the verification skill, I go, yeah, but it has value. But I'm at the same time like, yeah, I knew that. But yeah, I mean, yeah, exactly. Yeah, that said, my depreciating they make it very applicable they make it very applicable comments aside, it's very good. And like the other things that she's doing around formal verification. I think we talked about that last pod. I use her state machine graph and state machine like um Quint modelling and uh what's the other one TLA+ yes i run them i literally had an agent TLA+. last night doing a full because i had the race condition um and then i want to make sure that Ooh. the fix meets like the you know doesn't actually solve the race condition in a in a proper way so i usually the way it ends up with me i don't take on her stack but i ask opus uh here's a tweet where she talks about state graph and showing all of the ways that the state machine can end up in an invalid state or a faulty state. So I just point Opus at that tweet and also the repo where it lives, and I tell it, clone it locally, find the skill, and see how it applies. It's all on the cursor. the tweet Usually, yeah. Usually I ask Opus, like, find how I can apply it into this particular problem that I have and then it picks up the skill and then I inline it or it's a separate agent usually there's so much good stuff here actually I like the no comments one because I do I don't know about that one she could explain it better than I can though you know sometimes I can tell you AI just where I don't agree with it AI adds a lot of comments which can cultify the project. So this one is about removing those comments that sort of make no sense anymore, which is well i can tell you i agree a lot with what she says but and i and one of the biggest issues that ... biggest concerns that i have is you know how agents are working on independent pieces of work right so sometimes and this is a very common pattern that they do is they have to put a mock somewhere or they will put like a little proxy or like a function um like a placeholder that does like actually mocks the function doesn't do anything like not implemented error which Yeah, I work around. not a workaround um of a seam that you don't have the other side of the implementation or somebody else is working on it you have a clear contract and you may have an api mock between them Okay, okay, yeah. Right, right, right. right right so it's not a workaround it's it's it's a placeholder it's a temporary like function and i'm always afraid that they they will be lost and so one of the biggest things that she says is if you give the agents permission to put to do's then another agent comes along sees the to do and follows that pattern which obviously absolutely you know to do's get out of hand. But I want to have a way for agents to really know that, like, okay, I can put a to-do Mm-hmm. here. And it's very tricky, right? The to-do should be... So this is where I think comments still need to be there. Yeah, that's quite a lot. I mean, again, a lot of these skills... You know, another yeah i love this i love this skill here i gotta use this one bro bro the presentation that I don't know if I shared it with you, but there was this five days ago, AI engineer released a video from Anthropic. I forgot her name, but she shares, she talks about hmm how you used to use Opus 4.6, how that evolved towards 5.5 and 5.2 Fable. So the video came out after they released 5.2 Fable. And she talks about, this is like how you change, because this bro skill, for example, I think it was very relevant when you have Opus 5. mm But then they really trained Opus 5.5 to make it better. I will admit I'm still having pages and pages of agent text that I have to go through where I'm like, also bro, I want you to shorten it. For this particular question that I have, I want you to keep it short. Um... yeah the on the topic of skills uh in in my in my client they have a skill repository but it's gone stale like there's no one really actively updating it and uh and hence i'm always careful to disable the company's skills and what's my point here I don't know, Clay. Sorry. these these skills from a third party i.e potato are great and and i would i would use them judiciously and I have my own skills of course but the I really feel I wanted to basically make a point that the skills that are produced by by my client really need to be updated Same edition as So this is the thing that I wanted to go over, which I think highlights the problem that you're mentioning. Skills need to be... I think that's what PeaceTech also... Sorry, Potato, Laurentan really highlights and Matt Pocock, I really like the word that you use. Their interviews are interesting. But it's called gardening, right? We're basically making sure that the agents and the plants can grow and are flourishing. So we are responsible for maintaining those skills. Hmm. identifying where the agents are repeating mistakes Hmm. and curating the environment for the plants to grow nicely. Yeah. Matt, one thing that struck me about what Matt said is the vocabulary, the power of certain words is so critical. And I often find that every company has their own glossary, right? But sometimes it's just not really well stated anywhere. And it needs to be, right? I went very hard in on the glossary concept and the grill me and the domain driven. The danger is you need several iterations on getting it right. And it's very annoying. You know, you can do the glossary definition with the agents of terminology. But you have to be careful because what happened with me is some words, I forgot what they meant. So when we talk and then we use this glossary, I was like, can you repeat to me again what exactly that word means? mmm Or is that including that particular concept in that word? Haha Because I'm not sure. So this glossary, yes. That's what Matt talks about. He says what people always really like about my course is that they actually really like domain-driven design. They write the shared language. And it's a rediscovery. and i think it's a really like a lot of these original um practices uh of of like software development get a new breath of fresh air like life uh in this new new age so they become a bit Yeah, but just a quick aside about glossary. more practical when you work with agents but but In the real world, in enterprises, especially when they operate in different markets, the meaning of different things becomes pretty blurred depending on a lot of environments or a lot of context. So it's a hard task for any enterprise. yeah that's kind of what my my team members also were like i have um he's kind of a bit of a thought leader for me at least in terms of like domain driven language he highlights how how it really cuts at the edges like you just highlighted he said the same thing um but but right now it was Hmm. more a language that me and the agents were sharing for a very like i'm working on one one platform and there's different systems within and they each have their responsibilities so we don't have too many other people it's just me and the agents and i'm trying to get more people working on this but like it's not that like when we defer define one term um if i if it's a if it's an overlap between the bounded context and the agents will very quickly highlight and i don't have a lot of problems like disembogating those but this is the talk the one that i wanted to to mention this is the Rethinking Claude Code best practices um by lydia from anthropic All right. So, oh, so you've taken screenshots and you've... what I wanted to You're sharing that with your team or something. yes I was sharing it with my team but I didn't take the screenshot where she talks about the curating part because cloud code is very well designed based on many many users within Anthropic and how they use it so she goes over some of the practices that changed this is how we used to do things this is how we do things this is how you migrate from your existing Claude Code setup towards the new way of how you should steer the model and so she highlighted one of the skills was like slash doctor um is it prompt improve or or it's basically a skill that helps Claude highlight old ways of doing things that opus 4.8 needed or cleaning up your CLAUDE.md cleaning up your skills and the words within your skill if you have like big important, important. Yeah, that's... She says today, that was necessary back then. Yeah. But today, it actually causes the model to get distracted and to maybe do steps that it can actually skip because it knows more. Yeah. Like Fable 5.2 is really good. And also, this is really how I've been using Fable. And she's highlighting and that's a pattern that they are making available as a new feature called projects in beta. But you can do this already without, without and we've talked about this with with herder and and rex maybe but you have your fable as the orchestrator that keeps the overall goal and then you have it can spin up a separate Yeah, that's been up. session like you highlight i actually have dedicated agents that um like i have an opus 5.5 agent that needs to deliver the work and it does that by actually like spinning up sub agents but even then they say you shouldn't ask opus to spin up too many sub agents anymore apparently it's better when it does itself these days, himself, themselves, I All right. So, do you manage to use Rex or you just orchestrate inside Claude yourself? don't know. So, so we have this initiative about always updating Out of curiosity. your tools and finding time to breathe. So I haven't taken a pause. I think this is another thing you need to make time for. I spending a lot of time to read and I try out skills to fit them into my existing workflow. For example, you asked me to use beats, I really like a proper task tracking that I can hand off to the agents. Right now, I feel like I regressed because Mm-hmm. of because of pressure. So I really need to take some time to to re configure. Yeah. to consolidate yeah yeah there's there's a lot of i mean i love this actually i love uh rethinking or looking at the way i do things and improving on it and you have to do a lot of that sort of activity It's a luxury you have if you don't have like, maybe if you're in an enterprise environment where there's a lot of approvals and paper pushing, but if you're in a very like, delivered this yesterday environment, it's very hard to stand still and improve your tools. maybe yeah like It's a luxury to stand still and change your workflow. yeah here's something I could do with your advice on I was struggling to explain myself. A colleague had a PR for driving an agent in AgentCore, like one of those strands based ones. And his prompt was like a page long, a page of text, like do this, do that, do this, do this, do that, do that. Ultimately, all I was doing was working out if a security finding was a false positive or not. And instinctively, how do I explain this? But if you're using Opus 5.5, you don't have to give it much to go on. You know, this is your context. This is the source code. This is the finding. Is it a false positive or not? You know, in my opinion, I could use Jev almost. um but but though my colleague was like like a full page like thing oh i need to uh what do you call it reach consensus maybe run this with other agents to find out if it's a finding and i was like no no you've made it too complicated dude like models are good nowadays how old is the code well this this pr was was one day old but you know what i mean it's like like yeah i'm trying like today I mean like this week like this month yeah this week i'm trying to tell my my colleague here that like you don't need to engineer something like you would do maybe three months ago you can quite easily get by Did he write everything himself or did he ask Opus to write it? I think he must have got Claude to write it. So the next thing I need to figure out is that this is my task today, Yeah. my stretch goal, is that given his extensive prompt, it's a bit like using Claude's skill creator. I need to basically prove a point that his longer prompt is not any better than just a simple prompt. Yes, evals. And obliration, what's it called? Obliration? uh how to operate It's called obliration. Prompt obliration. It's chaos engineering on prompts. You basically delete paragraphs and then run an eval and see if the agent is still working or not, randomly. okay i don't know if do Strands or AgentCore give any frameworks for that because i need i need to do that now that's that's my one of my things i need to figure out But I was struggling to argue with my colleague, like, please, it doesn't need to be so complicated. Just have it simpler. i mean instead of telling him did you ask him how he evaluated and came to that prompt Well, I think he's gone through a whole bunch of iterations. It's just built on one, just a cue. Yeah, because I had the same with Opus when I was working on an AI reviewer for Terraform plans, where it was really constrained. And we wanted to get like kind of a fixed response as the end result. So part of it, oh, no, I remember where it was. It was in elicitation. So when an agent needs to stop and elicit more information, I've had a deterministic, like the result from the agent would go into another, you know, into a deterministic function. So it needs to be able to parse that, right? So actually, I think it was using a tool to verify that the response was matching the rules before it then resends it out back because then it goes into Slack that gets formatted. into an um an interactive card that gets presented to the user to say a b or c or another response it's a bit like ask user question but from AgentCore to Slack right so um and i also ended up with Yeah. with with an iteration an iterative loop where he kept saying okay i'm running the agent uh and i'm i'm verifying um if it's responding according to the format that we expect and then you know it keeps growing, keeps, keeps, you know, modifying and it says all the instructions aren't good enough because the agent came back with something that wasn't right or it took. yeah another thing that caught me off guard i didn't realize Strands had memory so i was just like i was a little bit yeah i was i was i noticed that it was building up on its It's AgentCore memory, I think. memory and i'm like i don't actually want that i want it to be lean and mean because i'm like when I switch up a model, you know, I'm developing on Sonnet and then I might switch to Opus. I feel like all this memory that you've gathered here is just dust, right? I mean, it's not going to help us in the future, really. memory systems are a huge market as well It might be useful in the sense that, like, it's maybe seen the finding before and it doesn't need to re-evaluate it, but then still it's kind of garbage in that sense. oh it's obliteration prompt obliteration yeah so anyway that's my task today and I should actually get back to work Vincent I need to do this eval on agent call and this Obliteration. It's obliteration. I need to do some prompt evals No, yeah, but but strengths has a lot of eval libraries and capabilities for you to run. I wouldn't be surprised if they even provide this as a capability. I know that Omni from AWS is an offering that allows you to take you can even get all of the sessions from like actual live sessions and then feed that into your agent setup and then rerun evals over and over to to see if you hit the mark on your quality ok control um and then yeah the the Claude skill- creator skill does that like locally very quickly yeah yeah okay yeah i'm gonna i'm gonna do that right now i'm excited uh but you have whole solutions yeah you know like judge evals even master like back more than a year ago March 2025 was doing yeah i guess the one issue with invoking the AgentCore is that it is a little bit slow that it's it's not definitely which one the Bedrock AgentCore when you invoke it on the cloud it's a bit sluggish what is slow just getting up and running I think I have some ideas to make it fast anything else you want to say in the podcast besides like the video comment on the pod I promised you a demo on policy, like on basically, yeah, the PII one. PII logging stuff. I have it all built, but it's the second part of the workshop. So the first part is running the workshop, getting information, like the dashboard, the trackers of the humans, the people, Mm-hmm. and then doing agentic work without actually exposing PII data by enforcing strict policies that block these calls. I've got the demo working, and thanks to your questions, I created it as an example of my talk. okay we can look into that yeah this this topic of of logging Next time. if you're a mature enterprise right you you have your production uh boundary in place right so that people don't view the logs of production unfortunately um it's hard to get there sometimes well it looks like like for example state graph their positioning is you can't trust an agent so you scope the token so they generate short- lived tokens scoped for certain tasks so that the agent can't do simply that thing, which is similar to having a policy that blocks certain tool calls, right? An MCP tool call. yeah but but there's a lot more to it because like because people But it would also solve your code mode problem because if you execute script and the token isn't allowed to do things, then even if the script runs, the token will block it because the token doesn't have the scope for it. So it could be one way of doing it. Your agent lives in a sandbox. It doesn't have any credentials. The requests that go out, the tool calls that go out Ok are injected with tokens that are scoped for that particular task. So if that token cannot read the logs then even if the agent uses a script to do A, B, C, D and one of those steps is read the logs the token will simply block it. So you don't need to do your complicated setup. yeah i i i get that but there's more to it like for example uh it's a human problem here really because like you have a human who who owns a particular service in a production or enterprise environment, you don't want that engineer to have production access to the logs. But, of course, depending on where they are and the maturity, they might need access to the production logs because they're debugging a problem, right? Because the software... and then a very common setup in an enterprise is to have zero trust and to have break rest access so the people don't have standing access to production except if there's an incident and Yeah, yeah. they raise the ticket. yeah but that but but but it gets a bit so so people maybe break that glass and then they copy in the logs into into claude and then you kind of have the same problem you know what i mean it's like you could you could have all these controls around your agent not doing it but then many in many cases uh engineers nowadays will use the break glass and just copy the freaking logs into Claude. but then your enterprise device management system has set up all these rules inside Claude and these monitors and metrics. And even if they use a not allowed cloud enterprise, sorry, cloud session that gets private logs, then they get flagged. Which you said people are not... Yeah, like a detective control. Well, let's not talk about it now. It's actually pretty fresh stuff at my workplace, so I don't want to talk about it right now. it's a common problem it's a problem I hear a lot Exactly, exactly. do you think if you move to background agents you remove that problem No, I don't think it's a technical problem. do you understand background agents It's more of a... No, I don't. it's an agent that doesn't run on the engineer laptop it doesn't I think it's one of the talks of the evolution that she talks about how people work with Claude like you used to steer it you used to need to see the session you used to need to read the messages to understand the concept we're moving away from steering the agent is no longer running on your laptop the agent is no longer running with your permissions there's no way for you to copy logs into it interesting that sounds very mature though okay i need the toilet it's my talk at Red Hat Singapore on the 27th it's about background agents and the different enterprise agentic adopting patterns Okay, cool. Okay, well, I'm glad to talk to you because you can help me solve these problems. And if anyone's listening and they know better, please comment below. And if you learned something... and if you're in Singapore come to the Red Hat meetup about the enterprise agentic adoption patterns Awesome. with background agents to solve these problems Okay, cool. Thank you. Okay. Cheers, Vincent. yeah Thanks. Thanks ever so much. See you.



