WEBVTT 00:00:01.040 --> 00:00:03.679 Hey, I also shaved. 00:00:03.679 --> 00:00:07.040 Oh. Oh. Oh. Oh. 00:00:07.040 --> 00:00:10.240 Oh, there's a bit of an echo. 00:00:10.240 --> 00:00:13.519 Maybe it's my headset feedback. Test. 00:00:13.519 --> 00:00:15.200 Test. 00:00:15.200 --> 00:00:18.000 Better. Um. Oh, damn. What have I done 00:00:18.000 --> 00:00:21.760 with my AirPods? 00:00:21.760 --> 00:00:26.560 H. 00:00:26.560 --> 00:00:28.960 I should have this this Na'vi signed 00:00:28.960 --> 00:00:32.000 jersey more in in in view. 00:00:32.000 --> 00:00:33.680 >> What is that about? 00:00:33.680 --> 00:00:36.079 >> This is Oh, this thing. I I don't know. 00:00:36.079 --> 00:00:38.160 This just Zeus 00:00:38.160 --> 00:00:41.520 >> the goat Na'vi team 00:00:41.520 --> 00:00:42.719 >> Guardian Edward 00:00:42.719 --> 00:00:44.559 >> talking about things I have no idea 00:00:44.559 --> 00:00:46.160 about. 00:00:46.160 --> 00:00:49.360 >> Counter Strike Global Offensive lineup 00:00:49.360 --> 00:00:54.320 from 2011 or something. 2013 maybe. 00:00:54.320 --> 00:01:00.239 Yeah. the first uh okay so 00:01:00.239 --> 00:01:02.239 I've just taken a more uh what do you 00:01:02.239 --> 00:01:05.439 call it security thing and like I get 00:01:05.439 --> 00:01:08.240 asked things about things that usually I 00:01:08.240 --> 00:01:11.360 would probably frankly not care so much 00:01:11.360 --> 00:01:14.479 about like is this for real should I be 00:01:14.479 --> 00:01:17.040 worried about it an active attack is 00:01:17.040 --> 00:01:19.920 happening claw right now 00:01:19.920 --> 00:01:23.119 >> yeah I've seen that 00:01:23.119 --> 00:01:25.840 uh what was it again you There's so much 00:01:25.840 --> 00:01:26.640 packages 00:01:26.640 --> 00:01:29.040 >> with security stuff. So much like what 00:01:29.040 --> 00:01:29.680 do you call it? 00:01:29.680 --> 00:01:31.280 >> Well, 00:01:31.280 --> 00:01:32.479 it's just like guys, 00:01:32.479 --> 00:01:34.880 >> it's just continuation of team PCP, 00:01:34.880 --> 00:01:36.640 right? Isn't that like the North Korean 00:01:36.640 --> 00:01:38.960 linked hacker group that 00:01:38.960 --> 00:01:42.000 >> did the shy hulut and they basically 00:01:42.000 --> 00:01:43.759 made the the worm public and there's 00:01:43.759 --> 00:01:46.159 been several worms similarly and 00:01:46.159 --> 00:01:48.399 apparently one of the redhead employee 00:01:48.399 --> 00:01:50.560 GitHub login was compromised and some 00:01:50.560 --> 00:01:54.479 packages were published 00:01:54.479 --> 00:01:58.000 but but has it I mean like what isn't 00:01:58.000 --> 00:02:00.240 this the responsibility of the clawed 00:02:00.240 --> 00:02:03.360 code maintainers to fix why is it 00:02:03.360 --> 00:02:06.560 blurted out to the wider community. You 00:02:06.560 --> 00:02:07.759 know what I mean? 00:02:07.759 --> 00:02:12.239 >> It's not Claude Code specific though. 00:02:12.239 --> 00:02:14.560 It's just like the problem is that these 00:02:14.560 --> 00:02:16.800 packages are often propagating through 00:02:16.800 --> 00:02:19.120 popular frameworks like in this case 00:02:19.120 --> 00:02:22.560 it's like a spring uh or Java type of 00:02:22.560 --> 00:02:25.680 framework and also often via VS code 00:02:25.680 --> 00:02:28.400 plugins because VS code plugins are or 00:02:28.400 --> 00:02:31.920 the market base is also like not really 00:02:31.920 --> 00:02:32.560 properly 00:02:32.560 --> 00:02:36.319 >> so is a theory that like uh Claude code 00:02:36.319 --> 00:02:39.599 would just install these things while 00:02:39.599 --> 00:02:40.160 vibing. 00:02:40.160 --> 00:02:41.840 >> Yeah. Yeah. Sorry, I'm putting my legs 00:02:41.840 --> 00:02:44.400 up because I'm I want to chill. So, 00:02:44.400 --> 00:02:45.920 yeah, that's that's one val that's one 00:02:45.920 --> 00:02:47.360 thing though, but cloud has been trained 00:02:47.360 --> 00:02:49.360 on it. Like opus 4.8 when I was trying 00:02:49.360 --> 00:02:52.239 to do a hindsight uh setup immediately 00:02:52.239 --> 00:02:54.480 said like hey this package that I'm 00:02:54.480 --> 00:02:57.120 trying to download for pi from Python I 00:02:57.120 --> 00:02:59.840 cannot verify that it is the official uh 00:02:59.840 --> 00:03:01.519 Python package for this framework that 00:03:01.519 --> 00:03:04.000 you want to work with and then it um it 00:03:04.000 --> 00:03:06.560 blocked it. I don't know if it was auto 00:03:06.560 --> 00:03:08.720 mode, but I think the the model itself 00:03:08.720 --> 00:03:10.080 kind of had this 00:03:10.080 --> 00:03:12.560 >> incentive to not just randomly like it 00:03:12.560 --> 00:03:15.440 it it it suspected uh name squatting. 00:03:15.440 --> 00:03:16.959 But yeah, name squatting is scary, 00:03:16.959 --> 00:03:18.879 right? That's been people have been 00:03:18.879 --> 00:03:20.720 warning about name squatting with LLMs 00:03:20.720 --> 00:03:22.400 for a long time. Basically, how do you 00:03:22.400 --> 00:03:24.239 know the LLM isn't just like 00:03:24.239 --> 00:03:25.760 hallucinating a package name and then 00:03:25.760 --> 00:03:28.319 somebody has has, you know, published 00:03:28.319 --> 00:03:29.920 that package and then the LLM just goes 00:03:29.920 --> 00:03:32.560 ahead and downloads and installs it. 00:03:32.560 --> 00:03:34.400 >> Yeah, it's it's interesting. It's 00:03:34.400 --> 00:03:37.599 interesting to think that 00:03:37.599 --> 00:03:38.959 >> but I think overall 00:03:38.959 --> 00:03:41.280 >> attackers are going for the uh the the 00:03:41.280 --> 00:03:45.519 tra the train models um sort of allow 00:03:45.519 --> 00:03:47.200 and it takes some time for them to 00:03:47.200 --> 00:03:48.159 update 00:03:48.159 --> 00:03:49.920 >> but it's also because of the ecosystem 00:03:49.920 --> 00:03:52.080 stance like for example VS code plugins 00:03:52.080 --> 00:03:55.120 have um way too many permissions they 00:03:55.120 --> 00:03:57.360 nothing in VS Code prevents a plug-in 00:03:57.360 --> 00:03:58.799 from reading whatever it wants on your 00:03:58.799 --> 00:04:00.319 disk and and maybe exploring 00:04:00.319 --> 00:04:02.319 >> I guess it's the same with it's the same 00:04:02.319 --> 00:04:04.560 sort of black hole of uh browser 00:04:04.560 --> 00:04:06.640 extensions, they also had like a poor 00:04:06.640 --> 00:04:08.080 security model for the longest time, 00:04:08.080 --> 00:04:08.480 right? 00:04:08.480 --> 00:04:10.239 >> For the longest time. Yeah. So now that 00:04:10.239 --> 00:04:12.319 becomes more apparent. Same with like 00:04:12.319 --> 00:04:14.319 all these packet manager managers for 00:04:14.319 --> 00:04:17.359 npm. Like PNPM is now very like by 00:04:17.359 --> 00:04:20.400 default does not execute hooks which 00:04:20.400 --> 00:04:23.199 breaks some package installations. So uh 00:04:23.199 --> 00:04:24.400 but they're they're becoming very 00:04:24.400 --> 00:04:26.160 aggressive. I think PNPM is probably the 00:04:26.160 --> 00:04:27.600 most aggressive in disabling these 00:04:27.600 --> 00:04:29.600 things for security reasons which is 00:04:29.600 --> 00:04:31.280 also why a lot of frameworks are moving 00:04:31.280 --> 00:04:34.080 towards PNPM because they're very 00:04:34.080 --> 00:04:35.759 aggressively blocking this type of 00:04:35.759 --> 00:04:36.960 exploits and works. 00:04:36.960 --> 00:04:38.479 >> Cool. Cool. Cool. 00:04:38.479 --> 00:04:40.160 >> Yeah. I mean I guess this stuff is all 00:04:40.160 --> 00:04:44.720 all entirely predictable now. Um I I 00:04:44.720 --> 00:04:46.400 messaged you on WhatsApp about like an 00:04:46.400 --> 00:04:50.240 inter do you think that you use AWS KOD 00:04:50.240 --> 00:04:52.639 factory or co coder whatever was it 00:04:52.639 --> 00:04:54.560 called? Kodi 00:04:54.560 --> 00:04:55.840 Ko. 00:04:55.840 --> 00:04:58.560 >> No, not Kira, but like code. Um, yeah, I 00:04:58.560 --> 00:05:02.320 use code art. Is it code artifact or 00:05:02.320 --> 00:05:03.840 >> Yeah, I think it's code artifact. God 00:05:03.840 --> 00:05:05.440 damn. Now you you put this in my head 00:05:05.440 --> 00:05:06.720 and I can't remember. 00:05:06.720 --> 00:05:08.160 >> Code artifact. Yes. 00:05:08.160 --> 00:05:09.440 >> But does that have 00:05:09.440 --> 00:05:10.400 >> artifactory? 00:05:10.400 --> 00:05:14.720 >> Yeah, code code artifact. So in light of 00:05:14.720 --> 00:05:17.600 PMPM doing its thing, 00:05:17.600 --> 00:05:19.440 is there I don't actually quite 00:05:19.440 --> 00:05:25.039 understand what the AWS code um 00:05:25.039 --> 00:05:27.360 what the the AWS code artifact does. It 00:05:27.360 --> 00:05:30.400 just gives a local copy of known good 00:05:30.400 --> 00:05:32.479 stuff, right? That's in your that's in 00:05:32.479 --> 00:05:35.919 your organization. For me, code artifact 00:05:35.919 --> 00:05:39.680 um came into play maybe five years ago 00:05:39.680 --> 00:05:43.840 when um I had to inherit like somebody 00:05:43.840 --> 00:05:46.880 left and there was this shadow IT uh 00:05:46.880 --> 00:05:49.600 what's it called? It's not the JROG one, 00:05:49.600 --> 00:05:54.240 but oh Nexus. Somebody had set up a 00:05:54.240 --> 00:05:55.919 private Nexus instance on a Kubernet, 00:05:55.919 --> 00:05:58.080 you know, like Helm install Nexus, 00:05:58.080 --> 00:06:02.400 right? The classic Helm false confidence 00:06:02.400 --> 00:06:04.479 of yes, just install this package and 00:06:04.479 --> 00:06:06.880 now I've got a whole service on my 00:06:06.880 --> 00:06:08.960 >> a huge stack. 00:06:08.960 --> 00:06:10.960 >> Yeah, but I mean it's not huge. it just 00:06:10.960 --> 00:06:12.960 it's it's not production ready like it 00:06:12.960 --> 00:06:15.840 doesn't provide SSO people were sharing 00:06:15.840 --> 00:06:19.280 and like the CI/CD was using um you know 00:06:19.280 --> 00:06:21.840 user credentials username and password 00:06:21.840 --> 00:06:23.440 and these credentials were shared all 00:06:23.440 --> 00:06:26.400 across the the teams and then it started 00:06:26.400 --> 00:06:27.919 failing 00:06:27.919 --> 00:06:30.240 um because it started falling flat on 00:06:30.240 --> 00:06:33.759 its face uh due to CPU memory and disk 00:06:33.759 --> 00:06:35.120 restrictions because this was also 00:06:35.120 --> 00:06:37.759 stateful and and and you know this is 00:06:37.759 --> 00:06:40.400 the classic somebody can't be bothered 00:06:40.400 --> 00:06:42.400 with waiting on um you know a platform 00:06:42.400 --> 00:06:44.000 team to do this properly and just goes 00:06:44.000 --> 00:06:45.759 ahead and and Helms installs it into one 00:06:45.759 --> 00:06:48.080 of the clusters and then it comes back 00:06:48.080 --> 00:06:51.600 to platform anyway. Um but the migration 00:06:51.600 --> 00:06:54.800 to AWS coder effect was just so smooth 00:06:54.800 --> 00:06:57.360 because it all piggybacks off the IM 00:06:57.360 --> 00:07:00.080 authentication and we were using AWS SSO 00:07:00.080 --> 00:07:02.800 which is tied into our uh entra ID or 00:07:02.800 --> 00:07:06.080 octa um so it's very to be honest 00:07:06.080 --> 00:07:07.440 >> you don't like what 00:07:07.440 --> 00:07:11.039 >> I I I love I am it just it's my 00:07:11.039 --> 00:07:12.720 >> it's the perk of being in the AWS 00:07:12.720 --> 00:07:14.319 ecosystem right the moment that you have 00:07:14.319 --> 00:07:16.160 a service uh and it's properly 00:07:16.160 --> 00:07:18.080 integrated with IM now you you can 00:07:18.080 --> 00:07:20.720 resolve a lot of the like um access 00:07:20.720 --> 00:07:22.800 permission roles. So the roll out 00:07:22.800 --> 00:07:25.360 becomes super simple like you I set up a 00:07:25.360 --> 00:07:27.520 dedicated I have a shared services AWS 00:07:27.520 --> 00:07:29.440 account set up code artifact do a 00:07:29.440 --> 00:07:32.400 crossorg trust relationship read only so 00:07:32.400 --> 00:07:34.160 that all of the other like environments 00:07:34.160 --> 00:07:36.960 like dev can download it and also the 00:07:36.960 --> 00:07:38.560 developers they do not get like push 00:07:38.560 --> 00:07:40.720 permission. uh you can with IM policies 00:07:40.720 --> 00:07:41.919 very clearly say this guy can 00:07:41.919 --> 00:07:44.080 authenticate. He can you know set up his 00:07:44.080 --> 00:07:47.360 local npm to use this private uh coded 00:07:47.360 --> 00:07:49.759 defect hosted registry but he cannot 00:07:49.759 --> 00:07:53.039 push and uh we only have like GitHub 00:07:53.039 --> 00:07:55.599 workflows with AWS configuration and 00:07:55.599 --> 00:07:57.919 this GitHub workflow IM role is the one 00:07:57.919 --> 00:07:59.360 that is able to publish to these 00:07:59.360 --> 00:08:00.960 particular repositories. You can even 00:08:00.960 --> 00:08:03.039 cross-link it from g this GitHub repo to 00:08:03.039 --> 00:08:06.560 this code artifact um repository. So 00:08:06.560 --> 00:08:08.960 your whole arbback all of the stuff that 00:08:08.960 --> 00:08:11.280 you have to start paying for with Nexus 00:08:11.280 --> 00:08:13.599 um just instantly gets solved. And yes, 00:08:13.599 --> 00:08:15.280 you can set up complete upstreams to do 00:08:15.280 --> 00:08:18.879 a complete proxy. So so you can force 00:08:18.879 --> 00:08:21.039 global registry setup with code 00:08:21.039 --> 00:08:23.199 artifact. means that every developer uh 00:08:23.199 --> 00:08:24.800 will download all of its dependencies 00:08:24.800 --> 00:08:27.360 through your AWS account and and and 00:08:27.360 --> 00:08:30.960 your AWS um code artifact registry is a 00:08:30.960 --> 00:08:33.120 full pull through and basically caches 00:08:33.120 --> 00:08:35.519 and protects against npmgs outages and 00:08:35.519 --> 00:08:36.959 things like that. Um 00:08:36.959 --> 00:08:39.760 >> so so the so the main risk that it 00:08:39.760 --> 00:08:42.800 solves is like a denial of service on a 00:08:42.800 --> 00:08:46.320 public uh package. Yeah, it could. But 00:08:46.320 --> 00:08:48.959 you can also have your complete um 00:08:48.959 --> 00:08:51.519 privately scoped uh packages, right? 00:08:51.519 --> 00:08:55.519 With npm or python or others. I you you 00:08:55.519 --> 00:08:58.880 can say this scope maps to my private 00:08:58.880 --> 00:09:02.080 codified registry and that you of course 00:09:02.080 --> 00:09:03.760 have to make sure that you also own the 00:09:03.760 --> 00:09:05.600 public scope because if somebody an 00:09:05.600 --> 00:09:07.279 engineer doesn't authenticate or doesn't 00:09:07.279 --> 00:09:08.959 set it up properly they don't want to 00:09:08.959 --> 00:09:11.200 accidentally uh end up downloading the 00:09:11.200 --> 00:09:13.600 somebody else uh packages put on the on 00:09:13.600 --> 00:09:14.800 I guess 00:09:14.800 --> 00:09:16.640 >> so we do always register make sure that 00:09:16.640 --> 00:09:19.680 we own the npgs scope the public scope 00:09:19.680 --> 00:09:20.800 even though we don't put anything in 00:09:20.800 --> 00:09:23.040 there and then when somebody onboards 00:09:23.040 --> 00:09:25.440 Um, we have like the AWS SSO register. 00:09:25.440 --> 00:09:27.200 It's a make target or whatever is fast 00:09:27.200 --> 00:09:28.720 as you know in your package or JSON 00:09:28.720 --> 00:09:30.800 script to to get the code artifact token 00:09:30.800 --> 00:09:32.880 which is valid for a day and it handles 00:09:32.880 --> 00:09:34.480 your offboarding because it's all tied 00:09:34.480 --> 00:09:36.399 into your SSO. So somebody's offboarded 00:09:36.399 --> 00:09:38.160 their tokens expired. They can't request 00:09:38.160 --> 00:09:39.680 a new one. So you don't have to worry 00:09:39.680 --> 00:09:42.240 about that either. Uh, and you have your 00:09:42.240 --> 00:09:44.399 full like GitHub actions IDC integration 00:09:44.399 --> 00:09:45.839 with IM authentication. 00:09:45.839 --> 00:09:48.959 >> It sounds like code co code uh artifact 00:09:48.959 --> 00:09:51.519 is a must. Does it support go to or is 00:09:51.519 --> 00:09:54.880 it it seems targeted at Node.js if I was 00:09:54.880 --> 00:09:56.399 reading the 00:09:56.399 --> 00:09:58.800 >> Go I'm not sure. So So I'm I used goat 00:09:58.800 --> 00:10:02.320 artifact for Maven, Nugat. Uh I no I'm 00:10:02.320 --> 00:10:04.000 not sure about Nougat but definitely for 00:10:04.000 --> 00:10:08.480 Java Maven Ruby Nougat. What is Nougat? 00:10:08.480 --> 00:10:11.519 >> Is the net uh package manager package 00:10:11.519 --> 00:10:12.399 system? 00:10:12.399 --> 00:10:14.399 >> Uh but I have not tried it with Golang. 00:10:14.399 --> 00:10:17.040 I mean honestly but overall Golang with 00:10:17.040 --> 00:10:19.440 private GitHub repos is quite horrible 00:10:19.440 --> 00:10:20.880 right I mean you have to set up this 00:10:20.880 --> 00:10:21.760 like 00:10:21.760 --> 00:10:24.000 >> private proxy or let's say 00:10:24.000 --> 00:10:25.680 authentication 00:10:25.680 --> 00:10:28.079 >> if you have poly repo or multiple it 00:10:28.079 --> 00:10:30.800 becomes a nightmare 00:10:30.800 --> 00:10:32.959 >> uh I've had nothing but good experiences 00:10:32.959 --> 00:10:35.040 with go package management 00:10:35.040 --> 00:10:36.880 >> yeah anyways it's been a while since we 00:10:36.880 --> 00:10:39.519 caught up I guess I've been busy with a 00:10:39.519 --> 00:10:42.560 new job you've been busy with a new job 00:10:42.560 --> 00:10:45.440 >> not yet no but um preparing for it. Um 00:10:45.440 --> 00:10:47.839 so I've been having fun. But back to 00:10:47.839 --> 00:10:49.200 because one thing that 00:10:49.200 --> 00:10:51.680 >> new role you you were like asking me to 00:10:51.680 --> 00:10:54.240 guess your new title. 00:10:54.240 --> 00:10:56.000 >> So one thing that code edifact doesn't 00:10:56.000 --> 00:10:58.240 do that if you were to look at JROG or 00:10:58.240 --> 00:11:01.040 the other like uh enterprise offerings. 00:11:01.040 --> 00:11:03.920 Um I think there's less of this full 00:11:03.920 --> 00:11:05.920 integration within with like 00:11:05.920 --> 00:11:09.440 vulnerability scanning and um like coded 00:11:09.440 --> 00:11:11.760 effect is very basic like it's just 00:11:11.760 --> 00:11:14.959 npmgs uh private host with IM off 00:11:14.959 --> 00:11:17.760 authentication and it does a lot already 00:11:17.760 --> 00:11:19.839 but if you want to have like super fancy 00:11:19.839 --> 00:11:22.079 enterprise type of stuff on top of your 00:11:22.079 --> 00:11:25.200 registry then I know that art code sorry 00:11:25.200 --> 00:11:26.959 artifactory from JROG and probably 00:11:26.959 --> 00:11:29.839 others they do uh a lot more there but 00:11:29.839 --> 00:11:32.240 they also cost I I think more money. So 00:11:32.240 --> 00:11:34.240 anyway, yes, in terms of the the new 00:11:34.240 --> 00:11:35.279 role, 00:11:35.279 --> 00:11:37.040 >> I I didn't even know what a super fancy 00:11:37.040 --> 00:11:40.160 enterprise feature is. 00:11:40.160 --> 00:11:41.440 >> Well, there's definitely a few where I 00:11:41.440 --> 00:11:43.360 I've been asked like, can we do this? 00:11:43.360 --> 00:11:45.200 And then I look into it and no, you 00:11:45.200 --> 00:11:46.800 can't really do that with code artifact. 00:11:46.800 --> 00:11:48.480 Like code artifact effect is really nice 00:11:48.480 --> 00:11:49.279 mirror pull through. 00:11:49.279 --> 00:11:51.040 >> Can you give an example? I mean, I can't 00:11:51.040 --> 00:11:53.440 even I can't even think of one. 00:11:53.440 --> 00:11:56.800 >> Can you give an example what of a super 00:11:56.800 --> 00:11:58.720 fancy enterprise? 00:11:58.720 --> 00:12:02.480 I guess I I see code artifact and AWS 00:12:02.480 --> 00:12:05.600 ECR very similar like OCI and one of the 00:12:05.600 --> 00:12:07.120 common requests there is whenever you 00:12:07.120 --> 00:12:09.360 publish uh an ECR image you want to have 00:12:09.360 --> 00:12:11.760 like a full scan on the file system you 00:12:11.760 --> 00:12:13.839 want to do vulnerability scan I mean you 00:12:13.839 --> 00:12:15.440 can probably do all of that but you're 00:12:15.440 --> 00:12:17.600 going to have to build like events uh 00:12:17.600 --> 00:12:20.320 maybe with uh I know that ECR has like a 00:12:20.320 --> 00:12:22.240 hook to automatically scan like there's 00:12:22.240 --> 00:12:24.959 a ECR registry scanner I'm not I don't 00:12:24.959 --> 00:12:26.720 think code artifact has it but you can 00:12:26.720 --> 00:12:29.200 also build on top it like at a point you 00:12:29.200 --> 00:12:31.120 got to wonder do I build in house or do 00:12:31.120 --> 00:12:33.200 I go for something if I need all of that 00:12:33.200 --> 00:12:34.800 I might as well go for this 00:12:34.800 --> 00:12:36.079 >> actually while I'm thinking about it 00:12:36.079 --> 00:12:38.399 have you ever had like discussions where 00:12:38.399 --> 00:12:41.040 you introduce a security you know 00:12:41.040 --> 00:12:45.519 layered uh feature like a scan and then 00:12:45.519 --> 00:12:48.240 it slows down your pipeline by let's 00:12:48.240 --> 00:12:51.360 just say 30% and then people are 00:12:51.360 --> 00:12:53.920 complaining. 00:12:53.920 --> 00:12:55.920 Do you have you ever like had that sort 00:12:55.920 --> 00:12:58.720 of like uh discussion where like yeah 00:12:58.720 --> 00:13:01.120 this new security thing isn't really 00:13:01.120 --> 00:13:02.639 working out for us because it makes us 00:13:02.639 --> 00:13:07.760 so much slower or something like that. 00:13:07.760 --> 00:13:11.360 Yeah. So I think the um the that's the 00:13:11.360 --> 00:13:13.600 classic problem with security right 00:13:13.600 --> 00:13:16.160 security has always to find the balance 00:13:16.160 --> 00:13:18.800 between making sure that things are done 00:13:18.800 --> 00:13:23.440 securely and um and and you know not 00:13:23.440 --> 00:13:25.040 slowing things down not being a 00:13:25.040 --> 00:13:26.959 bottleneck not not causing people to 00:13:26.959 --> 00:13:29.760 find workarounds and go um you know 00:13:29.760 --> 00:13:31.680 through the back of of security because 00:13:31.680 --> 00:13:34.880 of they are blocked by it. So yeah um 00:13:34.880 --> 00:13:37.200 and and yeah slowdowns like on the CI/CD 00:13:37.200 --> 00:13:40.399 I mean sec we had a very funny um you 00:13:40.399 --> 00:13:43.279 know when somebody new a new CTO came in 00:13:43.279 --> 00:13:45.279 and then uh there was like a double down 00:13:45.279 --> 00:13:46.880 because it's a fintech double down on 00:13:46.880 --> 00:13:48.720 sec on on like auditability and 00:13:48.720 --> 00:13:50.959 compliance and then we were looking at 00:13:50.959 --> 00:13:54.240 the costs I mean for a full sem 00:13:54.240 --> 00:13:55.839 ingestion of all of these events across 00:13:55.839 --> 00:13:58.160 the enterprise uh I mean the cost 00:13:58.160 --> 00:14:01.760 balloon amazing like even in in a recent 00:14:01.760 --> 00:14:04.000 role where I was like Echo, everyone was 00:14:04.000 --> 00:14:05.680 running Terraform off their laptop. 00:14:05.680 --> 00:14:07.680 There's this thing called Tacos. Um, 00:14:07.680 --> 00:14:10.560 they provide cool stuff, but you know, 00:14:10.560 --> 00:14:11.839 they have all of these initiatives 00:14:11.839 --> 00:14:13.279 around security that cost a lot of 00:14:13.279 --> 00:14:14.880 money. And then I'm like, okay, I can 00:14:14.880 --> 00:14:17.519 run this on a $5, you know, AWS instance 00:14:17.519 --> 00:14:19.600 or maybe $20 if it's a slightly larger 00:14:19.600 --> 00:14:22.800 instance and I I get P like pull request 00:14:22.800 --> 00:14:24.720 plans so everyone can like coordinate 00:14:24.720 --> 00:14:26.560 basically very simple tackles with 00:14:26.560 --> 00:14:28.560 Atlantis. I don't have to spend a lot of 00:14:28.560 --> 00:14:30.399 time setting up the GitHub workflows. I 00:14:30.399 --> 00:14:33.440 get a nice, you know, plan, approve, 00:14:33.440 --> 00:14:36.000 apply workflow with with Atlantis. Works 00:14:36.000 --> 00:14:37.920 really decent. Doesn't cost a lot of 00:14:37.920 --> 00:14:39.760 money. And then I'm like looking at the 00:14:39.760 --> 00:14:42.720 costs of security, you know, in putting 00:14:42.720 --> 00:14:45.040 up like IDS, IDP across all of your 00:14:45.040 --> 00:14:47.360 hosts, setting up the full wiz like 00:14:47.360 --> 00:14:49.440 cloud uh security posture management, 00:14:49.440 --> 00:14:51.519 CSPM, from the data dog. 00:14:51.519 --> 00:14:54.160 >> The costs are 00:14:54.160 --> 00:14:56.639 >> Yeah. I mean data dog alone I've just 00:14:56.639 --> 00:14:59.440 seen in countless clients now has been 00:14:59.440 --> 00:15:02.000 an insane cost. 00:15:02.000 --> 00:15:04.399 >> Uh wait until they then also need 00:15:04.399 --> 00:15:05.360 security 00:15:05.360 --> 00:15:06.079 >> the security 00:15:06.079 --> 00:15:08.560 >> because like because data log on its own 00:15:08.560 --> 00:15:10.800 is just like people start with just um 00:15:10.800 --> 00:15:13.199 you know maybe infrastructure. I mean 00:15:13.199 --> 00:15:15.680 originally data was pure infra only APM 00:15:15.680 --> 00:15:18.240 came later right and then now they have 00:15:18.240 --> 00:15:21.199 this whole security suite cloud you know 00:15:21.199 --> 00:15:23.120 security poster management the whole 00:15:23.120 --> 00:15:24.399 traces 00:15:24.399 --> 00:15:28.880 >> uh setup the synthetics like they have 00:15:28.880 --> 00:15:31.519 absorbed so many additional products 00:15:31.519 --> 00:15:33.120 that you know on top of the original 00:15:33.120 --> 00:15:35.839 basic like in monitoring that it was 00:15:35.839 --> 00:15:36.240 >> well 00:15:36.240 --> 00:15:37.839 >> it's insane 00:15:37.839 --> 00:15:42.079 >> thanks to AI I like to think 00:15:42.079 --> 00:15:45.199 security the whole the whole profession 00:15:45.199 --> 00:15:50.079 the whole the whole security um area I 00:15:50.079 --> 00:15:52.880 feel is being kind of re re looked at 00:15:52.880 --> 00:15:56.880 with a new lens like one thing I I I um 00:15:56.880 --> 00:16:00.959 I heard through uh work was that clients 00:16:00.959 --> 00:16:03.360 are now realizing that they can't hire 00:16:03.360 --> 00:16:06.240 the typical security people that you 00:16:06.240 --> 00:16:09.040 know are not engineer mind who don't 00:16:09.040 --> 00:16:11.279 have an engineer mindset you know they 00:16:11.279 --> 00:16:13.120 can't hire people who sit in a 00:16:13.120 --> 00:16:15.120 spreadsheet, you know, checking off 00:16:15.120 --> 00:16:17.279 boxes and things like that anymore. 00:16:17.279 --> 00:16:19.680 >> You need different types of security 00:16:19.680 --> 00:16:21.360 capability. 00:16:21.360 --> 00:16:22.880 >> That's a problem because when we were 00:16:22.880 --> 00:16:24.959 hiring security and like people, we were 00:16:24.959 --> 00:16:26.480 looking for engineers because I I 00:16:26.480 --> 00:16:28.320 absolutely did not want the security guy 00:16:28.320 --> 00:16:30.079 that would just be there uh you know 00:16:30.079 --> 00:16:32.000 pushing checklists through. But the 00:16:32.000 --> 00:16:35.199 reality is that we hired two guys very 00:16:35.199 --> 00:16:37.279 strong technical engineers and one of 00:16:37.279 --> 00:16:40.240 them slightly more uh senior was just 00:16:40.240 --> 00:16:42.880 stuck going through endless list of 00:16:42.880 --> 00:16:45.440 compliance on spreadsheets identifying 00:16:45.440 --> 00:16:47.360 proof that this was implemented like 00:16:47.360 --> 00:16:49.920 that's what the job divulges into 00:16:49.920 --> 00:16:51.839 unfortunately that's the reality like 00:16:51.839 --> 00:16:52.320 that's what the 00:16:52.320 --> 00:16:55.199 >> comp in the age of AI surely that that 00:16:55.199 --> 00:16:58.000 sort of stuff can be automated no 00:16:58.000 --> 00:17:00.880 >> yeah this was four years ago now AWS is 00:17:00.880 --> 00:17:04.240 like ramping up crazily in terms of well 00:17:04.240 --> 00:17:06.400 architectured framework and security 00:17:06.400 --> 00:17:08.799 advisories as well on your AWS account 00:17:08.799 --> 00:17:10.959 leveraging AI and providing all the 00:17:10.959 --> 00:17:11.360 context 00:17:11.360 --> 00:17:13.520 >> required. This is this is a big change I 00:17:13.520 --> 00:17:15.839 feel in in our industry. 00:17:15.839 --> 00:17:17.360 Um 00:17:17.360 --> 00:17:21.120 >> but still security is has got a new 00:17:21.120 --> 00:17:22.480 lens. 00:17:22.480 --> 00:17:24.240 >> Yeah. But like security moves super 00:17:24.240 --> 00:17:25.919 slow. Like often they are related to 00:17:25.919 --> 00:17:28.160 very like traditional industries and 00:17:28.160 --> 00:17:30.080 they will require you like if you need 00:17:30.080 --> 00:17:31.840 to get licensed to get access to a 00:17:31.840 --> 00:17:33.840 certain data set, you're going to have 00:17:33.840 --> 00:17:35.039 to spend months going through 00:17:35.039 --> 00:17:37.919 checklists. Yes. Today I can give that 00:17:37.919 --> 00:17:40.960 checklist to uh an AI and it can go 00:17:40.960 --> 00:17:43.280 through all of my AWS setup and identify 00:17:43.280 --> 00:17:45.760 or even my Git repos and identify we are 00:17:45.760 --> 00:17:47.360 compliant or we are not compliant with 00:17:47.360 --> 00:17:49.840 this particular checklist item. uh and 00:17:49.840 --> 00:17:52.080 he can find the source of where we are 00:17:52.080 --> 00:17:53.440 compliant and maybe what is the 00:17:53.440 --> 00:17:55.520 remediation and then we can go back to 00:17:55.520 --> 00:17:57.440 the auditors. 00:17:57.440 --> 00:17:59.440 >> What if I said to you that the checklist 00:17:59.440 --> 00:18:02.160 will now become like a unit test like 00:18:02.160 --> 00:18:04.000 for example one thing that I'm hoping to 00:18:04.000 --> 00:18:06.880 roll out is you know clawed auto mode 00:18:06.880 --> 00:18:08.799 you basically have all these rules and 00:18:08.799 --> 00:18:11.200 prompts to say that you know you can 00:18:11.200 --> 00:18:13.440 trust these domains and you shouldn't do 00:18:13.440 --> 00:18:16.320 that you shouldn't do that. I'm I'm 00:18:16.320 --> 00:18:19.760 thinking that um 00:18:19.760 --> 00:18:21.520 the rules of a business like what you 00:18:21.520 --> 00:18:24.160 can and what will become a test like a 00:18:24.160 --> 00:18:27.679 like I will have like a claw-p 00:18:27.679 --> 00:18:30.799 can you delete this file and and then 00:18:30.799 --> 00:18:33.280 and then we build up a test suite and 00:18:33.280 --> 00:18:35.600 that will be our checklist to know that 00:18:35.600 --> 00:18:38.400 our security guard rails are working for 00:18:38.400 --> 00:18:40.960 us or not you know like it's def it's 00:18:40.960 --> 00:18:44.960 it's completely shifted left I 00:18:44.960 --> 00:18:46.320 H that 00:18:46.320 --> 00:18:48.400 >> what what you're talking about is 00:18:48.400 --> 00:18:50.400 >> making sure the agent follows the rules. 00:18:50.400 --> 00:18:52.960 So even if the human is unaware, the 00:18:52.960 --> 00:18:55.039 agent gets like a guard rails around it 00:18:55.039 --> 00:18:55.919 is what you're saying. 00:18:55.919 --> 00:18:59.039 >> Exactly. So this is this is a big change 00:18:59.039 --> 00:19:01.039 in the industry. 00:19:01.039 --> 00:19:02.000 >> Yeah. I think I think 00:19:02.000 --> 00:19:03.600 >> this is different sort of this is this 00:19:03.600 --> 00:19:05.120 is different sort of engineering to what 00:19:05.120 --> 00:19:07.440 most security people have been doing in 00:19:07.440 --> 00:19:08.640 the past. 00:19:08.640 --> 00:19:10.720 >> Yeah. But I also think that one one 00:19:10.720 --> 00:19:12.799 important change is that we will have 00:19:12.799 --> 00:19:15.679 less people involved in the agentic 00:19:15.679 --> 00:19:17.280 loop, right? I mean, we've already 00:19:17.280 --> 00:19:18.799 talked about this like originally it's 00:19:18.799 --> 00:19:21.039 it's shifting left more and more. Uh 00:19:21.039 --> 00:19:22.960 when you talk about prompting an agent 00:19:22.960 --> 00:19:25.760 to do things, we we were originally 00:19:25.760 --> 00:19:27.280 talking about specri development, coming 00:19:27.280 --> 00:19:28.640 up with user stories, functional 00:19:28.640 --> 00:19:30.400 requirements, then breaking that down 00:19:30.400 --> 00:19:33.520 into tasks and so on. And then that over 00:19:33.520 --> 00:19:35.440 time we're no longer really checking if 00:19:35.440 --> 00:19:37.200 these tasks are like exactly what we 00:19:37.200 --> 00:19:38.880 want. We just make sure that it matches 00:19:38.880 --> 00:19:41.600 the the user stories and that we're able 00:19:41.600 --> 00:19:43.840 to define what good looks like and how 00:19:43.840 --> 00:19:45.679 you validate that good looks like that. 00:19:45.679 --> 00:19:49.280 And now with the dynamic workflows, 00:19:49.280 --> 00:19:51.200 humans are not even involved like nobody 00:19:51.200 --> 00:19:53.280 there's no human prompting the agent. 00:19:53.280 --> 00:19:55.760 It's just agents writing the prompts 00:19:55.760 --> 00:19:57.760 which is also Steinberger's reach. 00:19:57.760 --> 00:20:00.400 >> No no I think I think humans will be 00:20:00.400 --> 00:20:02.480 writing the prompts. I mean 00:20:02.480 --> 00:20:03.280 >> no 00:20:03.280 --> 00:20:05.120 >> I think I think you went a little bit 00:20:05.120 --> 00:20:07.440 too far there. I think you're not up to 00:20:07.440 --> 00:20:09.120 speed there. 00:20:09.120 --> 00:20:11.600 >> And it's it's easy to get like totally 00:20:11.600 --> 00:20:14.480 far-fetched. And the other thing that 00:20:14.480 --> 00:20:15.760 >> I think if you look into what's 00:20:15.760 --> 00:20:18.559 happening, it's all in on agents now. 00:20:18.559 --> 00:20:20.320 Like it's been all in on agents for a 00:20:20.320 --> 00:20:22.480 while, but things have changed a lot. 00:20:22.480 --> 00:20:24.320 >> I want to be the the voice of reason. 00:20:24.320 --> 00:20:24.960 Like 00:20:24.960 --> 00:20:26.480 >> like 00:20:26.480 --> 00:20:28.240 I heard 00:20:28.240 --> 00:20:30.240 a client wants to run agents in 00:20:30.240 --> 00:20:33.440 production and like my jaw was like 00:20:33.440 --> 00:20:35.679 >> I think it's very different between No, 00:20:35.679 --> 00:20:37.120 no, no. Listen, listen. You're 00:20:37.120 --> 00:20:39.440 conflating two things. 00:20:39.440 --> 00:20:40.799 >> Giving an agent to put it in 00:20:40.799 --> 00:20:43.840 >> agent writing prompts. That's I I I feel 00:20:43.840 --> 00:20:46.320 you because you can uh use agents to 00:20:46.320 --> 00:20:47.440 like refine 00:20:47.440 --> 00:20:49.360 >> Let me talk one second here because 00:20:49.360 --> 00:20:51.840 you're conflating putting agents in 00:20:51.840 --> 00:20:54.799 production as in I'm going to let random 00:20:54.799 --> 00:20:57.440 people consumers like you know you see 00:20:57.440 --> 00:20:59.520 all these meme tweets about you don't 00:20:59.520 --> 00:21:01.360 need to pay for cloud anymore. You can 00:21:01.360 --> 00:21:04.320 just uh go into a chat session with 00:21:04.320 --> 00:21:07.760 Chipotle's uh help menu and it's an LLM 00:21:07.760 --> 00:21:09.440 and you can ask it to write Python. You 00:21:09.440 --> 00:21:11.200 don't need to pay for cloud. That's if 00:21:11.200 --> 00:21:12.480 you talk about putting agent in 00:21:12.480 --> 00:21:14.240 production, that's what you talk about 00:21:14.240 --> 00:21:17.520 putting agent in front of consumers. 00:21:17.520 --> 00:21:19.280 >> Yeah. Or Yeah. Uh 00:21:19.280 --> 00:21:21.120 >> I think it's very different between your 00:21:21.120 --> 00:21:24.000 internal consumers like your developers, 00:21:24.000 --> 00:21:27.440 product managers, product owners. 00:21:27.440 --> 00:21:29.120 >> Yeah, you're right. You're right. You 00:21:29.120 --> 00:21:31.280 need to define it like perhaps using 00:21:31.280 --> 00:21:33.600 that lethal trifecta is probably the 00:21:33.600 --> 00:21:36.720 best approach like so so let's say let's 00:21:36.720 --> 00:21:38.799 say it's not it does it doesn't have 00:21:38.799 --> 00:21:41.600 access to untrusted data but but running 00:21:41.600 --> 00:21:44.320 an agent in production 00:21:44.320 --> 00:21:47.120 just internally I feel is just just kind 00:21:47.120 --> 00:21:50.159 of 00:21:50.159 --> 00:21:54.000 kind of risky. I mean, 00:21:54.000 --> 00:21:55.600 >> so basically, 00:21:55.600 --> 00:21:57.360 >> yeah, 00:21:57.360 --> 00:22:00.799 >> Peter Steinberger has put out a tweet, I 00:22:00.799 --> 00:22:02.240 think last week, and it's been 00:22:02.240 --> 00:22:06.080 repeatedly posted on Reddit. And it's 00:22:06.080 --> 00:22:08.480 basically like you do not write the 00:22:08.480 --> 00:22:10.480 prompt. You write the loop that write 00:22:10.480 --> 00:22:14.320 the prompt. And and um and I think this 00:22:14.320 --> 00:22:16.480 this is also what's happening with um I 00:22:16.480 --> 00:22:20.080 can't access uh sitter here. Um but but 00:22:20.080 --> 00:22:21.440 it's basically the same happening right 00:22:21.440 --> 00:22:23.440 now with these dynamic workflows right 00:22:23.440 --> 00:22:25.679 as long as I give it a a clear goal like 00:22:25.679 --> 00:22:28.240 even with codex goal it spins up an 00:22:28.240 --> 00:22:30.640 enormous amount of agents that all the 00:22:30.640 --> 00:22:33.280 work and they are able to validate you 00:22:33.280 --> 00:22:35.039 know they have their own loops feedback 00:22:35.039 --> 00:22:36.880 validation loops to validate that what 00:22:36.880 --> 00:22:40.400 they built works cuz if you look at like 00:22:40.400 --> 00:22:42.640 6 months ago you would do the spec you 00:22:42.640 --> 00:22:44.240 would do the plan you would go through 00:22:44.240 --> 00:22:46.640 sessions with the agent with the agentic 00:22:46.640 --> 00:22:48.320 shell right it's not an agent you would 00:22:48.320 --> 00:22:50.240 using agent and shells. It would 00:22:50.240 --> 00:22:51.679 complete the work. You would see the 00:22:51.679 --> 00:22:54.000 stream. You would realize what was 00:22:54.000 --> 00:22:54.480 working. 00:22:54.480 --> 00:22:56.080 >> You would write deterministic code, 00:22:56.080 --> 00:22:57.280 right? 00:22:57.280 --> 00:22:59.039 >> And and the agent would write code and 00:22:59.039 --> 00:23:00.159 you would have your verification 00:23:00.159 --> 00:23:00.799 mechanism. 00:23:00.799 --> 00:23:02.240 >> You don't believe in this stuff, do you? 00:23:02.240 --> 00:23:03.120 Come on, Vincent. 00:23:03.120 --> 00:23:05.039 >> I've seen this stuff. It works. You are 00:23:05.039 --> 00:23:07.120 the one that's not not paying attention. 00:23:07.120 --> 00:23:09.120 Read what's happening on Twitter in the 00:23:09.120 --> 00:23:11.360 last week. I I I know I know people, but 00:23:11.360 --> 00:23:13.440 like there's so much craziness on 00:23:13.440 --> 00:23:17.600 Twitter and X and like I I guess my 00:23:17.600 --> 00:23:19.520 philosophy is is my philosophy really 00:23:19.520 --> 00:23:21.039 that out of date? My philosophy is that 00:23:21.039 --> 00:23:22.960 you use the agent to create 00:23:22.960 --> 00:23:25.520 deterministic workflows, deterministic 00:23:25.520 --> 00:23:27.840 code to do the job. 00:23:27.840 --> 00:23:29.919 >> That's what dynamic workflows ultimately 00:23:29.919 --> 00:23:31.600 boggles down to. Yes. You you 00:23:31.600 --> 00:23:34.640 >> ship in production deterministic stuff. 00:23:34.640 --> 00:23:36.159 You don't want to ship 00:23:36.159 --> 00:23:37.600 >> that's not into production. We're not 00:23:37.600 --> 00:23:38.640 talking about like when you say 00:23:38.640 --> 00:23:40.240 production, what do you mean? Because 00:23:40.240 --> 00:23:41.760 this is not in front of consumers. 00:23:41.760 --> 00:23:43.600 >> Stuff that's going to hit like consumers 00:23:43.600 --> 00:23:44.240 and 00:23:44.240 --> 00:23:45.679 >> No, no, we're not talking about consu. 00:23:45.679 --> 00:23:47.039 We're talking about software factories 00:23:47.039 --> 00:23:48.880 and internal like software delivery like 00:23:48.880 --> 00:23:52.159 AI delivery. Um AI AI DLC. 00:23:52.159 --> 00:23:54.400 >> Okay. I think big shift. 00:23:54.400 --> 00:23:56.480 >> Okay. AI delivery. Okay. I can I can get 00:23:56.480 --> 00:23:58.559 behind that whole agents. You write the 00:23:58.559 --> 00:24:00.320 loop to write the agents. I can get 00:24:00.320 --> 00:24:00.480 Yeah. 00:24:00.480 --> 00:24:02.080 >> So why every time I mention it, you go 00:24:02.080 --> 00:24:03.520 you fall back to like, yo, but that's 00:24:03.520 --> 00:24:05.039 not that's not what you do. But then 00:24:05.039 --> 00:24:06.480 then you say, "Oh, no. I can get behind 00:24:06.480 --> 00:24:08.240 it like it's very simple. 00:24:08.240 --> 00:24:09.280 >> Okay. 00:24:09.280 --> 00:24:11.440 >> Claude Code shipped 00:24:11.440 --> 00:24:14.559 >> this this dynamic workflows agents have 00:24:14.559 --> 00:24:16.640 been doing the work completely 00:24:16.640 --> 00:24:18.640 independently and coming back with the 00:24:18.640 --> 00:24:20.240 solution because it has validated. 00:24:20.240 --> 00:24:21.360 >> I'm getting I'm getting a little bit 00:24:21.360 --> 00:24:24.320 fixated about running this sort of stuff 00:24:24.320 --> 00:24:29.039 in in in production with um with noobs 00:24:29.039 --> 00:24:31.039 or you know like non-engineers I 00:24:31.039 --> 00:24:32.960 suppose. 00:24:32.960 --> 00:24:34.640 I mean it's very closely linked to your 00:24:34.640 --> 00:24:37.120 your your worry because you say you need 00:24:37.120 --> 00:24:39.679 an agent you need an agent shell and you 00:24:39.679 --> 00:24:41.840 need guardrails. So if an engineer does 00:24:41.840 --> 00:24:43.520 not take into account security 00:24:43.520 --> 00:24:45.919 boundaries uh the agent shell guard 00:24:45.919 --> 00:24:47.840 rails will stop that. What I'm trying to 00:24:47.840 --> 00:24:49.679 say is there's no engineer sitting 00:24:49.679 --> 00:24:52.080 there. There's no no human there 00:24:52.080 --> 00:24:53.840 anymore. like what's happening. The 00:24:53.840 --> 00:24:55.120 humans are involved with the planning 00:24:55.120 --> 00:24:57.360 phase that creates a dynamic, you know, 00:24:57.360 --> 00:24:59.679 workflow which is a deterministic 00:24:59.679 --> 00:25:02.000 procedure of steps that includes up to 00:25:02.000 --> 00:25:04.480 10 12 agents minimum that are doing the 00:25:04.480 --> 00:25:06.559 implementation phases that are running 00:25:06.559 --> 00:25:08.559 the feedback like verification loops. 00:25:08.559 --> 00:25:09.919 But that only works if you can verify 00:25:09.919 --> 00:25:11.279 what good look like and what 00:25:11.279 --> 00:25:12.720 verification means 00:25:12.720 --> 00:25:13.760 >> and then you get back. 00:25:13.760 --> 00:25:16.159 >> I mean, we're getting a little bit stuck 00:25:16.159 --> 00:25:17.840 with terminology. I mean, so it's 00:25:17.840 --> 00:25:19.200 probably me that's the problem here, but 00:25:19.200 --> 00:25:21.039 like I guess what you're saying is like 00:25:21.039 --> 00:25:22.720 there's no more human in the loop. 00:25:22.720 --> 00:25:24.880 There's human human on the loop. I heard 00:25:24.880 --> 00:25:27.120 that expression. Would you get behind 00:25:27.120 --> 00:25:29.760 that? 00:25:29.760 --> 00:25:31.840 >> Whatever. What I'm saying is that 00:25:31.840 --> 00:25:34.000 there's no more human um you know 00:25:34.000 --> 00:25:35.760 directly reviewing the stream of 00:25:35.760 --> 00:25:37.039 commands happening and approving 00:25:37.039 --> 00:25:39.919 commands. That's kind of like agents are 00:25:39.919 --> 00:25:40.880 executing on their own. 00:25:40.880 --> 00:25:43.840 >> We're like all in on auto mode type. 00:25:43.840 --> 00:25:46.720 >> Yeah. Yeah, we are. Since I can say auto 00:25:46.720 --> 00:25:48.159 mode was one thing that was released 00:25:48.159 --> 00:25:49.840 maybe a few weeks ago. Then they 00:25:49.840 --> 00:25:51.520 released dynamic workflows and they made 00:25:51.520 --> 00:25:53.840 an additional follow-up post about all 00:25:53.840 --> 00:25:55.600 of examples of dynamic workflows and how 00:25:55.600 --> 00:25:56.799 they work. And they give you examples 00:25:56.799 --> 00:25:58.320 like hey you can call on dynamic 00:25:58.320 --> 00:26:00.000 workflows to go through all of your 00:26:00.000 --> 00:26:02.000 previous session and filter out like 00:26:02.000 --> 00:26:03.679 common mistakes the model make to 00:26:03.679 --> 00:26:05.520 exfiltrate better cloud rules global and 00:26:05.520 --> 00:26:07.200 locally within the project. There's many 00:26:07.200 --> 00:26:10.159 other examples in there. Um and and so 00:26:10.159 --> 00:26:14.080 dynamic workflows plus auto mode have 00:26:14.080 --> 00:26:16.559 significantly improved the accuracy of 00:26:16.559 --> 00:26:18.799 agents working in the like on their own. 00:26:18.799 --> 00:26:20.559 Like I can tell you back in November I 00:26:20.559 --> 00:26:22.559 tried to run like hey I have a beautiful 00:26:22.559 --> 00:26:25.200 plan fivephase implementation several 00:26:25.200 --> 00:26:26.720 modules that are completely decoupled 00:26:26.720 --> 00:26:27.919 and they need to do individual 00:26:27.919 --> 00:26:30.000 development work. Now spin up five 00:26:30.000 --> 00:26:32.080 agents and do them independently and it 00:26:32.080 --> 00:26:33.440 would come back with all these agents 00:26:33.440 --> 00:26:35.440 creating mock functions for contracts 00:26:35.440 --> 00:26:37.039 between the modules and it would be a 00:26:37.039 --> 00:26:39.919 complete mess. Um, now I have complete 00:26:39.919 --> 00:26:43.679 trust. Like it like it just works. Um, 00:26:43.679 --> 00:26:47.120 which is a massive shift. I mean, it's 00:26:47.120 --> 00:26:48.640 kind of in line with what like Stripe 00:26:48.640 --> 00:26:50.640 was saying that they have like minions. 00:26:50.640 --> 00:26:53.200 They just have tickets in in a in a in a 00:26:53.200 --> 00:26:56.400 Jira board or in a linear board and then 00:26:56.400 --> 00:26:58.799 the agents go off and do all the work. 00:26:58.799 --> 00:27:00.960 And and I think that 00:27:00.960 --> 00:27:04.799 >> well so it's like guest town is becoming 00:27:04.799 --> 00:27:05.039 uh 00:27:05.039 --> 00:27:08.080 >> yeah guest town is built into Claude Code 00:27:08.080 --> 00:27:10.400 directly now with with dynamic workflows 00:27:10.400 --> 00:27:12.799 >> you don't need like uh you know five max 00:27:12.799 --> 00:27:15.279 accounts and and and additional because 00:27:15.279 --> 00:27:16.960 that's kind of built into Claude Code 00:27:16.960 --> 00:27:18.400 with this dynamic workflows 00:27:18.400 --> 00:27:20.799 >> I mean my god things move so quickly 00:27:20.799 --> 00:27:24.960 don't they mean like oh my god oh 00:27:24.960 --> 00:27:27.360 there's something I want there's 00:27:27.360 --> 00:27:28.799 something I wanted to point out to you 00:27:28.799 --> 00:27:32.960 if I can just find the link. 00:27:32.960 --> 00:27:34.799 >> But I think like it changed so fast in 00:27:34.799 --> 00:27:38.240 just the last two months, but 00:27:38.240 --> 00:27:40.720 it just means that this aent like aic 00:27:40.720 --> 00:27:43.679 reality is is coming way faster 00:27:43.679 --> 00:27:45.039 to everyone. Like it's become a 00:27:45.039 --> 00:27:46.400 commodity. It's not just like, oh, 00:27:46.400 --> 00:27:48.799 there's this one stripe company that 00:27:48.799 --> 00:27:49.360 software. 00:27:49.360 --> 00:27:51.760 >> We're definitely like we're definitely 00:27:51.760 --> 00:27:53.360 uh 00:27:53.360 --> 00:27:55.279 the pioneers. I have to pinch myself 00:27:55.279 --> 00:27:58.000 sometimes. I I really thought I don't 00:27:58.000 --> 00:27:59.600 know if you saw this one how how we 00:27:59.600 --> 00:28:02.880 contain uh clawed across products. 00:28:02.880 --> 00:28:04.399 There's this footnote here that made me 00:28:04.399 --> 00:28:09.440 laugh. 00:28:09.440 --> 00:28:11.679 Claude code auto mode delegates command 00:28:11.679 --> 00:28:13.679 approvals to modelbased classifier. I'm 00:28:13.679 --> 00:28:14.799 not too sure why they call it a 00:28:14.799 --> 00:28:17.360 classifier as opposed to an agent, but 00:28:17.360 --> 00:28:22.559 whatever. It minimizes friction. 00:28:22.559 --> 00:28:25.440 So, and it interestingly says 0.4% 4% of 00:28:25.440 --> 00:28:27.919 ben of benign arguments or commands were 00:28:27.919 --> 00:28:30.960 blocked. So basically things that were 00:28:30.960 --> 00:28:33.520 pretty inconsequential, right? And then 00:28:33.520 --> 00:28:36.960 it said that um 00:28:36.960 --> 00:28:41.919 17% of ogre overeager actions which I 00:28:41.919 --> 00:28:44.240 which you can interpret it at you know 00:28:44.240 --> 00:28:48.080 like uh how how would you interpret that 00:28:48.080 --> 00:28:51.600 that you would you could say yeah risky 00:28:51.600 --> 00:28:55.279 17% of risky actions got through. 00:28:55.279 --> 00:28:56.799 Don't you think that's an interesting 00:28:56.799 --> 00:28:58.960 footnote to make on on on the topic of 00:28:58.960 --> 00:29:00.880 auto mode? Are you still there Vincent? 00:29:00.880 --> 00:29:02.880 Yeah, I mean in terms of like approvals 00:29:02.880 --> 00:29:05.520 to a modelbased classifier when when I I 00:29:05.520 --> 00:29:07.679 read classifier I thinks it has a very 00:29:07.679 --> 00:29:11.440 strict category like it can be A B or C 00:29:11.440 --> 00:29:13.679 and I mean it's not like an large 00:29:13.679 --> 00:29:15.760 language model that puts out pros a 00:29:15.760 --> 00:29:18.240 classifier would you know puts out a 00:29:18.240 --> 00:29:21.279 verdict approved reject 00:29:21.279 --> 00:29:24.559 um and then roughly 0.4% 4% of benign 00:29:24.559 --> 00:29:26.399 comments blocked which means like this 00:29:26.399 --> 00:29:28.640 comment is benign like I'm not English 00:29:28.640 --> 00:29:30.720 native but the way I understand it is 00:29:30.720 --> 00:29:32.880 that would be a comment that is that is 00:29:32.880 --> 00:29:35.279 fine but uh it was blocked anyway which 00:29:35.279 --> 00:29:38.480 is a little like you know it's a bit sad 00:29:38.480 --> 00:29:41.679 but it's better to be uh you know overly 00:29:41.679 --> 00:29:45.039 blocking things and then that's a cost 00:29:45.039 --> 00:29:47.600 to that's okay to to pay for missing the 00:29:47.600 --> 00:29:49.840 the what's the rest of the sense I don't 00:29:49.840 --> 00:29:53.440 know but then in terms of like 70% 00:29:53.440 --> 00:29:57.760 of uh riskier commands coming true. I 00:29:57.760 --> 00:29:59.360 think what the the main point is what 00:29:59.360 --> 00:30:00.799 they're making is you still need to run 00:30:00.799 --> 00:30:03.760 this in a sandbox like um you cannot 00:30:03.760 --> 00:30:05.360 trust this thing not to nuke your 00:30:05.360 --> 00:30:08.640 machine. Um you know 00:30:08.640 --> 00:30:10.240 >> that's basically what they're trying to 00:30:10.240 --> 00:30:12.880 say. And then on the on the topic of 00:30:12.880 --> 00:30:15.120 running Claude in a sandbox, I've been 00:30:15.120 --> 00:30:17.360 using that Docker sandbox tool and I 00:30:17.360 --> 00:30:19.440 think it's great actually except that 00:30:19.440 --> 00:30:23.039 it's proprietary and uh 00:30:23.039 --> 00:30:26.880 and that's a bit annoying and and it 00:30:26.880 --> 00:30:29.760 seems to require a login and that's also 00:30:29.760 --> 00:30:33.440 bloody annoying. Uh but otherwise this 00:30:33.440 --> 00:30:36.799 this sandbox tool to to run um agents 00:30:36.799 --> 00:30:39.600 inside I think is pretty nice cuz you 00:30:39.600 --> 00:30:41.919 can you can isolate the configuration. 00:30:41.919 --> 00:30:44.559 You can isolate the guard rails. Of 00:30:44.559 --> 00:30:46.480 course you have a nice little UI to show 00:30:46.480 --> 00:30:48.720 what network calls it's making. Not that 00:30:48.720 --> 00:30:51.520 it's actually useful to be honest but 00:30:51.520 --> 00:30:54.559 but I mean how else do you run uh agents 00:30:54.559 --> 00:30:57.919 in in sandboxes? Uh, previously I was 00:30:57.919 --> 00:31:00.720 doing it with like systemd and spawn but 00:31:00.720 --> 00:31:03.279 like no one has systemd on a Mac OS 00:31:03.279 --> 00:31:06.000 machine. Like how are you doing it? 00:31:06.000 --> 00:31:08.880 I mean 00:31:08.880 --> 00:31:12.240 when you use cloud the app on Mac OS and 00:31:12.240 --> 00:31:14.320 you use co-work 00:31:14.320 --> 00:31:16.320 I think I'm not I I haven't looked at 00:31:16.320 --> 00:31:18.320 all into any technical deep dives in how 00:31:18.320 --> 00:31:21.360 that works but the way it behaves it 00:31:21.360 --> 00:31:22.960 really seems that it's using some type 00:31:22.960 --> 00:31:25.440 of containers or some type of of uh you 00:31:25.440 --> 00:31:28.159 know OSX native kernel like 00:31:28.159 --> 00:31:29.919 virtualization 00:31:29.919 --> 00:31:31.840 uh because it cannot just read files on 00:31:31.840 --> 00:31:34.320 disk normally it needs to like mount 00:31:34.320 --> 00:31:36.720 files into the co-work see. So when you 00:31:36.720 --> 00:31:38.080 say sandboxes, you're talking about 00:31:38.080 --> 00:31:39.440 co-work cuz I think 00:31:39.440 --> 00:31:41.600 >> I mean I'm talking first about co-work 00:31:41.600 --> 00:31:44.880 which is one one uh you know consumer I 00:31:44.880 --> 00:31:47.519 guess less engineering focused solution 00:31:47.519 --> 00:31:49.679 and you give it access to to Gmail and 00:31:49.679 --> 00:31:52.080 crazy things like that. Um so that one 00:31:52.080 --> 00:31:54.960 does like very um very strong isolation. 00:31:54.960 --> 00:31:58.000 Again I don't care how it just works. 00:31:58.000 --> 00:31:59.279 >> You're actually using it. 00:31:59.279 --> 00:32:02.000 >> Yeah I am like I've got like this PDF 00:32:02.000 --> 00:32:04.080 file in Gmail. I hate, you know, filling 00:32:04.080 --> 00:32:07.120 things out. I just straight up, you 00:32:07.120 --> 00:32:09.440 know, asked the cowork to do it for me. 00:32:09.440 --> 00:32:11.600 Fetch the details. I got this contract 00:32:11.600 --> 00:32:13.120 that I needed to do some stuff on. I was 00:32:13.120 --> 00:32:14.559 like, "Hey, what's my action points 00:32:14.559 --> 00:32:16.240 here?" And you know, my details just 00:32:16.240 --> 00:32:18.000 fill out whatever you need to fill out 00:32:18.000 --> 00:32:20.320 in this thing and revert it, please. I 00:32:20.320 --> 00:32:23.200 don't even care. And um I mean that's 00:32:23.200 --> 00:32:25.440 where co-work is perfect for me like end 00:32:25.440 --> 00:32:27.840 of month type of stuff because I'm doing 00:32:27.840 --> 00:32:29.760 some freelance stuff. So then you need 00:32:29.760 --> 00:32:31.440 to do all this additional administrative 00:32:31.440 --> 00:32:33.440 things. I just go like, "Yo, here's an 00:32:33.440 --> 00:32:35.440 extra he I just literally go into the 00:32:35.440 --> 00:32:38.000 time sheet web page. I sometimes capture 00:32:38.000 --> 00:32:39.919 the network calls and drop drop in the 00:32:39.919 --> 00:32:41.279 HAR file." 00:32:41.279 --> 00:32:44.399 >> Well, I see. So, this Yeah, co-work has 00:32:44.399 --> 00:32:47.200 more UI functionality does, doesn't it? 00:32:47.200 --> 00:32:49.279 Um yeah, but I'm still approaching it 00:32:49.279 --> 00:32:50.960 very engineering like like I just drop 00:32:50.960 --> 00:32:52.799 dropping a network recording like 00:32:52.799 --> 00:32:54.960 network calls recording uh from Chrome 00:32:54.960 --> 00:32:57.679 into the into like the HAR archive and 00:32:57.679 --> 00:33:00.640 then I also I just write I do inspect 00:33:00.640 --> 00:33:03.440 and then I cl copy out the HTML elements 00:33:03.440 --> 00:33:05.360 and paste it in. I was like hey can you 00:33:05.360 --> 00:33:08.000 grap out the cuz these are stupid. I 00:33:08.000 --> 00:33:10.799 mean I I think why is is HR system still 00:33:10.799 --> 00:33:13.760 so horrible? Like I'm I have to work 00:33:13.760 --> 00:33:15.200 with several HR systems. 00:33:15.200 --> 00:33:17.120 >> Salesforce. 00:33:17.120 --> 00:33:19.039 >> Sorry, it's not Salesforce, 00:33:19.039 --> 00:33:20.320 >> isn't it? Oh, okay. 00:33:20.320 --> 00:33:22.640 >> No, it's HR. It's not CRM. Does 00:33:22.640 --> 00:33:24.640 Salesforce do HR now, too? 00:33:24.640 --> 00:33:26.799 >> No. Well, I I Well, I'm not going to 00:33:26.799 --> 00:33:29.360 give away how the HR time sheets are 00:33:29.360 --> 00:33:32.399 working in my company in my company, but 00:33:32.399 --> 00:33:34.640 it's something to do with a third party. 00:33:34.640 --> 00:33:36.960 I actually asked Claude Codes to look at 00:33:36.960 --> 00:33:38.640 the HR website and figure out how to 00:33:38.640 --> 00:33:40.720 enter a time sheet and it couldn't. It 00:33:40.720 --> 00:33:42.240 was such a nice software that even the 00:33:42.240 --> 00:33:44.080 HR couldn't figure it out. It tried to 00:33:44.080 --> 00:33:45.519 click on a couple of buttons and it 00:33:45.519 --> 00:33:46.480 didn't work. 00:33:46.480 --> 00:33:48.880 >> Yeah. I I feel like in every 00:33:48.880 --> 00:33:50.720 organization there needs to be like a 00:33:50.720 --> 00:33:54.559 test like how can you um can you tell 00:33:54.559 --> 00:33:57.039 buttons to on board? Yeah. C can the can 00:33:57.039 --> 00:33:59.039 the agents on board? Can the agents fill 00:33:59.039 --> 00:34:01.679 in a time entry? Can the agents uh 00:34:01.679 --> 00:34:03.279 >> that was actually very funny because my 00:34:03.279 --> 00:34:04.960 wife asked me she has to do some she 00:34:04.960 --> 00:34:07.600 works for product um what is it it's 00:34:07.600 --> 00:34:10.159 like it's textile labels and stuff for 00:34:10.159 --> 00:34:12.240 uh it's very very like manufacturing in 00:34:12.240 --> 00:34:14.159 Vietnam and they have to build some 00:34:14.159 --> 00:34:17.119 product development portal and she did 00:34:17.119 --> 00:34:18.720 some outsource work and she's like I 00:34:18.720 --> 00:34:20.879 need to do a usability study and I'm 00:34:20.879 --> 00:34:22.079 like 00:34:22.079 --> 00:34:24.560 >> let me install coowork point coowork at 00:34:24.560 --> 00:34:26.639 it and do a usability study with co-work 00:34:26.639 --> 00:34:28.399 seriously I'm not going to bother like 00:34:28.399 --> 00:34:32.240 >> that's actually a good idea. 00:34:32.240 --> 00:34:33.919 >> Oh yeah. Um, 00:34:33.919 --> 00:34:36.399 >> so back to sandboxes like the sandboxes 00:34:36.399 --> 00:34:40.079 in cowwork I I I I trust them. And then 00:34:40.079 --> 00:34:42.000 >> does the sandbox I was just looking 00:34:42.000 --> 00:34:43.520 through these notes here like for 00:34:43.520 --> 00:34:46.480 example the docker sandbox uh it doesn't 00:34:46.480 --> 00:34:51.200 do um it it basically by default it it 00:34:51.200 --> 00:34:53.679 has skip dangerous it's in yolo mode by 00:34:53.679 --> 00:34:56.800 default. Is co-work in co-work is not 00:34:56.800 --> 00:34:58.720 yolo mode by default is it? Yeah, 00:34:58.720 --> 00:35:00.960 >> I don't think so. But um but another 00:35:00.960 --> 00:35:03.200 agents that get spun up is is basically 00:35:03.200 --> 00:35:05.440 in Claude Code in Claude Code, right? It 00:35:05.440 --> 00:35:07.119 spins up a whole bunch of sub agents and 00:35:07.119 --> 00:35:08.320 they do whole bunch of things and 00:35:08.320 --> 00:35:10.640 they're all, you know, guarded by this 00:35:10.640 --> 00:35:12.880 auto mode and I don't think it does much 00:35:12.880 --> 00:35:14.640 sandboxing there, right? I think it's 00:35:14.640 --> 00:35:16.720 purely running agents directly in cloud 00:35:16.720 --> 00:35:19.280 code shell. Uh I think they really 00:35:19.280 --> 00:35:21.520 boosted Claude Code uh runtime 00:35:21.520 --> 00:35:24.240 environment to may make it possible to 00:35:24.240 --> 00:35:26.079 run so many agents. I think they're all 00:35:26.079 --> 00:35:27.680 running locally. It's it's crazy. Yeah, 00:35:27.680 --> 00:35:29.520 we we discussed this and I'm I'm being 00:35:29.520 --> 00:35:32.160 cynical like I guess they're they're 00:35:32.160 --> 00:35:35.839 going on they're leveraging agents just 00:35:35.839 --> 00:35:38.880 to basically get people to spend more 00:35:38.880 --> 00:35:42.000 money, right? I mean, I don't I I I 00:35:42.000 --> 00:35:43.599 don't know. But apparently I figured out 00:35:43.599 --> 00:35:46.240 that I'm on a max 5x account, which is 00:35:46.240 --> 00:35:48.160 why I could never which is why I never 00:35:48.160 --> 00:35:49.839 hit any limits. And I was like, "Oh, why 00:35:49.839 --> 00:35:51.839 is everybody hitting limits?" Um, yeah, 00:35:51.839 --> 00:35:53.760 I'm overpaying. But now I started 00:35:53.760 --> 00:35:54.880 actually hitting my limits and I 00:35:54.880 --> 00:35:57.200 actually start to use it properly. Um 00:35:57.200 --> 00:36:00.560 but but back to sandboxes, right? Um I 00:36:00.560 --> 00:36:03.359 actually was inv was looking into like 00:36:03.359 --> 00:36:05.599 using firecracker or there was this 00:36:05.599 --> 00:36:08.720 other um there was this other framework 00:36:08.720 --> 00:36:10.240 that that I saw on LinkedIn and I asked 00:36:10.240 --> 00:36:13.359 Claude Code like hey could I use this? Um 00:36:13.359 --> 00:36:15.040 and it was like nah man it's like two 00:36:15.040 --> 00:36:18.640 months old not worth it. Yeah cloud is 00:36:18.640 --> 00:36:21.119 pretty fun. Don't use it and you should 00:36:21.119 --> 00:36:22.560 go for firecracker. It's way more 00:36:22.560 --> 00:36:24.000 >> how are you doing your Wait a minute. 00:36:24.000 --> 00:36:25.760 How are you doing your re your research 00:36:25.760 --> 00:36:27.599 with core code? Are you just using web 00:36:27.599 --> 00:36:29.599 fetch to get the 00:36:29.599 --> 00:36:31.520 >> um 00:36:31.520 --> 00:36:35.040 I'm I'm so I'm on the cloud.ai like I'm 00:36:35.040 --> 00:36:36.960 on the app and on the browser for the 00:36:36.960 --> 00:36:39.520 chats. Uh and I always tell it to ground 00:36:39.520 --> 00:36:41.760 itself like go and web search and and 00:36:41.760 --> 00:36:43.119 find uh proof. 00:36:43.119 --> 00:36:44.640 >> You you shared I mean you've been 00:36:44.640 --> 00:36:46.240 showing me a lot and to be honest I 00:36:46.240 --> 00:36:48.240 haven't read through them all but like I 00:36:48.240 --> 00:36:50.000 think you like some time back you you 00:36:50.000 --> 00:36:52.320 mentioned firecrawl. Is that something 00:36:52.320 --> 00:36:54.079 to do with research? 00:36:54.079 --> 00:36:55.599 >> That's that's a funny thing, right? I 00:36:55.599 --> 00:36:57.359 was working on some curriculum for a 00:36:57.359 --> 00:37:00.720 course and it's a course on AWS. So 00:37:00.720 --> 00:37:03.760 there's a lot of AWS docs there and I 00:37:03.760 --> 00:37:05.599 had an initial like road map and 00:37:05.599 --> 00:37:08.000 curriculum with modules. Uh and then I 00:37:08.000 --> 00:37:09.599 wanted to break that down in like real 00:37:09.599 --> 00:37:11.760 content u you know that matches the 00:37:11.760 --> 00:37:14.640 actual product offering but I was like 00:37:14.640 --> 00:37:17.280 too lazy to go through every page. and I 00:37:17.280 --> 00:37:20.240 asked cowwork and it scraped maybe 80 00:37:20.240 --> 00:37:23.359 pages of like subsections of some some 00:37:23.359 --> 00:37:25.839 areas. So that code Claude Code does it 00:37:25.839 --> 00:37:28.240 automatically. Okay. But when you set up 00:37:28.240 --> 00:37:31.040 an agent so the another reason why I'm 00:37:31.040 --> 00:37:33.520 way way more in on agents um like I 00:37:33.520 --> 00:37:35.440 never liked open claw. I didn't see the 00:37:35.440 --> 00:37:36.880 purpose of it. You need to take need to 00:37:36.880 --> 00:37:38.079 go. 00:37:38.079 --> 00:37:40.000 >> No no no I'm just I just noticed my 00:37:40.000 --> 00:37:41.119 light wasn't on. 00:37:41.119 --> 00:37:43.280 >> Okay. I I I never liked OpenClaw because 00:37:43.280 --> 00:37:45.200 I didn't see the purpose of it. But I 00:37:45.200 --> 00:37:46.960 decided that because of my son is on 00:37:46.960 --> 00:37:48.720 holiday, he should, you know, set up an 00:37:48.720 --> 00:37:50.400 agent and then figure out if him and his 00:37:50.400 --> 00:37:52.160 friends can use it on a Telegram chat or 00:37:52.160 --> 00:37:54.240 whatever on Discord where they are to do 00:37:54.240 --> 00:37:56.320 something with it. Uh, I said I'll pay 00:37:56.320 --> 00:37:57.920 for the credits and actually we use 00:37:57.920 --> 00:38:01.440 Codex um budget and he set up Hermes 00:38:01.440 --> 00:38:03.119 and he's not really playing a lot a lot 00:38:03.119 --> 00:38:04.960 with it but I have it on discord so I 00:38:04.960 --> 00:38:06.720 just keep like asking it can you do this 00:38:06.720 --> 00:38:08.320 can you do that and it can do this and 00:38:08.320 --> 00:38:10.560 it can do that and it like it can do 00:38:10.560 --> 00:38:13.280 things uh it really started to sell me 00:38:13.280 --> 00:38:14.800 like when you earlier you said like you 00:38:14.800 --> 00:38:16.560 cannot like an agent's not going to be 00:38:16.560 --> 00:38:18.640 doing things on its own device but what 00:38:18.640 --> 00:38:21.119 I I really like with an agent like 00:38:21.119 --> 00:38:23.200 Hermes which by the way needed this for 00:38:23.200 --> 00:38:24.880 fire crawl because it says I need the 00:38:24.880 --> 00:38:27.280 fire crawl endpoint to do web scraping. 00:38:27.280 --> 00:38:28.720 I mean, it supports many other 00:38:28.720 --> 00:38:30.320 integrations, but firecrawl is one of 00:38:30.320 --> 00:38:30.960 them. 00:38:30.960 --> 00:38:33.200 >> Yeah, bright is a good one. But carry 00:38:33.200 --> 00:38:33.440 on. 00:38:33.440 --> 00:38:35.599 >> Yeah, I Yeah, I didn't pay for anything. 00:38:35.599 --> 00:38:37.440 And I asked like, can you run this on 00:38:37.440 --> 00:38:40.800 the on on your two CPU cores and 8 GB of 00:38:40.800 --> 00:38:42.480 memory? And it was like, yeah, I can run 00:38:42.480 --> 00:38:45.280 this on on on on two cores. And and what 00:38:45.280 --> 00:38:47.040 does it need? Well, Chromium is good 00:38:47.040 --> 00:38:48.560 enough, and you just need Docker and 00:38:48.560 --> 00:38:50.320 Okay, let's go. I don't going to pay for 00:38:50.320 --> 00:38:52.480 for it. So I'm running fire crawl stack 00:38:52.480 --> 00:38:57.280 local and um and it's fun uh and and so 00:38:57.280 --> 00:38:59.359 I started running a heras and I 00:38:59.359 --> 00:39:01.680 connected it with a I gave it a GitHub 00:39:01.680 --> 00:39:03.440 account and I connected it but it 00:39:03.440 --> 00:39:04.800 doesn't have any permissions on any 00:39:04.800 --> 00:39:07.599 repository. So it only it's like an 00:39:07.599 --> 00:39:09.359 external contributor. It has a GitHub 00:39:09.359 --> 00:39:11.200 account. It doesn't belong like it's not 00:39:11.200 --> 00:39:12.800 a collaborator. It doesn't have any 00:39:12.800 --> 00:39:14.960 permissions. It can only look at pull 00:39:14.960 --> 00:39:16.720 requests. It can fork the repo, create a 00:39:16.720 --> 00:39:18.640 pull request. I can do a GitHub search 00:39:18.640 --> 00:39:20.079 because you need to be authenticated for 00:39:20.079 --> 00:39:20.640 that, don't you? 00:39:20.640 --> 00:39:21.920 >> Yeah. Yeah. It has its own account. It's 00:39:21.920 --> 00:39:23.920 authenticated and they block agent mail, 00:39:23.920 --> 00:39:26.960 but I aliased it on another CNAME and 00:39:26.960 --> 00:39:28.880 and it worked. I was able to sign up 00:39:28.880 --> 00:39:31.520 with my one of my domains and it has its 00:39:31.520 --> 00:39:32.960 own account on agent mail. So, it has 00:39:32.960 --> 00:39:34.560 the ability and agent mail is pretty 00:39:34.560 --> 00:39:36.160 cool actually. You know, you you set up 00:39:36.160 --> 00:39:37.520 the agent and it has an inbox. It looks 00:39:37.520 --> 00:39:39.520 a bit like Gmail and you get quite a lot 00:39:39.520 --> 00:39:40.960 for free. Uh I don't remember how 00:39:40.960 --> 00:39:41.760 many,000 emails. 00:39:41.760 --> 00:39:44.160 >> Something else. I don't know. 00:39:44.160 --> 00:39:46.160 Why did you poo poo open claw? Because 00:39:46.160 --> 00:39:48.400 Hermes agent sounds the same as 00:39:48.400 --> 00:39:49.359 OpenClaw. 00:39:49.359 --> 00:39:51.200 >> Yeah. Yeah. Absolutely. Except they say 00:39:51.200 --> 00:39:53.040 I did a comparison. Apparently Hermes is 00:39:53.040 --> 00:39:54.960 a little bit older, uh, a little bit 00:39:54.960 --> 00:39:57.119 more security focused. I haven't tried 00:39:57.119 --> 00:39:58.640 OpenClaw, so I haven't really I don't 00:39:58.640 --> 00:40:01.599 really have a good comparison to it. 00:40:01.599 --> 00:40:03.200 >> But you enjoy. 00:40:03.200 --> 00:40:04.640 >> Wait a minute. We 00:40:04.640 --> 00:40:06.720 >> And and would you say Fire Call is 00:40:06.720 --> 00:40:09.200 better than Claude's native web fetch? 00:40:09.200 --> 00:40:10.480 That that's the thing that I was trying 00:40:10.480 --> 00:40:12.400 to get out of you. 00:40:12.400 --> 00:40:14.720 >> Yeah. So, so it came to the stage where 00:40:14.720 --> 00:40:17.760 Claude Code um which is like sorry cloud 00:40:17.760 --> 00:40:20.720 co-work which scrapes a bunch of pages 00:40:20.720 --> 00:40:23.359 but obviously it's using the entropic um 00:40:23.359 --> 00:40:25.839 infrastructure and it's limited to avoid 00:40:25.839 --> 00:40:28.000 getting blacklisted I guess. Um, and 00:40:28.000 --> 00:40:30.160 then I was like I was playing so much 00:40:30.160 --> 00:40:31.839 with this Hermes agent and it has this 00:40:31.839 --> 00:40:33.920 whole firecross tag and I moved 00:40:33.920 --> 00:40:35.839 everything to a local desktop. So it has 00:40:35.839 --> 00:40:39.599 like uh it now has 12 uh threads or six 00:40:39.599 --> 00:40:43.680 cores uh on an Intel um CPU. It's no 00:40:43.680 --> 00:40:43.920 longer 00:40:43.920 --> 00:40:46.480 >> ARM can now can thrash the internet 00:40:46.480 --> 00:40:50.240 >> and I was like go crawl. It's like how 00:40:50.240 --> 00:40:51.599 many pages are there? And it uses 00:40:51.599 --> 00:40:53.680 browser as well. So, so Hermes goes 00:40:53.680 --> 00:40:56.400 browser use, looks at the page, extract 00:40:56.400 --> 00:40:58.560 extracts, takes a snapshot, extracts the 00:40:58.560 --> 00:41:00.960 navbar, figures out all the I mean 00:41:00.960 --> 00:41:03.119 honestly crawling a web page, you could 00:41:03.119 --> 00:41:05.119 probably do that like 10 years ago with 00:41:05.119 --> 00:41:06.720 a basic Python script, right? You don't 00:41:06.720 --> 00:41:08.640 need an LLM for that either. Um, 00:41:08.640 --> 00:41:11.040 >> well, you do you do need it to run a 00:41:11.040 --> 00:41:12.880 browser because a lot of websites are 00:41:12.880 --> 00:41:14.800 JavaScript front ends and things like 00:41:14.800 --> 00:41:16.800 that. Yeah, but but yeah, I'm I'm 00:41:16.800 --> 00:41:19.119 crawling AWS docs and that's where I 00:41:19.119 --> 00:41:21.440 also asked you then why because I did 00:41:21.440 --> 00:41:23.200 the whole crawl. It was it crawled down 00:41:23.200 --> 00:41:25.599 400 pages into markdown with fire crawl 00:41:25.599 --> 00:41:28.160 in just like 5 minutes 00:41:28.160 --> 00:41:30.000 >> co-work didn't it it just was really 00:41:30.000 --> 00:41:32.400 limited it was blocked 00:41:32.400 --> 00:41:34.000 >> and so with with her 00:41:34.000 --> 00:41:36.319 >> with AWS docs MCP you definitely don't 00:41:36.319 --> 00:41:37.359 need to 00:41:37.359 --> 00:41:39.119 >> yeah so so then I I came to the 00:41:39.119 --> 00:41:41.040 realization because I was um I was 00:41:41.040 --> 00:41:42.480 having a lot of fun because this this 00:41:42.480 --> 00:41:45.280 desktop has a GPU so I was like it has 4 00:41:45.280 --> 00:41:48.720 GB of RAM in the CPU and apparently 00:41:48.720 --> 00:41:51.520 Hyperl is using 2.5 GB of it so I have 00:41:51.520 --> 00:41:53.760 like a little over a gigabyte of RAM on 00:41:53.760 --> 00:41:56.000 my GPU available and I was like yeah I 00:41:56.000 --> 00:41:57.680 can't do anything with that right but 00:41:57.680 --> 00:41:59.520 cloud was like no no you can run you can 00:41:59.520 --> 00:42:01.359 run embedding models locally and you can 00:42:01.359 --> 00:42:03.520 you can run a reranker locally and I was 00:42:03.520 --> 00:42:05.359 like oh that's cool and and I'm running 00:42:05.359 --> 00:42:09.119 like a full semantic memory layer uh on 00:42:09.119 --> 00:42:11.280 on a local stack like it's it's using 00:42:11.280 --> 00:42:13.200 another really cool find out of that is 00:42:13.200 --> 00:42:17.200 PG0 um from vectorz.io IO which runs PG 00:42:17.200 --> 00:42:19.760 vector embedded. Uh so it just spins up 00:42:19.760 --> 00:42:22.240 a Postgres. It's almost like SQL light. 00:42:22.240 --> 00:42:24.000 You know how annoying SQL light is 00:42:24.000 --> 00:42:25.839 because it doesn't have like prop I mean 00:42:25.839 --> 00:42:28.079 people love SQLite. SQLite is great 00:42:28.079 --> 00:42:29.440 because you just have a single file and 00:42:29.440 --> 00:42:30.640 you get started and you can run some 00:42:30.640 --> 00:42:32.640 integration tests but it's not the 00:42:32.640 --> 00:42:35.520 proper postgress diag 00:42:35.520 --> 00:42:38.480 schemas. Now you can PG0 you can run a 00:42:38.480 --> 00:42:41.680 embedded Postgres of a file just you 00:42:41.680 --> 00:42:43.040 know without having to stand up like a 00:42:43.040 --> 00:42:45.520 whole Postgres instance. 00:42:45.520 --> 00:42:48.000 Didn't know that. So PG0 is like a is 00:42:48.000 --> 00:42:50.400 like an SQLite version of Postgres is 00:42:50.400 --> 00:42:50.560 it? 00:42:50.560 --> 00:42:52.560 >> That's how I see it. So So this whole 00:42:52.560 --> 00:42:55.839 hindsight hindsight stack uses PG0 00:42:55.839 --> 00:42:57.760 embedded Postgres from the same company 00:42:57.760 --> 00:42:59.920 that built it vectors. Is that related 00:42:59.920 --> 00:43:01.680 to Hermes agent? What is hindsight? 00:43:01.680 --> 00:43:03.680 >> Yeah, hindsight and mem zero and others. 00:43:03.680 --> 00:43:05.440 There are memory systems. Um they 00:43:05.440 --> 00:43:07.359 basically create semantic links. So some 00:43:07.359 --> 00:43:09.520 of them are very basic. Uh there's 00:43:09.520 --> 00:43:11.280 another one that I came across. This is 00:43:11.280 --> 00:43:14.560 all in a use case of doing like 00:43:14.560 --> 00:43:16.319 researching and scraping the internet 00:43:16.319 --> 00:43:16.800 and stuff. 00:43:16.800 --> 00:43:18.480 >> I mean it's very very very closely 00:43:18.480 --> 00:43:20.480 related to like last year when you're 00:43:20.480 --> 00:43:23.680 doing um retrieval augmented generation 00:43:23.680 --> 00:43:25.440 basically when you have an incoming 00:43:25.440 --> 00:43:28.079 query you first go and fetch uh using 00:43:28.079 --> 00:43:31.119 those queries semantic related words and 00:43:31.119 --> 00:43:33.359 then you fetch the articles and then you 00:43:33.359 --> 00:43:34.960 inject it in the prompt so that that the 00:43:34.960 --> 00:43:36.960 the model the LLM is grounded with 00:43:36.960 --> 00:43:40.160 actual real um you know data. So this 00:43:40.160 --> 00:43:42.480 whole idea of doc of like chunking docs 00:43:42.480 --> 00:43:44.800 and putting them into a vector database 00:43:44.800 --> 00:43:47.920 to be able to do a vector search um you 00:43:47.920 --> 00:43:49.839 know semantically similar terminology. 00:43:49.839 --> 00:43:51.760 So how so how did you get started with 00:43:51.760 --> 00:43:53.520 all this stuff that you discovered it 00:43:53.520 --> 00:43:56.000 with Hermes agent firecrawl and then it 00:43:56.000 --> 00:43:56.880 just 00:43:56.880 --> 00:43:58.560 >> I mean I don't know about how open claw 00:43:58.560 --> 00:43:59.839 works because it probably has the same 00:43:59.839 --> 00:44:02.000 things but like Hermes agents has this 00:44:02.000 --> 00:44:04.400 basic memory file system like you have 00:44:04.400 --> 00:44:07.680 the memory MD the soulm the user MD that 00:44:07.680 --> 00:44:09.440 describes the user there's a couple of 00:44:09.440 --> 00:44:11.680 markdown files on disk but if you start 00:44:11.680 --> 00:44:13.440 with Hermes it has like quite strict 00:44:13.440 --> 00:44:15.359 limits on how many data there can be in 00:44:15.359 --> 00:44:17.680 the file and it very quickly like wants 00:44:17.680 --> 00:44:20.000 you to use some type of memory 00:44:20.000 --> 00:44:21.680 system and it advises you like if you 00:44:21.680 --> 00:44:24.319 use mem zero you just sign up you can $5 00:44:24.319 --> 00:44:26.000 credit out of the box. 00:44:26.000 --> 00:44:28.960 >> On the topic of of memory systems 00:44:28.960 --> 00:44:30.800 uh during that conference I went to last 00:44:30.800 --> 00:44:32.640 week there was this misilla project 00:44:32.640 --> 00:44:38.319 called CQ and the idea is that um 00:44:38.319 --> 00:44:41.680 >> yeah uh let me just 00:44:41.680 --> 00:44:45.119 launch the GitHub. I don't know if your 00:44:45.119 --> 00:44:46.800 memory program does the same thing. I'm 00:44:46.800 --> 00:44:50.400 just mentioning it uh because what they 00:44:50.400 --> 00:44:53.599 demoed was that you run CQ on like an 00:44:53.599 --> 00:44:56.240 organizational or team level, right? So, 00:44:56.240 --> 00:44:59.359 so you store memories as a team so that 00:44:59.359 --> 00:45:00.880 when you're when you're working through 00:45:00.880 --> 00:45:04.480 some issue, it it does it uses the CQ 00:45:04.480 --> 00:45:06.880 skill to grab what they call a knowledge 00:45:06.880 --> 00:45:08.960 unit, which is basically the same as a 00:45:08.960 --> 00:45:13.040 memory, uh to to basically do the job. 00:45:13.040 --> 00:45:14.000 And I thought that was really 00:45:14.000 --> 00:45:17.359 interesting. So have you explored using 00:45:17.359 --> 00:45:20.319 memories as on a team level? I guess you 00:45:20.319 --> 00:45:23.280 don't need to do that but I I do think 00:45:23.280 --> 00:45:26.880 it's very interesting idea 00:45:26.880 --> 00:45:29.359 shared agent because this is okay. So my 00:45:29.359 --> 00:45:31.680 exploration was like originally I was 00:45:31.680 --> 00:45:33.599 like should I just set up obsidian 00:45:33.599 --> 00:45:35.920 because this it lays out on markdown on 00:45:35.920 --> 00:45:39.440 disk right and then a human can browse 00:45:39.440 --> 00:45:41.760 the obsidian and see the relationship 00:45:41.760 --> 00:45:44.640 between the docs um and and it's very 00:45:44.640 --> 00:45:46.240 famous now people are like you use 00:45:46.240 --> 00:45:48.560 obsidian for a memory layer for your 00:45:48.560 --> 00:45:50.560 Claude Code sessions and things like that 00:45:50.560 --> 00:45:52.000 so I thought that's one option that's 00:45:52.000 --> 00:45:53.680 one of the first options I proposed to 00:45:53.680 --> 00:45:56.000 cloud say like hey maybe obsidian would 00:45:56.000 --> 00:45:58.160 be great to for me to talk to manage the 00:45:58.160 --> 00:46:02.560 memory of my Hermes agent. Um but or 00:46:02.560 --> 00:46:04.319 maybe I I have a Postgres stack because 00:46:04.319 --> 00:46:06.000 I have firecrawl. Maybe I can use PG 00:46:06.000 --> 00:46:09.280 vector or maybe and then it came down to 00:46:09.280 --> 00:46:12.240 what actual integrations does Hermes 00:46:12.240 --> 00:46:14.640 support and it supports me zero. It 00:46:14.640 --> 00:46:16.960 supports hindsight, it supports some 00:46:16.960 --> 00:46:19.839 other solutions. Um I don't see CQ in 00:46:19.839 --> 00:46:22.880 there. Um I then asked it to evaluate 00:46:22.880 --> 00:46:25.680 like local versus hosted because on one 00:46:25.680 --> 00:46:27.119 way I don't want to be locked behind 00:46:27.119 --> 00:46:28.880 like I don't want all my memor my my 00:46:28.880 --> 00:46:30.880 knowledge to be locked behind an API 00:46:30.880 --> 00:46:33.040 that is done very costly in retrieving 00:46:33.040 --> 00:46:34.800 information because a lot of these they 00:46:34.800 --> 00:46:36.800 make it very easy for you to put data 00:46:36.800 --> 00:46:39.359 but they actually try a lot to query to 00:46:39.359 --> 00:46:41.440 get data because then you get it's hard 00:46:41.440 --> 00:46:43.359 for you to get the data out and it's 00:46:43.359 --> 00:46:44.480 also one 00:46:44.480 --> 00:46:47.280 >> argument for actually running an agent 00:46:47.280 --> 00:46:49.920 is that like you use chbt sometimes 00:46:49.920 --> 00:46:51.520 Sometimes it's very annoying like Claude 00:46:51.520 --> 00:46:52.880 is very annoying with that when you say 00:46:52.880 --> 00:46:54.480 something he says like oh because of 00:46:54.480 --> 00:46:56.160 your other project this is very in line 00:46:56.160 --> 00:46:57.440 with what you want to do it's like I 00:46:57.440 --> 00:46:59.760 didn't ask you in line of my other 00:46:59.760 --> 00:47:02.319 projects right I mean I just asked you a 00:47:02.319 --> 00:47:04.000 different question um so I don't know 00:47:04.000 --> 00:47:05.520 how to tell cloud maybe it's possible to 00:47:05.520 --> 00:47:08.079 tell cloud like ignore all my me all the 00:47:08.079 --> 00:47:09.680 memories about me and my other project 00:47:09.680 --> 00:47:10.800 just answer this question 00:47:10.800 --> 00:47:13.839 >> yeah that I I I know what you mean by 00:47:13.839 --> 00:47:17.359 that I really know what you mean 00:47:17.359 --> 00:47:19.359 >> I think but on the other side the 00:47:19.359 --> 00:47:21.920 concept of tagging and domains. So like 00:47:21.920 --> 00:47:24.000 you might have a different a different 00:47:24.000 --> 00:47:26.160 organizational hierarch you know you 00:47:26.160 --> 00:47:28.640 could split your knowledge by the way 00:47:28.640 --> 00:47:30.800 your organization looks 00:47:30.800 --> 00:47:33.440 >> it has I think I think it's important I 00:47:33.440 --> 00:47:36.800 think markdowns on disks is definitely 00:47:36.800 --> 00:47:40.079 >> like getting limited at some point and 00:47:40.079 --> 00:47:42.800 also like a a confluence database and so 00:47:42.800 --> 00:47:45.520 on it's it's proprietary more and more 00:47:45.520 --> 00:47:48.000 of these memory systems have these type 00:47:48.000 --> 00:47:50.319 of features that are semantically for 00:47:50.319 --> 00:47:52.079 agents I I think this really interesting 00:47:52.079 --> 00:47:54.640 this confidence I don't know exactly how 00:47:54.640 --> 00:47:55.760 it works but I'm assuming 00:47:55.760 --> 00:47:56.720 >> the same 00:47:56.720 --> 00:47:58.640 >> assuming it has like a way of like if 00:47:58.640 --> 00:48:00.720 you use the knowledge and it actually 00:48:00.720 --> 00:48:03.280 was useful is sort of feedback that it 00:48:03.280 --> 00:48:05.119 was useful you know what I mean or or 00:48:05.119 --> 00:48:07.440 wasn't you 00:48:07.440 --> 00:48:08.880 there's some sort of like full life 00:48:08.880 --> 00:48:10.720 cycle management going on I think it's 00:48:10.720 --> 00:48:11.599 pretty interesting 00:48:11.599 --> 00:48:16.720 >> I should I I can ask okay I can we 00:48:16.720 --> 00:48:18.480 >> and you can still post this by the way 00:48:18.480 --> 00:48:20.720 you don't have to use their thing Okay, 00:48:20.720 --> 00:48:22.559 that that was one of the the important 00:48:22.559 --> 00:48:24.559 factors like there were some very basic 00:48:24.559 --> 00:48:26.079 markdown, there was some very basic 00:48:26.079 --> 00:48:28.960 lexical search solutions. Um I thought 00:48:28.960 --> 00:48:31.599 like do I need semantic like do I need 00:48:31.599 --> 00:48:34.000 uh embedding? Do I need this vector 00:48:34.000 --> 00:48:36.960 database? I don't know. Uh I tried just 00:48:36.960 --> 00:48:38.559 using markdown on disk for a week and 00:48:38.559 --> 00:48:40.880 then uh I thought maybe it would be 00:48:40.880 --> 00:48:43.200 better. Actually, I asked the the agent 00:48:43.200 --> 00:48:45.359 I migrated it from EC2 to my local 00:48:45.359 --> 00:48:47.040 desktop and then it shut down the 00:48:47.040 --> 00:48:48.880 gateway on the EC2 instance and it spun 00:48:48.880 --> 00:48:50.960 up the gateway in my local desktop and I 00:48:50.960 --> 00:48:53.119 asked the agent because it migrated 00:48:53.119 --> 00:48:56.240 cloud migrated everything cloud you know 00:48:56.240 --> 00:48:58.000 that was very nice to see it. It did an 00:48:58.000 --> 00:49:01.760 SSH tunnel to the um EC2 box and a tar 00:49:01.760 --> 00:49:04.079 stream down to the 00:49:04.079 --> 00:49:05.920 >> agent. This is the Hermes agent that 00:49:05.920 --> 00:49:06.160 did. 00:49:06.160 --> 00:49:07.760 >> Yeah. So I had I had been using it one 00:49:07.760 --> 00:49:09.599 week in the EC2 box. It has a whole 00:49:09.599 --> 00:49:11.359 bunch of like environment configuration 00:49:11.359 --> 00:49:13.520 memories it created. It had its own 00:49:13.520 --> 00:49:13.920 state 00:49:13.920 --> 00:49:14.960 >> shut it down. 00:49:14.960 --> 00:49:17.200 >> Yeah, it had its own state like um you 00:49:17.200 --> 00:49:18.720 know apparently there's a state database 00:49:18.720 --> 00:49:20.880 with all the sessions data in there and 00:49:20.880 --> 00:49:22.640 and then you know cloud helped me 00:49:22.640 --> 00:49:24.640 orchestrate the migration. It said and 00:49:24.640 --> 00:49:27.200 it actually did like a compression of 00:49:27.200 --> 00:49:30.000 all the files on the EC2 box streamed it 00:49:30.000 --> 00:49:32.480 down SSH and then decompressed it onto 00:49:32.480 --> 00:49:34.559 the local box which is like like 00:49:34.559 --> 00:49:37.119 >> Okay, you you're running code via 00:49:37.119 --> 00:49:39.280 Hermes. Okay. No, no, no, no, no, no. 00:49:39.280 --> 00:49:41.119 I'm not I'm not I don't want my account 00:49:41.119 --> 00:49:43.200 to be banned. And I was also very afraid 00:49:43.200 --> 00:49:45.119 that that Antropic would detect the 00:49:45.119 --> 00:49:47.599 Hermes keywords and and ban my account. 00:49:47.599 --> 00:49:50.000 What I was doing, I was running a 00:49:50.000 --> 00:49:52.800 control laptop that has SSH to both. And 00:49:52.800 --> 00:49:54.640 Cloud was orchestrating everything. 00:49:54.640 --> 00:49:57.680 Cloud was running SSH. Yeah, Cloud was 00:49:57.680 --> 00:50:00.720 writing shell scripts to migrate and to 00:50:00.720 --> 00:50:02.559 like get all of the data from the EC2 00:50:02.559 --> 00:50:05.040 box into my VM box. Yeah, I've seen I've 00:50:05.040 --> 00:50:07.760 seen Claude work over SSH before and 00:50:07.760 --> 00:50:11.040 usually it just goes S sh. 00:50:11.040 --> 00:50:13.040 >> It doesn't obviously it doesn't have the 00:50:13.040 --> 00:50:14.240 power to maintain. 00:50:14.240 --> 00:50:16.000 >> Yeah. Yeah, it does. But it works. It's 00:50:16.000 --> 00:50:17.520 insane. Like every because it's the same 00:50:17.520 --> 00:50:19.200 like running a command and bash on your 00:50:19.200 --> 00:50:20.800 local, right? It it writes this whole 00:50:20.800 --> 00:50:22.720 command script or whatever and it sends 00:50:22.720 --> 00:50:25.119 it over SSH and then waits for the box 00:50:25.119 --> 00:50:26.640 to settle and then gets back and says, 00:50:26.640 --> 00:50:27.839 "Okay, cool. This is all all 00:50:27.839 --> 00:50:31.599 configured." So it did the whole Q emu 00:50:31.599 --> 00:50:33.760 virtualization like VM setup, memory 00:50:33.760 --> 00:50:36.400 allocation, disk allocation, natural it 00:50:36.400 --> 00:50:38.800 created two bridges. Uh it gave the VM a 00:50:38.800 --> 00:50:41.119 local LAN IP bridge. It gave it a um a 00:50:41.119 --> 00:50:43.760 neted bridge with so there's host only 00:50:43.760 --> 00:50:46.000 communication. So the whole I want to I 00:50:46.000 --> 00:50:47.760 want to show you like the the hindsight 00:50:47.760 --> 00:50:49.359 dashboard because it looks really cool. 00:50:49.359 --> 00:50:51.440 You get like this semantic like 00:50:51.440 --> 00:50:53.040 knowledge graph of all of the memories 00:50:53.040 --> 00:50:53.920 of the agent. 00:50:53.920 --> 00:50:55.440 >> Sure. Sure. Although 00:50:55.440 --> 00:50:58.640 >> but it's all isolated. It's all locked 00:50:58.640 --> 00:51:02.079 away behind a virtual inter like bridge 00:51:02.079 --> 00:51:03.839 network that's only accessible between 00:51:03.839 --> 00:51:04.800 the whole 00:51:04.800 --> 00:51:06.720 >> do a tunnel cracky. 00:51:06.720 --> 00:51:08.960 >> No, no, no. I have to like I have to 00:51:08.960 --> 00:51:12.160 open the the UF the firewall to to open 00:51:12.160 --> 00:51:12.800 the port. 00:51:12.800 --> 00:51:14.640 >> Well, that's the it's the same way for 00:51:14.640 --> 00:51:17.520 openclaw I think. Hey. Yeah. Actually, 00:51:17.520 --> 00:51:20.240 in in the interest of time, I uh because 00:51:20.240 --> 00:51:22.079 I need to get to work actually. 00:51:22.079 --> 00:51:23.920 Actually, part of my work today is to do 00:51:23.920 --> 00:51:26.880 a risk assessment with the Slack MCP and 00:51:26.880 --> 00:51:35.599 the Gmail MCP. 00:51:35.599 --> 00:51:36.880 I mean, 00:51:36.880 --> 00:51:39.119 >> what's the SLE risk assessment like for 00:51:39.119 --> 00:51:41.520 people to use it and and risk 00:51:41.520 --> 00:51:44.800 >> for for for everyone to start using SL? 00:51:44.800 --> 00:51:48.319 I mean, I I'm a little like 00:51:48.319 --> 00:51:50.079 I wish I could say I'm not sure this is 00:51:50.079 --> 00:51:52.480 a good idea. Stop. 00:51:52.480 --> 00:51:54.880 Like what? Why use Slack MCP? Now 00:51:54.880 --> 00:51:56.880 everyone's going to launch agents into 00:51:56.880 --> 00:52:00.880 the Slack channel to do what? 00:52:00.880 --> 00:52:03.680 >> That's the thing, Kai. That's that's 00:52:03.680 --> 00:52:06.079 what organizations want now. like this 00:52:06.079 --> 00:52:07.599 Hermes agent that I'm doing is 00:52:07.599 --> 00:52:09.200 preparation for my job that I'm going to 00:52:09.200 --> 00:52:11.520 start because agents are going to be 00:52:11.520 --> 00:52:13.920 part of the of the of Slack of Jira of 00:52:13.920 --> 00:52:15.839 Confluence and we're going to assign the 00:52:15.839 --> 00:52:17.760 tickets directly to them which is I'm 00:52:17.760 --> 00:52:20.000 already doing with with the CDK terrain 00:52:20.000 --> 00:52:21.599 open source project. Yeah. 00:52:21.599 --> 00:52:24.079 >> Um I'm giving it access I mean it 00:52:24.079 --> 00:52:25.359 doesn't have any access but it only has 00:52:25.359 --> 00:52:27.040 read access. So I'm telling it there's 00:52:27.040 --> 00:52:29.280 actually there's actually a poll on the 00:52:29.280 --> 00:52:31.760 chrome that pulls request that pulls 00:52:31.760 --> 00:52:34.880 down a list of pull requests keeps it a 00:52:34.880 --> 00:52:36.960 tracker of when was the last time it saw 00:52:36.960 --> 00:52:39.200 it. So it's like a state on disk a 00:52:39.200 --> 00:52:40.800 simple JSON file that says I've seen 00:52:40.800 --> 00:52:43.440 this PR. It keeps a list of trusted 00:52:43.440 --> 00:52:46.400 authors and it it reads the PRs of 00:52:46.400 --> 00:52:48.160 trusted authors as long as they're not 00:52:48.160 --> 00:52:50.640 in draft and then it reads the common 00:52:50.640 --> 00:52:52.720 thread. It's all deterministic. No LLM 00:52:52.720 --> 00:52:54.160 involved, right? The script is purely 00:52:54.160 --> 00:52:56.960 terministic and it masks out any comment 00:52:56.960 --> 00:52:59.359 that is from not from a trusted person. 00:52:59.359 --> 00:53:01.599 So then it sends the full prompt to the 00:53:01.599 --> 00:53:03.920 LLM to Hermes and then Hermes does an 00:53:03.920 --> 00:53:05.839 evaluation which doesn't just read the 00:53:05.839 --> 00:53:07.440 code. It looks at the file diff, it runs 00:53:07.440 --> 00:53:10.400 the tests and it looks at the function 00:53:10.400 --> 00:53:13.280 of that PR and then runs its own things 00:53:13.280 --> 00:53:15.440 like for example I introduced release 00:53:15.440 --> 00:53:16.960 please and we wanted to validate that 00:53:16.960 --> 00:53:19.760 after it was merged to main um that when 00:53:19.760 --> 00:53:22.480 we merge a feature it will not bump from 00:53:22.480 --> 00:53:26.079 0.22 to 0.1.0 zero because we are in uh 00:53:26.079 --> 00:53:30.000 pre- major like alpha and so so I asked 00:53:30.000 --> 00:53:32.000 it after the release please PR was there 00:53:32.000 --> 00:53:34.640 like hey do a simulation and Hermes just 00:53:34.640 --> 00:53:36.480 creates a complete separate work tree or 00:53:36.480 --> 00:53:40.000 or directory simulates the merged calls 00:53:40.000 --> 00:53:42.559 this branch main adds a bunch of commits 00:53:42.559 --> 00:53:44.720 runs the release please command and 00:53:44.720 --> 00:53:46.480 validates the PR that would be created 00:53:46.480 --> 00:53:48.319 if it was not a simulation I can run 00:53:48.319 --> 00:53:48.960 release please 00:53:48.960 --> 00:53:51.440 >> I'm a little bit confused I mean surely 00:53:51.440 --> 00:53:53.280 this could have been done with Claude 00:53:53.280 --> 00:53:56.000 why do you No, you can't. What do you 00:53:56.000 --> 00:53:57.520 mean? I was I was eating breakfast and I 00:53:57.520 --> 00:53:59.760 sent a message on Discord. 00:53:59.760 --> 00:54:03.599 >> I see. So, so Hermes basically 00:54:03.599 --> 00:54:06.720 coordinated that work for you. Okay. All 00:54:06.720 --> 00:54:08.480 right. All right. But, but like 00:54:08.480 --> 00:54:10.160 >> the point is that this is the work the 00:54:10.160 --> 00:54:11.680 way we were going to work. Agents are 00:54:11.680 --> 00:54:12.240 going to be 00:54:12.240 --> 00:54:14.640 >> going to work. Yeah. But like I do feel 00:54:14.640 --> 00:54:16.720 that we're crossing a boundary very very 00:54:16.720 --> 00:54:18.559 quickly. Like for example, okay, Slack 00:54:18.559 --> 00:54:19.920 aside, never mind Slack. I can 00:54:19.920 --> 00:54:21.599 understand maybe that you would want 00:54:21.599 --> 00:54:24.000 your your agent to be aware of some 00:54:24.000 --> 00:54:24.880 context that's inside 00:54:24.880 --> 00:54:26.000 >> because that's where you define the 00:54:26.000 --> 00:54:28.319 conver like the the conversation defines 00:54:28.319 --> 00:54:30.400 the the product. But I think you need to 00:54:30.400 --> 00:54:32.400 dedicate those channels. You need to say 00:54:32.400 --> 00:54:33.359 only these channels. 00:54:33.359 --> 00:54:35.040 >> But then the next thing that the the 00:54:35.040 --> 00:54:36.960 that people I'm going to ask for now I 00:54:36.960 --> 00:54:40.240 want MCP for Gmail and then but now that 00:54:40.240 --> 00:54:42.079 Gmail is a completely different beast 00:54:42.079 --> 00:54:43.920 because because you can get external 00:54:43.920 --> 00:54:47.520 mail into into your Gmail inbox unlike 00:54:47.520 --> 00:54:51.040 So, so my my my Hermes agent has 00:54:51.040 --> 00:54:53.760 uh an email account, but it doesn't read 00:54:53.760 --> 00:54:55.440 the email. Like the email gets posted 00:54:55.440 --> 00:54:57.280 with a chrome drop into discord and I 00:54:57.280 --> 00:54:59.200 choose to act on it or not. Again, my 00:54:59.200 --> 00:55:01.520 agent cannot be directly talked to by 00:55:01.520 --> 00:55:03.359 anyone. It doesn't have any permissions. 00:55:03.359 --> 00:55:05.440 It's always an external contributor. It 00:55:05.440 --> 00:55:07.680 doesn't have any external like except 00:55:07.680 --> 00:55:09.359 for the one deterministic that I say I 00:55:09.359 --> 00:55:11.760 have trusted authors, but it doesn't 00:55:11.760 --> 00:55:14.480 like it reads the PR thread, but it 00:55:14.480 --> 00:55:16.400 won't respond. 00:55:16.400 --> 00:55:18.079 Well, if somebody asks and tags it on 00:55:18.079 --> 00:55:19.839 the PR thread and says, "Hey, can you 00:55:19.839 --> 00:55:20.319 tell me this?" 00:55:20.319 --> 00:55:22.880 >> I like I like to think that I set up the 00:55:22.880 --> 00:55:25.359 same rules for an open claw instance. 00:55:25.359 --> 00:55:27.200 So, basically, I got open claw running 00:55:27.200 --> 00:55:30.160 on my dad's uh WhatsApp. And the funny 00:55:30.160 --> 00:55:33.119 story was is that so um my sister had a 00:55:33.119 --> 00:55:35.440 WhatsApp group with my parents including 00:55:35.440 --> 00:55:38.559 my father and my father was basically 00:55:38.559 --> 00:55:40.079 talking and like everyone couldn't 00:55:40.079 --> 00:55:41.280 understand that cuz he's not on 00:55:41.280 --> 00:55:44.400 WhatsApp. Then later um I was invited to 00:55:44.400 --> 00:55:46.000 the same group and I was deadly 00:55:46.000 --> 00:55:49.359 surprised that my that my uh openclaw 00:55:49.359 --> 00:55:51.359 instance was chatting to my my sister 00:55:51.359 --> 00:55:54.720 and my mother u without without me in 00:55:54.720 --> 00:55:56.880 the mix. 00:55:56.880 --> 00:56:00.240 There's me screwed up that ACL or 00:56:00.240 --> 00:56:03.040 whatever. I mean all I'm doing is like 00:56:03.040 --> 00:56:05.680 uh open claw you only talk to me but it 00:56:05.680 --> 00:56:07.920 didn't work in this instance. Maybe 00:56:07.920 --> 00:56:09.359 that's where Hermes is really good 00:56:09.359 --> 00:56:11.119 because when you set up an a gateway 00:56:11.119 --> 00:56:14.400 integration, you have to identify the 00:56:14.400 --> 00:56:18.000 whitelisted like user ID on discord 00:56:18.000 --> 00:56:20.799 thread like channel ID. Uh we give we 00:56:20.799 --> 00:56:22.720 give the discord bot token permission to 00:56:22.720 --> 00:56:24.880 create a difference with open claw but 00:56:24.880 --> 00:56:26.720 through the upgrades maybe something 00:56:26.720 --> 00:56:29.119 something broke. Oh god. Okay. I I need 00:56:29.119 --> 00:56:33.920 to write some risk assessment and fix 00:56:33.920 --> 00:56:37.280 some stuff at work. But anyway, it's 00:56:37.280 --> 00:56:40.000 great talking with you, Vincent. Uh, and 00:56:40.000 --> 00:56:40.640 >> agents, 00:56:40.640 --> 00:56:42.880 >> hopefully anyone who's listening can 00:56:42.880 --> 00:56:45.440 weigh in and say that we're doing it all 00:56:45.440 --> 00:56:48.319 wrong. And 00:56:48.319 --> 00:56:51.119 >> like I I'm an absolute noob with agents. 00:56:51.119 --> 00:56:54.559 But I I was I was I was an absolute, how 00:56:54.559 --> 00:56:58.160 do you say, pessimistic? Uh, skeptic. I 00:56:58.160 --> 00:57:01.119 was very skeptic. Um, didn't trust it. 00:57:01.119 --> 00:57:02.319 Didn't think it would work. Didn't think 00:57:02.319 --> 00:57:04.640 it would give me any value. And in one 00:57:04.640 --> 00:57:08.319 week it just um I can't I can't find 00:57:08.319 --> 00:57:10.000 enough jobs to give to the agent. It's 00:57:10.000 --> 00:57:11.520 like can the agent do this, can the 00:57:11.520 --> 00:57:12.799 agent do that? 00:57:12.799 --> 00:57:14.960 >> You you need to record you need to share 00:57:14.960 --> 00:57:17.520 some of your workflow with me to maybe 00:57:17.520 --> 00:57:18.160 just 00:57:18.160 --> 00:57:19.839 >> I shared I shared the comment thread. I 00:57:19.839 --> 00:57:21.520 showed you the whole pull request back 00:57:21.520 --> 00:57:24.640 and forth between Claude Code web cloud 00:57:24.640 --> 00:57:26.240 code and the web basically while I was 00:57:26.240 --> 00:57:27.920 eating breakfast was implementing the 00:57:27.920 --> 00:57:29.839 release please. Then in this course I 00:57:29.839 --> 00:57:32.960 told the a hermes agent can you like run 00:57:32.960 --> 00:57:35.599 the simulation and it ran the simulation 00:57:35.599 --> 00:57:37.280 and then it commented back like I ran 00:57:37.280 --> 00:57:38.720 the simulation and these were the 00:57:38.720 --> 00:57:40.880 scenarios that worked that didn't work 00:57:40.880 --> 00:57:43.119 and but the worst part is after all this 00:57:43.119 --> 00:57:46.240 back and forth um after it was merged 00:57:46.240 --> 00:57:48.160 because of the markdown wasn't properly 00:57:48.160 --> 00:57:50.000 formatted in the change lock it injected 00:57:50.000 --> 00:57:51.920 the but release please is not LLM right 00:57:51.920 --> 00:57:54.240 it is derministically so the regular 00:57:54.240 --> 00:57:56.720 expression injected the next version 00:57:56.720 --> 00:57:58.960 release uh notes in the wrong section on 00:57:58.960 --> 00:58:01.839 the change lock which was after all this 00:58:01.839 --> 00:58:03.119 back and forth with the agent and all 00:58:03.119 --> 00:58:05.119 these simulations nobody looked if the 00:58:05.119 --> 00:58:06.640 change lock was actually you know 00:58:06.640 --> 00:58:08.880 properly generated like yes it's 00:58:08.880 --> 00:58:10.640 injecting it yeah but it's injecting it 00:58:10.640 --> 00:58:13.440 in the wrong bloody section 00:58:13.440 --> 00:58:15.520 so there's still still not like I think 00:58:15.520 --> 00:58:16.880 if a human would have looked at it he 00:58:16.880 --> 00:58:18.079 would say like yeah but it's not in the 00:58:18.079 --> 00:58:20.960 right place so yeah still still there's 00:58:20.960 --> 00:58:23.599 always some disappointment isn't it 00:58:23.599 --> 00:58:27.839 >> yeah I mean yeah let's don't trust talk 00:58:27.839 --> 00:58:28.960 about my sessions. 00:58:28.960 --> 00:58:31.839 >> The risk assessment. Don't Don't trust 00:58:31.839 --> 00:58:34.319 the agent on the risk assessment. 00:58:34.319 --> 00:58:36.240 >> I'm going to ride as much. I'm just 00:58:36.240 --> 00:58:37.440 going to 00:58:37.440 --> 00:58:38.960 >> It's so hard. 00:58:38.960 --> 00:58:42.000 >> It's so hard for me not like to just go 00:58:42.000 --> 00:58:44.960 like, you know what, let the agent do a 00:58:44.960 --> 00:58:46.960 first draft and then the draft gets so 00:58:46.960 --> 00:58:49.119 bloated that you go like, you know what, 00:58:49.119 --> 00:58:50.799 let another agent review the first 00:58:50.799 --> 00:58:53.599 draft. 00:58:53.599 --> 00:58:56.480 >> Are you not maxing out your max plan? 00:58:56.480 --> 00:58:58.079 Must be getting close. 00:58:58.079 --> 00:59:01.040 >> Not yet. Let me see. 00:59:01.040 --> 00:59:03.599 >> I have 100% left right now. Weekly I 00:59:03.599 --> 00:59:06.480 have 72 left weekly. Resets in two days. 00:59:06.480 --> 00:59:07.599 42 in reserve. 00:59:07.599 --> 00:59:08.960 >> Tropics surely are going to do a bait 00:59:08.960 --> 00:59:10.799 and switch. And this is 00:59:10.799 --> 00:59:12.640 >> they're they they're in the summer. 00:59:12.640 --> 00:59:14.400 They're doing promotions. school work is 00:59:14.400 --> 00:59:16.559 like increased limits and and and I 00:59:16.559 --> 00:59:17.920 think also 00:59:17.920 --> 00:59:21.599 >> once the IPO is done and dusted uh well 00:59:21.599 --> 00:59:23.040 fast forward a year you're probably 00:59:23.040 --> 00:59:25.040 going to be living under 00:59:25.040 --> 00:59:27.040 >> but that's why I want all of my memories 00:59:27.040 --> 00:59:29.440 out of Jet GPT and inside my own memory 00:59:29.440 --> 00:59:31.440 bank if I have a sufficient hardware to 00:59:31.440 --> 00:59:32.640 run it 00:59:32.640 --> 00:59:36.960 >> I believe that's CPU and um and let me 00:59:36.960 --> 00:59:39.599 know what you and yeah or whatever 00:59:39.599 --> 00:59:41.599 memory thing you use. Okay. Anyway, I 00:59:41.599 --> 00:59:43.839 really need to get back to to work. 00:59:43.839 --> 00:59:44.720 Great speaking. 00:59:44.720 --> 00:59:46.799 >> Read the blog. Read the Hermes blog. It 00:59:46.799 --> 00:59:48.880 tells you how to migrate out of ChatGPT 00:59:48.880 --> 00:59:51.280 using the entropic posts. Tells you all 00:59:51.280 --> 00:59:52.640 of the lessons learned in setting up 00:59:52.640 --> 00:59:54.480 hindsight on a local desktop with a very 00:59:54.480 --> 00:59:55.520 old GPU with 00:59:55.520 --> 00:59:57.040 >> I will give it a try. I will give it a 00:59:57.040 --> 00:59:58.559 try at some point. 00:59:58.559 --> 00:59:59.040 >> A lot of 00:59:59.040 --> 01:00:01.359 >> I do have a spare machine. I got a Oh, I 01:00:01.359 --> 01:00:03.040 didn't did I mention to you I have a new 01:00:03.040 --> 01:00:06.960 M5 48 GB beast in front of me now. 01:00:06.960 --> 01:00:09.040 >> That's That's plenty. 01:00:09.040 --> 01:00:10.559 >> Yeah, it's nice. And did you did you see 01:00:10.559 --> 01:00:14.799 that I was running uh Gwen LLM locally 01:00:14.799 --> 01:00:16.079 on Twitter? 01:00:16.079 --> 01:00:18.480 >> So, one thing I learned because I ne I 01:00:18.480 --> 01:00:20.000 never really cared about it, but the 01:00:20.000 --> 01:00:23.839 Deep Seek is actually really cheap. Um 01:00:23.839 --> 01:00:26.640 >> I think I sort of coach Flying by. 01:00:26.640 --> 01:00:28.319 >> So, you you're using that locally or you 01:00:28.319 --> 01:00:29.520 just use it? You got the little 01:00:29.520 --> 01:00:31.359 subscription? 01:00:31.359 --> 01:00:34.240 >> No, not local. I like I don't have the 01:00:34.240 --> 01:00:36.640 the laptop like you, but even then like 01:00:36.640 --> 01:00:38.400 you were saying that the the the token 01:00:38.400 --> 01:00:40.160 per second is extremely Oh, no. You were 01:00:40.160 --> 01:00:41.599 on Twitter and somebody replied the 01:00:41.599 --> 01:00:44.799 token per second is just uh abhorent. So 01:00:44.799 --> 01:00:45.760 So for me, 01:00:45.760 --> 01:00:48.079 >> the tokens per second is not terrible. I 01:00:48.079 --> 01:00:48.799 thought 01:00:48.799 --> 01:00:51.359 >> they said for for coding work you need a 01:00:51.359 --> 01:00:53.200 really high token per second because you 01:00:53.200 --> 01:00:54.880 will not you will not 01:00:54.880 --> 01:00:57.280 >> yeah actually to be honest you're right 01:00:57.280 --> 01:00:59.920 cuz I noticed my token tick claude goes 01:00:59.920 --> 01:01:03.520 to a,000 in one prompt 01:01:03.520 --> 01:01:07.680 so 50 into a th00and is uh yeah 01:01:07.680 --> 01:01:10.079 >> yeah you need really so so for me when I 01:01:10.079 --> 01:01:12.400 was looking at local LLMs I mean my 01:01:12.400 --> 01:01:14.400 hardware isn't good enough and then I 01:01:14.400 --> 01:01:16.000 was like you know worst case we fall 01:01:16.000 --> 01:01:18.240 back on like I put in a wallet on Deep 01:01:18.240 --> 01:01:20.240 Seat and let's see how he goes. And it's 01:01:20.240 --> 01:01:21.200 been 01:01:21.200 --> 01:01:22.880 >> three days and I barely used a dollar 01:01:22.880 --> 01:01:25.920 and I was like, "Huh, this is so good." 01:01:25.920 --> 01:01:28.160 >> Yeah, I actually configured Open Claw to 01:01:28.160 --> 01:01:30.720 use XAI and I was surprised how little I 01:01:30.720 --> 01:01:34.319 spent there. And XAI is some 01:01:34.319 --> 01:01:37.119 >> rightwing not drop off an LL 01:01:37.119 --> 01:01:39.440 >> and XAI is really good because it has 01:01:39.440 --> 01:01:41.839 all the Twitter info which is good for 01:01:41.839 --> 01:01:44.799 research I feel. Okay, that's ended. See 01:01:44.799 --> 01:01:47.359 you everybody. Please like the video. 01:01:47.359 --> 01:01:48.079 Comment below. 01:01:48.079 --> 01:01:50.160 >> Good point. I need X. I need I need some 01:01:50.160 --> 01:01:51.839 Twitter info because everyone post 01:01:51.839 --> 01:01:54.160 >> Exactly. So, get ready to for that 01:01:54.160 --> 01:01:59.760 SpaceX IPO. See you. Bye. Okay. Bye.