Generally Available

Hello and welcome to Yells at Cloud. This week brought to you by the generally available service that you couldn't use. My name is Gunnar Grosch and this is episode one.

Joining me today Danielle Heberling, AJ Stuyvenberg, Matthew Bonig, and Chris Williams. Welcome all to the first episode of a show we've been threatening to make now for quite a while. Hi everyone.

Hey, hey Gunnar, how are you doing today?

Well, I'm doing great.

Alright, we have a lot to get through, and I want to start with something that AJ has been waiting to yell about, I think. AJ, Lambda Micro VMs. Tell me what happened and whether I should care.

Gunnar. I'll try to land the plane here. I'm actually really excited about Lambda Micro VMs from a technological standpoint. It's I think their most creative compute product that the serverless team has offered it in years. It's the kind of thing that I've been begging them to make for a long time. What it is is a is a hosted version of

Lambda's underlying VM technology called Firecracker. And it's done in a way that is not tied to like a function response lifecycle like you get for Lambda, and is even more flexible than something like Fargate. So they give you you know, an actual running instance that you can connect to, you can send data to, you can get responses back in like an interactive session, and you can also suspend it and resume it as necessary. There's a couple of very cool aspects of Lambda that they applied to Micro VMs. I think the first is

When you deploy one, it will actually snapshot the running Micro VM so that when the function or s when the Micro VM is invoked for the first time, it it is restored. It's much faster to boot than it would be like on EC2 or Fargate where you to wait for the actual virtual machine to fully boot. In this case, it just restores the contents of RAM right back to the running machine. So it it comes up in milliseconds. That's a very cool application of Firecracker snapshot technology to a product.

What I think it's like particularly useful for, and Lambda's been very vocal about it underpinning a couple of other services. I think the biggest one that they talk about is Bedrock Agent Core. where I start to get into some of the the complaints is around their their pricing model. And while I think the the technology is super cool, the pricing model is is done in such a way where they give their own teams preferential pricing over other customers. So if you use Bedrock agent core, again built on top of

Micro VMs, you will pay only for the

active time where your task is scheduled on the CPU itself. So there's inside of a a Linux system there's a directory called Proc. It measures it holds data about metadata about different processes being scheduled and running on your machine, including what file descriptors are open, and what the task is doing if it's scheduled or not. So you you only pay in in the version of Bedrock Agent Core, you only pay for the time when the CPU schedules your task. And if anyone here has built AI agents or is using them, you know that a lot of your time is spent waiting on IO. So you're you're waiting on

on either like LLM inference, you're waiting for your your AI provider to respond to like a text query and then start a new turn of your agent. or you're waiting for some data to be loaded or stored from a data store or pulled somewhere. So the actual active CPU time is is quite small. And actually I looked at a couple of traces from a production service that I've I've been using at Datadog and it's like

point five percent of the time is active CPU time and the rest of it's dominated by either waiting for the actions that this agent takes, right? It like takes an action, it waits for that to respond and then it gets a response back. or it's waiting for the LLM. So I think like if you're doing pure agent development, this is probably not the product for you. That's my my my take on this. If you're doing something like they call it like if you want to run, you know, spark jobs or like glue like workloads where you have some big data compaction or data

analytic request that comes through every so often and you want kind of really large scale for a short period of time and you want to burst it and then you want to suspend it after a bit, that's great. there are some very cool and and very interesting aspects of this that are neat. So they will automatically suspend the task for you if it doesn't get any requests. There's like some

Interesting technological limitations. It's only like layer seven, so HTTP only. So you don't have like a raw TCP connection, so you can't host Redis, for example. myself and a couple other AWS Heroes Aiden Steel try to like deploy Kubernetes on top of this, which is kind of neat. and you can do you know, you can do TCP over WebSockets and a bunch of other little fun hacks. So there's like SSH on top of these, so it's cool. You get a real running machine. I I do like that. I do think it's a very cool dimension to serverless that we haven't had before. they also have a burstable CPU, so you you

provision low so say your task when it runs you need four vCPUs but when it's not running you want it to you know be around but much less so they'll automatically let you provision for one vCPU and then burst up to four x your capacity which I do think is like another cool aspect of this it does change the pricing but fundamentally I still think it's like roughly two point five to three point five x more expensive than like a Fargate. So I think if you're if you're on a smaller scale it's great.

Luc, another AWS Hero, replaced his his GitHub action runner fleet with with Micro VMs, which is like another cool use case. I did a little bit of napkin math and my my gut check is basically after you have like three or four developers working and you're doing enough jobs it probably makes more sense to

to build something on like more stateful computer you're use another product that's designed for that. but I think at low scale it it's pretty interesting. And I'm I'm mostly excited that it's a very, very different take on something we've seen from serverless. They're vocally departing from, you know, the the what serverless is supposed to be in spirit and they're experimenting with it, which I'm very you know encouraged about. I just really think it would be a better product if they would be able to be able to extend the per vCPU their the

active CPU pricing only model to Micro VMs because then you could throw a lot of these out there, let them suspend no matter what, and then when you're waiting for idle time or you're waiting for LLMs, you're not getting charged. And I think that just makes more sense. It's more reflective of the underlying cost to to AWS. and it's proven out by the fact that they bill their own teams based on that. You can you can buy Bedrock Agent Core and only pay when the CPU is actually scheduling your task versus what you would do if you actually try to build that yourself on top of Micro VMs. I think that right now customers probably don't care

so much. People are in a trying it out phase. But I do think what you're gonna see with time is people will have more and more specific workloads they want to run inside of AI agents, and that will necessitate a move from something like Agent Core or other competitor, you know, platforms that are that are agent builders to something where they have little bit more control over the compute. They'll discover Micro VMs, and I think the first thing they're gonna realize is like

their costs are gonna explode if they just kept kept running what they were doing and they'll have to kind of rebuild for that. So anyway, it is a very cool idea. I do like it. I've had a lot of fun playing with it. You can you know, we're we're still in the phase where where interesting projects are being contorted to run on those things like we had for a long time in Lambda. So I enjoy that very much.

But yeah, that's my take. I think it's a cool service. I I think it's kind of hamstrung by the pricing model. I do think eventually they'll come around on it or they'll launch like a flavor of this that is, you know, bills on active CPU time. Lots of other providers already do that, right? Vercel does that, Cloudflare workers do that. Notably Cloudflare sandboxes don't. That runs on their containers product, which it does bill, you know, by by like the hour. But otherwise, I think for the most part, it's a really cool service. You should check it out.

But in the state that it is in right now, who is it for then?

I have no idea. That's the the part that I cannot parse is when I ask about it and you and you read their blog, it's like, okay, this is for, know, you're gonna build an agent. I'm like, great, but

Bedrock agent core is that already. And it has the pricing model that I want. And again, I like I've the data. Like I have I have these agents that I'm running that I can show you, you know, 95 or 99.5% of the time is spent off CPU. It's blocked on IO or something else, right? so I I would never touch this. You're just gonna waste a bunch of idle cores. I don't know, you know, I I I would imagine this works great if you have like low density Flink jobs, for example. Flink is a data processing system for Kafka, very popular. And if you don't have a

large clusters of stateful flink compute, this makes a lot of sense. And you really do have that those like elastic needs. I think that can make a lot of sense. GitHub runners are probably another good example. Luc is doing this for his I think it's a solo project. You know, s small like not ... He doesn't have an army of staff as far as I know working on the AWS news feed that he's built. So I think that also makes a lot of sense. I yeah, there there's a couple of areas where I think it'd be it'd be cool. I can also see a couple of niche use cases where if they allow you to do something like TCP where you could you could host like

an in memory cache to like coordinate Lambda functions. There's like very cool things that you couldn't do on Lambda that you could now do.

You know, there was a project that built what is now a version of like the multi concurrent Lambda Managed instances. They built that on top of Fargate with Lambda. So they had like a Fargate coordinator that could multiplex requests onto sandboxes inside of Lambda. So you like, you know, don't waste those CPU cores. I think that was neat. That's something you could do with this too. Anything you need, you know, consistency with a capital C. Anything you need coordination, this could be useful if you're if you're doing some Lambda work. So I I don't have a killer use case for it though, 'cause it's certainly not

AI agents that are gonna spend most of their time waiting for inference. if your AI agent does something like a little bit of inference, then a ton of data processing, that might make more sense and then the burst capacities kind of work out on paper. What that is, I mean, we're in a world right now where people are overbuilding all kinds of things for a customer that they're hoping is gonna arrive at some point, I don't know. and and the clarity of responses I've heard from people is like this is kind of something they're using to underpin their own products and it made sense to like publish and and I think it does. And then maybe maybe we'll find that user. Maybe we will, but

Mm-hmm.

I don't know who it is now. It's not it's not me.

No. And the and the isolation thing in itself isn't worth four times the price.

Yeah, I I'm not, you know, I I do think that's like another target and we can kind of talk about that. There's like the do you want to run untrusted code that your users submit?

Right. That's like the you know, the Vercel question is always out there. and there are a few answers to that. Lambda famously backed Vercel for a very long time. They're now running on their own to some extent. I don't know what degree, but they're very very open about it on social media, that they're running their own fi hosted firecracker that I know powers all of their builds and it sounds like some of their invocations and some other stuff. But you know, beyond like if if people aren't uploading you know JavaScript code to you and you're executing it, it's probably not for you. There's there's a question of like what if my AI agent writes some code? I think that's what like Bedrock agent core.

is also pretty good at doing. the agent can write code and and kind of run there to some extent. you can also do very similar things on workers or like I said Vercel has a sandbox product that does that too. if it's only your agent you could probably get away with doing that on on Firecracker. I'm sorry on on Fargate because you don't care if there's some element of escaping or polluting within that cluster because it's all your AI agents. if you really don't trust that code then you're probably gonna use a VM anywhere as anyway, especially if you don't care about the other aspects. So I'm sure there's some companies that that

have this like killer intersection of they need snapshottable, restorable workspaces. I do think it's kind of cool if you're gonna have like a dev workspace that you connect to only a couple of hours a day and you want it to be there. that's fine. But again like companies that are at that stage where they're like everyone's gonna work on a dev box probably have enough compute where you're gonna run those statefully and then you can recharge your own machine, right? Like there's and there's like that does seem like a lot of work, but it really just isn't. Those are those are well understood paths right now where you can you could do that.

so I'm not I'm really not sure. I I I'm excited to see like the killer use case. I've seen a couple blog posts, I've tried it a lot. I've somehow racked up like a hundred dollar bill on it already, and I'm not exactly sure for what. but it is, you know, it is cool. It i i i i it I I will certainly say in the spirit of being constructive, it's like the most fun I've had with a Lambda branded product in a long time, by far. So and I and I hope you get the chance to play with it 'cause it is it's just neat. It's like it feels like the first time I tried Heroku in in it to an extent.

But i I d I just don't know where it lands in 2026. twenty six.

Yeah. Danielle, is this a problem or a solution to a problem that you have, or are you also struggling to find the the use case for it?

Yeah, for me mostly like it looks awesome, but yeah still struggling to find the use case. I also question how much it makes sense to have under the Lambda brand.

But yeah, I may consider using it for like lower, kind of like how Luc did it with his runners. Cause I use a CodeBuild build runners for GitHub and occasionally it takes some time to start up. that could like that alone would be a big win assuming my company's cool with eating the cost. But that's about a lot of time.

Yeah, there are some cool aspects of that. Right? Like you

can you can have if you have like hundreds of dependencies that you build ahead of time and your your runners are building them over and over again, it's hard to cache. Like great snapshots solve a lot of that, right? but that's not you know, those are also well understood things within within GitHub. Like GitHub has cache, you know, runner caches, GitHub has you know, you can you can cache layers of your container image also. That's like part of the OCI spec. so it's like it does solve a lot of problems that you could theoretically have, but I think there's already a lot of like solutions to them too.

so I'm I'm not sure how that how that plays.

yeah, like for my company, we need to be FedRAMP certified because we build stuff for like state governments. So yeah, like all self-hosted runners all the way and like, yeah, that's kind of the main reason why that stands out to me of like, this is another option aside from, you know, vanilla ECS Fargate versus CodeBuild. So I haven't checked it out yet, but wanting to experiment with it for sure.

Yeah. The pricing page is a history in itself as well. Just understanding how this is priced compared to to other type of Lambda functions, for instance, took me a while, but got there eventually, I think. Speaking of things that perhaps requires a support ticket almost to to understand we have CloudFormation Express Mode.

Yeah, so CloudFormation has been doing a lot over the last year or two to try to speed things up and to make the experience a little bit better for developers trying to build ephemeral environments or local environments, local environments. And this latest one is frustrating to me because they're sitting here claiming like you speed up infrastructure deployments by four times to by doing this new express mode.

The basis behind Express Mode is that they're no longer going to wait for resource stabilization before they say that the stack is complete or a resource is complete. That's not a finished deployment. Like they're just cheating at that point. They're not actually, you know, the resources aren't available. You can't hit you can't use them because they haven't stabilized, because they're not available yet. So to me it's it's just pushing the problem down

down the road a little bit. So now if I'm going to use this, so let's say I'm using this for an ephemeral environment. So I go and I do the deployment. And rather than waiting for a half an hour for an RDS instance to spin up, it gets done in five minutes. So then my CI process goes, hey, you're all done and the stack is done c running and everything. And I go to try to hit stuff and nothing is working. So I have to actually sit there and and now the ownership is on me

to understand, is the resource actually available and workable? So if you're talking about a fairly minor change, something that is very fast to go through, then this mode doesn't really buy you anything. And if you're talking about a resource that takes a long time to provision, a long time to stabilize, like a cloud front distribution, a Lambda attached to a VPC, an RDS instance, even an EC2 instance. Now it's on me to understand if that thing is up and

So it doesn't actually buy me anything. Now this actually puts more operational overhead on on me to care about. And I don't understand who this is for. They say, like very specifically for developers and for AI agents. Well, for an AI agent, I could kind of see it because giving the ownership to the AI agent to understand if something is available is kind of low cost. But from a development standpoint, this doesn't make any sense to me. Okay, so I have that ephemeral environment.

I can go run some tests against it, some end-to-end testing. no, I can't because RDS is still spinning up. And now I have to poll to see when that RDS instance is actually available or when the connection errors go away or when whatever. And it just doesn't make a lot of sense to me. I don't know who this is for. I I don't think this is something they should have released. And I think this breaks a fundamental contract that has been in CloudFormation formation since day one.

Which is if the stack is done, the resources resources are available. And so this should come with a huge warning label to people about, hey, don't use this unless you just don't care about doing any sort of access to that that resource once it's done, once it's it's complete, if you will.

So basically it reports done before it's actually done. That's the that's the whole thing here.

Mm-hmm. He lies. He lies to you.

Is there a I mean

Is there is there a way to like I I mean i i it sounds silly, but like, you know, bake in a depends on hook at the end. I mean, but but that's what the original version of CloudFormation does. So you don't you if you don't use that, then you don't have to do a workaround. Okay. I'm I'm struggling as well.

Build your own CloudFormation formation. Yeah.

Yeah. Yes. Right. Yeah. Like this is really just no now go build your own way of knowing when things are working.

every time I see something like this, I have to think that there's like one really big customer that has some horrific enterprise process where whatever you know, whatever's supervising the CloudFormation formation deploy has like a three minute timeout. And that's like unfixable because it runs on some old, you know, spark box or whatever, like some some machine in a closet somewhere that cannot be moved. And as a result, they're like, We just

We need a way to fire this off and not have to you know, the like the the actual CLI L I or whatever process is doing it from the from the local machine has to stop after three minutes. 'Cause I have no idea, you know, what the alternative is here. I don't know, you know if you're deploy like if you're deploying code to a Lambda function, for example, you can u like there's a different API endpoint just to update the function code. You don't have to go through CloudFormation formation for it. So I don't know what service takes so long to stabilize that they decided this needed to be done like globally through CloudFormation formation for everybody and not just through one service, but it's some

It's something we don't fully understand.

Perhaps the team got tired of everybody saying Terraform completes it ten times faster and they were like, you know what? Screw that. I'm gonna we're we're gonna do it, just not correctly.

Yeah, like there's a

there's

a metric somewhere that's like we made this ten times faster, and then there's like a tiny asterisk in in the bottom of like a quarterly goals document. There's like some summary that explains that they did this by just yeah, yellowing it.

Check.

Your mileage may vary. Please do not use in production.

I would also caution listeners as well that I was reading in the docs that it disables rollback by default. So be very, very careful or at least aware of that.

Yeah. Yeah. Yeah.

What the hell? Why?

Yeah, well because they no

That that

longer know that it's they're no w no longer watching for the stabilization. They don't know that they have to roll back.

Yeah, they can't clap.

What the hell is the actual point of this then?

There is there is one use case I can imagine where I would actually want this, but only and I say this bec b there are alternatives to this, but if you've ever deployed out ECS through CloudFormation, you know that it's it's very painful because a lot of times logically you want the cluster definition, the service definition, the task definition, all of these things to be in the same stack so that these things are a

one stack deployment and that's that's it. Well, if you've ever built an image on a local machine and then you push this thing and then ECS tries to run it and goes, exec error because you're running on a Mac, but you're x86 on your ECS tasks, then you know that that process of failing, failing, failing, failing until finally the circuit breaker kicks in, the deployment rolls back, now the entire stack rolls back.

And if you have things like an RDS instance also in that stack, now you're talking about a big pain in the butt there. But that is again, like they've already kind of solved that with the, you know, no rollback option. And so I could see how maybe this would be a slightly better experience in the ECS world. But this this definitely, I think AJ, you kind of nailed it. Like this feels like one big customer said we need to have this.

And since AWS builds off a customer requests, they built it, they deployed it, and now they think everyone else could potentially use it because it's more of a pain to gate this to only the one customer. And I just my general ex recommendation to our team is we can play around with this and try it, but I guarantee you we're never gonna find this in a in a production worthy use case.

I think I'm leaning Chris's way here. I I think it's just the the constant talk about CloudFormation formation being slow. This is just to to tick off that. Yeah. See, it can be very quick.

yeah, we'll fix your little red wagon.

And and it's and it's unfortunate because, you know, the CloudFormation formation team gets hit for that. They're the ones who get blamed for it. But it's not their fault. It's not their fault that it takes thirty minutes for an RDS instance to come up. It's not their fault that it takes ten to fifteen minutes for a Lambda attached to a VPC to come up and stabilize. They can't do anything about that. And I think the CloudFormation formation team has probably one of the hardest jobs in all of AWS because they're the one team

whose performance is not siloed to that team. They are dependent completely upon everybody else in the organization to support the CloudFormation resources, to have the proper APIs available to to handle these things, to make these resources provision quickly. You know, one of the things that AJ was saying with the Micro VMs, if they just did level four networking connections and now you could do T raw TCP, holy crap, I'm never using RDS again.

I don't know if that would actually work, but it would certainly be something I'd play with is how can I get Postgres running in these Micro VMs? Because I want these things fast, baby. I want them fast and I want them torn down quickly, 'cause there's a lot of cases where I only need an RDS instance for, you know, five, ten minutes a day and that's it.

You totally could too. I don't the the options are really, really open for that and I do think that there's like some thought to that. I don't I don't know if they wanna encourage it necessarily, but for like a dev system, for a dev box, I think it's like a really cool thing to play with when you just need a VM for a minute. It's it's very very easy to push a container image, have it build, you know, the the deployable unit from that image and then start it and it's yeah, yeah.

Our multi-million dollar production RDS was on a Micro VM. I'm sorry, what? You did

Better than being on Kubernetes.

Yeah, I mean it they still don't you know, it doesn't handle any kind of replication, any kind of consistency, any kind of like durability or availability or any of that. And I and they did that intentionally, like you'll also notice there's not a load balancer, right? So you don't have a way you know, there's no orchestration, you you have to kind of own that. but if you wanna build the orchestration layer, it's ready for ya.

And Matthew, you talked about CloudFormation formation now, but this is also then available for CDK as well, right?

Yeah, the CDK D K supports it. The the CDK D K team, I'll say, they've been doing a great job for about the last six months to a year of making sure that they are right in feature parity with CloudFormation on new stuff. I've been really impressed how it used to be the CloudFormation

would release some sort of change and you go to the CDK team and say, well, when are you going to support this? And they'd say, well we have a PR in flight and it's going to be out in a couple of weeks or something. Now it's day of and the CDK is really getting a lot of first s first support love and and feature parity with CloudFormation. So yeah, this is just an additional flag that you can now supply when you're doing a deploy through the CDK want to deploy my config mode is express.

And by default it rolls back to the proper slow way of doing things.

Alright, to to wrap this up then I guess the recommendation is

maybe try it out, but perhaps don't use it.

yeah, so I would say, you know, give this a try if you need some ephemeral environments or if you need something where they were already, you know, the resources were already deployed and available fairly quickly. This will shave a few seconds off here and there. but you know, I I wouldn't be using this for a lot of production. They even say like don't use this in production. I think they know that this isn't that valuable in a production environment.

Alright, something that I think is actually clever. Danielle, Cloudflare did something quite interesting with their CLI.

Yeah, so this one is less yelling and more, this is pretty awesome. Why don't more people copy this? So essentially Cloudflare Wrangler is their CLI that they have in-house built to deploy anything programmatically to Cloudflare. They essentially added a dash dash temporary flag to their Wrangler.

CLI. So you could do like npm wrangler deploy dash dash temporary. And essentially what that does is it will create a temporary Cloudflare account and it deploys it. And then you get a claim URL. So then you have 60 minutes, assuming you don't already have a Cloudflare account. If you're like, this looks awesome. I'm just trying it out. You can go to that claim URL.

sign up, it becomes your account, the resources stay in there. And the headline was really pushing, you know, for AI agents, like many things. But I actually think this is a very great use case for agents because if you're security conscious and don't want to give Cloudflare credentials to your agent, this is a great like lower barrier way.

to enable your agent to deploy something that you can actually test, but not give it your credentials. So I think that's probably what I would use this for. I've tried it out like quick dry run, work just as advertised with a simpler, just like hello world worker. And yeah, need to battle test it more with other bindings, but.

Pretty awesome. And yeah, I think my Yells at Cloud is why don't other people make things this seamless?

Yeah. No, I I think that framing there is interesting, that it's built for agents, not for humans. That the the the agents are then able to deploy its own environment and and it just expires unless you you claim it then I guess. I think it's a very different model, but I think it's appropriate for the time we're in right now. What do you think they're getting out of it then? Cloudflare? Will this bring in new customers?

Yeah, that's the question I had was like kind of what it looks like from Cloudflare's end from a maintenance standpoint, because they're going to have all these, you know, resources running and the account management on their end. As a user, it's great, but I'm guessing maybe it's just a way to get more people to sign up for Cloudflare accounts. But at the same time, I'm glad that I'm not tasked with being on the other end of having to manage and.

pay for all that compute that's just hanging out there.

Is this a case where you you have to claim this under a new account or can you roll it into an existing account?

What I was testing said it has to be a new account.

So this feels like just a way of artificially propping up their new account numbers then.

I mean if we're gonna bring the Yells at Cloud back into this, if we're gonna get salty then growth hacking, yeah, that it feels like some serious growth hacking.

Potentially.

Whoa, whoa, whoa. We we we call that growth hacking.

Hey this isn't Wells Fargo, what are you talking

about? They wouldn't do something so nefarious.

No, but I mean it like if I that feels like a a a barrier because I don't want to sit there and spin one of these things up and then have to create a new account to use it. I may want to roll it into my existing stuff. So if there's a path for that, that's great. But otherwise it feels like this is actually burdensome on anyone who isn't a brand new customer. Or as you said, temporary ephemeral environment and I just let it go away after a little.

Yeah, I think like how I would use it as someone who has a Cloudflare account would be like, do it per credentials in my account that I'm using, or my environment that I'm using the agent in, have it do the temporary deploy, and then do an environment where it does have the credentials. It's all, you know, IAC, so that would deploy into the other account. But I do definitely agree with your point though. It would be awesome if the claim URL could just be like, hey, move these deployed resources into like...

my actual account and this is the one I wanted in.

Yeah. Yeah, because vr especially for like enterprises, that would be absolutely necessary. And if if I was starting to adopt one of these processes and in these new temporary accounts, but I had no way of running it or elevating these things under my enterprise payment plan and I had to somehow own these things. I would be frustrated. I would be a little annoyed by it and probably just look elsewhere for these things or been sp spin my own up but

It is very to your point, like these are some innovative things from a billing perspective that a lot of other cloud providers need to start getting on board with. I feel like Cloudflare is unique in that they kinda can kind of get away with these things.

The neo clouds are so good at this. Like not just Cloudflare, Vercel continually does this. I mean v Vercel built their fluid compute primitive on top of Lambda and sold it before Lambda was able to do their Lambda managed instances, right? So they gave you multi-concurrent Lambda functions first. I think it's like such a good thing for for AWS to have that kind of competition because for better or for worse, big clouds are stymied by big politics. And when a smaller, more innovative

Customer in many cases, but also user is able to introduce a new concept that takes off like like fire. It gives them a path to at least like, okay, we can you know, we can also do this. We we have some validation that the market's there for it or like whatever, customers won't flee or someone will try and those types of things and and I I think that's just like genuinely a good thing in this space. Even if you I mean I'm I'm

I'm fairly sure that even some of the the high level engineers inside of of AWS are happy when they see you know, a Cloudflare or a Vercel or, you know, Modal or another like AI kinda lab or small company introduce a new dimension of pricing that people are excited about 'cause that goes straight into their doc that they had had on ice for the last six months, right? Like that's that's the forcing function that I think is a good thing. So I'm I'm stoked to see Cloudflare do this.

family.

Yeah, another cool thing is like students if they want like low low barrier to entry. I you know you don't want to be given that scary big cloud bill like this did not even require me to enter credit card info or anything like amazing for that demographic as well.

Totally.

Well Chris, you have been building up to something, and that is AWS WAF. AI traffic monetization.

Have I? built building up is a strong word. I

mean there's there's so there's so much it's

it's interesting how many I'm not gonna say reactive things that we've been doing as a result of the of the influence of AI, but this seems to be I I don't know if this is a good one or a bad one yet. I'm my the jury's still out. I don't know what the pricing is gonna be for this. I haven't looked that closely into it, but what's going on is

Currently, according to AWS, AI traffic, AI crawler traffic is now more up to 50%, maybe more, of all internet traffic total. and what that means, especially for people like me, content creators, people that you know get revenue from clicks, is we're not getting that money. the the click-through rates are dropping, the CPCs, CPMs, they're all they're all cratering. And I don't know if this is

I I under I understand the rationalization behind it, but I don't know who this is for, other than once I see the pricing, maybe I'll understand that it's okay, this is just for AWS's coffers. What they're what they're doing is they AWS WAF has now shipped an AI traffic monetization feature, which allows you to let's publishers, people that are creating content, to charge AI bots per request instead of just blocking them, which by and large they ignore anyway.

when the bot hits protected content, the WAF returns a 402 payment required. it uses c currently uses stablecoin via Coinbase, but it's going to then start including Stripe verification, start Stripe integration, and it classifies the bots in a bunch of different ways. You've got GPT bot, Claude web, you can do verified and unverified tiers, what to do with them, and have like reactive actions based upon that. So

If you want Perplexity to be able to scrape your site, but only for a certain price, you can basically set all that stuff up. This can potentially be a good thing for content creators that have been losing money because AI bots are not feeding back through the from click-throughs. it will be a different revenue stream than regular ad revenue. but again, I I have not implemented yet. I don't know what the I don't know how much of a

You know, how much of their pound of flesh AWS is gonna take off the back end of it. they are now basically playing traffic cop or traffic controller to allow folks to access or not access, or well not folks, but AI agents to access or not access. so I don't know if this is actually going to give publishers more leverage and more justification or or what's gonna happen with it. it's it literally just came out.

less than two, three weeks ago. So I have not seen any deployments of it in the wild yet. But you know, it's it's still early. So what what are what are everybody else's thoughts on this? D do Does anybody else have like websites that are that have lost revenue? What what what have you guys been thinking about this?

Too little, too late. This should have been done three years ago.

Even before, like when the base of the the models were were out. I mean it's it's such a cool idea and I I think like I I I do think there's like s a bunch of value in specific like bloggers and independent journalists like Substack folks that they can kind of capture some of that back that that some of that revenue back that was like ad sales, but for the the link aggregators and all those it's just it's far too far gone, I don't think.

Agreed.

Mm-hmm.

that that content is is deep in those models, it's never coming out. It just doesn't matter. I still see a lot of like onesie twosie traffic from bots on my blog, but I think for because it hasn't changed, like a lot of my content is static and it's just like AWS instructional stuff. doesn't have to get updated. So they don't you know, why why waste the time? I think post trading has really come a long way and I don't think like

the web stuff is not necessarily done in training anymore. as much as it is done in like for web access to look something up, right? In the same way, you know, you want to look at a pricing model for Lambda micro VMs, for example, 'cause you're about to go on a podcast and talk about it. the the AI will just fetch it live. Yeah, for it for totally, totally normal thing that you all do every day. that's kind of the the I think the the change of stuff. So cool.

For example, right.

price point actually, you know, being just pushed up the chain. Like l Perplexity is not gonna eat the cost of this. They're gonna they're gonna pass that on to their subscribers. So it's it it's it's an it's a another shell game, I feel.

Yeah, I feel like I'm wondering how this overlaps with Amazon Bedrock Agent Core payments. So if I'm able to put a WAF on, enable this on the WAF. Does it get charged twice? I don't know.

Mm-hmm.

Gotta find out.

But I I think what you said is interesting, AJ, as well, that many times it is the the actual agent that you're using that is fetching information and that traffic must be very high these days because it feels like my agents are doing that all the time.

Yeah, and keep in mind that

For a lot of people that's coming from a residential IP. You know, it it would be very, very, very hard to build. The same thing that makes like ClaudeBot or Hermes work great on a Mac mini that doesn't work well inside of you know AWS VM is that it's looks like your traffic from your house. So it sales through every kind of bot detection, every the the th we we could do a whole ninety minutes on how modern bot detection systems work, but your residential IP is so well fingerprinted to you that it is it's used for advertiser sales and decisions. Like when you see

Mm-hmm.

you know, someone opens Instagram and they're connected to your Wi-Fi, like y'all are gonna get things that you talked about that day because that person was talking about that already and you showed up to their house, you open the app, they know that you're connected to that Wi-Fi, bam, you're tied together and you're tied for life. And like those things all factor into your internet reputation, which goes the other direction in the case of the AI bots on my laptop. Like they had the same reputation as me. It's like unless you're unless you're an AWS C score viewer, it's Sterling. So so everyone, you know, like the the the the ability for some WAF to figure out that it's not a user

and and in fact an AI bot will be very, very hard for for when your c local Claude Code is doing anything. everything else about cloud agents, I think it's gonna go a different direction, but the internet is extremely mature in this respect. And it's very you know, people are millions of dollars in payroll goes out to to fingerprint what you're looking at with your eyeballs and I think that goes a little bit in service of what an AI agent can can crawl for you in an anonymous way or in a way that masquerades as you and not not an AI bot. So

Wild wild world out there.

That explains why I get all those weird ads when I'm over over at Bonig's house on my phone. I I did not put two and two together.

Yeah, they

know they like they they they're like this guy is on the weird train. Yeah. Yeah. They they sorted you into that bucket.

it's Bonig? He's a freak. Here. Just give him this.

I'll often throw my hotspot on in public places just to mess with people so that they get screwed over on all their ads for for Transformers and G.I. Joe's.

Mm-hmm.

This is the this is the mad genius I came

here for. This is exactly the mad genius I came here for.

So many leather shops. It's weird. All

right.

It's not un it's not unwelcome, just a little weird.

All right.

Hang it.

No, no,

no. I you know, I s I mean f at my house is different.

Yeah, yeah, exactly. Giant drums of let never mind. Okay, moving on.

Yeah, you're all latex at your shop.

So AWS is monetizing the AI economy at the edge now with that, but apparently these hyperscalers are also now willing to send thousands of engineers to your office basically to build your AI apps for you. So this is a topic that I find interesting.

Recently Microsoft dropped two and a half billion dollars to spin up a six thousand person subsidiary that they're calling Frontier Co. And AWS countered by I think it was something like a

A billion or a couple of billion of dollars with their forward deployed engineering program, a new org within AWS. So what that means is that these hyperscalers are now sending their own engineers to sit in customer offices and build the apps. And this is something new. Sure, they've had like AWS has had ProServe before, but this is something more than that. I think that traditional hyperscaler model now is kind of we build a primitive.

you figure out how to snap it together, but that model has broken down now with this.

The uncomfortable part of this is that many of these AI tools and the things you need to to use to build AI kind of require hyperscalers today to send a bunch of engineers to client offices to get it working. And if that's the case, I don't think they really have a platform that's more of a consulting gig at that point. So I think this shows that

Maybe AI adoption in these large enterprises isn't really going as fast as AWS and Microsoft and the others want it to.

I I think so ProServe is this is just the the next step in ProServe type work, right? And and but the problem that I've always seen with professional services work is that you've got these people that'll come in to an organization, try to become the technical experts and work with SMEs to implement some sort of functionality.

Write some code that that organization can't write themselves. And then they leave. And that organization is now responsible and has to maintain that code long term. And they didn't have the expertise in the first place to do it. So why do we think that they're going to be able to maintain in the long term? So either those contracts continue on and on and on and on for a long time, which organizations like AWS and Microsoft don't want that type of work because you are then just a warm body and you're just augmenting an existing development.

shop. Like the power of these things is to be able to come in, do a high impact feature development, and then leave. And if you don't leave, then you start losing your margins over time. Now, and and you lose your engineers. Engineers get bored with that and they move on and they do other things. But there's also it feels like an inherent conflict of interest here, too. Because if they're going to go in and they're going to implement these AI things, and they are doing this on behalf of Microsoft or

or Azure or AWS or whoever, then it is going to behoove them to make the most token inefficient AI agent that they can. And then how do you back away from that? Like I think a lot of why companies aren't adopting AI at the rate that they that these hyperscalers want is because A, it's very tough, but B, there's no way to know how

Mm-hmm.

effective these things are. And we've been seeing a lot of this floating around for the last like three months or so of like companies who are going, we just dumped a whole ton of money into AI. And we don't really know if we have anything to show for it. And now I think these big hyperscalers who are heavily invested CapEx wise on all these programs are going, how we got to fix this. We got to get people working these things and using them well and continue their token usage up and continue their spend up because if they don't, we're all going to collapse. So they're going to

Mm-hmm.

Throw these billion dollar programs at these things, but it doesn't really solve the problem, which is this isn't a technical problem. This is AI was oversold. And we're seeing now that people are starting to realize, AI can't fix everything, and it certainly can't do it at these magical numbers where we're spending a tenth to get the same productivity out of a person. Like we now know they've been trying it for a year and it's not getting us anywhere. And now they're panicking. This feels like a panic.

And maybe I'm just salty because I'm seeing a lot of people I know get laid off because of AI. It was supposed to take their jobs and it did. And now suddenly they've got nothing to go to unless they can somehow pass a magic AI HR bot to get hired somewhere. And that's a horrible experience. And the whole thing has just really made this industry a lot a lot of unfun from a corporate standpoint. I love the stuff that I can build.

with A on a AI on a day to day basis. But at work, it's incredibly frustrating when I'm sitting there and I'm working with clients and I'm talking to their AI bots and explaining to their AI bots why what they're asking for is not the simple thing the AI bot is saying. And then their response is always like, Well this is what the AI tells me and you know, this is supposed to be easy. So there's my rant. There's my yelling at the clouds. Like, ugh

I I saw a post on LinkedIn just the other day, someone was defending FDEs and basically said that well this is different than consulting because this is everything from ideation to production.

ha ha.

And I think that's worse,

honestly. If you need to bring someone in to come up with the idea what to use AI for, to then build with AI and then deploy it into production, yeah, you're not really solving a problem. You're you're creating a problem for yourself and you're paying for it. And then you end up paying for the tokens as well, like you said, Matthew. Yes.

so they really are on Accenture. I'm sorry, never mind.

That that's

what I'm saying. Billions of dollars of OPEX plowed into getting engineers to talk to their customers for the first time and it's all done and and you know, the idea is all being borrowed from your McKinsey's and your Bains and your BCGs and so on. It's beautiful, beautiful capitalism at work. You gotta just embrace this one, Matthew. I'm gonna give you the L here.

That's fair. And I could be very wrong about this. I'm just I'm just really salty on it right now because it just feels like every time I turn around there's another thing happening that's going that's not gonna pan out and everyone just put a half a trillion dollars into figuring that out.

Mm-hmm.

It's so I I don't think you're wrong, Matthew. I'm I'm actually on record saying that at least once now. And the the the problem that I have with this, aside from Gunnar's point and Matthew's point.

is is and AJ alluded to this a little bit, was the these people are getting paid to figure out how to increase your bill with that customer. I wouldn't trust somebody walking in my in my front door that said, Hey, I'm from Azure, I'm from AWS, and I'm here to cre create a s create make your life easier by creating an AI bot to help you spend more money with us. They don't say that last part out loud, but that's exactly what I'm thinking. It's it's the w it's the reason why I always go third party with ProServe to begin with, because

a a different company isn't going to say, well you need multiple RDS instances and you need them to be this size. They they'll say you can get away with a little bit smaller and and here's and here's an ECS instead of EKS. So there's and that that's an entire other different conversation. So yeah I don't I don't like that that that process and and them dumping all that money into it for all of those reasons.

And I don't have a solution or a fix for it otherwise. So I I don't have a a sh a silver lining for the end of that one. Sorry.

Don't hire

Don't hire Let them waste that that two billion dollars or something. It's just added to the pile of wasted money. I saw something the other day and I don't know if it's true, but something like the amount Yeah. Well it's that but it sounds legit. It sounds legit, so I'll let AI tell me if I'm wrong. that that if you think about back in the 2008 thousand eight collapse and all of the money that was put into all of these different housing market

That's a w that's a way to st I saw something the other day. I'm not sure if it's true or not, but

financial devices. We're two X that now on AI investments. And at least housing has an underlying value behind it. What happens when, you know, all of a sudden we don't have a use for all of those NVIDIA chips that they've been all buying up like crazy for the last three years? What do we do with those things? Are they reusable anywhere else? I don't think so, but

I think that'll go. Like I think it's gonna be used. I it's crazy. I do think we're overbuilding inference and I know we're we're spending a lot of time on this topic, but like it it'll go. I I think the the demand is insane. There the leaked financials from all these AI labs are s indicating that inference is actually a wildly popular and profitable industry. So they there's like margin to go down, if you're like a reseller and that's introducing, you know, Baseten ten and all these other competitors that are ho or providers that are hosting

like open weight models, I think it's like a ton and ton and ton of room for for capacity there. So even if like a lab doesn't eat up or something like a big a big lab implodes, they'll it'll they'll be use for sure. I i it's wild, but I think it's true.

Well, that also means that people have to start paying the actual price of tokens as well. But I think we'll get to to that. Yeah. Alright. I think we'll know the answer to to that question, I guess, in a year or two, see if these FDEs are still around or if it gets absorbed back into something else. So I th

yeah, very soon.

can we can look back at this

recording and say, see we were right. I love that.

were right, I'm sure. So I think we're gonna pause here and then when we come back, we're gonna have burst mode.

Alright, we are back and it's time for burst mode. Each of us brought one thing we noticed. It's not a deep dive. People, just something that caught your eye. Quicker pace, and Danielle, you're up first.

So much pressure. Yeah, so the thing that I wanted to bring up was there was an announcement post about Anthropic Claude Sonnet 5 available on Amazon Bedrock. And two interesting things about this. I think in the past, like it would say, generally available. The wording on this just says, is now available.

But I will say from my personal experience on my personal AWS account, I had tried to use it and I got a access denied warning and then it asked me to contact AWS sales in order to use this model. I'm not sure what available means. So yeah, just curious if anyone else had that experience.

Not support.

Well first off the wording is interesting because we've had a lot of those instances read recently where they posted that something is generally available. It's not really generally available though.

generally

it is.

In us-east-1 one.

For some.

And with people with better C scores than AJ.

Yeah, I mean that's a given though.

Welcome to the permitted underclass.

Yeah, maybe I've been talking to AJ too much, I don't know.

access point at reInvent, Danielle. Now they they they added us to the same high risk cohort.

It could

be, yeah, who knows? Blame AJ.

Yeah, no, it's interesting. the entire thing with rolling out these models it's I guess you have to be an enterprise to to start using them on on day one. Matthew, you're up next.

Yeah, so AWS changed a the IAM identity center. So now there's like programmatic access that you can do through things, which I think this may open it up for someone to actually create a better version of the UI console and make something that's, you know, not set twenty years in the past and you know consistent and functional and nice. But we'll see. I still think I am identity center is probably the best way to go when it comes to access to AWS and

certainly programmatic access would be is a nice additional feature there. So I'm looking forward to seeing what people create with that. we're gonna look at it internally at my company to see if we can't start creating some interesting tooling around that sort of thing.

I guess the question is why why did it take so long to get there, right?

Like a lot of things with IAM Identity Center, I think a lot of the times it's w why did it take so long? But it's

it's so core to your access, they probably have to be incredibly careful about how they they change and maintain that software 'cause you know, you're gonna absolutely kill everyone's access if you get one bug out there.

Mm-hmm.

All right, AJ, back to tokens.

Yeah, absolutely. So there is a new report according to the information that Anthropic and AWS are experiencing are experiencing a rift as they renegotiate their next contract. quoting the the information specifically, it says the change could s significantly increase Amazon's cost. today Amazon is paying for the number of hours of compute they use from Anthropic, and it sounds like they're next going to be paying based on tokens, which is what basically everybody else pays.

This is the first I've heard about. I A AWS and Anthropic have had a special relationship. A lot of Anthropic runs on top of AWS, so I can see why they probably have like a very, very special deal. tokens per second is how as far as I know, Anthropic does all of their like big enterprise deals. In in fact, like even if you're a startup and I think you're

the limits like five seats or six seats, like you get off of the like clawed max plans. So I think when everyone read this report, there's a paywall on the information. So unless you pay for it, you you probably couldn't read it and then it got re tw posted and tweeted. And I think a lot of people were reading that Amazon internally was somehow being allowed to use like Claude Max plans for their developers. I don't think that was the case. I think they were they were basically paying for the underlying compute that they were using from, you know, Anthropic. and now now they are gonna switch. Broadly though, there is obviously a theme.

The Brian Armstrong, the CEO of Coinbase, had a viral tweet that talked about using open weight models and how they're like

Increasing by a order of magnitude their amount of tokens spent and decreasing by an order of magnitude the amount of money they're spending because they they switched over to these like op open weight models. there are lots of AI gateways that are starting out there now where they're going to use a scoring or ranking algorithm to determine where to route your request. So if it, you know, you ask it to like change the color of a button, it's gonna go and switch it to a cheaper model versus if you ask it like some deep thinking task. I think this is gonna become part and parcel for using model inference in the next year. I think it'll be super common.

to have open weight models that are as good or close to as good as like a a much earlier version. I I did use GLM 5.2 five point two for a while in the last couple of weeks and I think it's really, really good. I for a like a joke had a friend to try it out on my laptop and see and they couldn't tell if it was, you know, Opus or or 5.2 point two. It was it it's they're really quite good. So I do think this is what we're gonna see a lot more of coming forward. which I think is just a cool part of the cycle.

Yeah, no, it's gonna be interesting to see also what happens with with a lot of the tooling. Like Kiro, for instance. Obviously you have been able to to pick some models, but it's heavily been leaning towards Claude.

You may have. I don't think I've ever tried Kiro.

I was talking about people in general. The ones generally available people, that kind of people. No no.

Not my not my account.

think you can use Claude Sonnet 5 on Kiro I've noticed that. Anyway. Yeah.

Special relationship, you know.

Yeah. Alright. Chris, we hinted at some booing before. Set the scene for us.

She has a high C score.

so I was at the AWS... I I go to a lot of Summits, obviously. I was at the AWS DC Summit and it was interesting because it was something that I hadn't experienced before. I've I've heard anecdotally of of people you know protesting or jumping up in in the audience and you know quietly being hustled out of the side of of different keynote speakers and and presenters.

At the AWS DC Summit, in the heart of DC, at the Javits Center, the Secretary Chris Wright, the US Secretary of Energy, was David Levy's ge guest speaker. So Dave came on, he talked about AI and security and

government and all of the things that he was supposed to talk about. And then the Secretary of Energy came up on stage and was r resoundingly roundly booed as he and and if you if you Google this, you'll see like news clips about how he was booed when he came on stage. What what those news clips didn't follow along with, and and I was in the audience, was the was the general scoffing

Of like the assertions that Chris was making. And I don't I don't know if these were like politically motivated or or or what their angle was in in regards to that. But when he said things like along the lines of building AI data centers is not gonna be materially impactful on our energy consumption or water consumption, it was like everybody was just calling BS in the crowds. Everybody was like, bullshit, you know, it was there there was

I don't think anybody's buying this. I I I don't think there's there's a a lot of folks in the in the general populace. And th and this was DC. I I don't I don't know if people are supposed to be extra spicy in DC or or if they're supposed to be extra

flexible, malleable or whatever. But like t to hear that was was interesting. I I will admit I'm also in the camp that I think that adding, you know, 30 to 40 kilowatt per hour racks for a data center with 6,000 racks is going to be materially impactful no matter where the hell you put it, much less in the middle of outer space or whatever absurd notion they have about that. But it was interesting to see how many people were like

It at an AWS Summit, you know, people people who are like, you know, they've drunk the Kool-Aid, they believe in cloud, they've they've they are techno nerds through and through, and they are not having it with AI data centers. So or or at least the suppositions of the Secretary of Energy of Secretary, that guy, say make making assertions. So yeah, I thought that was interesting.

Yeah, that AI is good for the planet argument. It's quite a hard sell right now. Yeah. Alright, I get to round off Burst Mode and I wanna talk about a paper that I think

Yes, that exactly.

Everyone that's shipping multi-agent systems right now should read. So Apple and Stanford researchers they they put LLM agents into teams and gave them classic coordination tasks. and they then asked whether the team beat its smartest member, and the teams didn't, which is super interesting. So what was clear is that these

Read this. Yes.

It's been quite common now to use experts basically within so you have different agents, multi-agents, and some are experts at certain things. And what was clear from this research is that the experts weren't really doing better. it's kind of the the compromise tax.

And teams don't ignore the expert, they average the expert's answer with everyone else's. So it's not disagreement, it's dilution So these agents would rather be agreeable than correct, which is super interesting. So they tried labeling the expert explicitly in the prompt, aggressively optimized prompts, still underperformed by 6 to but from 6 to 41%. So identification of what agent is.

An expert isn't a bottleneck either. And what I find quite strange as well is that that same consensus seeking that kills the expertise also makes these teams robust to sabotage. So when they added in a bad actor to the team,

it barely mattered at at all. So it's one knob that controls all of this. So I find this super interesting. I I've been building quite a lot of these multi-agent solutions recently and and this kind of changes a lot of things. So this is something a paper that I recommend everyone to to read.

I mean it it's it's

it's somet like like you you say hum humans figured this out decades ago. The larger the group of people trying to make the decision, you know, death by consensus. It's a it's a it's a phrase in business that you the more people you add to making a decision, the the worse that decision comes out to be. It's interesting that you you can apply it to AI too though. it makes sense.

Maybe multi-agent systems is just another thing to get people to to adopt AI. Maybe. Or burn tokens, yeah. Alright, people. That is episode one of Yells at Cloud. We made it.

Or burn tokens.

So, if you want to yell at us, you should then go to yells at yellsatcloudpod.com. You can obviously subscribe wherever you get your podcast. You should tell a coworker who has opinions about cloud and nowhere where to put them.

Like and subscribe.

Smash it smash that like button.

Smash that subscribe

button.