📆 ThursdAI - Oct 16 - VEO3.1, Haiku 4.5, ChatGPT adult mode, Claude Skills, NVIDIA DGX spark, Wordlabs RTFM & more AI news

for October 16th.

Can you believe it's the middle of October already? Welcome everyone to Thursday. I, my name is Alex VEOlkov. I'm an AI Avengers with Weights, & Biases from CoreWeave, and I'm adding to my stage my co-host Yam Peleg and Wolfram Raven Wolf. What's up guys? Welcome to the middle of October, one day after the middle of October. It's been, it's been, Big, big AI month for sure. we have a lot to talk about, obviously. don't remember a week where we didn't have a lot to talk about. I wanted to say welcome, to you folks. Yam, what's the most important AI thing that you saw this week

I'm waiting for, Gemini three.

Mm-hmm.

And I saw some mind blowing demos, allegedly.

Some hype.

Yeah.

But, but not just a little bit of hype, like full operating system. A website.

Yes.

just for listeners who are not following the same feeds that we do, there's this group of folks who allegedly get early access to models somehow through AB testing, whatever. And this one guy kept posting one shotted full, operating system build out. So like on the web, they built like the iOS one and Windows and Mac and whatever. And, yeah, that does look impressive. I saw the iOS one. I have built one a long time ago when I was a web developer, I built a clone of, iPad and it took me like three months and so I really feel it if this is gonna be one shot Alright. Wolfram, how's your, how's your, Yahoo? What got you excited this week?

Oh, there has been so much stuff going on.

So, of course the VEOs thing, the announcement of open may I to open up a bit more and become more open, when they bring the adult mode or something like that. So a lot of stuff is happening and, I'm very excited. Didn't have chance to get, look into all of this, but, yeah, it's a cool weekend. I'm looking forward to Gemini three the most.

Hey folks.

Alex from the editing floor here. We had an incredible show today with four interviews with incredible people. we had an interview with Jessica Gallegos, a senior product manager on the Google DeepMind team, all about VEO 3.1. We had finally the interview with Kyle Corbett from Open Pipe Weights, & Biases about Server, RL and RL in general. It was a great interview in depth. You'll hear it at the second hour of the show. We then had Queen Slack join us from AMP to talk about their new offering for AMP for free with ads. It's very interesting. And then finally, as breaking news, cognition released SWE Grep. And we had Swyx, our friend of the part jump in on the last part of the show to talk to us about how those speed ups are possible. So very exciting show, uh, in front of you. I hope you enjoy the rest of it.

for, October 16th.

we'll start with open source, as we usually do. No big, huge breakthroughs in state of the art open source. As far as I saw. There's one very important thing for us to talk about. we have, a new bilingual, model from Cist, Korean Institute of Technology. And it's a bilingual Korean English at 10 billion parameters called cmo, with a nice paper as well. Our friends from Qwen added, a new release for Qwen three vl, two small vls, which is always appreciated, like, 3 billion and 8 billion parameter vls. And then, we will have to absolutely cover the latest Google release in open source. It's, Google's C two S scale. It is a 27 billion parameter, Gemma model that apparently, reached a novel scientific discovery. I'll say this slowly again, because it's something that we've been all waiting for. Something that Gary Marcus said, that's never gonna be possible. A novel scientific discovery cell to sentence scale 27 B is, discovered breakthrough of how cancer cells behave, which is why Yam puts on the glasses right now. This is absolutely what we're looking forward to, and we'll definitely discuss what that means and, how this works in the world where, AI solve cancer. this is what we're going, so definitely worth the glasses moment. Alrighty folks. this is all in open source. I think the biggest highlights for this week is the big world, big companies, LMS and APIs. Claude Haiku 4.5 was release. Shout out to Anthropic. Let's go. We love the cloud family, the biggest brother, the middle brother Saana, that everybody loves 4.5, and, haiku 4.5. It's a fast, cheap model. I think it's like twice as fast as the previous one. so definitely Nisten would love to hear your thoughts about this. I saw you work with it already.

this too for later.

and then we'll move on to some OpenAI updates.

there's a big thread that, that Sam Altman started where, he talks about Chad DT Will, will start loosening restrictions and enable adult mold and potentially even erotica, which, you know, in December, as close as December, this is following some news that they've, they had a expert council on wellbeing and AI and researchers from Harvard, Stanford, et cetera. so we'll cover both these news. But basically we, we, like, we definitely have been waiting for adult mode. All of us, like, I'm pretty sure, this is something that, you know. We've talked about multiple times how GBT is maybe overly restrictive, and, they have been promising us this adult mode. That's by design. And I also absolutely want to cover the stuff we ended the show last week with from X AI and how they compare. We should definitely talk about, what adult want means and why it's important. they also updated Memory management. Automatic Memory Management is in J GPT. And then this morning, Microsoft is announcing, ai, co-pilot on the Windows 11 with every Windows 11 will be in A IPC right now. And this co-pilot will, will be able to execute tasks for you. So kind of like come at the browser, but on a OS level. And that's new from like, you know, an hour ago, I don't know if you guys already had the chance to see this 'cause we jumped on a stream, but it looks cool and it looks like Microsoft is absolutely gunning for folks, to talk to their computer. They tried with Cortana a decade ago. It didn't really work. but about time

now it should work.

Yes.

And, but hopefully, you know, with ai, it's gonna work. I also have an update, for folks, that due to our efforts, some things have, got solved. So, advanced VEOice mode, if you guys remember, we started talking about how advanced VEOice mode got really bad because it kept repeating the custom instructions. They were leaking into the context. And the advanced VEOice mode on OpenAI, would just repeat everything you were told it to. we raised this issue. Justin Berti, from OpenAI actually noticed and then come back to us with a fix. And so I tried, dance source mode and I like it again. So if you are aVEOiding dance waste mode, 'cause it was annoying you, they fixed it. So shout out to OpenNet for that. And then we're gonna go to hardware. we don't often cover hardware, but weeks like this, we, we should, Nisten Nisten, do you have the notes open in front of, do you wanna cover the hardware? TLDR Super quick.

Yeah.

Nvidia released this device, which on its own it is actually pretty impressive. Like, it's a low power device with 120 gigs of ram and it is optimized for four bit inference. It's the digits, workstation. but because at the end of the day, watts are watts and there's only so much efficiency that you can pull out of a chip. So if it is very low power and you've made sacrifices in terms of the, the amount of bandwidth, the LLM performance is going to suffer. this is a $4,000, 128 gig, GPU and CPU, put together by Nvidia. And it's tiny. It's like a Mac mini, a bit larger. a lot of people feel like that should have cost like a thousand or like 1,500 for what it was, because, it performed a lot worse than what was advertised in terms of flops. Would I want one? I would probably buy it if it was like 1500, but,

Okay.

So, j just to get back to the TLDR of this Nvidia, DGX Spark has launched, Jensen Hand deliver this to Elon Musk and the semi, and, some of them and like other folks as well. those handers by Jensen. but, the excitement aside, it looks like, you, you could get more performance from an M three Ultra. For example, apple also announced a new thing in hardware while we're in hardware. M five chip, M five, not M five pro, not M five, ultra or max, whatever. so this M 5G supposedly is like two, twice better for, for AI performance, but they didn't show any specific, AI things. You go ahead.

this one, even George Hot's commented on it, is, they've put

an accelerator inside each of the course. Mm-hmm. And this first came out On the iPhone. On the iPhone,

yes.

Chip.

but that's a very good design. it might end up being a lot faster, even the, I think it might be a lot faster even than what they're advertising there. And I think that might end up being a lot more than just 30%.

And so we're waiting for tests.

But also we're waiting for the pro and the ultra version of this, because this one I think goes only till what, 62? 32 gigs. 30 gigs of ram. Yeah, we want more. but it's a great chip, designed from Apple and, and it's coming into MacBook Pro, iPad and Vision Pro, by the way. all right. And the last thing in hardware is OpenAI and Broadcom. Set to deploy 10 gigawatts of custom AI accelerators. OpenAI is gonna make their own chips with Broadcom. they announced this this week. I think there's like rumors for a while, and now OpenAI is going to, make their own chips folks. we're gonna move on to this week's buzz where I'm gonna update you about everything that happens in, Weights, & Biases, CoreWeave. so we have a new model on our inference. It's open pipe Finetune of Qwen three 14 B. it's a instruct version, Finetune friendly instruct version of Coin three 14 B, because the, the main release did not include an instruct non-thinking model. So, the open by folks made it into a non-thinking model. Great for fine tuning. you can use this model on our interim service. We also, hopefully finally chat with Kyle Corbitt about ael and several launched from open pipe and just general, excitement from Kyle. if you are in Europe somewhere, our fully connected, event in London, it's coming up in 20 days, and that's a great way to meet with and biases, folks, some coral folks as well. This has been the hottest week since last week, actually, since two weeks ago.

Exactly.

VEO 3.1, folks, it's finally here.

It's next gen, video model. it has cinematic updates, audio, and a bunch of other stuff. we're gonna chat with, Jessica a senior product manager at Google DeepMind, about via 3.1. and that's gonna come in like 25 minutes, so definitely stay around with us. we have, I think around like three interviews here, which is great. We work really hard to bring you those people to tell you about those things. all right. OpenAI have updated SORA Pro to give you up to 25 second generation with their new storyboard feature. It's very similar to how, the storyboard works in flow from Google, and they've updated the default generations to everyone to 15 seconds on the default model. so I think from 10 seconds to 15 seconds or eight to 10. so that's impressive. and the quality there is not great, and I will definitely talk about this, because I've tried both and both are lacking the thing that we want. we're not quite there yet. But, it's great. And then, also I noticed that there's a new. Over 22nd video generation model from called Muse Streamer. I wasn't able to test it out because, if any of our listeners have a Chinese, phone hookup for me, I, I definitely need one. I feel like I'm restricted from the world of AI by not having a Chinese phone number, so I can't register to Bao, to, to WeChat. Like all these things. if anybody has a hookup for me, I would love to because I need to test out all the things. But, Baidu released new streamer, a while ago, I think in August. And now, they have over 22nd generation without extensions. in VEO three, apparently you can go up to one minute with extension, so that's great.

Thanks.

That was a wild sentence. I'm restricted from AI because I don't have a Chinese phone. Just want to point that out. that's true though. Sentence. That's true, man. Everything. That's true. That's true.

I can't log into different things and I really want

to, and you know, I don't have WeChat. Hi folks. We're almost at the end of the TLDR AI Art and Diffusion segment. We have, a very interesting entry to the world of, AI, art and Infusion. There's a new top one image editing model, like nano banana and C Dream. it's called River Flow one from Source full. and it's stopping the artificial analysis, ELO rank, over nano banana and Sea Dream and, et cetera. It's really hard to use it. I wasn't able to find this. There's like one place that has this, I wasn't able to register. so it's very interesting, that we have these models now that even I can get access to. So, source full folks, if you're listening, I want access to a river flow to test it out. I promise to be fair. And then also a very cool paper came out, that I haven't seen stuff like this before. D 360 is a state of art panoramic image generation. It like, unlike regular image generation, it generates panorama panoramic 360 images. So that's super cool. and the paper is great. And I think last thing we have to cover, unless we have, breaking news is, our friends from amp have launched a free tier. And this free tier is powered by ads. This is a definitely new and interesting entry in the world of, AI agents coding. I know many people are locked out from like Cloud, maxes, whatever, students, et cetera. And AMP launch the free tier. So we're gonna chat about them, with Quinn, at the end of the show. And I think it's time to hit open source because, we have a bunch of interviews. let's go to open source. All righty. So let's get going. Let's get started with open source source. Let's go

Open source ai. Let's get it started. All right, let's get it started. with, instance, prompt. We're gonna go straight into Qwen three, the smaller ones. we've talked about Qwen three. VL acquired, acquired a lot already. Qwen, VL generated great models. they finally the smaller ones, Right. So we have Qwen, VL four B and eight B two small. Let me see if I can open this in the tab. Okay. Those benchmarks are kind of like really good. Let's take a look. So we have, Qwen, let me just make sure that this is clear for folks who are watching. we're looking at the table least from Qwen full of benchmarks. And I dunno how many there are, but there's definitely more than 25 benchmarks. Qwen is crazy testing their models. I don't know of many other labs that do as many tests on their models and other models for comparison as Qwen. they have stuff like, and this is a visual model. So they have general understanding but also like a lot of visual stuff. So they have Hallus bench, and then they have MIA bench, mm star real, real world create math vision, MMMU Pro and Full. they have a lot of benchmarks and they compare most of them to the 72 billion parameter, 0.2 0.5. And I just. Comparing the tiny models of this generation to the main big model from last generation was Qwen 2.5. and seeing that the 4 billion parameter, 8 billion parameter matches 72 billion in the span of six months, when was 2.5 least, maybe seven, eight months. it is never gonna be not a problem.

Me.

That was a, very good model. I used that for a bunch of medical stuff. That was a very good model too. it's hard to run, but yeah.

we have, some examples, Qwen three VL 8 billion parameters.

The smaller one, it looks like it's beating g PT five nano, the smallest one on pretty much all benchmark that I can see here, on stuff like, real world GA gets 71 versus 60, on MMU and Pro 55 versus 36. this tiny model outscores GB five nano, if you need that vision kind of part. it's incredible. Now, as we hype up smaller models, for general intelligence, you would like different models for general intelligence. This is definitely not gonna answer questions in the way that, the bigger models answer. But for vision stuff, those models are very impressive.

I did a small talk, a small workshop a few months

ago, here where I just showed like vibe coding with a four B model. And I used like the previous four B eQwena stuff. It needs a lot of work, but it's weird that it can, and we're just taking the tech for granted now. I think the bigger deal with this is it understanding video because, you want like the smallest, fastest smart model you can for video just because of how heavy it is otherwise. Yeah, and its Apache too, so, yeah. I think this is like the on device model, right? Yep.

8 billion and OS world is the one that we're looking

at, for agent tick stuff, right? if you want the model to do computer use, we're looking at, screen spot. 33.9 score on OS world for the 8 billion parameter model means it's like a nagen thing that can click buttons. And we're gonna talk about clicking buttons in the Microsoft segment. 'cause Microsoft Cop out now does this, supposedly, but like 33% of Westworld is, really high. they compare it to the previous Qwen vl, 72 billion parameter had only 8%. So it's significant jump in O world. It looks like they just added a bunch of training data specifically for clicking buttons. yeah, this is a great model. We have breaking news in a second. Thank you for our listeners who bring us breaking news. all right, anything else about Qwen focuses? Apache two, it's available on hug and face. and shout out to the Qwen team for these releases.

Just for comparison, SONET four on OS world, which

is everyone was using it for that, that was only getting 41%. So it's pretty up there for an on-device.

SONET four is 41%, and Qwen three vl 8 billion

parameter is 33% on this world.

4.5 jumped a lot, but that's all right.

Yeah.

Yeah.

Sonnet 3.7 at 35. So you already got basically a 3.7 cable on, on device one for, for agent use?

runs like much faster on, you know, your local GPU.

'cause it's like only 8 billion parameters. all right. and then the next thing that we have to chat about is Google's C two s. Do you guys see this? Google released an update about, sell to a sentence, a model called Sell to Sentence Scale, 27 billion Parameter. I think this was a huge deal to not be missed, and we want to bring this, huge deal to you. So, basically, as far as I understood, it's a Google and Yale collaboration, based on Gemma, the open series of models from Google and DeepMind. they hit a breakthrough in generating a new hypothesis, a novel scientific discovery hypothesis. This is what we're waiting for. This is what Gary Marco said. That's never gonna happen. this is what, like David Deutsch says, that's never gonna happen. This is the type of stuff that, the people who assume that LLMs or stochastic paras repeating what they read, will never be able to achieve. And now I think it's one of the first signs of, let's fucking go type AI where AI hypothesizes something you'd never seen before. Never. No, nobody suggested before. And, scientists validate it experimentally in living cells, it turns out to be true. So in this case, it's a specific thing, because, you know, this Model 27 B open model was able to accelerate drug discovery for cancer treatment. and it's a novel hypothesis about cancer cell behavior, that nobody previously documented. they use GM open weights. let's read about this. Like, I think it's worth like stopping for this for just a second, I think because it's, it's, it, it feels to me like a big, big moment. meanwhile, folks on stage have you, have you read about this already? Have you looked into it?

Check the summary and they release this model and

the resources as open source. Just why we're covering it here. And that has to be applauded because, this is very beneficial for humanity and releasing it instead of just trying to make money outta it. yeah, kudos.

hundred percent.

you had a comment as well on Nisten.

I'm,

I'm looking

Okay.

I was trying to see how exactly, yeah, I think this is worth doing like a paper reading, and, and stuff for, we're seeing more and more because I've noticed before when they, study the cancer cell, apparently, like they take a picture with a spectrogram type of thing, and then they would train, a vision model and then combine that with an LLM to be able to just look at the dots of this spectrogram from a cell. And then from that they were able to train a model and do something. it just felt to me that when I saw those before, they were all done with very small models by like really broke grad students with very old GPUs. And they were still like getting all this, like major stuff done. So there's a lot more to do here to apply the combination where you have the data, you train it, you use a bigger model, and, you're able to do basically computational biology at scale and pick up patterns that you wouldn't normally pick up. Yeah, we're gonna see more.

A hundred percent.

This is only the start we had. our friend Dr. Daria, uh, I wanna say is the right way to pronounce his last name. he's immunologist and he is all, you know Yeah. As much as possible. And he's been on the show a couple times and he's saying, we're gonna solve cancer. AI is gonna solve cancer. And, you know, this is not solving cancer, what we're seeing right now, but it's a first kind of sign of what is potentially coming. the quote that I found from the blog, I don't wanna read it out. The result also provides a blueprint for new kind of biological discovery. It demonstrated by following the scaling laws and building larger models like C two s, scale 27 B, we can create predictive models of cellular behavior that are powerful enough to run high throughput virtual screens, discover context conditioned biology and generate biologically grounded hypothesis. Context, condition biology, I think is the word here. I think this is kind of what humans lack in expertise. And it looks like with bigger models, that's gonna come more to fruition. So I'm very, very excited about this one. and it's open source like Wolf and like you said, so it's great. It helps that folks can validate those results. and then not everybody has the lab, I think, I think it's absolutely incredible. Absolutely incredible. So shout out to Google folks for releasing this and, Yale folks for getting to a point where we're getting, novel discoveries. Yeah, go ahead.

Look at the size of this thing, like you're doing this with a 27 B model.

It just goes to show that if you train those things on, it's, I don't think it could even be called fine tuning because there is some free training here, that if you train such a model on only one thing and not trying to build a general model, you can get, pretty incredible results with a 27 B site model. You can run this at home. No, seriously, you can run this at home on a single GPU for inference and it's like solving cancer at the same time. And don't you think that's crazy?

That's insane.

That's absolutely insane.

super interesting thing is that, basically it's an LLM for

the language of individual biological cells instead of human language. So that is. Wow. You can use LLMs for so many things that can be interpreted like a language. Amazing.

I

Apparently they took the SC RNA sequence profiles and

they turned those into sentences, and then they just ran the RL stuff on, whether it replied with the right, RRNA, sequence of the cell. So it is like, They just used the same rl text techniques but instead of feeding full sentences, they just felt like long strings of the, DRNA.

look, I see that they have all the three stages

of, pre-training S-F-T-N-R-L-I. I'm in no way, shape or form, real biology, nor do I understand the complexity of the task itself. But just technically, I mean, do you know, from your knowledge, how much data are we talking about? How long did they train this? Because that's surprising. I mean, yeah. I get the idea. you can interpret anything as text and you can just prime everything into the model where you get incredible results. But how much data are we talking about? Let's say that I want to apply this to a different domain.

I found is that it is over 57 million cells and that was

more than 1 billion tokens of cells sentences that they converted it to.

Oh, that's not

bad.

Then biomedical text and metadata.

I wanna,

a

I wanna cover this like, last thing before we move on.

the scale is important here. They said this requires a level of condition reasoning that appear to be an emergent capability of scale. I like those three words. I like absolutely the ethos for everything. And the reason for us to put the, the let's go of sunglasses for sure.

It's all you need.

appeared to be an emergent capability of scale.

Scale leads to emergent capabilities. Novel things are happening. so incredible work on CC two s scale from, from Google Scale is in the, the, the name of this model. So very, very much incredible. Allright folks. for folks who are interested, scientists, et cetera, we w will add the links to the model here. They have a bio archive, which is like a, you know, biological version of the regular archive. full scientific preprint. If you are into this, definitely go check it out or also on hugging face. That's great. folks, we're moving on, because we have, we have an exciting interview for you right now. we're moving on to, vision and video, which is very, very hot for this week. And for this, we have a special guest that wasn't here on the show before. So we're gonna welcome Jessica, welcome Jessica, to the show, your product manager at DeepMind. first of all, thank you so much for joining us. please feel free to tell folks a bit about, who you are, what you're working on before we're gonna get to the actual exciting. features that you guys released.

Awesome.

Yeah. Thank you so much for having me. Um, I'm Jessica Gallegos. I'm a product manager in, DeepMind. I focus on our editing and control, side of the house. I also work a lot on some of our external engagements with, filmmakers and the creative community. I was part of the launch that we announced earlier this week. So excited to be here and joining you.

Awesome.

So thank you so much for joining. I think maybe one of the first things I'll say, there was so much speculation before you guys officially announced VEO 3.1 specifically about it already being there about supposed features And I was waiting for an official announcement, for a while. And then when it came, it was very interesting, to see how far the speculation went and how much hype the model was getting. and then we got a significant upgrade to VEO only a 0.1 from you guys, so not like VEO 3.5, the 0.1. could you walk us through the main differences from a person that works there and not some things that people invented? what changed, what is new in, VEO 2.1?

Yeah.

thank you so much for, also addressing that there was a lot of hype ahead of the launch. but yeah, this is an update that we were, very excited to bring. As you mentioned, it's not, a brand new model upgrade. It is not a VEO four. We're talking about a, 0.1 improvement to what we announced back in May. the main areas where we drove, improvements were on, better prompt adherence, and specifically, better quality on our image to video. for that would be for the base model. And then of course, what is, most exciting for me is, bringing a lot of new, creative, control through capabilities. So, we took some of our existing, controls from the May launch and added the audio component, which had been, a big request from the community, especially the creative community. so that would be for, ex scene extension. So that allows you to, condition on the last second of your previous scene and extend, We also have first and last frame. that one is really exciting. It really gives you a lot of creative control. Being able to, provide the first, frame that you want on your video, the last one, and then using a prompt to guide, what that transition looks like. we also, added audio to ingredients to video, if you're not familiar with it, allows you to bring in characters, objects, scenes, and combine them as you would, combine ingredients. and it gives you, amazing control in terms of timing and how those ingredients combine in the video. And now we support audio, which we're very excited about. so that would be on, generation, control. we also added, two new improvements to editing, within an existing video. I think this is pretty exciting. we, brought in, object insertion, as well as object removal, which are about control within an existing scene, so I think it can really open up a lot of new possibilities for creatives. so that's a recap of what we announced this week.

That's, that's first of all.

Thank you so much for the official, this is what we launched, folks. please make sure That, don't get too, I would say disappointed maybe. 'cause people do look at some previous like, oh yeah, it's gonna be this. And then, you know, they meet reality and that's not what they have. But you guys launched a lot of stuff. So one of the things that I do, definitely, I wanna talk to you about extension. You mentioned that, extending a video, which was a feature flow, which is the main interface for many folks, extending video, previously was possible with the last frame. you can, like, you make those last frame and asset and then you kind generate from this last frame. that obviously wasn't, enough for many folks. The main issue for me back then was audio. Like I would generate, my own face, for example, and I would get some very funny, very Americanized, person talking instead of me. And then, the next extension based on last frame would just be a completely different VEOice, a completely different like behavior, et cetera. So is video now a conditioning? So like the video can extend videos? Is that, is that how that's done?

So the extension feature allows you to extend any, any clip.

what we are looking at though, what the information that the model is receiving and conditioning on is based on the last second of that video frame, that you give it. So whatever visual and audio information is present in that last frame is what would be, taken on to the rest of the extension. So it's really useful for things like, for example, if you wanted to continue ambient noise, if you wanted to continue things like, audio and VEOice, then you need to make sure that the audio is present in that last second that you're conditioning on. But it also works, pretty well if the data's present.

Amazing.

the second thing that connects to this as well, you mentioned the ingredients, which I loved The features were only like VEO two for a while, and then we had to catch up. these are absolutely marvelous folks. ingredients to video lets you add, multiple things and then generate one video with these multiple things. So, you can add, an image of this jacket, for example, and then an image of me, this is how I generated, this video, this jacket, I think comes standard with the presets of video. and then the model will take those into account. Jessica, so you said that audio is now a part of this, so I can upload my voice and we'll generate it within my voice. is that what we're talking about here?

So to be clear, we're not using audio as a conditioning, but

the output will have audio integrated. we think of this capability, as. Very useful for when you're trying, for example, to compose a scene, and the amount of detail that it can really take in, like you could, as you mentioned, like not just reference a person, but you could reference very specific outfits, very specific, accessories that you want to be present. You can, another really interesting use case for this one, is you can have multiple characters and with the prompt, you can even bring them in at different moments during the generation, which I think is really exciting for storytelling. it's like a step change difference from what you would be able to do with Image two video, where you, you really have to stick to whatever's on that first frame here. You can really creatively bring in ingredients at different moments. for now, we're not yet, supporting the, audio conditioning as part of the ingredients that you're bringing in. But, you know, we're continuing to, build this capability and I think it does open a lot of, creative possibilities,

So, you know, I'm invited to, to tell us about updates,

not about feedback, but one thing that I absolutely saw from the community is, I want myself in there. This is like where, where I want to go myself with a, with a, with a picture of myself. That's, that's great. But then VEOice is a big component of this. And so this conditioning, once it comes does be great. But the extension feature, I absolutely noticed this, and I wanted to commend you guys and whoever worked on this because of the last second extension folks. Now, when you create longer videos, the same character speaks. with the same VEOice. I think it's very, very important because, before this you'll have to go and send it to like a different lip sync model, to dub, et cetera, and for creators that, that just added a, significant pain. So as you heard from Jessica, if the last second of your, video segment includes the VEOice, it will continue, speaking with the same. So that, that's incredible. let's talk about the remix. Uh, whatever you call this, the remix feature or like the edit feature. that's also fairly novel and new, right? before this, this wasn't necessarily that possible. Could you talk about this?

Yeah.

And just to clarify, are we talking about the insertion and removal feature? Insertion

removal?

Yeah.

Yeah, absolutely.

I am really excited about this feature because I think it will really reduce the iteration that is needed to get the shot that you want. Like, sometimes you generate something and there's like a specific detail that's missing or something that's extra. it also, doesn't only work on generated content. You could also have a video that you recorded of yourself, for example, where you had, someone crossing the background while we were giving an interview and then that totally ruined your shot. So now all of these things can be, generative Lee. And I think what I'm really proud of with this release is the quality. Because you're not just adding an object, but you're adding it realistically in a way that fits the scene. So we take care of things like shadows, we take care of things like occlusion. Same thing when you're removing the object, we make sure that it feels natural and sticks to the scene composition. So, actually a short backstory on this capability is, we announced back at io our partnership with, Darren Aronofsky, and his production, group, primordial Soup. And, one of the ways in which we actually built this capability was working with them. We were trying to put together a shot for a live action film, where they needed to have a baby. And, I don't know how much folks know about filming babies. but it is, it's very delicate and there's a lot of ethics around it. And it's also very difficult just from like, being able to get the right shot and, you know, you can't really like direct a baby to act. and so this capability actually allowed us to get that perfect shot. we actually have a blog post about it. I encourage you all to take a look at it, but it allowed us to insert, this virtual baby into the shot and really capture, the vision of that director. So that's an example of a really complex, and again, it was for an actual film, so it, really needed to get the lighting right, it needed to fit with the rest of the acting on the scene, the shadows, and really make it feel natural. So I think that one for me was like a magical moment of like, wow, this could really change some of, how we're doing things or even expand the creative possibility, of what's possible today.

I think absolutely, for creatives in an actual, production, these

capabilities are getting more and more in line with what they expect and what their, viewers expect with this quality. but also we should absolutely acknowledge how much power this gives to just regular, small, just as me trying to do some stuff. It's incredible with how much cinematic things I can achieve. And I think, the sound was enhanced as well. some cinematic motion was enhanced as well in VEO 3.1. So for 0.1, this is like a very, very impressive release. last thing maybe I'll ask you feature-wise is, last, first and last frame, interpolation, I think it, it existed there before, but now it's, it's a little bit better and I think none of bananas involved somehow.

Yeah, for sure.

So, first and last frame, again allows you to condition the first and the last frame of your video, and then, you can guide with a prompt what happens in the transition. the key upgrade here again is that we're bringing audio to those transitions, and also the better prompt adherence is also impacting all of these capabilities. So we're also being able to stay closer to the user's intent. the reason I was bringing up nano banana. y'all are familiar with it. So that's our, image editing, model is once you combine these two features, you really have like a power tool, right? Because you could really take any starting scene, edit it with Nana to turn it into whatever you want the ending scene to be, and then use VEO to guide that transition. And so I think we've been able to see some of our creatives do really amazing things when combining the two models for this feature.

One of the things that I saw, that works really well with this is taking

an object and then using Banano to have like an explosive version of this object with like the pieces flying everywhere and then like we interpolate this, or the IKEA box explosions that like was hippy for a while there that like somebody would like bring a box and then it would explodes with furniture all over the place. and so the model interpols and sound is now part of this as well. So generally, incredible, release. one thing that was kind of hyped but didn't really get officially, we didn't get an update on length. is that correct? Could you clarify the length things? I think we could extend, but the default length generation is still the same eight seconds.

Yep, that is correct.

Yeah.

So eight seconds. but then extension I think works way better. So I think this is one, one example from our own Nisten, uh, getting raccoons to talk about some stuff.

Well, it got the raccoon phase perfectly.

Like I wanted that exact raccoon. I didn't want other ones. So I, I recognize it.

All right.

Well that Jessica, thanks so much for joining, the show.

Thank you so much.

Thanks for having me.

folks VEO three reactions besides Nisten Nisten, your thing is

like super, like the recruit looks sick.

Yeah, the audio Sora just kind of has blown

away the audio be benchmark. It's like, but it's not like it's bad. The audio's good enough.

audio's good.

control that they now have so many options to upload

multiple images, give it references. The prompt adherence is better. So yeah, it unlocks a new capability. Similar to language models. And I think now that is great that they're focusing on this, giving you more control, extending videos and so on. So I will play a lot with this.

Yeah, a hundred percent.

we did have a second, like a little bit of breaking news. the breaking news that we now have is we have, world apps releasing a new thing called RTFM, realtime Frame Model. Did you guys see this already? you had a chance to go take a look, for leading the time. You can try a live demo of RTFM yourself, host on cloud gps, blah, blah, blah. this is RTFM from World Labs. We talked to you about World Labs and they're like world models, et cetera. real time frame model, a highly efficient world model, generates video frames in real time as you interact with it. Powered by single H 100 G-P-U-R-T-M renders persistent and 3D consistent worlds, both real and imaginary. What, let's go, let's take a look. So we're seeing, what are we seeing? We're seeing a, there's a button here that says, what am I seeing? But there's a, there's a world here, that we are connected to and we can move around this world. We can move forward. This is unlike the previous thing the World Labs does. This is like live rendered things. is that what we're seeing here? let's take another one. Okay. We're clicking the picture for folks who are just listening. It's called Frame Boy. we're click the picture that has like, some, reflections. Very good

name, very good.

Name

frame.

Boy. Like Gibo. Yeah, yeah. Frame boy. and then there's reflections in the ground. There's some water leaking, and we're like, in this bathtub thing. And usually these, real time diffusion models, they lose coherence on the objects that are in the scene. But this one seems, let me turn around all the way. 360 and see if the bathtub is still there. Oh, yeah. Okay. Yeah, that's, that's very impressive folks. it's, it's a small window so we don't get like a full window. Oh, okay. There's a toggle for quality and speed. So quality is higher, but like a little bit less of a refresh rate, but still it's like fairly real time. But if we remove and hit speed, we can see that, this generates as much as I rotate.

What if you go through the window Get to the outside?

Yeah, let's go see.

for now I cannot move there. let's see.

Yeah, I think it's limited in how much you can move.

Yeah.

I think there's like a little circle that I can move within. so I can turn around and look and then move a little bit forward and backwards. so folks, compared to the previous World Labs thing that we saw, this is like an interactive, real life. Kind of like the diffusion thing that happens on the fly. let's go and look at this library thing. Okay, we have this table, we have this chair. We can move to this chair. the consistency of the objects in there is very impressive. not quite Genie three level impressive, but again, we haven't played with Genie three and it's not real time, I believe. this feels like real time. So shout out to World Labs on this release. it's called, what is it called? RTFM. Generative world models will inevitably be computationally demanding, potentially scaling beyond even the requirements of today's lms. We believe a crucial research direction to explore the future of rendering. RT FM does not build an exclusive 2D representation of the world. So again, the thing that we saw from world loss before, is it builds a 3D world for you the pre generation step generates a 3D world, and then you walk in a 3D scene. this is similar to the card and some other folks where, you stream pixels and they're generated on the fly all the time. And so, instead it takes over one or more to the images. This input directly generates a new to the image of the same scene, from different points of view. and it's a, basically a renderer, a learned renderer, auto aggressive diffusion transformer, diffusion transformer that trained end-to-end on large scale video data. This is very impressive from them. Like I know they went to a different direction with the previous release, but this is very similar to the Cardinal Health. great breaking news comments on this, folks. are we walking into a simulation anytime soon with this?

I try to walk through the mirror.

it wouldn't let me, so I think it knows the world pretty well. that's where it's going. Is the whole UI and stuff just gonna be built for you on the fly.

Yep.

I'm excited to see this in virtual environment as well. I wanna talk about continuation of video. so we just talked about zora. we just talked about RTFM from World Labs, which I probably will add to 3D as well. world Labs. 3D Well, the thing that I wanna mention, is that after VEO 3.1 or because of it, or maybe not because of it. Sora also had an update, a fairly significant update as well. so here's the update from Sora. everybody can now generate up to 15 seconds up from the default, eight or 10 seconds, 15 seconds. Videos are generated in Sora, obviously with Canvas, et cetera. For SORA Pro, though, they've extended the video generation up to 25, 25 seconds. And the way to do this is on the web. you now have Sora Pro, you have this like new storyboards feature, new to SORA two has been there in the previous. So you can describe scenes, you can allocate this time to these scenes, right? So like scene one, ultra, you know, ultras dancing. Scene two, ultra, like falling, falling behind. as you can see, I'm tagging my name is because cameras work. although they don't show up as cameos, like, you know, when, when you generate a story, usually you text someone, it shows you their avatar. So cameras work here, I discovered, not that great, but they do work, falls down. But then you have this button here, 10 seconds, and then you can go all the way up to 25 seconds. I will say from my experience, my examples, the coherence of the last. Frames is really, really bad. Okay, so I'm gonna show you this cinematic movie that I did. It starts really good. It starts really good with the pixel frames. The text is really Garbo. This is so of pro. And then the longer it goes, the worse it gets. And the one thing I wanna focus on is like, okay, this shot is great. You can see my like hair, you can see eyes, whatever. and then now it starts to be very weird, like the cowboy wobbles in the weird thing. But then look at the face. This is just like, I don't know what this is. it's all getting worse progressively. And I wonder if this is the limitation of just like, video models going over certain minutes or just the way they build the extension feature. I dunno. So, the summary is SORA 15 seconds by default, 25 seconds for pro users and a new scene builder that you can use to build out scenes and an AI helper for you to help you with scenes. anybody of you folks use SORA? After we gave out codes and everything? Did you guys give up on it? Did the excitement pass? Where are we with SORA?

I've been having fun with it over the past few days.

I've been sometimes even hitting my, 30 video a day limit a video I generated yesterday, for example, was a Shiba Inu in a fighter jet getting pulled over by the police on the highway, and then it takes off using the highway to take off as a runway.

Nice.

Like, things like that.

I've been fooling many of my non-AI friends who

don't know that I have a podcast. I generated this, overflow from Vegas where I put myself on in the Thursday. I, ad and I said, I took out a little ad and, and people comment on this is though, this is real. I was like, folks, w we, we don't have that kind of budget on Thursday yet to take out the globe in Vegas. Yet we're gonna get there.

I would have left to get more into Sova, but, I even got an iPhone

from a friend and, you need a US account. So I couldn't create my own cameos or only a few Amy videos on ZOA for me now.

Well, hopefully it will get there soon.

but I think video is something that you'd have. moving forward to. we should talk about big companies. I think it's time for the main, updates as well. let's talk about Haiku. Stan, I want you to announce the Haiku news, if you don't mind, and let's talk about this.

so Haiku three had not been updated for, a while and, this

update is something completely different. maybe we were expecting some kind of small, tiny model to just do things here and there or parse data. But, when we tried it, I didn't even look at the benchmarks. I just set it in the workflow that I have. And, honestly, I like it better than Sonet 4.5.

Just to be very clear for folks who are listening now,

we're talking about Haiku 4.5 that was released by a topic just a few days ago.

The price or,

yeah.

Anyway, yeah, this just came out yesterday as a model.

finally an update. it was extremely fast. It wrote very, very good reports. But the most important thing, which I liked a lot better than sonnet is it did not have bad opinions. And, Gemini is the worst on this. It just sounds like a redditer that like is just completely wrong when it comes to things and tries to lecture you on how the app, should be It chooses not, not to do stuff. I could just obey to do the job. I, I told it, look through 16 pictures of the app and then take screenshots for each one of those 16 design pictures. And then it went, and it actually did that job exactly as I asked. And I told the Do not ask me any questions or make any statements until the work is done. And it was going very, very fast and it was going for like an hour and it actually com did the work as, as I asked when I asked it to edit and change the styling, the output, it was like, eh, you still need hundred or Opus for that. But, overall extremely impressed. I, I, I use this, this, this model a lot. I, I'm just surprised at how, surprisingly good it is. so I think they just made it for agent workloads. this is trained just to be an agent, I think.

Yeah.

So Swyx Eng verified for this model is, again, haiku 4.5. It's like a very, very cheap, tiny model. we should talk about the price as well. 73.3% on Swyx bench verified folks. You remember when we talked about Swyx bench getting like 25% and we were like, oh, 25%. This is the new soda. Jesus Christ. This is a tiny, super fast model getting 73%, on topic absolutely tops. The whole, top line of, Swyx bench looks like, 73.3% is close to 74 for GPD five Codex. It's more than GPD five and way more than Gemini 2.5 Pro. so, you know, we're waiting for Gemini three, but still people are connected to 2.5 Pro. it's three times cheaper per token than Sonet four, not 4.5, but Sonet four. It's twice as fast as previous cloud models. and then optimize for reasoning, like we said. this is a drop and replacement for Haiku 3.5 and sonet four on major use cases. So if you use Sonet four before, not 4.5, if you use HighQ 3.5 still, this is a drop in replacement now. the thing that we want a comparison to and we don't get often is kind of like other models. So I think the comparison to something like a Grok Four Fast, for example, which is I think is still cheaper, is something that people should do for themselves.

I started the, Eric, the ventures,

repo bench,

the re prop benchmark.

I see some people only got like 50. It was lower than, Quent coder. Bit higher than, than deep seek. I, that didn't feel like that when I was using it. But, again, this is, I've only used it, genetically where I just gave it a big list of tasks for it to do them. so I'm gonna run that bench and it'll be done in like 20 minutes or so, I think.

Nice.

actually have some not so positive feedback about hybrid.

Yeah, please. Because, they compared it to Sonet four and I've been using Sonet for a lot before and I gave it just one point in German. And the German of this model is much worse than the older sonnet one. It's a haiku after all. Haiku has always been very bad in other languages. Yeah, this is still true. So don't, expect the same quality that the benchmarks show if you're using a different language. Unfortunately, that's not the case.

Yep.

HighCo 4.5, drop in the replacement three times is cheap, compares to other fast quick models. It's really fast. not great if you use multimodal. So they do have sacrifices but significantly boost on O world. let's move to talk about this, and this is more of a conversation than an announcement. the announcement there is that posted on his feed that, open the, I as promised will release a check mark for an adult mode and supposedly give, treat people like adults if they want to. This includes, their period of strict restrictions for preventing folks with mental health risks and, different things, mitigation, that they took an OpenAI to make sure that JG GPT doesn't lead people to horrible outcomes. they are announcing that in December, they will roll out more customizable personalities, which is something that people missed from 4.0, and optional access to adult mode, potentially generating things like erotica or soft erotica, which blew the internet. Completely. People started thinking like, okay, here, here we go, we're done. Nobody's gonna leave their house, et cetera. Simon actually addressed this, with another tweet and said like, the reason why we're doing this, and like, here's the reason for some of this, we've seen those requests. Obviously people want to talk to Chat GPT in a less restrictive way. I feel fairly okay with making sure that, I'm not gonna one chatted by Chat GPT into insanity. I want it to talk to me the way I talk to Wolfram or Yam we should be able to cover topics. They're not deemed sensitive by somebody else because this is how, this is how we want our AI to be, other AI open source definitely have less of those restrictions, especially after some fine tuning. and we want the powerful models to also be and have this, basically the equivalent of Chat GPT right now. Assumes that everybody that watches a movie is PG 13 restricted. And so even the adults in the room are only limited to watching to interacting with PG 13 type chats. OpenAIs is basically releasing a rating system where there's like the 18 plus whatever that you can, r rated, R rated chats that you can decide to opt in into. Wolfram your thoughts on this?

I probably already have that mode unlocked

because if you're no prompting, you can get this anytime with all the models, even the big online models. So it wouldn't change anything. But I guess it's more about how customizable it's for the people. I'm not sure what people are doing with these models, but if health issues are really a topic then I understand that they need some safeguards. so adults can do stuff that teenagers for example, couldn't. in general, I like this and I have noticed that they have relaxed their restrictions on the models already. So over the couple past months even so personally, I see no censorship anymore in there. Nothing that has been there before. But I think, XAI has shown the way with being more open about this, treat adults as adults. I think that is a big and important concept we're paying for this. And as long as you're not breaking laws, doing anything illegal or. It's really, really detrimental to your health. You should be free to do whatever you want.

The one thing that is a caveat for some folks with

this, OpenAI does go into adult verification in order to do this. Like, for some of these safety controls that you will have to opt into, they will ask for your government ID to make sure that you're an adult to connect to your account. And that like triggers some people incorrectly. I don't care. 'cause I already have, via the API have been had to verify myself to use some of the image generation, the API. when they launch. So RO also, they, they launch parental controls when they know that SORAR is gonna be used by teenagers, but then their parents will be able to like restrict them for specific things. So it looks like, open eyes is definitely going towards, let's make sure that, you know, teenagers are not getting one. Shouted by ai, with some safety controls. And also together with this, they announced in eight member expert counsel on wellbeing and ai, including from Howard, Stanford, Oxford, to talk about, like mental health, risks in talking to these models. like because, people use J GPT as their emotional support, help. And so we don't want the AI to go crazy and tell them to do some stuff or do not seek support when they need, et cetera. so there's gonna be a council on this. I don't know how much this council is gonna be, deciding, but. those efforts are intertwined to get us more human-like emoji rich ai, that can, do some stuff. one call out there is some specifically said we're not usage maxing, which means they're not building these models to have your eyes on them as much as possible. and I think the line that I wanna draw is many folks reached out to me after last week's episode, where in the end we finished with at least I did share my, thoughts about, any, engagement features from X ai gr where, you have to talk to Annie every week to unlock a new potentially scampy, outfit. and it was really, really against this because that, that, like, that's engaging maxing, that's usage maxing, like asking a user to come in in order to receive some like, like specific, you know, outfit for your anime character that is supposedly your step into our social relationships. that to me, building something like this into the app, into the features felt, it's really great to see that, you know, OpenAI address that as something they explicitly are not doing. so, you know, those are two different things. Take it as you, as you will, which, which type of AI would you want? but I think opting in versus opting out I think is very, very important distinction. Anything else on this, on the, on the policy update? Yael Dja, I haven't heard from you in a while, on this. any thoughts on where OpenAI is going with this and how they're treating the new GBT release?

did it say, I remember a while back, I think it was even, I wanna say

December, 2024 when he had this threat about what do you guys want in 2025? And then he gave a summary of, okay, this is what it looks like people want in 2025. And one of the things there was treating adults more like adults and allowing them to do more adult things. And he's made some comments on and, and other places about wanting to eventually treat adults more like adults. So I think this is just inevitable and a lot of people are treating this as if it's just some sudden rash decision that Open is radically doing for revenue, but that doesn't seem to be the case.

I think as of last year or two years ago, the company that was making

the most money in AI, I think was replica. So this was the most profit generating side of the project. I think it's both. Like he does want to treat adults as adults, but also he doesn't wanna lose market share to grok in the, yeah.

In the more open type models that people actually interact with.

I think Simon talked about this as well when they introduced five, the amount of outrage towards removing all the models, like 4.0 with specific personality capabilities that people really loved was huge. People were like really mourning the passing of 4.0 as though it's like their friend passed. I think OpenAI definitely should contend with the fact that people are treating these models as something that's part of them and they want specific, things. And when they replaced the model, with not a version, it kind of does feel like they're replacing a friend with a different friend. so for many people this is now the reality and, being able to, Get back to that level of personality in open models, I think is important for some folks. folks, let's move on to, to talk about Microsoft is making every Windows 11 in the IPC. do you guys see this, super quick. Basically, the interesting thing is, they're adding a new agent copilot feature. we'll take a task and complete it in a separate desktop environment, and you can watch it while it works. So very similar to how Comet works. but let's, let's take a look. basically you'll have this is built into Windows and you can upload a bunch of files for it, like jpeg, et cetera. and then they will have a, like a, this is actually a very interesting approach. They will have a Windows environment is like standalone. As you can see. It launches like a mini windows in there. and they execute the stuff in that like closed environment that you can watch and supposedly protect it from using different apps that it doesn't need. It runs a browser in there, I believe, but it also runs, the explorer with files. It goes and finds files and, it launches, mail and sends a mail to somebody. I'm just like narrating the video that we're watching. This is built into Windows folks. So like, this is, like, we've talked about AI being built into Windows. Will folks use this versus typing it themselves? I dunno, but you can take control. Like it's very similar to how Comment operates, but on the Windows operating system level folks, will people use Agentic use on a noise level and what are the benefits for

I've been waiting for this because I'm a Windows user I

think any of the os makers can make an agent that works with you alongside you. co-pilot is a great term for that because, we all know the screen viewing agent that can give you, context and input if you have an agent that is risky because it could mess up your computer and you can't do anything else while it's working. So having it as a second user basically, that you can outsource some stuff to while it works in the background. That is a cool concept. I think that is a great thing and I look forward to use it.

I think it's a great differentiation of the benefits there.

An agent that takes control of your computer, your mouse, your keyboards, essentially prevents you from doing anything else. It's a foreground agent. Not only is it risky, like we said, that you can like click things, delete things that you, oh no, it's also. Preventing you from actually interacting with the Jupiter, which kind of sucks 'cause then you feel unpowered and you're like just watching, an agent on the background, like what they have with the solution of a, of a, of a encapsulated sandbox environment that's still windows. This is a big, a big difference, for folks. 'cause you can put it in the background like Wolfram said it and go do stuff. I tried to order some food with Comment yesterday and and then I decided to completely rely on this, and then only, ask it to like, let me know when I need to like hit send. The agent listens. and then, yeah, this week I ate a meal that was fully ordered and decided for me by, by the ai. It wasn't, you know, fully the best, but it was all right, from a place I never ordered before like that. That was great. And I wonder if like most Windows users will actually start using more Gentech stuff with this. I also wonder what models are there. but generally it feels like a big, big step for Windows to like put a gen AI in nobody's hands. Anybody else would use this? Folks, I don't know who's here on Windows, but like what do you think about like, agent, AI being built on the system level?

It's got no nowhere else to go.

Yeah.

as much as I want to criticize Windows and OneDrive

and all of that, we're in a bubble. The majority of the people, that's what they're gonna use like more than two thirds of everybody. So Google

is preparing that for sure because they just released

their computer use model, remember? And Apple will have to do something like that as well.

Yeah.

And I think the most important thing is this is OS level and a secure sandbox environment on a OS level. I think it's very important. And also like Nisten is saying, this is in people's faces. This is literally the default AI for them to use. if you think about this, there's the OS level and then there's the browser level, and then there's the application level. Like J GPT. J GPT is currently third in the stack. We have the OS level where most people exist. Not many people will keep exploring that upwards. So there's definitely that, Alrighty, we are go to this week's buzz and interview with Kyle Corbet from Open Pipe, who recently joined with Biases. Let's do this thing, folks, and then we still have some stuff, to talk about. we also have another interview of stories, so please don't go anywhere and let's go.

All right, folks, I am very pleased to welcome to the stage Kyle Corbett, who does AEL at Open Pipe, and is the CEO of Open Pipe as well. Kyle, you've been here before, and I remember we chatted with you before, so folks, may remember it, but for folks who don't, please, can you give us a little bit of, introduction to who you are and what's your background and, How are you finding yourself in this world of rl?

Yeah, absolutely.

so I'm one of the founders of Open Pipe. we do reinforcement learning for LLMs, to make agents more reliable. we've been working on this company for the last two years. about a month and a half ago we got acquired by CoreWeave. So now we all with Weights, & Biases, get to hang out, with the team a lot. got into this space because it was really cool. before this I was at Y Combinator. I actually worked under Sam Melman there, for part of my time there. And then, as I saw just everything that was happening with LLMs in 2022, it just seemed like too cool not to get into. So that's how I ended up here.

So that, that's like a super speedy, intro to you.

And then, we, on ThursdAI cover a bunch of open source models for an extended period of time, the covering of open source models also included fine tuning of those models. So like Yam here and Nisten and a bunch of other friends of Thursday Eye, we are all into looking at the hug and face open source, leaderboard, if you remember those. And like fine tuning and then open pipe you guys offered before ael. This was like a part of the service as well. That's how we talked to you, like, I think last time. Yeah. we saw with the increase of, Instruct models getting better and better. somewhere around LAMA three it was harder and harder to see like Finetune outperform the base models. and then we saw kind of a low in fine tuning and now it seems to like picking back up. Could you talk about this for a little bit from how you see this and, the offerings that you guys started doing in open pipe according to this?

Yes.

so there was, a lot of buzz around fine tuning around, the sort of LAMA two era and then Misra released some great models that were pretty popular to Finetune. and then yeah, it seemed like, much less interest in that. I think, it really makes sense. And you know, for the record all the way through this, I told people, in fact I gave a talk that was pretty well received at the, AI engineer conference in 2024 saying you probably should not Finetune. I think that, for a lot of use cases with SFT, it really didn't make sense. it mostly made sense if you had to move to a smaller model anyway, typically for latency reasons. So we see a lot of people doing real-time VEOice applications, where you can just get a much better user experience. If you move to a smaller model, you get far more tokens per second, you can have a much more natural sounding conversation. so that's the one niche where it feels like sort of like. Traditional SFT style fine tuning, still has a strong place. but for other things, I think like broadly the reason I see that happening, a couple of things. First of all, the price point on Frontier models dropped by, several orders of magnitude over the course of about a year. and a big part of the reason why people, did do fine tuning was in order to deal with, you know, not have to spend those frontier model very expensive tokens. So that became less of an issue. and then also, yeah, just, the quality got much better, from those frontier models. So you didn't have to do much task specific work to get good results. and also like fine tuning the way most people deployed it, and the way that kind of our workflow at Open Pipe worked was in this distillation flow. So you would, typically the way it would work is you would deploy some big model. You'd deploy, you know, GB four or four oh or something like that in production. You would capture request and responses and then train a smaller model, through distillation to perform similarly. now that can get you lower costs, that can get you faster speed, that can't really get you better quality than the model you're distilling from. and it can't give you better performance, can't give you better, you know, reliability. And as a result, as, as we move into these more agentic things, the real issue people are running into now is, is that kind of like reliability, right? It's the, that's, that's the big blocker when we talk to enterprises, when we talk to startups like, Hey. If we don't want a cheaper model. sometimes we want a faster model, but the biggest thing we want is we want a more reliable model. We want a model that can do these longer gen traces, and successfully complete them without getting confused and provide a bad user experience. And, SFT can't really get you there, or at least it's much harder to get you there. And we can talk about why RL can get you there. RL can actually get you more reliability. it can have, better performance on your task. And I think that's why we're seeing this resurgence, this interest again, because, it's solving a different problem that is one that's really top of mind.

reinforcement learning for folks who are listening and, part

of why I wanted to bring you here is to educate us a little bit as well. a year and a month ago, somewhere there, open air released like the first versions of oh one and like the main thing there was they use the rail for verifiable kind of outcomes, like math and code specifically. And, like train their models continuously on like chain of thoughts to like, think for longer and then bring us like different results. And like, I think the open source community also started like going into RL then we've seen incredible investments in RL from different companies. Brock four, famously like half of its training budget was on, on rl, like unprecedented amount of time spent compared to previous models as well. and then kind of after deep seek and GRP Hope. folks started getting really excited about like RL four smaller models as well. And we've seen, and we talked on the show about multiple very tiny models like in the 1.5 billion parameter, 3 billion parameter models that on very specific things that's verifiable, with not a lot of training. RL supposedly like reduces the time you have to, outperform major models. We saw this with the, deep research from, multiple models. We saw this on a bunch of coin fine tunes. could you talk about why RL achieves those goals and where they come from suddenly and then we can talk about the offerings that we are releasing to people.

Yeah, totally.

So I guess the first thing I would say is like, I'm a little bit bearish on just like, you know, I think that there's no fundamental reason why RL makes it so that smaller models are better than larger models or something. I think that's sort of like the existing scaling laws from before is still mostly true. So I don't think we're gonna see, start seeing, this big wave of tiny models that are actually better than big models on general tasks. Right on, on very, very specific things. Yes. And I think even when we're talking to customers, typically, we don't try and move them to those very, very small models. Like you're talking about the 1.53 billion parameter models. I think, our sweet spot is like maybe 14 billion parameter models, and our biggest focus right now, at Open Pipe is moving up to larger models. you'll see announcements for us soon on training much larger models, hundreds of billion parameter models. because I think, the RL works there just as well. in fact, it works better because those big models are more data efficient, because they're already so smart. I think, the reason why people are investing so much in this is because it is a way to get better performance out of, task specific rl, the big labs are doing RL on, many, many different tasks. And then also just kind of like general purpose reasoning and math. the incredible thing about rl, relative to where we were before with just kind of pre-training and then like instruct training is that it lets you get better performance than your training data has. Right? So like, if you're appreciating data on a model on the whole internet, you're sort of like the upper bound in some ways of how good their model's gonna be in practice is kind of like limited by the quality of the text on the internet. You're training it on. whereas when you use rl, you know, you're basically what you're doing with rl at a very high level is you're saying, Hey, here's the question or the context or whatever the task you're trying to do. Go do whatever you want. You can reason, you can, use tools if there's tools available, you can search the web. Like there's all these different things you can do. you, as the model can do whatever you want and then you just gimme back a final answer and I'll tell you if the answer was good or not. And so, it can get, a lot more creativity and freedom because, you're giving it that freedom, right? To do whatever it wants in between. And so it can learn really creative things that were potentially things that weren't even in the training data. It can learn to use those techniques effectively. And so it ends up just giving you much, much better performance, than you could get by just pure imitation learning. especially since there isn't probably like a lot of really, really high quality data imitate on your specific task in, in most cases.

I think specific task is the name of the game here as far as I saw.

And, we saw the RL and thinking models get significant improvements on coding, which is verifiable. you can run the code, see if it works, verify, and then give this result back. And also math, we saw. the competition math sky high, like models are now solving different things, because it's verifiable and they've been trained to do those verifications as well on general task. It doesn't seem like, that's a big step function improvement scale still works there, like you said. alright, Kyle, lightning round. I want to ask, a few more questions. when should folks think about Ara fine tuning versus just prompt engineering for example, because both can make my model or agent do what I want. Do you have a gut sense to tell our folks about when to use it,

So I always say take prompt engineering as far as you can first.

it's worth spending significant effort on that and getting to the best performance you can. Make sure you're giving your model the right tools. Make sure you're giving it the right instructions. do as much as you can there. If you've done as much as you can and you're sort of hitting a limit and it's like, hey, my agent is doing what I want 80% of the time and then there's like these 20% of the cases where it's not, that's where I think it may sense to bring in RL and you can get that last 20% of the performance.

next question is currently in October 16th, 2025, which is the

best open model to use for fine tuning?

Yeah, I mean I think the one that there's the most

kind of ecosystem familiarity with is the Qwen three, family. That's where it's sort of like the kind of standard in kind of the academic and just like. Yeah. Kind of like, RL ecosystem.

Yeah.

Okay. And last question that will take us to the next, segment and probably the last segment of what we're here to announce. if I am new to RL or newish and I don't have infrastructure, is there an offering for open pipe? Wow. A great question. So what is the, now we did, we talked about service RL last time, but like I would love to hear from you, what did we launch last week? What did you guys launch last week? it took us, around the month from the point that you integrated into CoreWeave to actually launch this, talk to us and walk us through server rl, why we built it, what is the thoughts behind this and what it actually does for folks.

Yeah, absolutely.

So I'm very happy to talk about that. one thing I wanna say is, like at Open Pipe, we don't wanna launch something just because we can, like we wanna launch something. 'cause it's actually solving a real problem. I feel very deeply about that. And so we've been doing RL for like the last year. I'm excited about serverless rl, which I'll talk about because for me as a researcher, as just an individual trying to use this, this makes my life so much easier and I'm hopeful that that'll be the case, for other folks as well. Basically what it is, is it's a way, so when you're running rl you don't have to manage any of the backend, any of the infrastructure, any of the servers, GPUs. So you write your, it looks like Alex has got our blog post announcing it up here. I won't go into the technical details 'cause people are at different levels. You can read the post, but, the upshot is, you end up with faster runs, you end up with cheaper runs. You end up not having to provision GPUs. this graph, I think is probably the coolest thing in my mind where when you're running, 'cause when you're doing these jobs, it's not like, you get it all set up and you run at once and it works. I mean, there's always gonna be unfortunately iteration. And so that cycle time is super important to getting good results. traditionally if you're managing your own GPUs and starting everything up that way, you might take like a couple of minutes every time you start a run to get the weights of memory and everything, and then you notice something's wrong, you change it. With Siro rl, since all that's happening on a remote computer, you are just editing your very lightweight Python file that's defining, your reward function, your tools, things like that. And so it just takes a couple seconds to start up and you can be off to the races.

the thing that I didn't understand would love to, for you to

explain to me just a little bit is that RL has basically two steps in there. Like you talked about this a little bit. There's the concept of rollouts and there's concept of like the training stuff as well. so you need kind of both the training loop and the influence loop. Could you just like, speak just a little bit about how they intertwine? Like what, what's the process there?

Fundamentally the way RL works is, let's say you're

training an agent, like let's say you're training a deep research agent. So what you're gonna do is you're going to have your deep research agent go off and research a hundred different questions or something like that, It comes back to the final answer. And then you're gonna take all those final answers and you're gonna have to grade them some way and say, okay, for each of these answers was, was this good or bad? Or kind of somewhere in the middle. And then once you've done that, you've got that batch together of answers from your model and their grades. Then you're going to use that to train and update your model and make it so your model, is slightly more likely to produce the answers that were good and slightly less likely to produce the answers that were bad. That's at an extremely high level, kind of like how this loop works. So you can see there's kind of two phases here. There's the part where you're actually using your model to, in this case, run all those deep research tasks. and then there's the second part where you're training, Those, trajectories we call them from your model, like the, agent run and the scores we gave it, or rewards, which is what we call it to update the model weights. so you're kind of bouncing back and forth between these two phases all the time as you're doing an RL run.

So you have to have them very reliable as well.

for many standard RL practices on one machine, you have to remove the way, upload them for inference and then run the training loop and then do some stuff.

You're using like VLM or SGL or something like that.

And then for training you're using MEGATRON or FSDP, but they're, they're kinda like different tools. And so yes, you end up having to kind of like, all right, we're gonna turn off our inference stack and remove the weights there and then turn it on for training. or you can be doing it with different sets of GPUs, but then you generally get pretty low utilization 'cause it's hard to have both of them. it's very messy.

would love to hear from you about that thing that allows

multiple folks running their own custom things on kind of the same base model. Absolutely.

So it's not, entirely new.

this is something that has existed, in BLM and other inference service for a while. But basically it lets you, it's based on this paper called PICA from late 2023, that did some really excellent kind of cuda kernel engineering. But basically what it lets you do is if you have a bunch of different Loras and loras are just, it's, it stands for low rank adapter. It's kind of like, a way of customizing a model without changing all the parameters, just changing like a very small, defined subset of it. if you train your models in this way, you can actually at inference time, still batch together inference requests from different requests that are using different Lauras and it can all be run efficiently in the same batch, as if they were all running against the same model. So that's how you get the inference wins. the big criticism or, complaint people have about Lauras traditionally is that, since you're only adjusting a very small number of weights, you may not get the same performance from your model or the ability to, push it as far as you could if you were adjusting all the weights in the model. there was a really excellent blog post, which maybe we can put in the show notes later from thinking machines, about a month ago at this point, where they did some. there was actually a really good one, a couple years ago as well. the thinking machines post about Loras, they did a lot of ablations. they spent, thousands of GPU hours running all these different things and basically showed, when you're doing RL at least, there's zero discernible difference in training all the parameters versus train alo. This is something we've been doing for a long time, frankly, at Open Pipe and that's been our experience as well. but I think that that just gives us like a lot more ammunition, to, when people have this objection to say, well, you know, there

yeah, we've talked about this, blog post a little

bit here, WNB infants did not have lo Laura of van tuning. Now you guys came in and hopefully we'll also release this separately. but at least for now, you guys can use the whole stack of, CoreWeave powered, inference service with ways and biases. and you can also use those traces and rollouts to see them in Weave the tracing and evaluation software that I talk to you about all the time on the stage to train with open pipe. So I think that the highlight here is, first of all, me and Kyle are coworkers. the speed with which you guys implemented this once you came into Core was incredible. Like less than a month, I think, till the final rollout while onboarding, while doing so, like tons of stuff. So shout out to the whole open by team, is quite incredible. And we are now generally at CoreWeave, are offering a, a significant. Kinda like full coverage of, use cases with AI that are now possible. Some of them are powered by the core GPUs, some of them are just offerings from us. WB models is still used for training some of the stuff. So like in the, our inference, you would see this in the WNB like interface standard. And the rollouts will be tracked with Weave, which is specifically built for seeing exactly like, how agent behaves in multiple steps, et cetera, and tracking like et cetera. And then, the loop is powered by core of GPUs. I think the last thing that we didn't mention, we definitely should mention, art and ruler Kyle, before we drop off. how's art, which is your open source, library that you guys released, and ruler, plug into all of this power.

so art, you can just search open pipe art, it'll come up.

yeah, it's very popular library for running reinforcement learning. it does work with the serverless background. I was just talking, or the serverless RL stack I was just talking about. it also works on your own GPUs. It's fully open source. You can just use it for your own RL in either environment. it just makes RL way easier to use, right? it's, this stuff is all finicky. There's a lot to get right and we have a bunch of heuristics that make it so it's easier for practitioners, which is really our North Star. Like, we wanna make this stuff really easy. it isn't that easy right now, and we're solving for that problem. and then Alex also mentioned ruler, which is something that's very cool. so Ruler is, basically like an automated LMS judge. we do a bunch of work to make LMS judge work really, really well for RL and in practice. this works phenomenally well, surprisingly well, where, because a big part before we released this, a very hard part in, doing RL was getting that reward function, which I've sort of glossed over up to this point. But actually it can be hard, right? If you've got a bunch of, traces from your agent to say, Hey, which of these are good and bad? ruler is a way of solving that problem in, a very generalized automated way. We've tried it on a bunch of different tasks internally and on customer tasks, it works really well. so that's why we released it, and that's fully open source. if you just search, open Pipe ruler, you'll see that as well.

So as folks heard, we're bringing you pretty much all of the pieces

of the stack that you need together in one service, and you definitely can try this. One last shout out to the team is that this week we also released a new model, Qwen three, 14 B from Open Pipe. this is a Finetune for Qwen three that does not do the thinking, which is helpful for AEL and fine tuning purposes specifically. so not only does the open pipe team release products, they also now release models on an infra server. So definitely folks can check this out. Kyle, thank you so much for coming. I'm very, very excited about, the offering that we now have. I had a chance during our hackathon to actually see people interact with this and give a bunch of feedback. Folks definitely should. go check out service. Ro, we have a great notebook for you, to run this, and then give us feedback. Send it to me, send it to Kyle. We'll definitely, be listening to tomorrow. ARO is very important. we have to move on. Kyle, any last shout outs to the team or anything that you want to say while, ending the Yeah, this

has been great and yeah, it's been really fun working together.

All righty, Kyle, thank you so much.

Always welcome by the way to the show as well. folks, Kyle, if you don't know one of the top folks in the real, in the, you know, the talks about it openly, definitely go check out his talk, from before and then we'll bring you more RL news from CoreWeave Thank you Kyle. Cheers, folks. As this interview ends, we have a new person here on the roster. is this, and it's Queen Slack from amp. you may know m from, from another person that's not with us. obviously Ryan, our co-host, often co-host is, recently started working in AMP and like hyping amp up. Qwen, welcome to the show. hey, thanks for having me and, welcome. the thing that, we talked about like folks here know about AMP specifically. we would love to hear about your new release, like it made quite a lot of waves at least on my feeds. Could you briefly give us a summary of what is going on with amp recently?

Yeah, we made AMP and you know, for the last six months it's on

the frontier of being the very best, using the best models, unconstrained. And we learned so much. We were able to build something and it's awesome. And we saw all these new models coming out. so many new models, there's really great cheap tokens. So we made amp free, there's a new free mode and the, the other one is the smart mode. Yeah, people love it. you know, I think I'm probably gonna use the, the smart mode like 80% of the time myself, and that's okay. Different people have different mixes, but this new free mode means that magenta coding is now accessible to everyone. And we can all get to explore what you can do when you're not feeling like that something is ticking, you know, your wallet down in the background. And the way we did it is with ads, the oldest business model on the internet. So, crazy idea, but it's been received a lot better than we thought, frankly.

Ryan reached out to me like, Hey, something big is coming.

And then when you guys released it yesterday and you hopped on livestream, Ryan was like, Hey, this is the, and I looked at it, I was like, HADS, really? And then I had time to think about this and I'm like. I'm looking at my, like, AI bill generally from OpenAI is $200, two cursor, two, like all of these things. And this stacks up and like, I'm pretty sure that folks, wanna get into this are not there. And you know what? I don't mind watching ads, especially if they're contextual. Like, I recently get to a realization that like Instagram ads are net value add to me. Like, sometimes they are targeted so much, they're like, oh, I actually didn't know this thing exists and I do need to buy this. And then I looked at some of your ads and they're like, AI engineering ads, like Base 10 and Axiom and like, whatever. So I did, like, I had the whole thing where like EO ads, like, oh, it's actually not a bad idea from the beginning of the day, the, the end of the day.

Yeah.

I think everyone's going through that. And there's so many people that they're paying for so many subscriptions now and they're not sure if they fill it out. Or if one subscription is good, then they hit the max and people are managing that. So, you know, part of it is, I'm so excited about making it so the people that cannot afford this, now I have access to agent coating, but also it even if you can. It's so much easier if something is free. Like with Google, Google is free and it just, it's the default, it's achieved that kind of status. And it turns out with ai, if you fast forward a few years, all the data centers that people are building out all this scale, that is what is going to drive down tokens. And everyone is betting on tokens eventually becoming too cheap to meter. So we're starting to build the product that can kind of assume that, which is cool. But, you know, it's definitely everyone is looking toward a future, you know, a few years from now. And the numbers do not work out right now, but we've seen this incredible surge of advertisers that are interested. 'cause we're the first ones to do an ad supported model like this, which is kind of crazy. Kind of funny that it arrives in the CLI in a agent coding tool first, right?

last thing that I wanna ask you Qwen is like, how's

the performance on the free tier? What models are getting into there? what people can expect? 'cause I know for a fact we're here on Thursday, I we're reporting on the latest and various models. We talked about hiku 4.5, et cetera. what models will people like be able to expect, you know, on the free tier of, of amp,

it's gonna be a rotating mix.

on the other side we're going to all the model providers and saying who's got excess capacity, who's got a new model in training? Or who can just, give us a discount because they wanna be a part of this. Maybe they want to show their ads on it. And so the whole other side of this business is us going around and hustling for tokens. So we're not gonna switch which model is being used in thread, like while you're coding a specific task as much as we can. But one day, it might be, XAI cra code fast, which is an amazing model. the next day, it might be Kimmy K two on gr. it's gonna switch a lot so you can expect. to try a lot. And in some cases we actually will not be able to show you what model is being used.

Yeah.

The variability is, is very interesting. Qwen, thank you so much for hopping on and giving us a little bit of the energy that led to this like crazy potential release. Alright, Gwen, thank you so much for joining us, folks. We're almost time, inviting back folks on stage here to chime to basically went back to back interviews, reaction time for some of the interviews. we would love to hear about AEL as well. Also the, the free code. Let's just have like a little free, free conversation while I shut up a little bit.

I mean, they all offer the free one.

Anthropic doesn't, but Coin and Google do with their tools. So yeah, that's a pretty interesting, way for a company to go. I'm pretty sure for some of the open source res I wouldn't really care that info goes out, but, it does make me a bit nerVEOus that there is a chance, I might make a mistake and then just push the whole code base towards ads. it will be interesting to see how that works.

I wanna see,

I'm more interested like what Open Models AMP is using

because, I thought it was just sonnet. That's what I wanna know.

Did I understand correctly that they

can switch it at any time?

Yeah, it looks like every request is getting switched.

I use amp by the way.

whenever I finally do need something done on the code base, you just like foot the bill with that and get amped to finally fix it.

Look, first about skills.

from what I see, I must say that, the implementation is quite weird, there are folders that hold the skill, inside the folders there are MD files that, from what I see, they're like conditions, conditional statements. Like, if you need to do this, follow this MD file. Like you need to extract text from a PDF. Here is another MD file that will tell you how to do that. Or if you need to, create a PDF from text. I'm just saying, let's say that PDF something, then you have another MD file that explains how to do that. And also you have some data sources, which is kind of weird some pre-made Python code that the agent can use. Python. I get the idea, I get the idea because you learn a skill and you want to not re-implement the skill.

Yeah, From my point of view, because I've been creating skills

while before when I have something like how to take a screenshot cloud code on my system, knew how to take screenshots using command line tools, how to save them, where to put them, and all stuff. So I put that in the Cloud MD file, and, I, the more skills I call them that now, the more skills I added, the bigger the file gut, and I don't need to have all that in context all the time. So taking them out like Subin and putting them as modules in a different file makes sense. And I wouldn't want to write a MCP server just for such basic functions. So I think it makes sense.

yeah, I think that if you can generate skills automatically, like take

a log or a chat or, or I don't know, CLI log of you running just casually doing your thing and then extract, reproducible skills from that automatically. Like with cloud code, some sort of, I don't know, maybe a mode of cloud code.

they have a skill builder thing that's what I used.

but it's only in the ui. I think they have exactly that. They have a skill builder skill.

All righty, folks, breaking news from cognition.

and we have Swyx here who recently joined Cognition. he is on the blog of this release of Swyx Rep. what'd you guys release? What is this about? we showed the demo, but like, we would love to hear more.

Oh, did you do the playground?

We did the playground,

yeah, I I'm curious what you think.

sorry, I should have given you guys a heads up. we only decided that today was the launch date, like 12 hours ago. so I guess it was a big, big, big sprint. so we basically we're launching. two models. We're announcing two models and launching it as a feature inside of Windsurf You can't really use the models as an API, mostly because it's very, very tuned, in terms of the two calls to windsurf, right now. we hope to generalize it in future, but, that's future work. so the two models are SWE grep and SWE grep mini, they're basically like, RL tuned models for, rapid agent searches which is what we call it. and so we basically, we have here the, cognition code, search eval where, there's like internal benchmarks of, code retrieval, and basically showing like you have the frontier models like on 4.5 or GT five or, what have you. and basically we would be able to match those performance numbers, but do it, something like three to four times faster. the idea is that in sort of like an ID situation, you kind of need to keep people in flow. speed is actually very, very important. And so like trainings fast, small models is very good. I think there's the number one question that I had, and like most people have when, when seeing this thing is like, okay, what are the sources of speed? one is yes, it's a smaller model, two. it is hosted on cerebral, I think. that is, you know, if you see the first image with the tokens per second thing, the C GRA mini goes up to 2,800 tokens per second, which is, pretty crazy.

Oh.

the last one actually I think is the most interesting, which

is, natively parallel to calling. So, the RL tuning was, was trained for multi turn and, natively parallel, up to eight, to parallel two calls at once. I think, like, this is like the core insight that made me go, aha. Like if you look in the blog post, I have a skeleton draw chart. the shitty drawn charts are mine, and then the nice charts are everyone else Yeah, that one. Yeah. So this is you. Yeah. Average parallel. average parallelism of most models is like one, 1.2, 1.5, right? we observed that like the newer models like sonnet can do a lot more parallelism. but so you grab starts at like six and it goes up to eight. the more parallelism you have, the more you can like sort of mass search a code base. And actually it's very, very good custom model for a sub agent. And I think like that's something that, we're pretty excited about in terms of exploring as an era research, cognition has been famous for saying don't build multi-agent. but subagent as a tool, seems relatively underrated. So I'll stop there. But, that's the main framing.

That's very exciting.

We saw the playground, we compared it. one thing that I'm trying to figure out is, because the comparison is between like solid 4.5 and then Swyx grab. We got mini plus sound book 4.5. So, how is it just the speed up is just in the context building stage and then the token stream from sound 4.5? Correct.

You can choose whatever main model you want.

there's one piece of messaging that we had in the blog, but we actually dropped it 'cause it was getting too long. It's basically like, you know, if, what is the fundamental equation of coding agents, right? First you have to find the right files, you have to read the right files, and then you have to edit, generate the edits for the, right files. That's it. Read and write, right? this solution is focusing on reading. we found in most Asian queries across DeVEOn and Windsurf is about more than 60 to 70% of, the complex queries that we find. and so what if we could speed that up by three to five x? Like, that's a no-brainer. and it is, it's like, reading is a very fundamental thing. if you can make it cheap to read, you can actually get human input because humans stick around faster. So it is focusing on code-based understanding. there's about two more launches that we have, that are following up from Swyx grab, that we're doing. the next one I'm pretty excited about. It's already being teased. It's called Code Maps. you can try it, you can find it in winds if you nice look for it. but yeah, this is only focused on agent retrieval of code bases, for coding agents. And the next one is, for humans as well.

is this a result of, windorf joining, cognition and

kind of the power up of two teams? Would you say that like some of this research is like cognition coded versus like wind surf coded, or is this something that Windsor had on this roadmap before and like could have delivered before?

no.

I think this is net new. yeah, I would say, this is where part of the thesis or the due diligence of commission and buying Windsurf was that they were like, okay, what can we do if we took over this code base, so this is the first of what is being planned. and we'll see more, you know, like the tab models being worked on, code maps is being worked on and, more models coming.

We just had, queen Slack from AMP on here talking

about their free offering. Oh yeah. Are ads coming to Windsurf at any time?

No.

plans for ads. windsurf has a good tier. I think ads is actually very useful. I think it's one of the best, most interesting ideas in coding agents I've seen in a while. honestly, I think you can even get more fun and ridiculous by like, injecting, like, here's one of my faVEOrite UI library of the day. Like, sponsored by whatever brought to you by, RAD ui.

Yeah.

which, just to call it specifically something they don't do. They, they don't, the ads currently in they just show up in the popup. Yeah. They just show up. They don't interfere with like the, the, the responses.

Yeah.

But you know, it can, it can be fun. Like do it for Halloween. Anyway, I think it's a very good direction. Right. But it is still, not a huge business in the enterprise because, they're not gonna want ads at work. and I think that's where cognition is still more focused, in terms of enterprise stuff. I think the goal is the analysis that I wrote about when I joined cognition was a lot of coding agents started $20 a month, and then they have margin issues and then they have to rug pull or they have to fuck with the context or whatever. Devin started at 500 and then goes up from there. there is a self-serve thing, but honestly, the real thing that has PMF is enterprise, and you don't mess with ads when you do that, you just build a very, very good enterprise business that's positive margin. then you end up like, winning there. So I think the thing about ads is it has to be very, very B2C, in the scale of Google to work. And I don't know how, I don't know how like the, the sort of B two developer side will, will work, but I mean, I think it's a good experiment. I'm excited to see them do this.

Yeah.

I changed my tune, on the ads. in the beginning I was like, EW ads. And then I was like, oh,

it's actually not a bad idea.

It's fine. They have taste, you know, queen And they have taste.

Yeah.

Yeah, taste and also ads. Like I definitely put a Weave out in there. Like, it makes like perfectly target sense. I mean, so

like, there's a spectrum of like, I think all the coding agent companies,

there are a lot, you know, a IE code we're gonna feature like all of them. I think the free is not the valuable. Property as far, sorry,

AI code.

What was that? What

is that?

Uh, uh, so my next, my next, uh, the next AI engineer is in New York. Again, focused on coding agents for the first time. obviously I'm interested, but I think this year has been a year of coding agents for sure. can you believe that Claude code launched this year?

I still can't believe that reasoning as a concept came a

year and a month ago in, one preview. so yes, we're moving very fast. Yeah, so Ai e code in New York, folks definitely check out. yeah, tickets.

I wasn't technically gonna plug that, but anyway, like you can

own terminal bench, you can own like free, I think where cognition is, is moving towards is, as I put at the end of the blog post is fast. and I think that's at a very interesting, dimension that hasn't been explored.

Yep.

Sounds great. Well, Swyx, now that you're in cognition, but even before, feel free to come up and, talk to us about some of your experiences and also the learnings that you have in these labs. a code is coming in November. We're actually a proud sponsor, so folks will see us there as well. definitely worth coming to. thank you so much for jumping on last second, for getting some breaking news and give us, the update yourself folks as we, close out on, ThursdAI for October 16th. I don't remember. An episode where we had four different interviews. I think four is our like maximum, and two of them are like impromptu. we had Swyx here, we had, Qwen Slack from amp. we, we had, Kyle Corbett from Open Pipe and at the beginning we talked with Jessica Gallagher, senior product manager for Google DeepMind. This is achieving TVPN level interview stacks, in one stream. So that's great. and we are all looking forward to getting the third version of the Big G at some point. with this, thank you all so much for joining. there's a lot that has happened this week. If you missed any part of the show, ThursdAIs always getting turned into a newsletter and a podcast at the end of this everywhere where you get your podcast. we're, we're available. The amount of stuff we have from week to week is incredible as well. so definitely feel free to subscribe and share with your friends as well. And with that, We'll see you next week, folks. Bye-bye everyone. Cheers.