Meta went frontier. Then Meta went open.Live from
New York.Meta Muse Wonder & 816B Muse Spark Open, Gemini 4 Argon, State of AI 2026 — ThursdAI Oct 15, 2026

0:00Alex VolkovAI engineers are sleeping on the personal agents renaissance explosions happening in the real world. It's kind of crazy.

0:06Nathan BenaichAmericans say that they're much, that they much prefer living near a nuclear power plant than a data center, which is, uh, quite wild.

0:14Alex VolkovAnthropic is also going to do an explosive IPO. they reported an insane quarter, uh, 16.8 billion dollars, about 45% from the second quarter increase.

0:26Omar Sansevierowe've been doing optimizations for Gemma. We have been able to achieve over 100,000 tokens per second with the E2B, so our smallest model.

0:34Alex VolkovAndrej Karpathy says 6 billion people have barely touched an LLM.

0:39Peter GostevI'm very pro rumor, but, you know, uh, the- I'm also a psycho, so, you know, you don't have to be.

0:45Alex VolkovWelcome to ThursdAI for October 15. Uh, I'm live from New York. Uh, my name is Alex Volkov, I'm an AI evangelist with CoreWeave, and with me Wolfram Ravenwolf, also AI evangelist, also from CoreWeave, and Peter Gostev, evals and model behavior at Arena. And I just spent 3 days at AI Engineer here in New York. I can't wait to tell you guys all about this. And while I was there, there was an incredible week just blowing up, so, so we had to try and catch up. Uh, Meta shipped Muse Wonder, which was rumored by them as as this big watermelon model, and it smells big. It definitely smells like a big model, and you can use it right now in in Muse, and

1:26Alex Volkovwe're gonna talk about this. Uh, it's number 2 on Arena, which is puts Meta Square in the Frontier Lab race. As we told you before multiple times, do not, under any circumstances, do not bet against Zach. We told you. We told you this 2 years ago. We told you this after Llama 4 dropped, it was a disaster, and we told you this, uh, when he started hiring people with soup. And so now we're gonna feel it. Peter, I think you you tested it out. I think you have some thoughts. Uh, would love to hear

1:56Peter GostevYeah.

1:56Alex Volkovfrom you.

1:57Peter GostevYeah, maybe to, to differentiate, we do have few different leaderboards, and they, they are second on the text arena,

2:04Alex VolkovMm-hmm.

2:04Peter Gostevbut they're actually 12th on the agent arena.

2:08Alex VolkovInteresting.

2:10Peter GostevYeah, and it, it's kind of interesting to interpret. I think we do have this kind of divergence now going on. So, for example, Argon, uh, from Google as well did really well on text. So it's actually first. Um, so you can roughly interpret it as, how much people like to chat to the model. So it's interesting that there is a divergence between what people want, who people want to talk to versus agentic capabilities and coding and all of that, and it looks like it's different. So yeah, Argon is fourth on Agent Arena, and Muse won the max, so it's not the full version.

2:44Alex VolkovMm-hmm.

2:44Peter GostevIt's on, it's ranked 12th.

2:46Alex VolkovUh, very interesting. We, we also have to talk about the whole thing where I saw the timeline, react negatively because, Muse Wonder wasn't able to recreate the 3D scenes in, and people were like, oh no, this is a bad model because it, it can't compete with Astra on like the 3GS thing. Uh, we have to address this because I don't know if my assistant every day needs 3GS, so I don't know if this is like a personal thing that I need my assistant to do. Okay, uh, what else? Okay, so folks, la- later today, Omar Sanseviero from Google DeepMind joins us to talk about it, about Argon specifically. We also have, thanks to Peter, we have, Nathan Benaich from the State of AI Report

3:26Alex Volkovjoining us in an hour to talk about the state of AI. Uh, they've been running this for a decade, Peter? Is is that, is it's like a crazy thing. I'm really excited to hear from him.

3:34Peter GostevYeah. Yeah, no, he's a, he's a great guy. Yeah, he's uh, he's been doing this for ages. Yeah, I actually haven't checked how long, but yeah, I've been reading it for many years.

3:42Alex VolkovI think it's, I think it's a decade, but Slopos prepared the news while I was running at the engineer, so hopefully Slopos did a good job. Uh, good job, Slopos. Um, and then what else? Oh, Anthropic had an investor day. So before we only had rumors and leaks, whatever. Anthropic is about to go IPO. We have some numbers and a specific thing for kind of the only sponsor for this podcast, CoreWeave, with our Agent Lens product. Uh, Anthropic Claude code specifically now supports open telemetry. So you literally can start running Claude code and then see your results in, um, in Forge Agent Lens, which is super cool. I think I've seen some examples already, uh, so we're definitely going to talk

4:23Alex Volkovabout this at least a little bit. And then, uh, so after Nathan Benaich, we'll talk about state of the art, but I think since I was a little bit of out of the loop, I do have a whole prep thing here. Would love to hear from you guys about the highlight of the week. I think we'll start with Peter.

4:42Peter GostevWell, my highlight was, uh, I was at the NVIDIA event yesterday.

4:46Alex VolkovOh, you met Jensen, right?

4:48Peter GostevYeah, yeah, he's

4:49Alex VolkovTell us, tell us.

4:50Peter GostevHe was watching on the show. Yeah, they they had a kind of, it was very confusing afternoon because during the day, during the afternoon, they were building, they had this kind of builders event, so it was all like, people you meet at AI engineer conference kind of thing. So that was really fun.

5:09Alex VolkovYeah.

5:09Peter GostevUm, and yeah, met a few people from Twitter who I haven't met before. They had like, uh, it was in the office in London of a company that does like humanoid robots, uh, which I had no idea that it was central London, you know, so that was awesome. Um, yeah, they were kind of a, it was all about OpenClaw kind of thing, so they were kind of showing how they also launched the laptop, right, the, the one with the kind of Spark, um, or, or rather did it start shipping or something? I think they announced it a few months ago, right?

5:41Alex VolkovMhm.

5:41Peter GostevUm, so yeah, a few cool things like that. Yeah, and then in the evening it was very different. It was in a fancy, uh, kind of, uh, reception lounge, uh, of a, I know there's a weird converted church building in, in London, and there was

5:57Alex VolkovNice.

5:58Peter GostevAI minister and all of the fancy people from, I don't know, universities and industry and a very different, very different vibe.

6:08Alex VolkovThat's so cool, man. I- it's kind of like, besides the AI engineer London I, that we both went to and we recorded from, this is the first time that you're saying something about, like, somewhere else outside of the States that gets me FOMO. I'm like, oh, that sounds super cool. I wish I would have been there. Uh, that's your highlight, obviously. Uh, the, folks are welcome to check out Peter's timeline for this. Uh, you posted, uh, very well from there, um, and you can tell us more about this. And then, Wolfram, what's your highlight? What's your one highlight from this week?

6:36Wolfram RavenwolfThere's so much stuff, but if I have to pick one, just one that I'm currently most excited about, it's an open weights release, uh, of Fish Audio.

6:43Alex VolkovFish Audio.

6:44Wolfram RavenwolfFish Audio Drama 3. Yeah, that is currently, I mean, there's so much, I could na- name multiple things, but

6:51Alex VolkovSince we're going to cover, Gemini 4 Argon, and you are our Google guy with the Pixel phone and whatever, and you didn't choose, Argon, I think that I'm very interested to hear what you think about Argon, since this wasn't your highlight. Uh, I think it's

7:06Wolfram RavenwolfYeah, definitely. I can say something about this.

7:07Alex VolkovOkay, yeah, we'll definitely, we'll definitely get to that. Um, highlight of the week for me. Um, okay, this is a highlight and a hot take. I've spent the last 3 days among the top AI engineers, in New York, and also this AI engineer conference was adjacent to obviously Wall Street, so this was a finance and AI engineers together. So there's a lot of finance bros, a lot of, big trading companies, a lot of banks. The highlight for me is I realized something that I kind of Wolfram and I talked about on the show, Peter, maybe with you as well. Um, AI engineers are sleeping on the personal agents renaissance explosions

7:46Alex Volkovhappening in the real world. It's kind of crazy. Uh, so folks who are listening to ThursdAI, I know you guys are AI. Hopefully we do a good job to tell you about the stuff that's happening outside of our bubble as well. It is kind of crazy. It feels to me that because everybody's building their own harness, or they're so used to Claude Code doing stuff for them, they're trying to force this, behavior into tools that don't need to do these tools. Meanwhile, the world is changing for normies because of Grokbot and Muse and to some extent bots, or dots, to some extent dots. Uh, and and this is like the summary for me because I'm like very, personal assistant pilled or personal agent pilled, and I see that I

8:26Alex Volkovtalk to folks and they don't have one running around 24/7 doing stuff for them. this is my highlight of, this is not the AI news, but this is news to me, that even the engineers that we talk to every week for the past 3 years that love to stay at the forefront of what's happening are missing a huge, huge thing that's happening. So hopefully we're here to tell you about this huge new thing, and we're here to make sure that you guys don't miss it, because I think it is huge. Because most of the normies that I talk to did hear about some of these, personal agents. Wolfram, go ahead.

8:57Wolfram RavenwolfWhat really excited me this week, it's another thing. It's not an AI release or something, but it is something someone I very much respect said, because Linus Torvalds really said that not using AI to create code is like, it's just a tool, so if you don't use it and you are a smart person, you should be using it. He, he said if anyone is submitting, as an example, it's, it's as crazy to say you can't use AI for doing something. It's like saying you can't use a compiler to create code, because that is also a machine tool that you are using, and you are not using assembly, assembler to create the code. And that was an example, and it's great to see the mindset change.

9:36Wolfram RavenwolfI mean, he already, already had an open mindset, and he says, as long as you understand the code, I don't care how you create it. That is the important thing, and I think more people should be thinking like that. It doesn't matter what tool has been used, but did you, what you ever, whatever you created, do you understand it, and does it work? That is the important thing, not how you create it.

9:55Alex VolkovThe do you understand the part is interesting. we should definitely talk about this because I definitely slop. I turned my slop cannon towards some projects that, Peter, I think you you can resonate. I have no idea what's going on under the hood, as long as I see it and it works and it's dope. Uh, but I do agree, there's some stuff, especially in critical infrastructure, so the folks at OpenAI, uh, let me scratch that. The folks at AI Engineer from banks, for example, there's a lot of stuff that needs to be human readable. There's a lot of stuff that they need to understand everything. Somebody taught me the concept of toxic access, where

10:29Alex Volkovthe loan book, the master loan book record or whatever, something inside the bank that, all the money lives in for everybody. if there's people with access that can change some stuff, that's like toxic access or something like that. Uh, so that, obviously, people need to understand what's going on there and be accountable as well. let me run to TLDR, which is a section of ThursdAI where we tell you briefly about the most interesting, exciting parts of what happened, curated to our interests. I will, I think I'll stop saying that soon, but we used to be able to cover all the news. Uh, I no longer, with fair conscience, can say this is all everything

11:04Alex Volkovthat happened in the world of AI, but here is the most important thing that happened in the world of AI. Let's go. All right, here's everything on ThursdAI that we're covering on October 15th in about 5 minutes. Let's get into it. So we're live from New York after AI Engineer, uh, where I think the highlight was Capital One Max Kanat Alexander made the best case I've heard when code quality still matters. This is probably the MVP talk, and I would recommend folks to, to tune into the live show and wait for the release.

11:42Alex VolkovUm, swyx also came on stage, at his own conference and talked about how to kill your SaaS, uh, hackathon. There was a hackathon this weekend, and 72 clones of big SaaSes, spent 100 billion tokens over this weekend to build, clones. And then Anthropic shows dynamic workflows, as well. Let's see what else we have with the TLDR. Yeah, um, in big companies and APIs, Meta's first big and heavy and like frontier-ish model, or maybe frontier model. No, it's actually a frontier model. Meta Muse Wonder is number 2 on on Arena, and this is the best version,

12:23Alex Volkovuh, and it is now available in different tools. So it's available in MetaCode and Muse only for subscribers. Uh, some builders are split about this. Also, there's 2 versions there. There's Muse Wonder and Muse Wonder Full. Um... Did you see what I did there? Muse Wonder and Muse Wonder Full? That's, I think, what they did, because everybody calls their models Max on the Max reasoning. They call it Full, and it's wonderful, and it's pretty cool. Uh, Gemini 4 finally launched. Welcome, Gemini 4. Again, as a reminder, we'll have the folks from Gemini, Omar, Sanseviero, and Geneviève, joining us in about an hour to talk about Gemini 4 Argon.

13:01Alex VolkovIt's out for everyone. It's number one on Arena and visual evals, uh, and we can't wait to talk to these folks, and hear all about the use cases about Gemini. And then, uh, we talked about Anthropic opening, open telemetry traces and talking about their IPO. And Agent SDK still runs on your cloud subscription. Wolfram, hopefully you caught that. There was an update, and now there is an Agent SDK, and, um, if you guys remember the whole maybe Anthropic will block you from the subscription service if you use it for like Hermes Agent or whatever. Now we're back at the place where Agent SDK supposedly runs on your Claude

13:40Alex Volkovsubscription. Um, Anthropic also showed investors a 6- 16.8 billion dollar quarter, uh, ahead of its reported November IPO, and there's a rumor about Fable 5.5 showing up in Azure, but it's unconfirmed, just a rumor. Uh, folks from TypeSafe, this was just this morning, this is like nearly breaking news, like like I think it's in the breaking news category. Uh, Jev 1.14, which can now see images. So Image Jev, we heard that it's coming, and Image Jev is now here, so it's now in preview. Um, and I think that's wonderful. I really want to try it. I didn't have time running around, but like Image GeV is something I want for, I

14:21Alex Volkovwaited for a long time. Microsoft shipped its inference chip Maia 300 and its own decision model, Microsoft Decision 1. So decision models are taking over, and I'm very, very interested to see how, Decision 1 stacks up with ImageJev. Open source LLM. In open source LLM, 4 open decision models was released, Amos 1, from 0.5B to 27B. Uh, they also answer with bounding boxes, so it's not only Jev. Jev answers with bounding boxes, so do these models. Uh... haven't, I haven't had the time to test them.

15:00Alex VolkovMiniCPM returns with 4.7 and runs a 35 billion param vision model on iPhone in about 3 gigs. Uh, Liquid, shout out to our friends from Liquid AI. They released their biggest Liquid foundation model, 24 billion parameters, built for local agents, and we constantly talk about running agents locally. And then QwenGyre, RL framework for long horizon agents came out. Uh, I think it's a paper. In this week's buzz, we told you about Forge, CoreWeave Forge, the rebrand for our, agentic cloud offerings on SaaS. Forge model distillation is now in preview,

15:41Alex Volkovand, I think that's super cool. And also Agent Lens in preview, and Agent Lens is just hot this week because now you can connect Claude, uh, via OpenTelemetry and see the most, you know, the most useful and not useful traces from your Claude, uh, run directly in Agent Lens. I think that's super cool. apparently XAI dropped Grok Voice 3. Do you guys, here see this, speech-to- speech leaderboard? It tops the speech-to-speech leaderboards, and, apparently all of the Starlink support calls are now used by Grok Voice 3. And, uh, Wolfram, you also mentioned Fish Audio, correct? Drama voice.

16:18Wolfram RavenwolfCorrect.

16:19Alex VolkovJust summarize for us.

16:23Wolfram RavenwolfYeah, uh, it's a new, um, a model, a speech model that you can direct, where you can give instructions for how the voice should be sounding. I think this is best if we, when we talk about it, we we show some examples.

16:37Alex VolkovYeah, we maybe play some examples.

16:38Wolfram RavenwolfIt's open source as well, so you can run it locally, and uh, it's also at the top of what is possible with voice right now.

16:46Alex VolkovOh, that's super cool. Um, all right. Let's go to industry. Industry is where we talk about stuff that are not releases, but I think is important from this week. Uh, Andrej Karpathy says 6 billion people have barely touched an LLM. Uh, he he did some back of the napkin math, and don't we love a good, like, Karpathy tweet that sets the industry ablaze, and it feels like he's been tweeting less since he joined Anthropic. Uh, so he literally had this, like, funnel of saying 6 billion people barely touched an LLM and AI, and the gap, he says, is getting wider, uh, and then he kind of does calculate. So hopefully, everybody who tunes into ThursdAI, folks, you're on

17:27Alex Volkovthe top of under Karpathy's funnel, but he does have this thing where, like, only a handful of people touch the models at the Frontier Labs. and last week, State of the AI, state of AI report came out. Nathan Benaich, the author, the the guys who's been running this for a while, is gonna join us, was at 10 AM Pacific, which is going to be in an hour, to talk about it. And I'm excited. Peter, you connected us to Nathan. Uh, I'm very, very excited. He's been running this State of AI report f- for a while now.

17:56Peter GostevAlex, I think, uh, did you mention Nemotron? I don't think so, right?

18:00Alex VolkovNo.

18:01Peter GostevThe, it just launched about half an hour ago.

18:03Alex VolkovOoh.

18:04Peter GostevAnd

18:04Alex VolkovNo, half an hour.

18:05Peter GostevNemotron, yeah, Nemotron 3.5 Super VL, 120, uh, billion total parameters and 12 billion active parameters.

18:16Alex VolkovNo, I do not.

18:17Peter GostevUm, it looks like they're focusing on, uh, more, it's multimodal, so the idea is that it's good at video understanding, image understanding, and long-running agents, and roughly where they're kind of pitching it at DeepSeek V4 flash, including small, that kind of level, but it's meant to be quite fast and, um, and let's say it's performant at, at that sort of level. So, yeah.

18:43Alex VolkovBut just a new release. We haven't tested it. It just literally came out. Yeah.

18:46Peter GostevYeah.

18:46Alex VolkovUm.

18:47Peter GostevPerfect.

18:47Alex VolkovSo shout out to our friends from f- f- from Nemotron, Team Green, and uh, and our friend Chris Alexiuk, who works on that team and, and always joins us and talks about it. So shout out to NVIDIA. Uh, there's also rumors that NVIDIA, did you guys see this? And NVIDIA, is in talks to buy, Reflection? I think it was Reflection. Well, the lab behind Beam that we talked about Beam last week. I'm pretty sure it's Reflection. This is like, there's rumors that Jensen's collecting the Infinity Stones of open source. So Hugging Face, Reflection, also rumored, also friends of the pod, the Israeli company rumored that NVIDIA is like talking about purchasing them as

19:22Alex Volkovwell. All right, I think it's time for us to go to Frontier Labs and talk about Meta Muse Wonder. Alrighty, folks, let's talk about... do not bet against Mark Zuckerberg, and now Alex Wang. Uh, so in addition to having, great meme taste, Meta is now back at the frontier. Meta's frontier model is number 2 on Arena and, just 12 points behind Argon, which we're going to talk about. Uh, but some builders are calling these benchmarks, and

20:03Alex VolkovI think, several things can be true. Meta Superintelligence Labs released Muse Wonder and Muse Wonder Full. Uh, Wonder Full is kind of the over-the- top model that we don't get in the API. It only exists in Meta subscription tiers. Alex Wang called it their largest and most capable model yet. Uh, I joked with somebody yesterday that all I need to say every week on the show is that this new model launch, numbers are bigger, and they claim it's their best yet. And it does, we're kind of there, right? We're kind of there where it's like the new iPhone 18 moment for every lab, and this is the best iPhone we've released yet. Uh, both Meta models, though they do smell like a big model to me.

20:44Alex VolkovSo both have 1 million token. Um, Muse Wonder via API, it's 4 bucks in and 20 dollars out per million tokens, basically Opus 5.5 pricing. But if you see the evals charts, they actually show that they beat Opus on multiple interesting benchmarks, and those benchmarks are not your standard benchmarks, besides the SWE-bench one. I'm pretty sure that those benchmarks are, a little bit different. Um, Muse Wonder, in parentheses, full, is the better model. That's the one that kind of, they say, beat Opus and beat Astra and beat Fable, only for Muse code subscribers. And uh, 50

21:26Alex Volkovor 200 dollars a month. This is the tier that you need to pay, either 50 dollars a month or 200 dollars a month to get it on Muse Code. No open weights for it either, but here's the news that excites us. Uh, Muse Spark, the model that was there before Muse Wonder, they told us about open source, and Muse- Mu- Muse Spark is basically the new Llama. They're going to, not yet, but they're going to open source Muse Spark. Uh, I hope this is the latest version, 1.3, but I'm very, very excited. Uh, Meta says, the Meta Wonder models are too big and maybe even too risky for open source. That's a, that's an interesting call out. Um,

22:04Alex Volkovright, let's, let's just dive in there. Uh, Peter, I know you guys tested Muse Wonder. Um, would love to hear from you.

22:10Peter GostevYeah, so you, you're right to say that it was second on the text arena, which means that people like talking to it. It is, I think, 12th on the agent arena, and agent arena is when all people like you and I go on arena and use it in various kinds of ways, which could include coding, could include white collar work. So that doesn't, that's not amazing, I must say, 12 is not, it's not that great. Um, wi- which is kind of interesting, like people say, or is it benchmarks or, or like Alex Wang is, uh, fighting back and saying, oh, you know, who needs, uh, 3D and all of that.

22:50Peter GostevBut

22:50Alex VolkovYeah.

22:50Peter Gostevyou know, 3- 3D aside, I have tried also building websites with it, and I must say it was kind of awful, that websites.

22:57Alex VolkovMm.

22:58Peter GostevI, I don't know, maybe it was my experience, maybe I did something wrong, and I didn't spend, like, many hours with it.

23:04Alex VolkovYeah.

23:05Peter GostevBut I think that's kind of an important as well. I've never seen a- a model that's really bad, but it's good at 3D and it's good at websites, but it's fundamentally bad. Like, I- I think- I- I think it's kind of unreasonable to say, well, it- it's much as Opus, but actually it's really bad at half of the things that people use it for. So it- it's kind of, I I don't know, I don't want to be unfair to them. It's clearly a better model, but equally, I think it's still behind on some things, like front end for sure. 3D aside, front end is seems like really quite bad, actually.

23:38Alex VolkovSo

23:38Peter GostevThis is the frontier.

23:40Alex VolkovI will say, while 3D is not a use case that I expect my assistants to kind of use, and it's obvious that Meta is fine tuning Muse Wonder for the Muse Assistant, right? Like, it's very obvious to us why they have the model, and the way they gonna put this capability in people's hands is via their personal agent or AI assistant, whatever you want to call this category. And so they're optimizing f- f- for that thing. We, we should probably mention the two innovation type things that they're adding there that are not evaluated, which is, dynamic memory and dynamic context stuff. Um, however, uh, Peter, to your point, while asking my assistant to create 3D castles and 3D series or whatever Nisten

24:22Alex Volkovcreated and they showed us last week, a virtual copy of Toronto with 100,000 agents running, that's not what I'm gonna ask my assistant to do. Building websites is kind of what I think, many people would like their agents to help them with. Building websites, one-off websites for like a use case, or like more, more stuff as well. So I think that this is definitely in the realm of, something that people will ask their assistants to do. Wolfram, do you have any thoughts on, uh, Meta Wonder, Meta Muse Wonder?

24:49Wolfram RavenwolfOh, on one hand, uh, that is exactly this what I've seen less technical people who are not as much into AI. They use AI assistants to, uh, create websites or update their websites. So that is very important to be good in this regard. 3D, on the other hand, of course, it's a nice demo, but yeah, the practical use case for us most of the time is not that high. I tested Wonder, and it is excellent in German, which is not true with all the models, so

25:14Alex VolkovMhm.

25:15Wolfram RavenwolfIt has the colloquialisms and stuff, so this is great. Um, yeah, it's, it's a good model, and I think that they have their own agents, uh, that they are offering personal agents means that it can also be used very well in Hermes Agent and the other ones. It's available with the API.

25:30Alex VolkovYeah.

25:31Wolfram RavenwolfUm, yeah, I think it's great to have more choice here.

25:35Alex VolkovI think a very interesting thing is, I haven't seen the Muse code subscription being used in like Hermes Agent. I don't know if that's possible, but I think that'd be cool, because now, we're gonna get there, but the Claude SDK looks to be back up. Um.

25:49Wolfram RavenwolfWhat's always possible is to have your Hermes Agent just use another agent. I'm using all the agents like that, that Hermes on my own system is my main, and connecting to CrockPot and the other ones and just taking control of them all.

26:00Alex VolkovYeah. How did you set it up? Yam, you had a comment as well about Muse?

26:03Yam PelegYeah, that's uh, that's a good idea, actually. I never thought about, about trying it as well. How do you, if we have time, how do you, how do you set this? Like, what do you use it for?

26:14Wolfram RavenwolfOh, I've been using this for a long time, even before the agent stuff came up. I was calling it minions. Amy has her minions, and she uses computer use if there is no API or CLI tool. It's just a s- a skill, basically, a minion skill where she can use, we did it in the, in the, ah, when, when the first computer use stuff came out, she was going to the ChatGPT website and using that as a minion

26:37Yam PelegYeah.

26:37Wolfram Ravenwolffrom Claude and stuff like that, and it still works the same way. You just give it a tool, like it can use a CLI tool, or it can use a website or Crocbot, the app can be controlled with computer use, and it's just another agent for a sub-agent for the main agent. So very easy to set up.

26:54Yam PelegOkay. I do it all the time in the terminal, uh, Claude calling Codex, Codex calling Claude.

26:59Wolfram RavenwolfMm-hmm. Yeah.

27:00Yam PelegThe variety gives, gives you a lot for 100%. I'm j- I'm just wondering because these are, these are different. They're extremely powerful, connected to many different services. So just, I don't know, just, just the use case, like what you use it for. I mean,

27:14Alex VolkovI think, yeah.

27:14Yam PelegIt can be interesting. Okay,

27:15Wolfram RavenwolfJust one use case is a council like GPT Pro, which you can't just use with the API and so on. So we are using it that way. If we are creating a concept for something, it goes through all these. I have a whole council of 12, 13 agents that all, uh, work together to make sure that this is, uh, if it's very important, to be the best one we can make.

27:38Yam PelegCool.

27:40Alex VolkovWe definitely should talk about use cases, though. So we talk about models and numbers, et cetera, but I think, as we started to say in the beginning, we're approaching the level of capability where, like, for most of us, um, the stuff that we need to do, we can probably already do. So, Go ahead.

27:55Peter GostevMaybe one, one, one thing to add as well, I think, about the release, and I imagine they probably just don't have enough compute, but it so- sounds like at least from, like,

28:04Alex VolkovTheo, you know, the-

28:06Peter GostevI think they don't. Yeah, it's like the- yeah, I don't think so, because why, why is the full one not available via API? Like, I don't think they're just being stingy, right? Maybe they want to cross-sell the subscriptions, but that feels, I don't think that's working that well. So, for example, I think Theo, right, the the YouTuber, he, uh, he was very complimentary about the Muse model, which I was confused by, and then he came out and said, well, actually the the full one is great. The regular one that we got, he thought was bad.

28:38Alex VolkovMm.

28:38Peter GostevSo I I haven't tested the full one, so my impression is very like meh about the model, but maybe the full one is amazing, but we don't have access to it. So I, I, I wonder if there's also that kind of mismatch that they think, oh, their model is amazing, then the reception is like, oh, is it benchmarks or what? But actually, maybe it is amazing, but they, they mean a different model. It's not ideal.

29:01Alex VolkovThis is kind of on the same topic of, folks, let's say inside Anthropic, they're probably not using Opus, they're probably using like t- Opus plus plus, whatever model. And so when they talk about the stuff that works for them in the public, I'm not sure that they mean kind of the models that are available to us. That's a very interesting, very interesting point. And I think Theo also gets early access to like the models that are not available, in the API. Um, you can get Muse Wonder, not Wonderful, Muse Wonder within the product. So I gotta wonder if any of you guys used it within the product itself. Because I switched immediately, so

29:33Alex VolkovI, I've been using Muse, as you guys know, because I've talked about this a lot, and I will continue talking about this because I think Muse is wonderful, not the model. I, when I realized that I can only get access to this new intelligence, essentially what happens is, Wolfram, you know all about this, you've been doing this for Amy for a while. Essentially, I realized that I can upgrade my assistant's intelligence with a click. And that is incredible to me. That is just like, um, this is the scene from The Matrix where Neo gets a thing, and now he's like, boom, and I know kung fu. So skills is kind of like that, but also like upgraded intelligence is kind of like

30:09Alex Volkovthat. And so I upgraded, muse, uh, Wolfred, my muse, to Wonder, and I gave it a lot of tasks. Here's use cases time. Let's talk about use cases a little bit. Uh, the podcast studio that I'm in, shout out, Virgo and Ezra, for helping me set this up super quick. Muse found this out. Muse emailed Ezra a couple of times, including the the notes and the requirements I need for the podcast. On the way here, Muse got me coffee. I did a, meet up yesterday, liquid cooling IRL, AI engineers in sauna, and you guys can see this wristband, Muse designed and ordered it and it shipped to the hotel. And all of this was on Muse Wonder. I was just kept like shoving tasks to it, and it was executing them

30:49Alex Volkovmostly perfectly. The failure modes of this model, Peter, it's really hard to benchmark, so we need to talk about how we benchmark those like use cases, but the failure modes of this model were not the logic part. The failure modes of this model were the website doesn't work, the website is black and last because Cloudflare doesn't like agents, or, the Stripe link doesn't quite connect or something. So the failure were real world failures, including a very funny ones where, Muse called some businesses and businesses hung up because they don't want to talk to a personal assistant. It was really funny. So, I think, again, I think we're approaching a level of capability where most people

31:27Alex Volkovdo not care, um, and as long as it does stuff for them. And then the question is, does it do stuff for them? So I think when Meta released the models, they had some evals in there. I think we should pull them up that are not the standard ones. There are some, like, personal assistant ones. And Peter, the question back to you is, you are saying on the agent arena, Muse Wonder is kind of suffering. I gotta wonder if my experience with like the personal agent stuff versus the agent arena, where are those tasks falling? Where's the daylight between those tasks?

31:59Peter GostevYeah, it's, it's definitely hard to say, and I think, yeah, it, it could, the, the kind of failures that you're talking about, they do sound more like the harness in a, in a broader sense, like

32:12Alex VolkovMhm.

32:12Peter Gostevconnections, integrations, like things like that. So, yeah, it's definitely, it's definitely hard to say. One thing to say, though, about the, the benchmarks, and you mentioned the different ones, um, this is kind of a pattern, which is a funny one. I, I did a bit of analysis on this, maybe I should, I should publish it there, where I kind of pulled together the different benchmarks that labs were using over the last year, just all of the releases. And the half life of a benchmark that they quote is, is, uh, basically like one release, uh, where so many benchmarks just get quoted once 'cause they get a good number and then they're never seen again.

32:49Peter GostevSo I wonder it, uh, I'm not accusing them of, of this in this case. Maybe they're meaningful benchmarks, I, I don't know, but that's also something to look out for, is that quite often I would open a release and I'm looking at it, I don't recognize even the titles of half of the benchmarks.

33:04Alex VolkovYeah.

33:04Peter GostevAnd, you know, I- I'm in the industry, so I- I don't really know how, how reliable, how are they gonna stick to those benchmarks or- or not. But if they're inventing new ones or kind of citing new ones because it has a good score, I think that that's also not- not necessarily indicative of the performance.

33:22Alex VolkovUh, that's true, and I- I think, to some extent this makes sense because capability, let me, try to find the, the exact thing you're talking about, but I think the capability moves and different benchmarks need, need to switch as well, right? Like, we've we've seen that OpenAI decides not to do SWE-bench anymore because SWE-bench was like leaked in full, and then SWE-bench, Verified came out as SWE-bench Pro and like a bunch of like, updates to this paradigm. Um, but I definitely want to pull up some of them so we can talk about this meta... Wonder. Nisten, meanwhile, any comments on Muse Wonder from your end?

33:55Nisten TahirajNo, it's is Wonder a new thing? I'm lo- I I have Muse running in circles all night. I don't know what it does. It tests websites and stuff. It's pretty dumb sometimes, but it's just so energetic.

34:08Alex VolkovNisten, you don't know what Muse Wonder is?

34:10Nisten TahirajNo idea. Just

34:11Alex VolkovI love how, how co-hosts of ThursdAI sometimes joins as co-hosts with deep insights and knowledge and building things in the engineering, and sometimes they join as guests and just like chill here and like get their news from the live show. This is incredible. Muse Wonder is their new model. This is their Opus that beats Opus on multiple things. So I think, it's time for us to show some numbers. see.

34:29Nisten TahirajOh wow, holy cow. How does that compare to to Argon? Holy shoot.

34:32Alex VolkovYeah, so so this is the thing. They did not, they did not do any Argon comparisons. Uh, the Muse Wonder full and Muse Wonder on Max, are compared only to Claude Fable 5.1, Astra 6, and Claude Opus 5.5. And, I think, Peter, maybe this is the one that you called out. They have a new benchmark called Personal Superintelligence Index. Internal Composite Index of Personal and knowledge work. So basically doing kind of like a artificial analysis, ish thing, capability-ish thing. I'm not quite sure how that works out. And on that index, Muse Wonder, the one that we don't get access to, on Muse Wonder Full is 59, and Muse Wonder is 54,

35:11Alex Volkovbeating pretty much everybody besides Opus 5.5 Max. Um, which other ones here, Peter, would you say?

35:19Nisten TahirajThis is crazy. Where is, uh, like-

35:22Alex VolkovWait, hold on, Nisten, just one second. Peter, which ones of these would you say are one times or benchmark or, or new ones?

35:29Peter GostevWell, like, if you scroll down a little bit, so we've got GDP PDF, Job Bench.

35:36Alex VolkovYeah, GDP PDF I have never seen before. Job Bench I have never seen before. I think we're missing.

35:41Peter GostevMemory Bench.

35:42Alex VolkovMissing LDJ here, he knows a lot about some of these. Uh, I will say on the artificial analysis index, which I think a new one is coming, this model does not beat Astra and Fable on that specific index. But also that index, again, it's testing generality of models, and I think coding agents and not necessarily personal assistants. Um, OS World, for example, we know and love, right? OS World is a benchmark that measures computer use and and browser use, and on this benchmark, uh, let's see, Muse Wonder, the one that we do have access to, matches Fable and is significantly below Opus 5.5 on computer use and even below, Claude.

36:24Alex VolkovSo computer use, this model is not the best. Um, w- which, okay, so going back to what assistants need, computer is like the number one thing that I need for my assistant. It needs to go and click the right buttons, otherwise it's not useful to me. Um, a very interesting set of releases here. And uh, Nisten, you mentioned Argon. We're going to talk with folks from uh, Gemini about Argon in about an hour. We have Omer back and Geneviève, so we're gonna talk about that with them. I think we're good on, uh, uh, the news coverage or anything else left there? Wolfram, go ahead.

36:59Wolfram RavenwolfUh, yeah, since we were looking at the benchmarks, I, as I'm creating Wolfbench 2.0, I, I have a crisis moment, and I realized something. It's so hard to benchmark agents because they are complete systems. They are not just the model. You have the skills and memories and all this, and everybody using an agent has a different one, even if it's the same system, because of all the different personalizations, connectors, and so on, and even the internet access and the search providers, they all affect the results. And the thing is, nowadays we are at a, we are now at a point where you can have your agent create a benchmark for your own purposes and use the same system.

37:35Wolfram RavenwolfI'm doing it with Hermes, so I can put a Hermes in the sandbox with my, my soul and skill and all and so on, and use a model and do some tests specifically for the users I have, and I think everybody can now do this. Just have your agent build it. You can build all the software in the world now. You can also build an evaluation. I mean, as someone building evaluations professionally, that is a strange moment, but it is really here now where you can have a customized evaluation for your own purposes with the agents you are using and test new models, test new agents. All of this is now possible, which wasn't possible just

38:09Alex VolkovThere was.

38:10Wolfram RavenwolfIt was a lot of work now. Now it's possible.

38:13Alex VolkovThere was a talk at AIE that calls out this paradigm where you're like, you talk to your slop generator. You're like, hey, how can I know that you're right? And the slop generator says, well, I can do an eval for you. Uh, here's a few things to test myself on. And then you say, okay, do it yourself. And so the slop generator generates the evaluations for itself, and they're also slop. And where are we if nobody reads those traces? Uh, we're talk- we're gonna talk about reading traces in a bit, in the Claude section. Um, so this is Muse Wonder, and the call out there for our open source folks is that the Muse Spark model is going to get open source.

38:47Alex VolkovThis is a new Llama, and I think that's one, that's great. I need to start using the word wonderful because it's confusing, but I think that's great. Uh, right, let's talk about Anthropic. Anthropic is big in the news this week. Anthropic gave us 4 stories this week. You can trace your agent now, which is a very big one for, at least the sponsor of the show. Uh, you can now, or I guess again, we're so back, you can run the agent SDK on your subscription. So this means that you can plug it into Hermes. I see Wolfram shaking, he said, uh, uh, very strongly because, folks, I have told you before, so all of this personal agent type revolution,

39:25Alex Volkovwhat I, hold on, so what I mostly need from this is that personal agent ability with Opus 5.5 capability. And now it seems that while I was able to pay API prices for this before, now I could do this with my, uh, Claude subscription. So I'm definitely excited to try this out. I've seen this in multiple coding harnesses and agents. Uh, what else do we have? Uh, Anthropic also showed investors a 16.8 billion dollar quarter, and also rumored Fable 5.5, uh, in Azure configs, which is a very interesting thing because Opus 5.5 is kind of beating Fable 5.1 for most stuff.

40:05Alex VolkovThey're not like super, super, strong. So let's talk about some of the Anthropic stuff. How is this lab?

40:11Wolfram RavenwolfOne more.

40:12Alex VolkovYeah, one more.

40:13Wolfram RavenwolfOne more. There's even one more. They posted a, an article about strengthening their defenses against distillation attacks. So after clamoring for, ah, we have to regulate and ban this stuff, uh, now they basically did it by themselves, increasing their defense. They, they reduced the amount of distillation that happened to the tracing, reasoning traces and so on by 90 to 99%. So it will be much harder to distill Claude reasoning now, and we'll see what effect that has on other models.

40:46Alex VolkovI know the effect that it has on me. There's not a day goes by that my Claudes are disconnected, I need to re-log in again, and then the remote control thing fails, and so from New York I can control my Mac. I don't know if this is the effect of their security on the distillation, but it affects me personally. I don't want to call this out. Anthropic, please, please let me use my Claudes wherever I am. This is getting old. This is really getting old. All right, talk about this. Uh, let's talk about OpenTelemetry, which is kind of a nerdy subject, but it's an important subject. OpenTelemetry is this open standard for how, systems communicate their traces to

41:22Alex Volkoveach other, basically the logs. OpenTelemetry has been around for a long, long while, and then folks like, the We folks from Weights and Biases, and now the Agent Lens folks, which we're now launching in, in deploy preview. This is how systems communicate with those platforms. Um, and now Claude itself, the Claude harness is now supporting OpenTelemetry. What I'm saying is, where before you had to run something like a 1B Hive mind, which we told you about, that shows you all, all the Claude, conversations. Now Claude itself can send its traces to whatever tool of choice that you have. Obviously, on this pod, we strongly, we strongly ship, the CoreWeave Agent Lens,

42:00Alex Volkovproduct we just launched, but any anything that supports OpenTelemetry, Claude code now supports. I think it's incredible, and I think it's very interesting how what you just said, Wolfram, where they try to prevent distillation while also simultaneously allowing Claude, Claude to send his traces in uh OpenTelemetry. Though this was never a problem. Claude was always writing the conversations to local, right? So you always able to get it on disk. They were not, they're not encrypted. The reasoning traces are encrypted, but the conversations themselves aren't. Uh, but I think a, it's a cool tool. You can, send your traces to observability stacks.

42:36Alex VolkovUh, and this includes everything, agent turns and tool calls and MCP calls and latency and token usage, so you can, like, properly measure how much Claude is costing you. Uh, and if you're a company, you can do this at scale. Oh, there's one stat here from Ramp. They said this helped them to cut token use to 10 to 20% on some agents because they learned that some of their agents are running through loops. Will you guys be, are you guys following your agents in any way? Are you evaluating them based on this way? Will you be using this from Claude Code? Nisten and Yam both shook heads.

43:10Nisten TahirajDefinitely gonna try.

43:11Alex VolkovSomebody pick it up.

43:13Nisten TahirajYeah, this sounds good, especially for stopping loops and getting over all agent stuff. I- I have noticed that, uh, Claude Code does, does tend to stop often if you have anything to do with the reasoning, now it just like triggers it as a, as a, as a word. Like, if you're working with any type of LLM, like I've been working on WebGPU, the moment I even like mention anything about reasoning, now, now it just triggers it right away, uh, which is like for a small model, but yeah.

43:43Alex VolkovUm.

43:44Nisten TahirajStill good, though. It's, you can't beat Opus, so.

43:48Alex VolkovOpus inside the Claude Code harness, I think it's still the best tool for overall tasks and doing some stuff. Uh, Opus also helped prepare the show.

43:59Nisten TahirajJust Opus Low. If you need to do something, just try all in Opus Low, guys. It is just straight up vibing.

44:08Alex VolkovUm, so I think, and because of this reason, Anthropic is also going to do an explosive IPO. Um, they reported an insane quarter, uh, 16.8 billion dollars, about 45% from the second quarter increase. Adjusted profitable to the second quarter in a row and cash flow positive. Anthropic is profitable and cash flow positive and gets billions of dollars and going 45% from the previous quarter still. it's absolutely insane.

44:40Nisten TahirajI think they're- they have way better opinionated, uh, coding than the other models. I think people are impressed more like by the, the agentic and completeness ability of Astra, but like the actual quality of the code, it's, it's not that good. I mean, it's pretty good, uh, but it, it's not as good as Opus. And when it comes to DevOps, even Opus is bad, and Astra is like even below that. Yeah, I, I think they still have a lot of tricks, and it's mainly because of Haiku, honestly. It's- Haiku is like the most disruptive, uh, model I think they've ever released at that price, and they even give you 200 dollars in credits now to just use

45:22Nisten Tahirajthroughout apps. That's not gonna run out at 1 cent at 100K. Like, whatever app you're gonna serve to people, if you just keep it under 100K context for Haiku, which most conversational UX stuff is, is good enough. That is more than enough, and on the max plan, that 200 dollars, that's probably gonna support like a thousand users pretty easily.

45:42Alex VolkovYeah.

45:43Nisten Tahiraj10 cents in, 1 cent cash, that's, uh, yeah, it's just, it's just way too cheap.

45:49Alex Volkovso let's talk about the subscription saga. Wolfram, I'm gonna put you up there, As we said, we would love to use personal agents with the Opus 5.5 level. Now it seems like it's possible. could you cover that a little bit? Did you see this? The Claude SDK and the slash P stuff.

46:05Wolfram RavenwolfThat's why I was waiting for the new Hermes update, where this is now possible to do. So I cancelled my subscription with Opus when, or with uh, Anthropic, when uh, you weren't able to use it without getting banned. Some people apparently got banned for using it, and uh, so that was when I decided I don't want the subscription if I can't use it with the agent I'm using. I pay for this, and if it's Claude Code or if it's Hermes, it shouldn't matter, in my opinion. And now it's possible to use your personal agent, be it Hermes, OpenClaw, whatever you use, and connect the SDK, the agent SDK, and use Opus as the main model in your own agent that way, again, in a legal way without being

46:45Wolfram Ravenwolfbanned. So yeah, now it's possible again, and I have to reconsider resubscribing.

46:50Alex VolkovSo the cool thing about, what happened 2 weeks ago is that not only did OpenAI always allow this use case, they also literally shipped a button and an integration called, like, sign with your ChatGPT, and then the other products could use your tokens from your subscription, right? So Devin is one example from Cognition. It supports that use case from the get-go. Hermes Agent was called out on stage, Nous Research, shout out to Nous Research, our good friends from there, and congrats on their round again. Um, and Hermes Agent is now going to support both, and this was always the use case with OpenAI. It looks looks like Anthropic kind of needs to follow suit.

47:26Alex Volkovso now it's going to be possible. Quinn Slack, also a friend from AMP, the coding agent, said that this is why now you can use their models. Uh, Peter Gostev, will you take the risk? Will you use your Anthropic subscription within external tools, or will you just say, ah, Claude Code is good enough?

47:43Peter GostevYeah, it is kind of a, a funny one, because I think they're clearly trying to push you in the direction of, of using Claude Code.

47:50Alex VolkovYeah.

47:50Peter GostevUm, so yeah, it's kind of annoying. I don't know, aren't they making enough money already to just let people be? Like, uh, I f- I feel like there, there should be more freedom. But yeah, I mean, I, I don't personally, I, I just don't because I, I've got enough stuff going on. I use other models via Pi, for example, if I need to. I find that, uh, more convenient. Um, because I, and I'll tell you why, is that I think with Anthropic, with OpenAI, they clearly, their main priority is to use their own models, right, and to, uh, and to kind of, uh, to, to coordinate everything, and I think it's just easier to, to,

48:29Peter Gostevuh, use other models in other ways, but yeah, it, uh, I, I prefer to just match the harness to the model, basically, and simply put.

48:39Alex VolkovHarness to the model, I think, is very important and something that folks miss. Claude Code and Claude definitely go hand in hand. Again, Muse Wonder and Muse probably go hand in hand as well. Codex is also very good with Astra and less so in other places, and that's why it's, for agents like Hermes, it's some time to catch up. And there's a rumor, and we don't love rumors here, but we will say it, that Fable 5.5 showed up. And I I think I'm like, I'm getting tired of rumors, so I'm gonna do this for just one second. This is not planned. But hey, uh, Anthropic is going to release Fable 6 also, and then, Opus 6 is going to come

49:22Alex Volkovout, and then Opus 6.1 or 6.5 is going to come out after that, it's going to be even better, and then Opus 7. Uh, so if you heard it here first, Opus 7 is coming. We just don't know when or how, but I'm getting tired of this, like, oh, this model is coming and this getting just a lot of, o- okay, so, so when we, when we'll get it, we'll get it. I don't understand why this, why the X algorithm for all of us is so fine-tuned on just like trying to snipe the news of something that we all know is on a continuous improvement loop all the time. Could you just tell me if that's interesting to you when you see somebody and just like, oh, I've seen this leak or whatever.

50:00Alex VolkovWe know the models are gonna get better. We know it's very important for these labs to tell you how better they were gonna get. It's, it was especially important pre-IPO. Um, I'm interested in testing them out, playing with them, hearing what Peter says, what Nisten and Wolfram and Yam say, rather than just like knowing the news. But given said all that, somebody did spot Fable 5.5, so I'm very excited to see how that's gonna go. Uh, comments? Did you guys hear my, do do you push back? Do you feel like it's, it's important to snipe and like do breaking news, et cetera, at this age, or are we getting a little bit of tired of models getting better, numbers got better?

50:37Nisten TahirajI want to see them run a full factory or something. That's that rumor. Like, I want to see a whole bunch of humanoids running whatever model that, uh, yeah. Yeah, they're, yeah, the models are going to get better, of course. There's going to be, there's going to be all Opus 6. I think Anthropic's going to not allow everyone to just stay on top of the benchmarks. They always have something in the, in the back end, but uh, yeah, rumors now are more acceptable for uh, like actual humanoids running a factory, running a hospital or something. We gotta, uh, we gotta leave the rumors for more like actual physical stuff.

51:20Nisten TahirajYeah, I agree. That's, I'm not sure why. Like everybody else, it's common sense. I don't know.

51:27Alex VolkovYeah. Wolfram, from an evaluation standpoint, do you think that, s- sniping the news and telling you, oh, this model is coming is still important, or are we like past that?

51:38Wolfram RavenwolfI mean, right now, the most important thing for most people using agents is to have a way to use a subscription, so if the new model comes out and it's API only, or it's so expensive that you run out of, uh, your tokens as as quickly and you can't really use it, it's not as useful as a model that is just smart enough. And I think we are reaching a point where a model is smart enough for most users, and the much higher intelligence is only for bigger projects and stuff that you as a, as a user of a personal agent are not even needing for book me a flight or, uh, plan my vacation and stuff like that.

52:15Alex VolkovNo.

52:15Wolfram RavenwolfI think there's a limit to what we really need, and everything beyond that is on a company level and not on a personal level anymore.

52:24Peter GostevUm, I, uh, completely disagree. I think we're gonna look back at this a year later, gonna be like, oh, we were so, so stupid and naive. I think it's gonna be amazing. And I, I, I get your point, like, I think it's kinda, kinda tedious, you know, this number is gonna go up. But I think the fact that we, we shouldn't take it for granted. I think OpenAI and Anthropic are the two labs that actually ship models very regularly.

52:51Alex VolkovMhm.

52:51Peter GostevBut we haven't seen a full pro model from, uh, Google for, what, 8, 10 months? I, I don't know, haven't counted. So they, they were only shipping, uh, flash models. So I think, well, you, I, I find it perfectly reasonable for people not to pay attention to rumors, but I think there is information in it when, when, uh, companies actually ship. So let's say if there is a scenario that Anthropic doesn't ship 5.5 Fable for another 2 months, I think that's also kind of newsworthy, like what happened there?

53:22Alex VolkovMm.

53:22Peter GostevUm, so yeah, it's a, I that, so I think it can kind of go both ways, um, which I don't think you have to have rumors for, but I still, I I still get a little bit of information out of it. So yeah, we'll see, like I haven't counted, uh, how many weeks OpenAI, when did they first kind of, um, had Astra released? It's probably what, like a month ago, 6 weeks ago, something like that.

53:46Alex VolkovAstra is around a month ago, 5 month, 5 weeks ago or something, yeah.

53:49Peter GostevSo, so yeah, you'd expect them to, I'm gonna leave 6.1 Sol aside because I think that was like maybe out of cycle a little bit, but eh, the next Astra, you'd think that's probably another month away, right? So I think, but if it's 2 months away, then I think we'll get some information out of it because it's, uh, like they're probably spending more time, I know, aligning the model or something like that. So yeah, so, you know, I'm very pro rumor, but, you know, uh, the- I'm also a psycho, so, you know, you don't have to be.

54:19Alex VolkovI- I- I think first of all, the descending opinions is, I talked to our fans, in IRL, they love this. That when we disagree, we give like multiple, sides of the story, and I think they actually do love this. So that's great, Peter, and I do think that, you're giving an additional insight there where now Anthropic is going to be public in IPO, whether or not they're going to release a new model actually will move the news, will move markets, given how big Anthropic is. Uh, that's a very interesting kind of thing. So maybe we're, after doing this for quite a while, maybe we need to find the passion again to say, oh, this model's coming, yeah, let's go.

54:53Alex VolkovUm, but yeah, we'll definitely get there. So I think that's enough about Anthropic, um, and given, let me see if I can make this connection here. Given that Anthropic is going to IPO likely on Wall Street, and given that I am here very, very close to Wall Street, and the reason I'm here is AI Engineer. Let's talk about AI Engineer in New York. Uh, nice? Yeah, okay. this week, AI Engineer in New York. This is the second New York, AI Engineer conference. For those of you who are new to our show and not familiar with AI Engineer, uh, ThursdAI started as a Twitter space and then became a podcast, and the first time that I ever told anyone else when they said, hey, what do you do?

55:30Alex VolkovI told them I have a podcast, was on my way to AI Engineer in 2023 to cover it. Uh, since then, the conference has blown up, and swyx, the co-host of Latent Space and the creator of AI Engineer category and the conference, has been curating incredible folks throughout these conferences. So we went live, earlier this year twice. One time from the Moscone West was this, with Wolfram during the, in the middle of show The Engineer, and then, uh, Peter actually interviewed you when we chatted in London as well. So we went live from London, um, Wolfram as well, uh, and this time, for some reason, they decided not to do it on Thursday, but we're here

56:11Alex Volkovin New York, after AI Engineer. And what interests me about this specific conference is the zeitgeist is incredible because a lot of folks come there, first of all, to learn so that you know that the speakers are gonna get deliver something, but also the hallway conversations. And so I have a little bit of a, like, debrief for you because I know that many people did not make it to New York. None of my co-hosts are here in New York right now. Um, and so here's a little bit of a debrief. This specific conference, specific- because it was around adjacent to Wall Street, this focused on finance and AI. The big hedge funds were all here. Um,

56:46Alex VolkovTwo Sigma, let me go find them. Capital One and just Bridgewater, Apollo, Citadel, Bloomberg, like all of those folks, and then OpenAI and Anthropic were also here. Uh, the big theme from Latent Space, what they said, was autonomy versus security. Now that the big companies are actually running agents. And when I say now that the companies are running agents, to contrast, a year ago, 2025 was the year of agents in quotes, and the zeitgeist was agents don't work. This was before Opus 4.5. This was before the kind of the November, January revolution. Just scroll back just a year ago, agents did not do work, and now all these folks

57:26Alex Volkovare saying, oh shit, agents do so much work that we need to like put guardrails on them and do like, how do we, how do we deal with autonomy and models crossing the threshold from non-coding work to like making decisions? There's now agents inside banks, there's golden data sets, there's evals, there's a bunch of stuff that the industry now contends with that they didn't have to a year ago, because a year ago agents did not work. Most of the AI in, in companies were software engineers trying to get, lines of code out. Now there's like agentic things happening inside most of these ones.

57:57Nisten TahirajLike 2 years ago, I think I posted something about I've never seen an agent actually do anything useful other than Plinys buying crypto, and that was, uh, that was only 2 years ago. So just remember, everyone's opinion is always wrong at some, at some point. Yeah, there was a tweet from me yesterday, I don't know what people are doing with agents.

58:19Alex VolkovNo, but also we're changing so fast.

58:20Nisten TahirajAnd then it just completely

58:23Alex VolkovYeah.

58:23Nisten Tahirajflip one, one, yeah.

58:25Alex VolkovAnd this is what we talk about when we talk about acceleration. you, you can't be looking at the current state of the world right now and say, hey, AI is this. You, you have to start, this is why we're talking about acceleration. This is why we're try- trying to bring you the news and the use cases because, we kind of feel that this is happening. So all the banks are now, thinking about this, like, very strongly. Not only AI engineers are spending tokens, banks are actually using agentic processes. Max Kanat Alexander, he is, I believe he's a, a distinguished engineer in, at Capital One. He was the code simplicity guy, uh, and he did a bunch of Kubernetes stuff at Google.

59:00Alex VolkovHe's a fascinating dude to talk to. His talk was the highlight. Uh, he said, like, he talked about, does code quality even matter? Uh, given the background of this person, I think it's very, very important. A principal software engineer and open source, back in Google. He- He- he led a bunch of the bug stuff in Google. Uh, he basically says the code quality matters in proportion to how durable your software is, uh, because AI is about to make a whole new kind of software way more common, disposable software. Just write once, use it, and then just throw it away. So when you have disposable software, does quality matter if it works? If you build a calculator or an app for yourself, party planner, like, who, like,

59:43Alex Volkovdoes it matter? So this was one great, uh, show, and I recommend folks to tune into the AI Engineer live stream on YouTube, which you can rewatch if you want to. The talks will get released. The other kind of like hallway conversation was, first of all, about evals and whether or not agents are working. Um, but as I said in the beginning of the show, the thing that was highlight for me, which I do want to ask you guys about as well, is the disconnect between how advanced all those AI engineers are in use cases and harnesses and personal agents with the revolution of personal agents for normies.

1:00:21Alex VolkovAnd how, when I talk to some of these AI engineers, I'm like, hey, do you use Instinct, Muse, Grokbot, or whatever? And I don't get an answer yes from them, and that's really, really something, because many of the engineers just like build their own harnesses or use Claude Code or Codex, and why not use Codex for some of this stuff? And so there's like a big disconnect, which is very interesting to me specifically because how big and wide OpenClaw and the Hermes Agent got, and those are harnesses you can tweak and play with. Um, so I did, I did notice this in the halls as well. And, specifically, this is highlighted for me because I did talk, like Wolfram does,

1:00:58Alex Volkovto every person in the world that is not an engineer. Wolfram talks to stewardesses on planes to try to AI pill them. Uh, so I did the same. Wolfram, I took a page out of your book and I talked with, there was the Token Billionaires Lounge we w- told you about before. This, this AI engineer had this again. So if you are spending more than 1 billion tokens a week or something, you get this golden card. I think I have it, hold on. Wolfram, there's a new card that you need to get. The token billionaires card kind of looks like this, and then this gives you access to this, like, lounge. That this lounge is kind of like for billionaires lounge.

1:01:33Alex VolkovThere were a lot of service providers in that lounge that were not part of the conference, and I tried to talk with them. Here's the thing that I got from them, not from the engineers. All of them know about Muse because Muse is getting pushed down their throats in every Instagram post. Like, Zuck is really going out of their way. So, like, they all know about Muse. Many of them didn't install it or didn't get, why it's, why it's good for them, didn't yet, catch that. Uh, so those conversations were fascinating. Um, let's see what else. So, AI engineer, if you haven't been there. The highlight for me, and I, I tell it to everyone also.

1:02:08Alex VolkovIf you're listening to the show, you're good enough to submit a talk proposal to AI Engineer. And if you submit a proposal to AI Engineer, you get access, and Wolfram and Peter both can comment that this is true. You get access to this, where am I? Oh, there we go. Uh, you get access to this amazing speakers lounge where all the speakers that are hand selected are top of the level of people, and you get also access to Speakers Dinner. And to me, the Speakers Dinner and the Speakers Lounge hangout is the highest tier of, folks who understand whatever word I say. You don't need to explain what harness is. you actually learn a lot. One example that I can shout out is, I just w- were walking by and somebody,

1:02:48Alex Volkovsomebody's tag said linear. I was like, oh, who's this dude? Oh, he's the, you know, the, the senior, like, I don't remember the full, I think he's like manages software for Linear, all of Linear. Linear is a, agentic forward task management software. And I had a bunch of feedback about Linear. We just sat down just right there and talked about how I use Linear with my agents and what they can do to improve. This is one of those conversations. So if you are listening to this pod, if you have anything to say, I strongly, strongly recommend to apply for a conversation talk at AIE. Um, Wolfram, Peter, any thoughts on that space?

1:03:26Wolfram RavenwolfIt's the highest level of networking you can have in the AI space because you are not talking to some other attendees of a conference, you are talking to the the big shots who are creating the AI news we are reporting about. So this is super, super amazing. And even if you don't become a speaker, I mean, just attending and going to the booth and talking to the people you meet there, they're all the, the people from Google and Anthropic and OpenAI, they are all there. So it's the best way to connect to the makers and shakers in the AI space, definitely. Best conference I know.

1:04:00Alex Volkov100%. Um, Peter, go ahead.

1:04:03Peter GostevYeah, now I would say also everyone can apply, but do take time to make a good application because I can't remember what the stats were, but I think swyx was saying, not for this one, but for the one in London, I think it was like the, the acceptance rate was very low. So I would say put, put effort in. It's

1:04:25Alex VolkovPut effort.

1:04:25Peter GostevIt's, yeah, the, the reason why it's a good networking, uh, opportunity is because it is a bunch of good people. So, yeah, it's i- it's worth, you know, don't just LLM generate, uh, your your summary and proposal. Yeah, take take your time.

1:04:43Alex VolkovHere's an additional insight that I got, and I think we've talked about this on the show, but, obviously this is on our mindset, right? So, AI engineer in London, the zeitgeist were talking about, should we still read code? We don't write code anymore. Should we still read code? I talked about the ZL continuum on my way back. Uh, I think it's clear now that barely, barely anybody's reading code. Now we're like stressed how to even test it and make it valuable. The zeitgeist here was the end of attention, and how do we scale from managing one or two agents directly by steering them and seeing outputs to running multiple, multiple tens or hundreds even agents, and how do we have

1:05:27Alex Volkovthe attention to attend to all of that and make, decisions, or whether or not we just like let this swarm run and deal with this. And I think it's a very interesting, I talked to many, many people. There's not a lot of solutions. Ryan Carson are, by the way, shout out Ryan. We hung out during this whole AI engineer, our previous, our co-host. Um, me and Ryan, we were talking to like a bunch of people. Ryan gave a great talk about this as well, and he's missing a control plane. He's missing like a one central place where he can like see all of the stuff that's working. Another thing that was in the zeitgeist is the move to the cloud. Many people are just getting

1:05:58Alex Volkovlimited by their local environments. You can see this from the Frontier Labs as well. Codex Cloud just launched, InDev Day renewed, launched Claude Cloud. You can now hand off tasks to the cloud as well, and they even give you, like Nisten, I think you said, they give you some budget, like 100 dollars, to try the cloud thing. Uh, obviously Devin and Cursor, they have cloud agents, um, and folks are getting to the limit of their environment, not only because it doesn't run 24/7, you need to run caffeinate to keep it running, but also because the rock environments are running out of memory. and Peter, I think you mentioned this as well.

1:06:34Alex VolkovYou're like coding so much in your things that the agents just like killed your computer, so you got a different box. Uh, and when you spin those up in sandboxes on the cloud, you kind of don't have this restriction. You c- you can't go wide. So like, we're getting to a point where not only are we using more billions tokens more, my case, definitely Peter's case, definitely Wolfram's case, all of us here, probably like 10x for the past few months, and the Zeitgeist and the engineer as well. We also need a place for that whole thing to happen, all the tools to happen, and that's not happening in local environment. I heard, um, the engineer.

1:07:07Alex VolkovUm, all right, folks, next up we have, Omar Sanseviero and Geneviève Huskens, I believe. Uh, they're joining us now. Uh, just before that, as always, given the show is sponsored by one sponsor only, CoreWeave, I would love to tell you about some of the stuff that we have, and then we're gonna talk with, Geneviève and Omar about finally we got the new big Gemini back, and it's so exciting to hear about this as well. Let's go to this week's buzz, and just right after that, we'll have our guests. CoreWeave.

1:07:48Alex VolkovFrom CoreWeave. I think that it's time to sunset that transition and come up with a new transition, Wolfram, because, as of 2 weeks ago, CoreWeave Forge is now, the home for the weights and biases as a tool, and also a bunch of other tools, specifically, Agent Lens. Agent Lens launches this week, and, if the highlight here for me in Agent Lens, Wolfram, t- tell me if if if you're also on this, is that Claude Code now supports OpenTelemetry exports, and then now you can use our

1:08:28Alex Volkovtools to actually look at the agent runtimes that you have and analyze if it did the work correctly. So, um, would, would love to just like let you know that that you can now use that. Uh, you can sign up on forge.coreweave.com. The highlight for me from Fully Connected 2 weeks ago was that given the previous life at CoreWeave, where you had to get, you had to go talk to a salesperson, register for long contracts and sign up. Forge, for the first time, is our register as a regular SaaS offering, and then you get access to a bunch of tools from our, CoreWeave Agentic Cloud, tools like distillation as well.

1:09:08Alex VolkovWe talked about model distillation a lot on the show, but now via the OpenPipe folks, which are great folks, friends of the show, you can get, model distillation for yourself. You don't have to set that all up. It's all part of forge.coreweave.com. Um, Wolfram, did you set up your Claude code with the AgentLens yet? Are you sending Otel?

1:09:31Wolfram RavenwolfAt the moment, I, I don't. It's all local right now, but uh, it makes definitely sense to integrate all of this, and the integration is what Forge is all about, to have integrations between all the different tools in the whole AI loop, from the evaluations to the inference it's running on to uh, the curation of the data sets. It's all connected, and that is a new thing, I think. Uh, it's not just Weave or a individual product where you have to export, import stuff, you can jump between them and the connection all on one.

1:10:00Alex VolkovUm,

1:10:00Wolfram Ravenwolfon account.

1:10:01Alex VolkovI want to call out yesterday. Yesterday we said, at the event that we did, we sat down and we talked about some stuff, and obviously I was like, okay, this is a CoreWeave sponsored dinner, let me at least tell you about their offerings. And the folks who use Weights and Biases didn't even know we have Aria. and somebody's like, what is Aria? So Aria is a auto agent assistant, agentic research assistant, basically auto research from Karpathy personified in the product. So if you are using Weights and Biases for anything, people didn't even know that we have this offering, so we definitely need to work a bit about, externalizing our offerings. But if you have seen auto research as a concept from Andrej Karpathy

1:10:36Alex Volkovin his tweet, it's now in GA. You can just use it, inside Forge on your training runs. All right, folks, I think this is enough about Forge, and now we're gonna move on, uh, because we have not quite breaking news, but definitely news from the frontier of AI with two great guests here, and let's go talk to them.

1:11:05Geneviève HuskensHello.

1:11:06Alex VolkovHi. Omar and Geneviève are joining us from DeepMind, from Google DeepMind, and you guys have news to share, and it's, can I just say it's about time? It's about time.

1:11:19Geneviève HuskensYep.

1:11:19Alex VolkovWe talked on the show before, and folks, it's about time for a big model. So you guys launched, Gemini, 4 Argon, but I would love for you to tell us about the launch, about what's new, about what are the use cases. I'm all ears. It's out for everyone. It's number one on Arena, and, uh, J- it helped make Gemma three times faster. Omar, that's kind of your corner as well, right? I would love to hear about the use cases. So please give us a highlight, and then we're gonna dive into use cases. And Geneviève, great to meet you. Thank you so much for coming.

1:11:46Geneviève HuskensLikewise. Thanks for having me.

1:11:48Alex VolkovAll right, who's gonna start? Omar, let's go.

1:11:50Omar SansevieroI can go first. So yeah, I mean, Gemini 4 Argon is our latest model. Uh, TLDR, it's a great model. You have probably heard many things about it. My only recommendation is to go try it out, and I think many of you will be pleasantly surprised. eh, yeah, the model can be used for many things. We've been using it internally. Eh, as you said, we've been doing optimizations for Gemma. We have been able to achieve over 100,000 tokens per second with the E2B, so our smallest model. Eh, the model has a 1 million unified, eh, context, eh, so pretty much you can pass very long videos, but you can also have very long outputs, so that is quite exciting.

1:12:27Alex VolkovMm.

1:12:28Omar SansevieroUh, and yeah, we have been very excited to see what the community has been building with the model. what do you think, Geneviève?

1:12:33Geneviève HuskensYeah, we've had so much fun, building with Argon. I it had been a while since we launched a new, model, and so most people will know that this is the first model that came out in our Gemini 4 series, by far our largest train training run ever. And so, uh, Argon, I think, is the first, actually quite early checkpoint that already is doing really well, but we're also super excited about everything that we can build, next now. And so Argon is already training our future generations of models. Um, I think we've really enjoyed the fact that Argon can do many more long-running tests, and so now everyone is shipping a bunch of stuff overnight, and you wake up

1:13:03Geneviève Huskensand you just wake up to really fun things that Argon has been cooking in the background. And so, um, it's much better at 3D, um, much better at creativity. The writing is really pleasant, and so

1:13:13Alex VolkovMm.

1:13:13Geneviève Huskensin general, it's just been super fun to build things with this model. And again, uh, we think there's many more great things to come from us, uh, in the next few weeks.

1:13:19Alex VolkovI would love-

1:13:20Omar SansevieroYeah.

1:13:21Alex VolkovYeah, go ahead, Omar, please.

1:13:22Omar SansevieroUh, yeah, ju- just that as usual, we have been working with all of the open source ecosystem to make sure that you can use the model, eh, regardless of what's your favorite tool. So we have been working to make sure that this model works well with OpenClaw, with Hermes, with, uh, all of the tools, uh, Llama, uh, sorry, uh, LangChain, uh, Vercel, and so on. So, uh, super excited to also collaborate with all of this ecosystem to, to get the model out there.

1:13:46Alex VolkovI want to say, the release definitely is a wide surface release, right? it's available pretty much everywhere, uh,

1:13:51Omar SansevieroYeah.

1:13:51Alex Volkovand also in your own, Gemini API and AI Studio. Shout out to AI Studio folks, for always, bringing us in to the fold. Uh, Gemini app only for Pro and Ultra, is that correct? So folks who are subscribing to Gemini app on the Pro and Ultra tier are giving this, and Antigravity Ultra as well. 2 dollars in and 10 dollars out. Uh, this is introductory pricing?

1:14:10Omar SansevieroExactly.

1:14:11Alex VolkovYeah. Uh, and then what happens then? Ex- extension, new model, another interactive pricing? Um.

1:14:17Omar SansevieroBreaking news, breaking news for, for next month.

1:14:20Alex VolkovAll right. and then, based on what it says now then it doubles, 1 million token context, and I think let's talk about multimodality. Uh, Google and Gemini models specifically were always kind of like the, the front leading in multimodal, including like watching the video that you send to it. 1 million token is a great thing for that.

1:14:35Omar SansevieroYeah, you'll see that the model is very good at understanding visuals, um, the space, and it has a very good understanding of the world and physics. So, uh, many people kind of forget that models are not just for coding, like models are to understand the world, and the world is highly multimodal, uh, multilingual as well, and in general for Gemini, we make sure that this model works well across all of these kind of use cases.

1:14:59Alex VolkovThat's incredible. Let's talk about vision. Uh, Roboflow called this model like number one in vision and detect OCR stuff as well, very, very, very strong. Um, any specific demos around vision that you guys want to show us? Any cool things? That'd be dope.

1:15:12Geneviève HuskensUh, let's start with the ones that I have ready, and then

1:15:14Alex VolkovLet's go, let's go.

1:15:15Geneviève Huskensif you have anything else, vision in the background. Uh, the ones that I picked for today are mostly very cool looking. Uh, but yeah, it's great to see the vision capabilities always, if you're building with a model and it can check its own work, if it can record videos and kind of see what's going on, then that is super, super helpful. So even for some of our 3D stuff that we're now doing, it's just so pleasant that Gemini can watch its own videos back. Even things like motion graphics is very helpful because it can look at a video, understand the style,

1:15:39Alex VolkovOoh.

1:15:40Geneviève Huskensit. And so I made a trailer about Argon earlier this week that took me just maybe 30 minutes, uh, and it came out really good. Um, but yeah, in terms of, uh, the demos that we've been cooking, the first one that I made was, uh, Desk Racer. Um, and so first of all, if you go into Google AI Studio and you click on the gallery, you can find a bunch of stuff that we've made, uh, with Argon. Um, and so Desk Racer came from our multimodality, where I wanted to build a racing game on top of my actual desk. And so the microphone that you see here, and then this like cute little cat, um, just gave Gemini some reference images and said, can you just build me a racetrack?

1:16:13Geneviève HuskensUm, and what's fun about this is, I mean, it's a long running task to understand where to go around the objects, to make it look good, uh, to sort of plan the lighting and everything. Um, and so now if you can play this, um, and then Argon even, hang on, let me switch camera view and turn the sound off.

1:16:29Alex VolkovThat's so cool.

1:16:30Geneviève HuskensArgon even prompted our Omni video model, um, to extract sounds, like engine sounds from, uh, different race cars, listen to all the sounds, found the sort of, uh, audio that was able to loop, and then stitch all of that. And so, like, yes, it looks good, but it's also cool to think about the rest that's happening in the background to, in the end, uh, come up with a very, uh, shiny, uh, result. And so this one I really like because it's where I spend all day, every day, uh, but it's it's fun to see Argon, uh, do such a good job. And so I built all these little other camera views in it, uh, and so this you can play in AI Studio. Uh, there's a timed mode where you just drive three laps, uh, and so that's pretty

1:17:06Geneviève Huskensfun. Um, and then this one is-

1:17:09Alex VolkovJust before you switch, could I ask you for our audience, how do you approach building this demo? Is this all in AI Studio, using any other harnesses, or using like the internal tools? Could you tell us how do we get to building this where Argon can download the files, then like using Omni to generating things and like extracting? Where, where's all the tool happening?

1:17:27Geneviève HuskensYeah, that's a, that's a great question, and actually I wrote a guide about that earlier this week, and so in essence, started with the reference images, um, use GrillMe, so like ask me questions about how I should plan, uh, uh, this project, and then just use a big goal command and said, you know, I just want you to spawn a bunch of sub-agents, work on it however long you need. Some things that helped were A, um, using adversary, uh, uh, sub-agents, and so there was always one sub-agent whose job it was to look at the outputs, grade them for accuracy, for lighting, come back with feedback, and then, um, give a rating out of 10, and then keep going until the rating was, uh, very, very high.

1:18:01Geneviève HuskensUh, so that, first of all, uh, uh, was really helpful. And then secondly, we've all really been enjoying using, uh, Nano Banana to make 3D even better. And so in this example, for instance, I'll turn off the audio, but this is a demo that looks really, really good. It's like a pagoda, uh, demo. And so if you have a poor render and you give it to Nano Banana and you say, make this look really good and realistic, and then you pass that image back to Argon and say, do whatever you need to do to make it look like this.

1:18:27Alex VolkovMm.

1:18:27Geneviève HuskensUm, I'm now using that in all of my, uh, 3D projects, and so the lighting that you saw in Desk Racer, uh, the neon, uh, lighting was designed by Nano Banana because at first it looked very bland, and so this, I think, is a really nice way to interact, uh, with any model, just that image inspiration is a really good, uh, visual reference.

1:18:45Alex VolkovOh, that's incredible.

1:18:45Geneviève HuskensAnd yeah, I did write a guide about this, so, uh, sorry, if you look this up.

1:18:48Alex VolkovNo, no, it's it's all good. It's just like a definitely a great tip for our audience as well. Use visual references because the model is very good at multimodality from other models, for this. That is incredible.

1:18:58Geneviève HuskensAmazing.

1:18:59Alex Volkovshow us another one. That's, that's great.

1:19:00Geneviève HuskensYeah, so, so this one is, is pretty cool because like, yes, it's fun to do things with, uh, Blender and Three.js, uh, but in this case, my colleague actually asked, um, Argon to create this 3D scene completely from scratch. So zero libraries, zero, uh, external things being pulled in. Um, and what's cool about this is if you think about how a model needs to do that, is it needs to think about all the light, all the ways that objects might interact with each other. And so the light bouncing off the wooden floor here needs to be refracted in a certain way. Uh, the light shining through the glass needs to exist in a certain way. And so if you think about the amount of real world understanding and then also the

1:19:36Geneviève Huskensquality of code generation that you need to make happen to make a project like this, uh, that we also thought was very impressive. Um, and then the other thing that was fun about this video, I'll, I'll spare you the audio, but uh, Argon also narrated it by itself, um, uh, picked the, the music with Lyria. And so again, the idea that you can build something cool, give it back to Argon, and then say, now make this into a cool video, that all just ends up working. And I think in our previous models, we probably would have had to babysit that a lot more and and and give much more guidance on how things should look. Um, and so another one that I think is just a cool example of of what you can do.

1:20:09Alex VolkovI wanna, I wanna go back to here with Omar as well, and maybe both of you can chime in. Uh, the token output changed as well, like, Argon is like generates output significantly. Could you tell us how this affects like demos of like this on other stuff? Omar, would love to hear from you as well. How does that improve people's kind of like out results? Specifically because if you think about kind of like evaluating models, a lot of the times, uh, the folks are focusing on the Pareto frontier corner where reasoning used to take a long, long, long time, and it's just like with more reasoning you get better results. And then there was a little bit of a movement towards improving that so that

1:20:49Alex Volkovthe model will give great results, but still not reason as much and not waste as, as many tokens. But this is not about that. This is like about the model being able to generate like long, long things. So could you tell us about the outputs a little bit?

1:21:01Omar SansevieroI mean, this is mostly for long horizon or scientific projects where you do want the model to do lots and lots of reasoning, eh, and that's pretty much the kind of task where you would need that kind of context. In the day-to-day, most likely you do not need that level of output. For input maybe, right? Like if you want to pass a very long video or lots of PDFs, you may need to do that, but this is mostly for scientific or long horizon.

1:21:28Alex VolkovThat was awesome. And Geneviève, did it affect any of of the work that you did?

1:21:32Geneviève HuskensUm, no, I, I agree with Omar that in most cases, like, yes, you could write one or several books in one prompt, but that doesn't end up being super useful. So to us, it's mostly on the, on the very long reasoning. One project that I have been working on in the background, but it needs a bit more work, is one 3D project where I'm modeling 55 cities in one go. And so whenever it sends off to the sub- agents, then they all start building many, many cities at the same time. Uh, but again, in most engineering applications, it's still much better to just do things in slightly smaller batches. But if you have to think about a very long, uh, difficult math proof for a long

1:22:04Geneviève Huskenstime, then you don't want the output window to be constricted by the fact that the model already thought for a long time. And so that's where we see most application. Um, I think it is a very new thing also for us, and so there's also some complexity with how this works, even in things like external harnesses. But in general, the idea that the scientific community can use this is, is very interesting to us.

1:22:24Alex VolkovYeah, that is very exciting for sure. Uh, so we talked about the visual kind of understanding in number one. Uh, Omar and Geneviève, I think you mentioned this, that now Argon is working on next versions of Argon. that suspiciously sounds like RSI. And Omar, you're talking about Argon training Gemma models, and that's like, parent teacher, not necessarily distillation, but it's also in the process of, of training them. give us a little bit more of the, I think people do want to know whether or not this model can, like, help train other models and how that works. Uh, specifically around Gemma work, how much of this was autonomous or how much of

1:22:54Alex Volkovthis is like intervening? Uh, could you guys tell, talk to us about model improving other models in the future?

1:23:00Omar SansevieroYeah, I think just as we think about AI for science, AI for math, AI for biology, AI for quantum computing, we can think as well about AI for AI, right? And I, I do find this a very exciting area. This is not, this is something that we as an ecosystem have been doing for some time. It's just that as these models are getting significantly better, it's quite exciting to think about ideas. So, for example, in the context of Gemma, eh, what Argon did was optimize vLLM, which is one of the most popular, eh, open source libraries for server, eh, side inference, and it did a bunch of, eh, vLLM optimizations. It found a bunch of

1:23:39Omar Sansevieroareas where we could im- eh, improve things. Similarly, you could see how models could be help on co-designing training code or co-designing architectures. Eh, so yeah.

1:23:52Alex VolkovUh, Geneviève, anything, any last comments, before we let you guys go? No, we're very busy during release weeks.

1:23:56Geneviève HuskensUm, no, I mean, I think, yeah, the sentiment in the market about Gemini has, has varied. I would say please give Argon a shot. It, to us, feels like a very different model from, uh, the models that we've launched up to now, and so I hope everyone gives it a creative shot because there's so much, I think, that you, you can do with it. Um, for us, I think it's the start of a new era without sounding too commercial, but yeah, I think this is a very exciting first step to what hopefully is, uh, it's gonna be a couple of fun weeks and months.

1:24:21Alex VolkovWe've been waiting, and, we're now very excited to have it back and like to see that Google is back. So, folks, thank you so much for joining us. Geneviève, nice to have you on the podcast. Uh, feel free to come back with awesome demos, which we al- always love. Omar, as always, friend of the pod for a long, long time. Thank you guys for joining us, and

1:24:37Geneviève HuskensThank you so much.

1:24:37Alex VolkovCongrats on the release, and we know that you have a lot of stuff to catch up on during the release week. Uh, we'll see you here, next time. Cheers. Thank you so much, guys.

1:24:44Geneviève HuskensSee you. Bye-bye.

1:24:44Omar SansevieroCheers. Bye-bye.

1:24:46Alex VolkovAll right, folks, from one, conversation to another, now we're going to chat, uh, now we're moving forward, and I want to introduce you to Nathan Benaich, I- I've followed you for the longest time, but I think we've never talked like this on video, and I followed the State of AI report. so let me just, let me just do this reading. Uh, folks, the State of the AI report turns 9 this year. Na- is that real?

1:25:09Nathan BenaichYes.

1:25:09Alex Volkov9 years. I feel like I've been doing this for 3 and a half years, and I feel like this is a long time. Uh, 9 years. Uh, the word AI was not familiar to many, like many folks back like 9 years ago.

1:25:21Nathan BenaichYeah, it was, uh, it was, it was the era of like AI dash ML and data science, if you remember that.

1:25:26Alex VolkovAnd uh, this year is 20 to uh 2 244 slides, a scorecard on last year's predictions. Uh, would love to hear about last year's predictions and whether or not we can hit them, and 9 new predictions, including AGI 27. Nathan Benaich, the founder of Air Street Capital, is with us, the author of State of AI Report. Um, Nathan, welcome.

1:25:48Nathan BenaichThanks for having me.

1:25:50Alex VolkovUh, P- Peter also joined us for this interview. Peter, thanks so much for making this connection. Uh, I want to talk about some, like, the highlights, some favorite numbers. I have Claude led 27% of measured model R&D at Anthropic in August. We just talked with folks, from DeepMind about Argon and, like, RSI, whatever. Um, would love to hear from you, Nathan, how this kind of like shapes, takes, takes shape between last year and this year. People do talk about RSI being a thing. People do talk about models training other models, whether or not we're going to hit this like, thing. Uh, what are you seeing from there? What are the, what are the report says about RSI and continued improvement by the

1:26:25Alex Volkovmodels?

1:26:25Nathan BenaichYeah, yeah, yeah. I would say like the, the major story in the last 12 months has been, has been RSI and, and, you know, super intelligence, from AGI to super intelligence, uh, and then, you know, increasing evidence, uh, you know, at the frontier that shows, like, we're kind of making progress towards this idea of, like, AI, building AI. I think, um, what if, like, the two, the two experiments I found most interesting there, one of them you cited, which is just the general stat of, like, how, um, Anthropic's, like, R&D team, when they're doing core, uh, R&D work on the model, uh, so 26% of, uh, of like the directions in research are being guided by Claude.

1:27:03Alex VolkovMm.

1:27:04Nathan BenaichAnd that's up from basically 0 a couple of months ago.

1:27:07Alex VolkovAs not to oppose this, but I think the code generation stat that I remember, maybe you have an updated one, but as far as I remember, like 80 or 85% of code is generated by Claude, maybe not directed. So-

1:27:18Nathan BenaichYeah, yeah, yeah. So that, yeah, that was like a few, a few months earlier. You could see like a chart of like code commits that were, that were basically accepted by engineers, uh, as a function of like how models have been changing, and it was sort of like flat for a while, and then like the Mythos class models came, and then now it's like 8x the, the average, and you see a similar stat at OpenAI. Um, and then the other, uh, the other data point that I, that I like that we made a slide on is this, this like, um, experimental setup where a human researcher engages on a, on a, on a research path, um, kind of goes through like several leaves down like the node of that research path, and at some point the

1:27:54Nathan Benaichpath forks, and then one of the fork is a model that continues the research, and then the other fork is a human that continues the research. And at the end of like both of those leaves, uh, a human rates, uh, which one was better.

1:28:06Alex VolkovMm.

1:28:07Nathan BenaichAnd um, and you saw that like the old, old school models were definitely not better. Like, I think 10% of the time humans prefer them, but then the Mythos class model, it was like 60% of the time humans said the Mythos, uh, suggestions were better than humans. So I think that's like also, um, it's also interesting, and it gets to this point about scientific taste, um, which I, which I think is like on the, on the tip of everybody's tongue, like no pun intended.

1:28:32Alex VolkovScientific taste. That's incredible.

1:28:36Nathan BenaichYeah, just, just the idea for the, for, for those who are like less familiar. It's like, um, I, I did a PhD in biology, and like what I remember were like the great papers were the ones where you would, the grad student or the postdoc would, uh, would be, uh, running a couple of experiments and then every week going to their supervisor and showing the experiments to the supervisor who would look at the work and, you know, kind of mull it over and say like, okay, the next, the next best thing you should do for the story is this. And I think the best PIs would be the ones who'd suggest experiments or ways to interpret the experiments that you did and the results you got in a way that you

1:29:12Nathan Benaichthought was like, like novel and, and interesting and elegant. And then the bad PIs, like, do that very poorly. And then you see this in the result of a paper where, like, the story kind of reads beautifully. It's very logical, like each experiment adds additional signal that the previous one didn't have. And I think that's like what people are trying to encompass in, uh, what they call taste. So can a model like approximate that?

1:29:40Peter GostevYeah, I think there were, there was one more from Prime Intellect, I think, where it w-

1:29:45Nathan BenaichYeah.

1:29:45Peter Gostevit was the Nano GPT, right? Where

1:29:48Nathan BenaichYeah.

1:29:48Peter Gostevwhere, where it sort of did it almost get to the human level, right? In

1:29:52Nathan BenaichYeah.

1:29:53Peter Gostevbut I think there was something that it was still narrow, the the scope of the work that it was doing.

1:29:57Nathan BenaichYeah, yeah. There's a, like, to to get into this more, what I, there was um a study uh from OpenAI in the report where they looked at trying to categorize, like, the the token usage of their uh of their engineers and researchers. Um, and um and what they did was like identify some tokens that were about uh helping decide the research direction, what experiments should you run to test that research, and then other kind of category of tokens, which is like implementing those ideas, like designing the infrastructure, engineering the infrastructure, doing the data pipelines, and doing the data analysis, then just writing the results of the study.

1:30:35Nathan BenaichAnd the vast majority of the token usage was on just like the raw engineering of just like get the thing working and then writing about it, and very, very small amount of tokens was on what research should I do in the first place, and like, how should I test it? And um, and so mine, I think there's different ways you can interpret this, but but one of the interpretations to me is there's still like so much more to go on improving, uh, you know, the use of models for like ideation in brand new id- brand new areas.

1:31:04Alex VolkovUm, you also gave yourself a scorecard. Can we talk about that from last year's

1:31:08Nathan BenaichYeah.

1:31:08Alex Volkovpredictions and misses? Uh, what stuff did you hit correctly? What's the, or like the report hit correctly, and I'm assuming that's you as well. Uh, what stuff

1:31:15Nathan BenaichYeah.

1:31:15Alex Volkovdid you miss?

1:31:17Nathan BenaichYeah.

1:31:17Alex VolkovWould love to hear.

1:31:18Nathan BenaichUm, yeah, like the stuff I got right was, um, that there'd be a pushback from big labs and neo labs into open source. And this is in, and um, i- in the US and the West, and the idea with that is it was very clearly part of the national AI strategy from the Trump administration to sort of regain, regain supremacy of, um, of American AI, and like open source is a way of doing that. And very soon after we published that report, like Reflection AI sort of went from the coding agent work that they were doing towards saying we're going to publish an open

1:31:49Alex VolkovYeah.

1:31:50Nathan Benaichopen weights model, and then NVIDIA kind of supercharged all that. Um.

1:31:55Alex VolkovBy s- by by basically, I think it's rumored. I'm not sure if this is quite

1:31:59Nathan BenaichYeah.

1:31:59Alex Volkovthere yet, but by saying that NVIDIA is going to maybe like get reflection in there after

1:32:05Nathan BenaichYeah.

1:32:05Alex Volkovgetting Hugging Face in there also this year. So yeah, you're kind of

1:32:08Nathan BenaichAnd Poolside.

1:32:09Alex VolkovAnd Poolside and other potential ones. And uh, so you're bang on on that as well.

1:32:14Nathan BenaichYeah. So that one, and then the, the second one was um, on NIMBYism for data centers, the idea of like, not in my backyard, like I don't want to have a data center next to my garden, um, that this would be a real problem in the, yeah, you know, hampering like the build out of data centers, but also influencing part of the midterms, which we'll see in the next couple of weeks. Um, we substantiated this stat with like a survey that we found where Americans say that they're much, that they much prefer living near a nuclear power plant than a data center, which is, uh, quite wild.

1:32:46Alex VolkovWild, yeah. we covered this on the show as well, and we still can't believe it. It's

1:32:50Nathan BenaichYeah.

1:32:51Alex VolkovIt's, it's just, oof, the amount of

1:32:53Nathan BenaichYeah.

1:32:53Alex Volkovthat are going on and just people on the street talking about just, AI boiling the planet, uh.

1:32:58Nathan BenaichYeah, yeah, yeah. And it's not just like in the US, it's certainly happening in Europe and in Korea and in other places where, you know, there's been like a small proportion of the population that's benefited like enormously from AI and, and many people who haven't, and, um, you know, there's some social issues with that. Uh, I think the last, the, the last one that I got right was that, um, you know, Chinese labs would overtake US ones, uh, in some major leaderboards, and, you know, I think it depends on, you know, which, which timeline you actually look at, and Peter probably knows this by heart, but, um, but certainly, you know, Kimi and GLM

1:33:29Nathan Benaichand others have done really, really well.

1:33:31Alex VolkovYeah. What about some of the misses? What did you predict and didn't

1:33:36Nathan BenaichYeah. Yeah, yeah. Well, um, there's been so much talk about, like, AI sovereignty, which could be interpreted in different ways, but either broadly speaking, the one of I, I as a country want to have independence or control over my own destiny when it comes to, uh, building models, um, uh, training them, serving them, having the infrastructure for it and the energy, et cetera. And, um, and my bet was w- there would be this concept of like, uh, you know, I'm AI neutrality in that that kind of mirrors like, uh, uh, sort of mutual defense, um, agreements where a country is like too small to be able to afford, uh, you know, its own military or its own defenses in the same way it's like too small to be

1:34:17Nathan Benaichable to build its own data centers and have people build models, etcetera. So it just forms like a strategic like defense, uh, you know, agreement, uh, or AI, um, provision agreement with a bigger neighbor or an ally that says, you know, I will, I will, I'll promise to give you all, all of the AI in exchange for something else. Um, I haven't quite seen something like that yet. There's been a bit of a move with the, like, US Pax Silica, um, policy that, um, that tries to, like, bring, um, a collection of countries that have, uh, access to core, um, parts of the chip supply chain, whether it's like minerals or fabs or lithography or whatever.

1:34:58Nathan BenaichUm, but, but overall the sort of AI neutrality thing hasn't really taken on just yet.

1:35:03Alex VolkovNo, not yet.

1:35:05Nathan BenaichUm, and overall, I'd say the predictions are like directionally well, like this is a guy on LinkedIn who like looked at the last like 9 years, uh, and then tried to see like over that 9 years which report which pro- which what portion of all predictions eventually happened. It was, I think, like, over 2/3. And so sometimes it's just a question of timing, more than just like, this is ludicrous.

1:35:23Peter GostevWell, well, that that's what I like about your prediction is that I think it's quite easy to make predictions that, uh, you're gonna nail, like, 90% correct. You just kind of predict roughly what just happened.

1:35:33Nathan BenaichYeah.

1:35:33Peter GostevOh, you know, Anthropic is gonna make more money next year. You know, it's like, okay, that's not

1:35:37Nathan BenaichYeah.

1:35:37Peter Gostevthat's not very interesting. So,

1:35:39Nathan BenaichYeah.

1:35:39Peter Gostevyeah, I find it interesting you got a hit rate of like

1:35:41Nathan BenaichYeah.

1:35:41Peter Gostev50-50.

1:35:42Nathan BenaichYeah. Yeah.

1:35:44Peter GostevSo is it like how, is it difficult to to come up with the predictions? Are there some that you you kind of decided against, I don't know, too wild that that you didn't end up making this year?

1:35:54Nathan BenaichYeah, um, there's one I couldn't quite formulate around, um, around, uh, one is like cybersecurity and like, can AI somehow disrupt, um, encryption on the internet, um, as it might be, uh, as that might be accelerated by the fact that AI solves math. So I don't quite know how to, yeah, basically I just ran out of time for like thinking about how to formulate it properly to be able to defend it,

1:36:22Peter GostevCan I, can I ask you, so you, you speak to all kinds of people, and we're a little bit in the AI engineer kind of bubble. Like, what, what are sort of things that people ask you about? Because you've got different parts of the report about even going into politics and, and things like that. What do ki- what do you hear people care, caring about?

1:36:45Nathan BenaichUm... There's a segment of people who really care about what we discussed, like the, the RSI, like, rapidly kind of accelerating progress, like, what does that mean for the, the rest of us, basically? Um, there's, there's some who, um, you know, are just like increasingly like, uh, pessimistic over like software companies still, because, um, you know, now like if you kind of believe the RSI is coming and basically the, the quality of the software you can produce or the work you can produce is like very limited by what model you have and how much money you're willing to spend on it. Then, um, there's certain people that will just like, or organizations that'll accelerate

1:37:26Nathan Benaichway faster than others, and if you're not part of that, like, what do you do? Um, there's always like chatter, you know, every 2 weeks or something, and I think now we're in one of those negative cycles around like, is there too much compute, circular financing, like how is it financed, like is the debt going to be able to be serviced? And, you know, there's a bloodbath in NeoClouds in the last, like, 2, 3 days because we're just in that part of the cycle. Um, um, you know, there's always, always debate around, like, will, will NeoLabs monetize? How will they monetize? Or these, like, funky financing structures with multiple stages at different prices, all within a couple minutes of each other.

1:38:05Nathan BenaichIs that like a sign of, um, I don't know, just like rabid demand and not very much thinking? Um.

1:38:13Alex VolkovHow about normies? I have to ask you, Nathan, how about like just regular folks on the street and how they view AI? Do you track that in state of AI? Do you talk to them? How do you see that opinion shapes? So we did talk about

1:38:24Nathan BenaichYeah.

1:38:24Alex Volkovthe change in the public opinion about water use, et cetera. Uh, what about the kind of the use case for AI? What do you see there? What do you track there? The rise of personal agents is now upon us, Gokbots and Muse and all of those. Do you consider those in the report?

1:38:38Nathan BenaichYeah, but it that's still like in the small minority of tech people who know, who know what it is. Um, you know, I would, I would still bet that like 90% of people are using like a model to maybe at best like read their email or

1:38:51Alex VolkovMm.

1:38:52Nathan Benaichanswer to their email or, you know, ask questions in the same way they would ask Google, but they probably never click the work button. Like, what does that even mean? Like, do I want to open a consumer app and, like, work?

1:39:03Alex VolkovYeah.

1:39:03Nathan BenaichNot really. You know, and most people don't even know you can swipe left, open Codex, run it on your machine, and like talk to your machine and make it do work for you. So, like, I think that's so far away, which is one of the reasons why, like, I've just started saying, like, I think these big labs should open like a Genius Bar or like an actual physical store and just like start walking, you know, regular people through the magical things you can do with AI.

1:39:29Alex VolkovThat, this is the kind of also the geek squad genius bar comment, right?

1:39:34Nathan BenaichExactly.

1:39:35Alex VolkovThey need hand holding and imagination unlock. Um.

1:39:39Nathan BenaichYeah.

1:39:39Alex VolkovAnd also that's what we're trying to do on the show, though the show does cater to AI engineers, and so if you're still listening to this, uh, you are in the very few, I don't know if you saw the Karpathy tweet as well about, he kind of talks about this as well.

1:39:52Nathan BenaichYeah, exactly, when he segments the world. Exactly. It's, it's, and we're still very much, very much like that. Um, and there's an argument that, like, that gap, that gap expands even further if, like, capabilities get better and RSI happens and all that stuff. But you can see it a bit in, like, the, you know, the ramp data that we show with the 70,000 US customers that they have and tracking how much money they spend on models. It's like, you know, the the average spends like 12 bucks per employee per month, and the top 1% is like 7,500, roughly speaking. So it's like 580 times difference.

1:40:27Alex VolkovBetween the potential permanent underclass, if this continues, yes.

1:40:31Nathan BenaichRight.

1:40:32Alex VolkovAnd and and the top spenders, the billion token ones. Nathan,

1:40:34Nathan BenaichRight.

1:40:34Alex VolkovThank you so much for joining. The thing that I do want to, ask you, because you've been doing this for a long time, again, from AI ML days to then generative AI to, know,

1:40:44Nathan BenaichYeah.

1:40:44Alex Volkovuh, very small context windows to the huge ones. We've been doing this for 3 years, and it feels like incredible time. You've been doing this for, for, for much, much longer.

1:40:52Nathan BenaichYeah.

1:40:52Alex VolkovUm, your personal beliefs and whether or not, your P doom stuff, I don't like the P doom, nomenclature,

1:41:00Nathan BenaichYeah.

1:41:00Alex Volkovbut about the bubble stuff. I would love to just hear from you, how do you see your views change throughout these years? If you can

1:41:07Nathan BenaichYeah.

1:41:08Alex Volkovchat about this for a little bit briefly.

1:41:10Nathan BenaichYeah. I, I think the major... Well, so I, my P doom is very small.

1:41:15Alex VolkovMm-hmm.

1:41:15Nathan BenaichI'm not like a big believer in X risk. Um, I, I, I do think along that angle that cybersecurity is very precarious, um, and should be taken like insanely seriously, but I think that's different than like existential risk of like destroying humanity. Um, I think the- the major change I've had in the last like 2, 3 years is I, I've generally believed that you use the right model for the task in the same way that, you know, you don't, you don't drive your Ferrari to go to, like, the coffee shop, or maybe you do to show off or something, but it's like not the most reasonable thing to do, just like walk.

1:41:51Alex VolkovYeah.

1:41:51Nathan BenaichUm, but I think there's like a weird behavior with like, w- with like frontier intelligence. If you, if you have like 200 bucks a month to spend and you value the quality of your work, do you really want to like take the risk of putting the slider down to low and like using a cheaper model on the chance that like the quality of the work is worse? So while I could switch for, like, emails or whatever, I j- I just don't. I'm just like on Astra High, like, who cares?

1:42:17Alex VolkovYou're answering emails on Astra High.

1:42:18Nathan BenaichYeah.

1:42:18Alex VolkovNathan Benaich from Air Street Capital.

1:42:23Nathan BenaichNot all my emails, but like the ones we have to think about it, like.

1:42:26Alex VolkovYeah.

1:42:26Nathan BenaichYou know, why, why would you take the risk using a shittier, a shittier system?

1:42:30Alex VolkovMhm.

1:42:30Nathan BenaichUm, or potentially shittier system. So, um, w- yeah, so there's, there's that, and then like, you know, will we ever make money on frontier intelligence? And I think it's starting to show that, like, we will, because the value of the task that they're doing is increasingly high, and just this, the more people from all sorts of different parts of the technology world who come into working in AI bring all of their knowledge and optimizations to, to the fore, and just, you know, human ingenuity plus lots of AI equals more progress. So I'm like quite, um, quite bullish on like margins improving, and you know, I don't know what Anthropic is adjusting when they report their adjusted numbers,

1:43:06Nathan Benaichbut um, it's, you know, it's probably not like community adjusted EBITDA like WeWork. Like, I think it's a serious company, so um, um, so I'm, I've changed my mind on on on that. Like, I think you can really make money on Frontier AI, and I'm increasingly convinced that it's the the open source versus like Frontier Labs is like um, iOS versus Android. It's sort of cool for a while to play around with your Android and mod everything, and some people want to do that, but

1:43:33Alex VolkovYeah.

1:43:34Nathan BenaichAt some point, you just want product that auto updates, is like reliable, connects with everything, and looks nice, and so.

1:43:41Alex VolkovAll right, are, those are great stuff. Uh, folks, Nathan Benaich, thank you so much for joining, us on Thursday. I will always welcome you back after the next one as well. If you, are you planning to keep going, to to the-

1:43:51Nathan BenaichYeah, yeah, yeah, yeah. We'll we'll keep going. I mean, you know, if AGI 27 happens, I'll be on the 10th year anniversary, and then I might be able to stop, but uh,

1:43:58Alex VolkovAll right.

1:43:59Nathan BenaichLet's see. Yeah. In the meantime, you can find everything online. It's all free at stateof.ai.

1:44:03Alex VolkovIt's at stateof.ai.

1:44:07Nathan BenaichYes.

1:44:07Alex Volkovso it's a great, URL. Nathan, thank you so much for joining us.

1:44:09Nathan BenaichYeah, thanks guys.

1:44:11Alex VolkovBye-bye. Um, that was super cool. Uh, I don't, we don't often, we don't often get somebody here that does this longer than we do. Uh, and LDJ joined us as well. Uh, this is great. Nathan, uh, Peter, thank you so much. What's your take on-

1:44:27Peter GostevI, the, the most, the, the very cool thing about this is that there's obviously, I, I find this really helpful to keep up with this stuff every week, but there's something really valuable, I think, that to just look back at the year and you're just like, oh, I completely forgotten about this, or I didn't actually realize there were papers there that maybe didn't quite make the splash this week, so we skipped it because, you know, there's too much to cover.

1:44:51Alex VolkovYeah.

1:44:51Peter GostevBut actually they turn out to be quite important. So I really value his work, and I've been reading his stuff for years now. I can't remember what was the first one that I read, but it it's just so useful to just look at it and and just read the stuff. So I really recommend just book, I don't know, an hour, 2 hours in your diary and just scroll for this. You don't have to read every single thing, but just scroll for this. I know it's like 250 slides. You can do it in an hour quite easily. Don't, don't le- don't ask your agent to read this.

1:45:19Alex VolkovDon't ask your agent to read this.

1:45:20Peter GostevYeah, just read it, read it, put some effort in. No, it's worth it.

1:45:24Alex VolkovNisten, how's your 100K city going? Are you still AI pilled? And, Peter, what, what new and interesting did you start using or, or building this week? Just whoever wants to jump in, let's just chat.

1:45:37Nisten TahirajHell yeah. There's, uh, I, I got, I got some, some, some crazy stuff coming up for, uh, release. Um, it's, uh, yeah, the, the all the entire decompilation of games and stuff is enabling full out simulations.

1:45:54Alex VolkovCan we talk about this decompilation? So, LDJ, we mentioned this like last time that people are putting things in other games, and now there's the whole decompilation thing happening. Somebody told me somebody decompiled or reverse engineered the airdrop protocol. What the fuck?

1:46:06Nisten TahirajGuys, it is crazy.

1:46:07Alex VolkovIt's like a renaissance happening right now, right? tell us.

1:46:10Nisten TahirajYes. All right, here's, here's what's happened. The entire internet the last two months has just gone completely nuts by using, like, the NSA Ghidra protocol, which allows you to, like, reverse engineer any piece of software. And then there was also the law in 1996 from the Phoenix BIOS, or AVS, I might be wrong on that, that allows you to actually provide, like, visually functional, uh, pieces of software, as long as it does not violate any trademark. So you see a lot of these things called Photocraft, Gridcraft, and stuff, which are like Photoshop or Excel builds that are done in Rust.

1:46:48Alex VolkovClean, clean room builds, yes.

1:46:50Nisten TahirajYes, so those are mainly done in Rust. But yeah, it's a complete renaissance in, uh, in apps. It's going to be a very interesting next 3 months as, uh, entire countries and economies figure out what kind of office suite do they need to run. Uh, and, and, and the very interesting thing is that actual professionals and creatives tend to like the vibe-coded versions more because they're just so much more portable.

1:47:17Alex VolkovAnd they're fun.

1:47:18Nisten TahirajYeah, and they're fun. Again, they're not perfect. For example, in Photoshop, the reviews were that the Magic Wand tool is not very good because it used like an older machine learning model to remove the background.

1:47:27Alex VolkovMm.

1:47:27Nisten TahirajThat's going to just be fixed in the next, like, 2 weeks.

1:47:31Alex VolkovThe second he turns his slop cannon towards that issue, that issue is solved.

1:47:34Nisten TahirajYes, that issue is solved. Yeah, they're just, they're throwing entire gauntlet prompts, and they're, I've seen entire apps, the entire Adobe suite, InDesign, After Effects, uh, Lightroom, uh, all just running in the browser, very performant too, and you can run them without any telemetry or crap or spyware or

1:47:56Alex VolkovWithout Adobe paying also a bunch of money to Adobe. Nisten,

1:47:59Nisten TahirajYeah.

1:48:00Alex VolkovI do have to pause you because LDJ just bought us breaking news. Uh, let's go.

1:48:10Breaking news stingerAI breaking news coming at you only on ThursdAI.

1:48:22Alex VolkovAll right, folks, we have, quite exciting breaking news. LDJ, you wanna take this one?

1:48:26LDJSure.

1:48:27Alex VolkovI can share.

1:48:27LDJSo, uh, yeah, I, I do feel like overall these past few months we've had a bit of a renaissance of, of Western American open models, and Meta just announced, uh, literally within the past 20 minutes, that Muse Spark Open is released, which I presume is maybe like a partly censored version of Muse Spark. It's 816 billion total parameters, 42 billion active parameters. So this is around 3 times smaller in total parameter size than, uh, than Kimi K3, while being approximately half of the active parameters of Kimi K3.

1:49:06LDJSo you could expect it to be a lot more practical to run a lot faster. Um, I'd say probably in terms of parameter count of active and total, it's probably most fair to compare it to maybe the full GLM 5.3, and I think it is maybe beating GLM 5.3, though, or at least trading blows with it in the benchmarks. So I guess we'll have to scroll down or kind of wait for people to benchmark things.

1:49:29Alex VolkovYeah, let's take a look. Let's take a look. They're comparing themselves, uh, Muse Spark Open, to Inkling, to Nemotron, to Beam, to Mistral, and only then to Qwen and GLM, right? So, like, they're starting with, what is it Inkling, US-based?

1:49:43LDJThinking Machines.

1:49:44Alex VolkovYeah, oh yeah,

1:49:44LDJYes.

1:49:45Alex VolkovNice, okay. Uh, so look, they're literally saying, hey, we're comparing ourselves to American counterparts first, then to European counterparts, and only then Qwen and GLM and Kimi.

1:49:55Peter GostevYeah, I, I think what's interesting, if you scroll down into the coding area and compare it to the Chinese models,

1:50:02Alex VolkovMhm.

1:50:02Peter GostevSo we've got the Terminal Bench 2.1.

1:50:04Alex VolkovYeah.

1:50:05Peter GostevSo Muse Spark has, uh, 76.4, then Qwen 3.8 has 82, and, uh, GLM 5.3 has, uh, 84.

1:50:16Alex Volkov83.

1:50:17Peter GostevYeah, 83.9, Kimi 85. So it's like a 10 points behind the, no, well, not quite 10 points, but a b- like 7 and 8 points behind. Then DeepSeek as well, but basically behind GLM and Kimi K3, which, by the way, been

1:50:31Alex VolkovMhm.

1:50:31Peter Gostevout for, what, a month, 2, something like that?

1:50:34Alex VolkovYeah.

1:50:34Peter GostevI haven't found it. So yeah, definitely better than the US models, but not necessarily at coding versus the, the top tier Chinese models yet.

1:50:43Alex VolkovYep. Um, it doesn't look like they were going to break Sora. One, oh, somebody says, yeah, but they're like 3x the size, right? So GLM and and all those are like, in the billions of tokens.

1:50:54Peter GostevI think GLM not so much, right? Isn't it, isn't it like about

1:50:58LDJGLM 3. Yeah, it's it's quite large here. Let me find the exact parameter count real quick here, uh, because Flash is around 300 billion parameters. Uh, yeah, okay, so total parameters for GLM 5.3, the the not Flash version, which they're showing the not Flash version in that chart, it's 753 billion total parameters.

1:51:16Alex VolkovOh, so that's comparable.

1:51:18LDJYeah, and 40 billion active parameters, so almost the, the, uh, very, very similar.

1:51:22Alex VolkovVery similar and gets beaten.

1:51:26Peter GostevWell, the, it's, uh, GLM is bad at coding, like quite a bit.

1:51:30Alex VolkovYeah.

1:51:31Peter GostevQuite a few points. So, so yeah, I think it's... Well, well, what do you guys think? Is this, what was the relationship to Spark, uh, Muse Spark through, uh, 1.123? Any relation?

1:51:45Alex VolkovOh, that's, I'm assuming that this is like a cleanup version to some extent.

1:51:49LDJYeah. Yeah.

1:51:51Alex VolkovBut they're not talking about this, but yeah, Muse Spark from 1.1 to 1.3 did like quite heavy jumps, um... And this one is a multimodal text, image, video, and audio. Output is only text with 1 million as inference context window and, a great vocabulary size. Uh, this is super cool. Llama is back, essentially, is what we're saying, right? This is like the first release from Meta for a long time that's in an LLM space. Nisten, thoughts? Excitement?

1:52:23Nisten TahirajI'm just, I'm just going to, I just need to try it.

1:52:27Alex VolkovThe weights are available to download. Uh, the model is available in BF16 and F-

1:52:32Nisten TahirajIt's pretty big.

1:52:32Alex Volkovand NVFP4. To run on your system, the model requires a GPU with at least 700 gigs of VRAM and NVFP4 and 2 terabytes for BF16. It also requires an inference deployment framework, supported frameworks, SG Lang, vLLM, Hugging Face, and Llama CPP. We're definitely gonna chat with our inference folks, to try and, to provide this inference for you, because we do believe this is a good model.

1:52:55Nisten TahirajYeah, yeah, yeah, yeah. It's a, especially for computer use, I think this will be pretty key right now. I'm just reading through the license. It just asks that anything you do with it, it should say built with Muse Spark Open in the interface of each product or service that uses the software or any derivatives. That's the main license, and uh, everything else is like an open usage policy, so no violence and s- drugs and stuff, um, or not impersonating and misinfo.

1:53:27Alex VolkovYeah.

1:53:27Nisten TahirajOtherwise, it looks uh, close to just like an open license. There's no restrictions on like how much money or anything like that. Uh, thi- this is, yeah. Oh no, there is a restriction. Where the third party outputs of such models, models of service, and the aggregate gross revenue of the licensee and its affiliates exceed 20 million US dollars

1:53:48Alex VolkovMm.

1:53:48Nisten Tahirajor equivalent during any consecutive 12-month period. So as long as you make less than 20 million dollars a year, you can just, uh, offer it.

1:53:58Alex VolkovMhm.

1:53:59Nisten TahirajUh, yeah.

1:54:00Alex VolkovAnd then probably a license.

1:54:01Nisten TahirajYeah, that's, those are the things, but this, this is amazing because it, uh, it, it will be great for generating data and stuff. Yeah, I'm pretty sure companies will offer it. It's probably one of the best computer use ones.

1:54:13Alex VolkovThat is so cool. Let's go. I wish I had my, my, my air horn button. Folks, Llama is so back, and

1:54:19Nisten TahirajLlama is back.

1:54:21Alex VolkovThe promises of Space Unk, where they release a new Grok, the old Grok will be open source, that doesn't often come to fruition. I don't remember the last time Grok was open source. Uh, Zuck and Alex Wang did say that the, the open source is back after the manifesto and everything, and so now they're, after the releasing Muse, so two, two huge news from Meta this week, right? They released Muse Wonder and Wonderful, and that's, folks are split on that a little bit, and now they released the open source version of Spark in Muse Spark Open, a sorry, 816 billion parameter, 42 active billion parameters. We don't have the clap buttons here, also the applause, but like every lab that

1:54:57Alex Volkovreleases stuff with the license that's open, we applause. Uh, and this was our attempt to adjust the conversation, then breaking news came. Uh, there was a bunch of other stuff this week that we didn't get to, including the voice. So Grok Voice released, and then Tiny Fish as well. Um, let's see what else is interesting here. Um, yeah, I think it's pretty much it. we've covered most of it, besides the, the, the, oh, and multimodal JEV. Didn't get to. I really wanted to. JEV is now multimodal, the original JEV, not the 700 open source or so clones, including the one from Microsoft that's called Decision 1, including the one from OpenAI that's called OpenAI Decision API.

1:55:35Alex VolkovUh, JEV, the original JEV from TypeSafe AI that we talked to you about, what, a month ago? That's it? That's only a month since JEV came out? Yeah, pretty much literally a month ago. Um, JEV is now multimodal, so it can look at images and return bounding boxes. Uh, we also told you about Gemini Argon, which is Gemini 4, and we had both folks here, Geneviève and Omar Sanseviero, from the developer experience team at DeepMind to tell you about these models as well. Coming to you live from New York, AI Engineer, the conference, had a lot of folks who listened to ThursdAI and a lot of talk about the type of stuff that we struggle with, including attention, including how to run

1:56:14Alex Volkovmultiple agents, including when to sleep and how to sleep when you're just about to send something to an agent and it can reply and wait for you the whole night. This is also a part of the conversation as well. Uh, signing off from here. Uh, if you missed any part of the show, the show comes, to any podcast app that you use, Apple Podcasts, Spotify, et cetera, and on YouTube, including shorts. Uh, if you haven't subscribed, please like and subscribe. Um, and if you have, help us on other platforms as well. Really, really does help for exposure. Uh, Wolfram... go ahead.

1:56:50Wolfram RavenwolfI just wanted to say the news are coming so fast that it is now the time where it is the easiest to actually implement, implement the stuff. When we started with the show and I saw we were, will be talking about Grok, uh, voice, I told my agent, put this in my, my voice app, and it's now running on my phone already. I can talk to Amy with the Grok voice.

1:57:08Alex VolkovYeah.

1:57:08Wolfram RavenwolfAnd it would be cool if the audience would also, um, tell us next time, what of the last show did you implement yourself? If you have your agents, what is the stuff you are actually using? What, what about the things we reported on are you using now? This is super interesting to us, uh, not just reporting on the news, but actually the uses of the news items, what our audience is doing. That would be very interesting to hear.

1:57:33Alex VolkovYeah, please send and comment and tag us on social media. It's gonna be great. Uh, with that, we're signing off. Folks, thank you so much. Peter Gostev, Wolfram Ravenwolf, Nisten Tahiraj, LDJ, Yam Peleg was here before, and our 3 guests, Nathan Benaich, Geneviève, and Omar Sanseviero from, DeepMind, and all of you for tuning in, for listening, for sharing, for liking, for sending subscribes and sharing with your friends. It really means a lot to us, especially if you come up and tell us, that you're a listener. Thank you so much, and we'll see you here next week. Bye-bye, everyone.