GPT‑5.1’s New Brain, Grok’s 2M Context, Omnilingual ASR, and a Terminal UI That Sparks Joy

folks?

This is Alex. Welcome to ThursdAI for November 13th, 2025. Can you believe it? we had a packed show today in open source. We have terminal bench with a new bar for agent evals, Baidu released Ernie 4.5 vl. Which is a 3 billion active parameter in punching way above its weights, WEBO, the Chinese Twitter, dropping the vibe Thinker 1.5, Qwen Finetune beating deep seek on math, which is we'll see And code Arena living models build web apps live to compete. With a ton of big company news as well. GPT released 5.1. Finally, an upgrade from GPT. It has a much warmer voice and personalities, is pretty cool. And Grok four Fast. Apparently now has 2 million context window tokens, which is crazy. I used it and I showed an example of how to use it in the show as well. And Gemini Live updated voice updates in this week's buzz, we had Dima from the SDK team. Showing us their side project called LEET. It's a terminal native dashboard for your runs, even offline ones. Dima did a cool demo for us and you definitely should stay tuned for that. We also had Paul from 11 Labs talk about their scribed auto model, which they launched Scribe two live, with a cool demo of transcription generally. Very cool. Show this week with tons of AI content. Let's dive right in.

Open Source ai.

Let's get it started. right. All right. It's time for us to get started, with Open Source ai. we'll start with Terminal Bench. So we've mentioned Terminal Bench a bunch, in the past, and, as you guys know, we're looking at different, agents. that can control your computer and also solve tasks, as well. And, terminal Bench is one of the, main hard ones now for agents to be able to solve, terminal focused tasks. terminal Bench is a a second version that's much harder and It consists of 89, hard test to test these capabilities. the very cool thing about Terminal Bench is that, you can get another example of which top models are doing the stuff that AI coding agents are really, really good at. and you can see in the recent chart, this chart has been updated. I think the Warp, got a higher score here than Codex CLI Warp is the console, the agentic console, and it got 50%. So this, term venture is an agent performance, evaluation thing. each task received several hours of human and LLM assisted verification to ensure that tasks are solvable and realistic. Some of these benchmarks folks, they scrape GitHub and some of them are not solvable. Like this is the reason, if you guys remember Swyx Bench, SVE bench. then this is the reason why there's like multiple versions. This is Swyx Bench verified Swyx inch hard, Swyx inch Ultra from scale. I don't remember exact name because sometimes, these ais go get lost in the type of the questions. so the new Leader, Bo Vitamin Terminal event shows, Cotex, CLI and Warp at on top and g BT five with 49% or so. and, I think it's a good one to monitor and watch,

they also stated that they have a thousand contributors

for this, uh, on their discord, which is amazing if you think about the number of people involved. So that is a benchmark to watch. Yep.

Yep, a hundred percent.

so shout out to these folks. I also wanted to mention there is a Weave integration in there as well. it's sitting in the pr so shout out to the folks on the website who built it.

Yeah, just, If I can add, like, just to give, to give

people perspective of what exactly are the models being tested on? you can see the leaderboard here, but here you can see the actual questions, the actual challenges, one after the other. What exactly, the models are being tested on. There are plenty of them. And just, for example, like these models are running in the terminal, so these tasks are targeting, use cases that are very hard but realistic. that's the prompt I just made, some changes to my personal site and I checked out to master and I broke the entire thing. So, help me out and, you know, it's a Git problem. It can be hard, it can be tough also for models, and it's very realistic. It happens all the time. Again, Git workflow hack and, implement GPT two, code golf, with the least amount of code possible. and you are measured by, obviously it's very, what you're measured by. these are the things that those models are being tested on The thing is that these benchmarks are getting more robust, getting more questions, because over time models just become better and better And it's really cool to see initiatives like terminal Bench, taking this to the next level and saying, all right, we are going to iterate as well. Once you, solve our benchmark, we're gonna make a new one, which is harder. we're committed to this. We got a huge community, and, this is how you get models that are so good, like cloud code or Codex that are running on your computer and can actually do stuff. So really, really cool. This is good.

Thank you.

This was a great, coverage folks. Terminal Bench definitely is the one to watch because, the second version is harder and, the top models are only getting 50%. I think it's worth mentioning the top score right now on terminal bench is Warp. the agent id, that I really like, is the top score right now is 50%. We love scores that are 50%. We used to talk to you about MMLU and AI E 25 and those are like 99% situation. When we see improvements, we see improvements in 0.1 scores and that's almost effectively meaningless. So, those benchmarks are effectively, saturated, which means there's no difference between the new models. Terminal bench is 50%, humanities last exam is not gonna be last for long. But also like the top models there don't get like a crazy, crazy score yet. So those benchmarks are important. Shout out to terminal bench folks for open sourcing and working on this and the hardness as well.

Absolutely.

Yeah. it's much more meaningful to optimize the benchmark where everyone is 50% on, than to, chase, I don't know, four basis points, out of a score that is already 90 something. we really need to look into what questions exactly are those 50% errors coming from, because I dunno if we can see the exact questions. I haven't, I tried to look for them, but I didn't. But it's really interesting to see because probably it's, I guess that there are full use cases that simply are not covered, like a full chunk of, kind of the same, use cases that are not covered. and now that you know, they're mapped into a benchmark, everyone is gonna bench max this thing. And these are use cases, so it's really cool to see. But yeah, absolutely.

I think worth mentioning on this topic that we sometimes

mention the benchmark called OS world, It was recent in 2024. And, like Yam you're saying it's very important to look at the questions. Somebody actually, I think last week looked at the questions in OS world and discovered the OS world does not exist as a concept. Like there's no one score. the questions there are difficult, to the point where they're not reproducible. there's multiple versions of this OS world that people don't really report. So effectively they're claimed that this benchmark is meaningless because everybody who posts like a score, the score is not comparable to others because it's not replicable. So terminal bench, for example, is a good example of a replicable and harness like eval that will return the same score if you run the same model on the same thing, which is very important if you think about this in agentic world specifically because all these agents with terminal with temperature higher than zero, will return sometimes different results. so it's honestly an art to be able to give us reproducible scores and we give companies shit for releasing sometimes a model with no benchmarks, but imagine how much work additional to the training of the model goes into benchmarking, and releasing reliable scores, which is something that, we also deeply understand here. Weights, & Biases, and so. shout out to,

especially when it's interactive, you need to write code.

Then you run the code and you need to check the output. It's not just chat completion. You're not picking an answer. That thing is interactive and you write code and somebody needs to run the code and check your output. I understand that's just complicated on its own to execute this in a replicable way.

Yeah.

I don't know if you,

one token off, you break half of the models.

Like that's not easy to, make it fair benchmark at all.

Wolf.

I want you to chime in, in here as the

So these one shot, tests that are usually done in the

benchmark, you have a question and answer and you can verify if it achieves that. That is usually just a theoretical concept. And in the practical world, you are iterating on these things. And the same thing happens with, if you make a benchmark and it gets, used by a model that you don't have control over, then it should be considered leaked. And then you need a new version. But once you have a new version, you can't compare the scores anymore directly. If you add new questions, if you change anything at the benchmark, you can't directly compare anymore. That different and need multiple, multiple runs, like Alex, even temperature get driver issues. That stuff. I had seen issues where new driver version totally changed. So there are so many factors and you need to know all the variables. If you want to make a very useful benchmark. If you change one variable, it cannot be compared anymore. That is, you need all these details.

This is kind of an argument for using the same tested benchmarks

that people already know and love, right? Because they've been tested a while and people are comparing the same ones. OS world specifically, shout out to epoch AI for looking at this thing. You guys should definitely give them a follow. they are doing these evaluations of evaluations, meta evaluations, basically. and they're saying that, os world is 361 computer use tasks. Many of them don't require even a, UI and, the contamination risk is medium. so very interesting.

Just a quick, Harbor, since you mentioned

harbor, which is, in container. So that is also important for benchmarking. If you want to compare the benchmarks, you need to have the same drivers even on different. Same software. So having it containerized is a very important thing to make it usable, comparable across.

Harbor from Terminal Bench is their harness for running, sandbox

container, agent rollouts at scale. And I think like full performance is very important because you have to run them, on the same environment as well. 'cause even environment changes the results of the benchmark. folks, let's talk about Baidu earning 4.5 vl and apparently Ernie five LDJ. Baidu, has been around us for a while, folks. We talked about Ernie, the series of models. They started out as closed source, but somebody in BA realized that folks like Deep Seek and Qwen are leapfrogging them in the Chinese open source and decided to start releasing their models publicly. we'll start with Baidu, Ernie Vl, uh, with 3 billion parameter. It is a reasoning model with 3 billion parameters. what's interesting about this, they claim to. Outperform close models like Gemini 2.5 Pro with just three thinking parameters. again, we're looking at some of these evals, and MMMU, for example, they're claiming oof, oof this chart. Oof this chart, but, okay. So, Baidu is getting very close to G PT five. Oh, this is GP five high. Ooh, that's high reasoning. This is a 3 billion parameter model competes with GPT five high on visual stuff, folks.

wait, what?

Me? I must say, what are we measuring?

take a look.

I understand.

I agree. I believe you. Just what, exactly are we measuring where performing GP five?

Let's see, let's, we're landing at some,

some Chinese 4 0 4 number. Hopefully the hanging space is up. No, we ended

up in the,

yeah, it looks like I

what?

Yeah, let me, lemme go find this, folks, visual, like

verify that your AI assistant put actual links in your stuff. let's talk. Okay, here, here's the actual chart. Let's take a look. Yeah. so Math Vista. And math vision and MMMU, MMMU, they're pretty far away still. Okay, so like 72%, at the 3 billion parameter versus, Gini five high, GBT five high. Gini five is another thing. there's a new benchmark here that says VMs are blind. That's very interesting. And they're, they're claiming to have a bigger one. I haven't heard about this one. These are all like visual tasks, visual Q tasks, simple VQA and halluc bench. very interesting video understanding. Oh, this is a video understanding model as well. we can probably just test this model live, super quick. They have a demo. Let's see if this demo works. it's a hugging face demo. How much did GPT five I get on SR. Blind bench. this is a visual model so we can send it some stuff and see if it's fast. they have a demo, hug and face, but this is an open source model. It's a MIT license, I believe. small vision thinker that's supposed to run like super, super quick. what's interesting in there as well is, they have, highlights include thinking with images into interactive zooming spatial grounding, which is what GPT five used to have. And, this is a reasoning model. So it runs, reasoning based on the charge. GP five scored 69.6 on the L lms or blind benchmarks. Do you guys think this is true? let's take a look at this. I need the image in the new tab so I can zoom in. where are we vs. Are blind? Okay. yeah, it can reach out, with the 3 billion parameters. it got the right score on VMs are blind. GP five gets 69.6%.

let's just give it, a screenshot of, an article, a paper,

something that, you need to convert into text and ask it equation a table, something to reproduce it or something.

I think it's a visual model, so like it will tell you some stuff.

So I don't know if, this is its strength, but let's take a look.

we know about this?

I mean, how do you get 3 billion parameters to be this good on vision? Vision specifically is, I don't know, it's weird. we always get tiny models that are crazy good.

We're getting better and better at vision.

a hundred percent. there's no doubt about that. I don't think they have posted anything besides the post, but let's see what Twitter has to say. this is the thing that I wanted to highlight. Innovative thinking with images. It can zoom, analyze, and reason visually. famously GBT five, when it was launched did the same thing. They had the tools that, you know, zoom into images and kind of like look around and move them around to like have a better understanding. this has also, it's a reasoning model. So has built in tool calling for dynamic, interaction. it's open source with Apache license, I believe. Yeah, Apache two. multimodal reinforcement learning, GSPO and ICE P strategies, which I haven't heard about GSPO.

So many variants

spatial grounding for real world tasks.

Let's take a look, dude. It's really honestly impressive. I just gave it the chart for folks just listening. Just gave you the chart from a breaking news release that we're gonna talk about in a second. and it gave me a summary based on the provided charts. Here's the summary of the scores. web word or bank shows the hollow models performing significantly better than the Qwen models. holo is from the age company, the French age company just released, performed 83%. Yeah, it reads charts like crazy, like nobody's business. This chart is not like easy to read for me even on the screen.

By the way, GSPO is the new version of GRPO that was made by Qwen.

it's group sequence policy optimization. It was a paper released by the Qwen people.

Oh, so it looks like these folks already, used it, which is great.

I want to summarize the news from Hollow super quick, because shout out to Lincoln, who posted us on breaking news from Hollow two, but I haven't seen any release for Hollow two. All right, while this happens, let's hit the breaking news button because we have breaking news in open source. AI breaking news coming at you only on ThursdAI All right, folks. Breaking news from, this week from, November 13th. We just got, a brand new Spanky release from the age company. Age company does, computer use agents and visual agents, they were releasing Hello two, the next generation multimodal. Family built our grounding navigation reasoning across web, desktop, and mobile. So it's open sourced, state of the performance on screen spot Pro Apache two. Thank you. And, we were just talking about OS world, but this is a OS World G version. I don't know what this one is, but we're gonna take a look. Maybe somebody cleaned up O world. 76% s world. Let's take a look at some graphs as well. computer use performance powered by whole two surfer outperformance predecessors, as well as open source baselines, across, computer use benchmarks, web verge or web arena. The same O world that we just talked about, that's not super reliable. it's a Finetune of Qwen three VL drop in replacement for hollow one, the previous ones, mixture of experts, but this is basically the result of Qwen three. Apache two license, four B and eight B models. 30 B and 30 BA three B. So 30 BMOE is a non-commercial model. let's take a look at some stats here, OS world G, which we need to go and take a look at. hollow two.

seems to be a state of the art, on some models that stuff on screen,

yes, but on some models that are not necessarily

like it's Comp Compare. Okay. Here it compares with Quin three VL thinking, and here they just compare it to themselves in Quin vl, which this is a Finetune of Coin vl. So they're showing that they're fine tuned. Well, but, let's go back super quick to, Ernie V that we just talked about. and earning compares to major, major, foundational models like Gemini 2.5 Pro and GBT five High, which is the comparison Alright, this is it for banking news. meanwhile, let's see if our agent was able to get us a thing. and yeah, I think that we have an update. Oh,

agent maxing.

Now everything is an agent.

but dude, you saw, I just clicked the link and fucking

didn't open the right link. So I, I don't know if like it, I don't know if Agi here folks, I dunno if, is there maybe

How well does it this job?

actually, yeah, like if you build an agent, you

definitely have to evaluate this. And if you evaluate this, please use says we for this, and which I haven't. And I have a, a demo for you at the this week's bus. Hi folks. We're still at the open source and we're like, nearing the one hour into the show. So let's go and talk about Project ala, project ala, which I think since then was rebranded. oh yeah. because of,

Name shadowing.

Thank you.

Name shadowing is the exact right. reason why. this was rebranded. It's a partner with Lay On Lion on ai, which, I love that. Lion is still around. truly OpenAI. They were the foundation for a stable diffusion with their training of the model and the comparison of data sets they're doing like open source for real. this is a open source initiative to make scientific research accessible. They summarized, a hundred thousand papers, I think on archive, and this is how it looks. This is a pretty cool, like semantic embedding cloud. they fine tuned two LLMs on top of the scientific research, which we can take a look at because they also released, these models on hack and face. this is the visualizer, sign data explorer, blah, blah. We can take a look at the set. This, it just, I just love embedding, I don't know what the axis here. I just like love embedding visualization so you guys can see how cool this is. science data explorer. we can take a look at material science, for example. This is the paper for material science. what do you have to say about this topic?

I mean, it's very useful to have embeddings for entire archive.

I wanted do it myself, back in the day, not long ago. so yeah, it's very useful. But, I'm not sure that we can, understand anything too deep from these plots, like you're saying, like what are the axis? Well, it's projection to 3D kind of,

I would just say the visualization is the least

interesting part about this. they released open source, which we love. they released models with it. They call it inference models. they released L Andron 12 B and Eloqua 14 B. So this is a fine tuned model, specialist for generating instruction summaries of scientific research papers in J format. So a very specific format, for scientific, summary. I've been trying to get with my summaries and been honing my prompts. They built extra models. they created the original dataset, which is about a hundred million scrapped scientific and research articles on inference.net. and then they Finetune these models. so we saw the Live Explorer. I just wanted to highlight that. This is a very cool release from folks who, did some very good work. they show some examples, like folks, honestly, like if you tune into the Thursday, I show to understand which models to use. I don't think that these models are for you to use for anything else unless you are summarizing this. This is like a very fine tuned specific example. maybe I'm wrong, maybe these models are best, but like what they're showing is every score on their own summary. What they're seeing is GP five is the best summarizer. but their models are scoring higher than the regular models. even MA 2.5 pro, so that's great. so they released the dataset, a huge dataset, a hundred million paper corpus. they summarized all of them with a specific LLM. This is a one hell of a research effort alright folks, we are moving forward because we have to finish up, like we have five minutes left for open source. I wanted to talk about Vibe Thinker. This is Webo. WEBO is the Chinese Twitter essentially. and they have released a, I think this one is a Qwen. I think this is a Qwen, Finetune. We can take a look at the hug and face link if the link works for us. yes, this is a, 2.5 Finetune. this is Vibe Thinker, 1.5, billion Parameters. it's very impressive. Look at this, on AI E 24. I don't trust 24 anymore. AME is competition math. If you guys remember, 24 is leaked at this point to oblivion. So any company that posts a ME 24, even 25 is already leaked, with a tiny, tiny model out bidding, major models. That's a red flag for me at this point, but hey, we can take a look.

I don't know.

I just wanna say

be 1.5 billion parameter model, supposedly beating deep seek R one

from earlier this year, maybe a year ago. Well, a year ago is crazy. But folks we're looking at very, very bold claims of a Finetune of Qwen, getting higher scores on some of these things as math, which thinking models are really good at. it's not than the 661 billion parameter model than GPT OSS 20 billion. These are bold claims. wolfen, what's your, gut check on this? What's your gut say on this vibe thinker?

Yeah.

It's a small model in the end, so it has to be used for this specific, part. if you need it and that, then it makes sense. But otherwise, it's not a model that will be very popular in white circles, I'm pretty sure.

Yeah,

yeah.

but look, they are claiming, 55% of live code bench. Like apparently it can write pretty decent code and this is what we're looking at. Yeah.

Life code bench 55,

use it for your case.

That's a stretch.

I do wonder how much, 'cause it, it's a, a reasoning model they say, right?

Yeah. so I wonder how many reasoning tokens is it using? Because it does make it a bit more realistic to me if this is, let's say this is using like a million reasoning tokens or something. So like, you know, that kind of changes the equation, a bit of the inference compute because, you know, a 10 times larger model using a hundred times less reasoning tokens, you know, like the comparison starts to change a lot there. but still impressive regardless, I think no matter how many reasoning tokens it's using.

Yeah.

It doesn't look like tons of folks, on x are replying to this a positive way. folks are calling independent valuations, which is what we're saying basically as well all. So shout to way before this. it's on hug and face folks if you wanna check it out. 1.5. Is there, I think we pretty much covered open source. early five was not open source, right? Was it? No, no, no. Open source. Okay.

Oh, there is something I think we're missing.

Tell us.

so play and from France, they've released, two cool small models that

are both open source called Monet. Yeah. And Baggetto tron. I see ya. I'm nodding. He probably remembers it too. Right here. I'll put the link in the side shot. But these are really interesting Monad, or sorry, baggetto Tron is about 300 million parameters. And then Monad is a 50 million parameter transformer, and it, it's really interesting architecture here. The 300 million parameter model is about, even though it's only 300 million parameters, they made it 80 layers.

Alright.

So Mo Monad from, I think it is from Alexander Doria. I think I you're talking about synth. Right.

so both of them are trained on synth and I think in that same thread of where

you might be seeing synth mentioned. They also mentioned Monad and Baggetto Tron.

LDJ, unfortunately we have to move forward.

Yeah, go ahead. Let's move on. Yeah, yeah. we have to move forward because we have two guests on the show. we will say hi to Paul Asjes and Dima Duev, which, both of them showed up at the same time, So, we'll first talk with Paul about 11 Labs release and Dima. we'll get to you and talk about lead a little bit, in a moment. Paul, the reason why you're here, is because you work at 11 labs. You guys released a cool thing. let's chat with you a little bit about the cool thing that you guys released and maybe take a look at this.

Yes.

Hello. hi, I'm Paul. I am an engineer at 11 Labs, as stated here. Great to be here. Wonderful. See you all. Welcome, welcome. Yeah, we did a thing this week. We launched on, what was it, Tuesday, something called Scribe V two Real Time. So if you're unaware who 11 Labs are, we are an audio ai engineering company. So we do everything that is to do with sound, which is both, you know, text to speech, speech to texts, a voice agents music, sound effects, all that wonderful stuff. But, what we launched recently is earlier this year we launched something called Scribe V one, which is a stage of the art speech to text model. And then on Tuesday we've launched the thing most people have been asking for, which is a real time version of that. So what this is, is, it's super fast run, 150 milliseconds. Where you talk into a mic or you give an audio file and it'll just transcribe exactly what you're saying in over 90 different languages with a very low WER or word error rate.

Paul, first of all, thank you so much for coming.

we talked about 11 labs. folks who are familiar with 11 Labs. know 11 labs as the place to generate the voices, right? Right. As the place to do the other side of, let's say the agent, voice chat, end-to-end flow, where you basically give some text and the model just says it with different sounds. We covered the Custom Voices thing. That was like last Halloween, I believe. I customized voices with a prompt, for example, and 11 Labs. what are you guys doing in the SR world? Like how and why are you stepping into the place where Whisper is and where, you know, other companies like Yeah,

thus, excellent

question.

So the reason we're doing this, the reason where this kind of sprung

from is when you're building something that we're very interested in right now, which is like a voice ai voice agent. You generally need three components to do that. You need the text to speech, which we've kind of been doing for a while, You need the brain, which is the LLM behind it, but then you also need something to turn human speech into text that the LLM can then consume. And this is the same for every single LM out that needs this kind of component. So 11 labs is what we are. we're putting a lot of time and energy and money building our agents platform out. And a part of that is this speech to text component. So, short answer, we needed it ourselves. Yes. Because we have this and we figured that we built such a good model that we figured we should be releasing this as an API for other people to use for the own purposes.

That's so cool.

Dude. How about the demo? I played with the demo a little bit and I was like very, very excited and I honestly wanted to overlay this here. would you be able to give us that demo that I saw and this is why I invited you guys?

Yeah, absolutely.

Let me share my screen real quick.

Yes.

We'll bring it up and then we'll bring it back, the host here to, discuss this because I think it is pretty cool. and also, you mentioned the amount of languages that it supports, so we definitely can look at the, at the translation component there as well.

So hopefully you can, see my screen right now.

Yes,

Excellent.

Fantastic. So here is a demo we put together. My colleague Thor did this, actually, what this is, is, it's called Captions at Events, and the idea was that you could have something like this for a presentation or a conference. Where you could have the speaker, be live transcribed as they are giving a presentation So let me start this off.

This is a real website, folks can access this at caption events

So you'll notice that as I'm talking, the, words are

appearing at the bottom over there. and what it's doing here is that it's going to detect the language automatically. And you see it's showing up on the right hand side as well. So like in viewer mode, this is gonna show up very fast. but what's fun about this model is that it's context aware as well. So not only is it super accurate, but it can actually do different languages. So, for instance, I'm from the Netherlands, so I can actually speak in Dutch and it will start to transcribe in Dutch as well.

This is so cool.

so just super quick for folks who are listening the Powell was talking, basically it live transcribed everything he was saying, with real, real time. And then he switched into Dutch and then we saw another section, that started just with translation and it auto switched and he didn't have to press any buttons to tell it what to do. It is basically we're saying like, Hey, the transcription should be in English, and it looks like your computer doesn't capture what I'm saying, obviously, 'cause of, you know, you're tuning into the show. but it would work for multiple speakers as well. I believe. when I was playing with this, I had Spotify on and then it transcribed the whole song, which is really cool.

Yep.

right now I've got echo cancellation on, so that's why it's not picking up you over my speakers, but it works pretty well for me speaking into my old microphone.

Yeah, you can see this is really cool and

this is like nearly real time. So this, is able to get streamed into an LLM as I'm speaking, I'm assuming, so that this agent that I'm building will be able to also reply to me with, with, 11 labs. this was built also with, super base realtime. Shout out to super base, friends, friends of the part for sure. Paul, thank so much for the demo. translation also is something that you did like one, like one way translation when you started speaking a different language, it picked up. Mm-hmm. how are you going guys comparing to something like Whispers for example, which is, you know, has been in this world of a SR for a while and it's, was pretty dominant up until a moment ago.

So the, I'd say the, what we think, I think I do, we do better than

Whisper is that we support more languages. It's over 90 different languages, which are much lower word error rate as well. So you can throw a lot of different, like, very complex terms at it. When I was testing it, I was like talking to it about code and it managed to pick up things like Initialisms and actual code as well, and was able to transcribe those in a very realistic way, which is great. And also it's very fast. It's, the average latency is 150 milliseconds. Oh wow. I've seen it go lower than that. It kind of depends as well, what language you're speaking. But for the English model, which is one that kinda like the flagship, language that we used, I've seen it go down to as far as like, you know, 80, 90, a hundred milliseconds before they get a response. So it's super fast, which is really fun.

Yep.

This is awesome. and first of all, congrats on the release. we love SR here on Thursday for some reason some of the hosts have been playing with SR for a while. And like every new release and excitement, we definitely wanna cover this, but also like real time. I think Whisper is not real time at all. you have to wait for whisper to process. You have to put it up. this is really, really fast. and then looks like you're bidding competitors like deep in assembly and from a company that doesn't necessarily do only this. It's really, really super cool. Paul, thank you so much for joining us. if folks wanna find out more about this or play with the demo, you wanna shout out some of the folks. Feel free. This is your kind of like, your

Level Labs io will be the place to go.

You can actually play around with Scribe on the site there as well of little demo. shout out, I guess just our devs Twitter accounts would be good one to follow. Just Level Labs at Level Labs devs on x.

Awesome.

So we'll definitely add those to the show notes. Paul, thank you so much for joining us. As we move on, to this week's buzz. Folks for this week for folks who are, joining the show, anew. we have a corner. So obviously I work at Ways and Buzz, that's now part of CoreWeave. And we have a, weekly corner here that's called This Week's Buzz, where we talk about everything, Weights, &, Biases for this corner. I'm not alone this time because, somebody who built a cool thing in Weights, & Biases, is joining us. Dima. So let's go to this week's buzz. Super quick.

Alrighty. In this weeks buzz, I'm joined by Dima Duev from the SDK team here at W and B Dima. How you doing?

Hey, what's up Alex?

Good to see you, man.

Good to see you as well.

I think you shocked the company today in the all hands when you presented something, super, super cool. We posted about this, but I wanted to, call on you to actually do the Cook. Cool. Little quick demo on this week's buzz here on ThursdAI, would you tell us, first of all, feel free to introduce yourself and what you do on WNB and then, give us a little demo of lead.

Sure.

Let's do that. I'm an engineer at Weights, & Biases. So I've been with, wandb for almost four years. I'm on the SDK team, you know, import wandb, that's us. so you are friendly wandb SDK, developers. from time to time you should do something fun, right? Something to spark joy. So this is, this is pretty much exactly what this is about. This is a little, side project, that we did over the past couple months. So I'm pretty excited to share with you guys. Yeah. So the problem statement, okay, so imagine you're a, you're building your foundational model, but the problem is that it's, running on an air gap, HPC cluster. So no internet man. And in addition to this, as an introvert, you, you know, you get, naturally anxious, when, leaving your terminal. So what you do, okay, behold, lightweight experiment, exploration tool or lead for sure. for short, wandb Elite.

We shall be able to see it.

Okay, so let's start this, fake, training experiment.

Offline. Okay. I promise. No internet. No internet, right? So

right now you have a training script running with,

no, it's not logging to YB at all.

logging.

It says that it's okay. It's in, offline mode and the data is, saved locally. So what you do, and you don't wanna leave your terminal, okay? You go into the, your other terminal wandb beta. Let's try and see if it works.

There we go.

It does work.

Okay, wonderful.

So, like I said, the main purpose of this tool is to spark joy right here in your terminal. But, it also provides access to real time, offline. Metrics for your run, again, delivered straight to your terminal. You could see your A system metrics here, and this whole thing is interactive. you see basically the stuff you would normally see in your,

before we get to features, for folks who are just listening,

Some folks are saying in the car they can see this, Dima, after you run the training round script, you're now seeing a terminal UI basically with no web browser or nothing. You're just like, run the command wandb beta leet it is gonna be out of beta soon. And what you're seeing is a full visualization of all your runs and metrics, very similar to how Weights biases kind graph works. Obviously we do a bunch of other stuff, but this is fully, completely offline. if you are in the air gap, HPC cluster, if you're training a very secure machine learning model somewhere, this is possible to you even now.

First of all, it's just, beautiful osky art.

Who doesn't like husky arts? there's a whole bunch of stuff here. yeah. So you would see your run review, you know the information about your environment config summary. You would see the main metrics screen that you could. configure, like if you don't like the grid, layout, you could change that. and you would see your system metrics. And then in this example, I am logging something like not too many metrics, like a thousand. And then you could of course, scroll through them real quick, but you're not expecting me to scroll through all this. So you can go ahead and filter them and for example, look at only your train metrics. and then you might be wondering like, what's this weird spike on this, train slash loss, plot. And then you could just zoom into it with your, with your mouse. And that's so cool. This, this spike looks suspicious. We should probably do something about that. Yeah. So, it is available, it is available now in the latest, 1D BSDK version, zero point 23.0. As always, you know, stay tuned for more and I just want to give a shout out to the, to the entire SDK team.

we, we also want to give a shout out to the whole entire team

and UD Dima for, not only building cool things like this and coming to Thursday, I talk about this, but also maintaining one of the, if not the most popular machine learning, experimentation framework out there. wandb which everybody knows and loves. So, shout out. Dimo, thank you so much for joining us and showing us a demo. Folks, again, this is one B lead, Dimo. What does lead stand for? Again?

Lightweight experiment, exploration tool, lightweight

experimentation, tool person for 1, 3, 7.

One B, lead.

So TUI in your terminal, folks were asking, Jeff was asking, in the comments, how is it offline if Dima is broadcasting it to us Over the internet, we are simulating an offline use of wandb, but in Dima, in your. Scripts case, you don't send it to Weights. & Biases, you are logging to a file and in fact, you don't have

a need an A wandb account actually.

So if you turn it to offline mode, you could just

don't tell them.

but yeah, you don't need to. diona, thank you so much for joining us. We're gonna continue with the show, but folks, if you are training machine learning models and you like T UIs, this is now available to you. wandb beta lead. This is the actual command. One B Better lead. We have some, documentation about this we're gonna list as well. And, if you've used one B and you like the SDK, Dima and SDK team are the folks to thank. So thank you Dima for joining us. it's been great to see the cool stuff that you release. All right folks, we're gonna continue. I'm gonna bring back some folks here. Dima, thank you so much. We'll take you off so you can continue, supporting the SDK and say hi to SDK team. we're gonna continue talking about big companies APIs That was cool.

That was cool.

I agree.

I have it open here in case Dima couldn't make it. and I played with it and it really is like, it really, really is nice,

really searching for it right now.

I don't find it on GitHub, so it'll be useful if you just post the link somewhere because I can't find it and it's really cool.

So I'll add this to the banner here.

All you need is the latest way eights and biases version. So people install dash U one B and then all you need is wandb beta leet. This is the command. And then after we remove, do some better, it's gonna be just one B lead. you run this in any folder that has Weights, & Biases running, and then it will just work. and it'll look super, super cool. I think that like screenshots of loss rates and spikes are like very viral for some reason on Twitter. I think we can move on from this week's buzz. While welcome ing as well, we have a bunch of stuff to talk about, including, big companies in APIs. So we covered voice and audio. Lemme just finish the voice and audio part, and then we'll move to big companies and APIs because invoice and audio, there's a huge release from this week also from Meta. And folks, I think that we talked to you about this multiple times. this is actually not meta, this is Facebook. Do you guys remember Facebook? which rebranded to matter to include the Instagram whatever. but Facebook, I think that actually, like the blog is on Facebook or something, or somewhere like on hugging Face or something. This was like a Facebook pin. Basically they're on the mission to remove language barriers from the world. And hey, we've been doing the world and have been doing great, Work at that for a while. And this week they released something called lingual, SR. This is a speech recognition, we just talked about 11 Scribe, which is an a SR model as well. Like this is a speech recognition real time. they support 90 plus languages, lingual, whisper. The one that they compare to also supports around 99 languages or so. Lingual SR is a speech recognition. They released a bunch of models as well. it supports 1600 plus languages, 500 of them never before served in any type of a SR. So they're like low research languages, significant low s languages. they have a character error rate of less than 10% for over 78 of their support languages. the lower the character error rate, the better transcription quality is. they released a bunch of models with this. some of these models are actually thinker models, what is important to understand about the Sr. Someone like Whisper has no intelligence at all. When Open Air is Whisper is a concept. This was not an LM, this was like a standard machine learning regression, whatever model, that knew nothing about what it transcribed. this, they released a bunch of fine tunes for their lingual, a SR that are actually smart and they are using the smartness to kind of like improve significantly the, the model, performance. So we can take a look, at this and, and they released all this in Apache two. So Huge shout out to everybody on the meta team for releasing the first of all, the cool ass demo. the full data set. They have released a bunch of, a bunch of transcribed, they paid a lot of people to go to these like remote places and transcribed, talk to these people, transcribed it, and they posted a dataset about this. the dataset, I believe is called Integral SR Corpus and it's 500,000. This is almost a terabyte of transcription, 500,000 rows, 550,000 rows of just like language and audio. So, we will not be able to hear, I believe. Gimme a sec. Actually, there's a new feature where you can hear if I just share a screen, which I wanna try, but folks, you may hear yourself, so hold on please. But do let me know, maybe in chat if this is working,

I don't think.

No, I think it's

really impressive.

Yes.

and then you see the, the protex. So this is like a huge data set, which is also a, honestly a huge release. But besides the dataset, they also released a bunch of models and this beautiful visualization, globe where you can see a comparison of the 600 languages that we have. Okay. I'll just go and show you the, the thing myself. they have, where's that demo? I have the school as demo and it's queued up, This is like a visual comparison of the language support in something like Open Air Whisper and Omni a SR. Visually you can see like, the crazy thing I wanted to cover this last thing in context learning, they talked a lot about training these models to, basically one shot understand languages based on provided information. So without training the model and they're showing this insane graph where they can go all the way up to 500, 4,000 supported language. I did not know there's so many languages existed in the world right now. where because it's an l and inspired system, it brings in context learning into the whole fold. So you can like, give it the context of like a diary or something like this, of a language that I didn't see before and will be very good, in context learner because these LMS are getting significantly better. and I think that this is pretty much it. Everything is Apache two, language exploration demo. This is the one that I wanted to show you.

We got models and we got demos.

You can try it.

Yep.

Live actually.

Okay. It's very hard to visualize the table from hugging face.

Yeah.

Okay.

So, we got, a fleet of models, from 300 m to, one B, seven B, and all the way to, the seven b, zero shot, which is, I assume the one that can do, in context learning and get, a bunch of examples for a new language it never saw and generates, and do speech recognition. just wanna say, we spur large, the large one is, 1.5, billion. So, and it's absolutely usable. in real life, people might, I mean, the obvious thing people are gonna say about this is that, okay, seven B model for speech recognition is quite a lot, but okay, it's gonna be good. But, if you compare the one B model is this phenomenal state of the art on its own, basically, nearly a drop in replacement is a having face model. It's built on ec, you could just use it, probably you can easily plug it into, faster whisper or other, supporting frameworks. And use it for picture recognition. Absolutely. Amazing. Yeah. Seriously. Absolutely. Be amazing.

This is a huge release, from meta and we

definitely wanted to cover this. All right, we're covering, we're finishing up the voice and audio and it looks like we're going back to open source. LDJ just came back in with some breaking news. LDJ, you wanna cover this after the opener? AI breaking news coming at you only on Thursday. we are actually moving towards the big companies in APIs segment, which we wanted to get to anyway. Alright, LDJ?

Yes.

So Qwen just released and I put this link in the side chat. it's a new deep research model. They had one a few months ago, but this seems significantly better. I'm looking to see if they have comparisons posted against other popular deep research models. overall looks good just in terms of search depth, average report length. Researching for that alone seems about doubled along with. How much it's able to do with that time. Really good.

Yeah, and so the comparison points would be open the

eyes deep research, and and Google. Google definitely does deep research and honestly, for folks who've been doing deep research, less and less folks are using this because these thinking models are now doing a lot of tool calling sequentially as well. So this is essentially what deep research is. Some deep research, uses different strategies for like even deeper stuff. so we can definitely test it out on the show. let's take a look. Gwen has been doing deep research for a while. Let me, lemme bring back. Yeah,

apparently they have a faster version too.

So they have a normal mode that is, that they say efficient and versatile for most needs. Than an advanced mode for the deep research.

research, yeah.

Even deeper research. Right. We are now logged in and we'll take a look together, I believe. okay, so now I need to click the actual deep research link from them, which is here. They have an app as well. Where is this happening? they have a Qwen app for mobile, only an Android, and, desktop has a Mac s as well. We're gonna take a look. Shout to Qwen on this release. so I want deep research. Where is it? Now that I logged in, I don't see it anymore. Ah, deep research advance. Okay. I mean, give me a research topic to, to take a look at folks. I'm gonna go to Hong Kong for a, for a week, and you help me plan an itinerary departing from Beijing.

to be clear, this is not an open source model.

can this platform work, and do deep research on data? I don't know, like data that I upload or it just plugs into the internet?

I think you can upload data as well.

there doesn't seem to be any connectors here for deposit or Gmail or something like this. next week. So it does ask me the same kind of follow up questions, very similar to J GTT research. And then, this looks like based on Quin three Max. the thing is deep research takes a while to run. but we can see the research planning stage and we can take a look at some of these thinking. and, we're clicking the

possible connectors in the chat.

I looked and there looks like there's upload.

so basically we have the detailed site pin here and, it asked me when do I want to go? I said next week. And it like it already in the thinking process. new that today's week is this one. So like it figure out the next week. It's pretty cool. and, it asked me about budget and stuff. All right, so we're gonna have it run because deep research usually is not, a very good visual demo. but we'll move on to other big companies in API news because we're running at a show and we have a bunch of stuff to talk about. all right. I wanna cover GR four fast, super quick. gr imagine and non banana is something else, but Grok four fast. They keep talking about grok getting improving and improving and it's really interesting and I don't like it, not the model itself. but you guys know, I go on, some rage, the rage, conversations here about grok. I don't think that a foundational model should upgrade a checkpoint at the, as the same number, like I don't think the ROC four should improve. Graph four should stay fucking graph four. And if they wanna improve in recent, another version like graph four, 4.1, which can compare between the two. This is how most of the other foundation labs are doing. This is how I think it should work. Why? Because then we can actually quantify the difference. What happens now is, they literally just improve graph four, with a fast version of Gok four fast, but then they improved gok four fast with 2 million contact window, which is great. Folks listening to the show, it's great. Thank you so much for this work. It's really impressive to have, support 2 million, tokens in the contact window. You're one of the few to do it. Even Google stopped. Gemini was supporting 2 million, contacts window for Gemini flash, light whatever, 2.5. And then for 2.5, they stopped and they supported for two. it's great, but. Stop updating the model without giving it a new version. We cannot evaluate this against each other. We cannot know if it's better or worse or different things. this is what happens there. Elon Musk says something about, Hey, update your Grok app weekly. 'cause it gets like daily updates. but Grok Forecast now has 2 million contacts. It's ridiculously fast. It's really ridiculously fast. It's really, really good. the thing that I wanted to show before, which is when somebody commented that Alex is like Agent maxing, is I have plugged in GR four Fast here into my workflow to prepare notes for the show. And now I have GR four, research agent. The thing I wanted to call out specifically is that Grog four Fast has a X AI search ability that is not matched by anyone else, right? So because of the incestuous relationship between X and X ai, graph four, which is X AI's product, can search X with semantic search, which you cannot as a user, you just can't, like, it doesn't work for you. The search does not give you disability. It can search top quotes. Which also API does not give you, you can do it in the UI I found out recently, but via the standard API to XAPI that you have to pay for, you cannot get the top quotes, you cannot get semantics. So I wanted to actually like show that GR four is not only, and the reason I'm calling this out is because we mentioned this before where grog.com can do this, but on the API, it wasn't possible the API was not connected to X. But now it is, and the things that it does is stuff like I wanna show you. so this is an example of the H company, tweet that I sent. we sent it to Grok with reasoning, and we told it, Hey, basically, if you wanna see my prompt super quick, it's kind of like this. it's a big prompt, but it's fine. you're a research intelligence, ex searcher. You have access to these things. Here is the summary of the tweet made by a user, and here is, the settings. You do x search from day to data, basically saying, from the data of the tweet until today, and then enable, enable image understanding. So Greg, not only can it search, it can look at the images in the post and understand them. It can also do video as well. and then it returns something like this, which is a, which is a, example of the tools that it ran, it ran this query. And then it ran, the, view image query. So like it actually looked at this image to understand it and looked at another image to understand it. and then I did a keyword search where it took this id, and then it gave me the top quote tweets, for example. and then it did a semantic search, something that you, again, as a person cannot do. All of this happened, one after another, and it did a bunch of other stuff as well. and then it looked for reactions. This is insane. It gave me like a deep, deep research with all these links. All of these folks are basically talking about, H Company. Let's take a look. if this is correct, sometimes it fails, but most of the time it's correct, like it's, again, it's semantic search and then gives you like a very deep researched, article based on, x data. So graph four fast, really worth playing with. if you're playing it with API, it can access web search as well as ex search. So we can do like a bunch of, default research as well. and I have implemented this into our research, flow. and it's very cheap as well. I think it's, significantly, low. So this is an example of how much it costs to run GR four on artificial analysis Intelligence index. GR four Fest takes around $32 comparable to where some like deep seek and Mistral medium. this is the reasoning one, and the performance on the index is around, higher than Dipsy and Cloud four Opus. So we don't often shout at Glock and they don't often release and tell us the, the new stuff, but it's really, really worth it with 2 million contacts as well. all righty, moving forward, the main release of this week, folks, open the eye released GPT 5.1, the first. Dot release since the, the major release of GBT five. this one has warmer conversational defaults. they have instant and thinking modes. So adaptive instant and thinking modes. so GBT five Instant can basically like re reply super, super fast and I think present, preset personalities. Have you guys played with the hit? What's your first reaction to GPD 5.1? Let's start with Wolf.

Yeah, it's actually my most disappointing release of

the week and in a long time I tested it, just my write checking so far, but it's much worse in German. It's no way really. It's terrible. So it's a big degradation. I'm back to sonnet 4.5 now is my main model, or GBT, four, I still would use that before I use 5.10 and disappointed, unfortunately, the German language understanding, it's terrible. It even removed the ums from my text because I had something about don't use topographical special characters. Yeah, like dashes I had said in my prompt and it even removed the, so I was really, really disappointed about this. Unfortunately. So I will test it some more. Maybe it feels like a Quantas model from China actually. That's my feeling right now. I would use Kimmy before I use this.

And which one did you use?

Did you use Instant or did you use the thing you want?

it was mostly with instant, so I had

a direct comparison to fall. I will do some more testing with the thinking model, but the instant, it felt like a flash model from Google or something.

Interesting.

Smaller one.

What are your comments?

Have you played with G BT 5.1 at all?

I must say that, first, first I think that it's a absolute

push in the right direction. GPT five, is a great model and gets stuff done a hundred percent, but it can drive you crazy by the amount of it not being, communicated. when you charge, you just, let it do stuff, and have a, conversation with it. You really feel it in Codex where sometimes you ask a question, like, what do you think about this and that approach. And instead of answering the question, the guy just goes and start doing the thing Like, it's not, it's not, it's not communicating enough. And so I really do think that it's a really pushing in the right direction and, you know, everyone knows what the public opinion was and the GPT five release, even though it's a great model. There is something about GBT four. I personally, just my, my use cases are split between, if I wanna get something I know, scientific done, I go for GPT five, usually g BT five thinking. if I wanna get, I know something, more phrasing text or, or help me write it, I, you know, more of my idea into coherent text I can send to somebody. I just immediately go to 4.1, because it's really good. And, dash? Oh yeah. Oh yeah, absolutely. We have a, we

have a comment from, Milo, in the comments that says, it's

good to see that it's m dashing in German as well as we showed on stage.

now the new model misinterpreted the prompt

and remove the ods as well.

don't you think it's a prompting thing that you have to readjust

the prompting to a new model as well,

Yeah, sure.

I have to check now, but the prompt works in the other models. The same one works in Gemma and in, Germany. And it works in, sonnet and everywhere else. Only here. It doesn't work.

Yeah, so, a few of the actual things we know,

so we don't know evals open. I did not release any evals about this model at all. Didn't compare anything.

That's odd.

It was a real weird release to that extent.

They did say, for example, that they've added these, personality dropdowns now. So you could have done this before with your own kind of customization prompts, but now we have something like, the default one, professional, friendly, candid, quirky, efficient, nerdy and cynical

on the setting the entire time.

I don't know if this specific dropdown, but the personality thing, which is completely isolated from your system. Prompt was on the internal setting for, for a while, and I did try to play with it I even tweeted once that, why don't you let me add a custom one? Like Yeah. All the others had, I mean, that's a good push in the right direction, but just let me customize it. okay. So we have it open in the, in the dropdown, and we have more personalities. Dude,

I played with them and I don't see the difference, honestly.

Like maybe it needs like a new test or something. Like, I honestly don't see the difference. I keep it on default. Maybe my, my customer instructions are here. I remember customer instructions were interfering with the advanced, voice mode. So like, I kind of like, don't really like, get excited about them. but yeah, we need to, we need to test this more. they also mentioned that it's great that creative writing and for this, we actually have a confirmation that this model was,

mm-hmm.

first on EQ bench. Absolutely. Eq. Hundred percent.

So, EQ Bench from our friend San Peach.

this, like checks.

Yeah, I saw that as well.

Clots on it. 4.5. I love this model. I'm not a fan of philanthropic, but this model, it's still my number one and Gemini 2.5 is behind it. And Kimmi is great as well, but not German as

So this model is confirmed to have been tested

on open neuro Polaris Alpha. This is the model. Mm-hmm. it's slightly better at different things like coding. It's not necessarily bigger, but for, the EL score for, creative writing. it's around the top one in creative writing and I did notice this and we'll see today. I'll test it out for some of the summaries for ThursdAI, and we'll take a look. he has also a SLAP score, and you can see that Gemma is the highest slap LLM out there. and you can analyze your text, your own text. So you can see if, G four is getting, more slot

if you are in LLM.

what else do we have as reactions from the community for OpenAI?

Lets take a look. they made it easier to customize. I think that this chart, we should mention this. they did release this thing. GT 5.1 spends less time on easy tests and more time on hard task. And here's an example of G BT five standard versus GT five one standard. And you can see the 10th percentile of tasks it, which is like the very low, like, hey. How are you? For example, it's not a task, but there's like a very, very, like low example, GBT 5.1 spends 57% less tokens on answering this while on 90th percent of difficult tasks, GBT 5.1 spends 71% more tokens. This is quite a big difference. Now, we don't know how they measure this, and we always give them some shit for evals. But if this is true wolf, I think that this is a very big difference for this model. because this is what we want to thinking model to do. We want to like give you a lot of reason, when it's difficult and react instantly. quick X enrichment summary from Fiji cmo, A warmer, more conversational personality. So essentially I think 4.0 was one of the models that when Open the Eye decided to remove. people got really upset. And I think open ice since then has been chasing the lightning to like, bring people the same kind of warm personality. I haven't seen this yet. I will say, I think they've been testing this inside Atlas for a while because I've noticed for the past week while chatting with Atlas, specifically the Atlas response faster and better for me than regular g BT five. And we know that they're a ab tasting like crazy sometimes You get this model, that model, especially in auto, and I, I notice myself like just opening Atlas and just chatting with the G GPT there versus the regular app. So, folks play around with this, probably gonna be a significant update in some of your stuff.

I wanna get a Mac just because of the way you speak about ala.

Seriously, sir, I wanna get Mac just because this man

Hmm.

Comet is pretty much the best agent browser there yet. Okay. let's talk about SIMA two. SIMA two is really, really interesting. CMA is Google deepminds release today of, a AI agent for 3D virtual worlds Gemini powered for the first time. This, they go beyond the basic instructions. They actually do perform task in the world. I wanna show this because it's super cool, and we've talked about this a little bit before. They're showing, game engines like, no Man Sky, this goat simulator thing. And SEMA is playing those games and it presents about the visual stuff in those worlds. They call them embodied agents. And supposedly this is like a great playground for robotics. so this shows like a bunch of stuff. Minecraft. Everybody plays Minecraft. but everybody who plays Minecraft, they take screenshots of the screen. here's an example. a user says to this model, find the object drawn in the sketch and jump on top of it. And they sketch out like a spaceship. And then Seima says, okay, I see the crash ship. I will jump on top of this. this is another example of the reasoning part. They basically took Gemini and baked it into sema. The, the agent that is able to play worlds. And, this is another example. They said, go to the house called like a rip tomato. And CIMA says, okay, I see the house. It looks like a tomato looking going there now. And it goes to the red house. the reasoning part of this is like very Gemini powered. So user wants me to go to the tomato house, I'm gonna go to the tomato house. And this is, it's hard to overstate how interesting this is when it fully understands the games.

how is it connected to the real time thing that controls the actual game?

Sound sounds cool, but

yeah.

How, something called like Chatted touches

the actuators in real time. It can't be, I don't think it's a chat model.

it supports chat, but I think they have controls that are connected,

like mimicking a game controller. I'm pretty sure that this is how it works, because it knows how to control this game controller as well. so it was trying to achieve high level goals where you can just give it the general understanding of the goal of the game and it will go and control this. here's the example of the self improvement part. It supposedly, also learns from mistakes as well. actually Google showed us this a little bit. So the agent, keyboard and mouse, not a controller, at least based on this graphic. it goes, it tries to play the game. there's another Gemini agent that is the task setter, so basically gives it out the task. And then they have a reward model figuring out whether or not they did the task correctly. So they have like a self-improving system, which they're supporting. You know, supposing that this is like another step towards a GI, type thing. this game engine is playing games for real. And the cool thing is, if you guys remember Genie three, which is the world model from, Google This and Genie Three together means that Google can generate virtual worlds and then have agents run and play in these virtual worlds for minutes on a time, if not more. And then learn from this and then keep generating the world. it's pretty crazy. We're obviously still waiting for Genie three and to be able to, to play with this. I haven't, I haven't yet. really excited. But here's an example of it playing a virtual world generated in real time. We'll take a look at this for a second. but again, I just wanna tell you like what's going on in this, in this video here. This is an AI agent playing in a world which it understands visually that was generated by another ai. This is like absolutely insane what we're getting, at this point. complete fabrication. None of this ever happened. This is an AI agent that play the world that was generated. No 3D Engine was involved in what we're seeing besides the learning part. Isn't this crazy folks? Isn't this like absolutely, mind blowing that it creates games on the fly. I'm just absolutely shocked by this.

It is.

Look, It is cool. I'm just, I mean, I really like the part where you have both reasoning running through, an LL, which is smart, a hundred percent and can plan and so on and reason and, find objects and so on. But something like it needs to connect somehow to something and presses the button. Just for example, there was another paper, the first paper I saw that tries to do something like this was, on GPT four, the first one, called Voyager. What they did is play Minecraft with GPT four, and it was really impressive. The way they did it is GPT four writes, small snippets of code. Yeah. And the code is the one that is actually like, it's automated code that touches like presses the button, move forward and so on. And it makes sense because g PT four then generates the code. But here you're talking real time and then on simulated environment. I get, if you close somehow the loop, You can simulate environment that, I don't know, simulate robots and so on and train robots and then, and so it, it's crazy. Don't release big stuff, guys. Like you did something cool. tell us how cool it is We don't know the details, like, how do you did this? Yeah. Why don't you just tell us? That's really, really cool. Seriously.

So we have a technical report from coming soon.

It's not available yet. Google. This is an announcement from Google, like a research announcement. They have an agent that plays games and it plays them well with visual understanding. not only screenshots but like they actually understand what it does. we are moving towards, the end of our show. We have some stuff to mention in ai. We mentioned Coursera raises $2.8 billion, insane amount, just absolutely insane amount. just, I don't dunno how it makes sense for 2.3, $2.3 billion at 29.3 billion valuation course of the. Vs. Code clone that is now used by pretty much every major lab, including us. Yeah, we also use Coursera. and then I wanna talk about the image edit. I wanna try those things, live. Wolfram, would you wanna help me with this, Qwen image edit? Have you guys seen this? so basically we're done with big companies in the LMS and APIs. Google also has like an AI shopping update. The updates that we're waiting from Google, are the, the three variety, and we still haven't seen them announce anything. Three, we were waiting for today, but hopefully very, very soon. so we're moving on from big companies, APIs. We have about five minutes to cover the no video. a bunch of thirsty eyes no video updates. major ones this week, not even from Sora. but the main update came from Qwen Image Edit. It's a multi-angle. Lauras that add. super cool le let's go here. so to Leno from hugging face, you can upload an image and we'll upload an image of, Wolfram what you wanna be.

but spaceship.

And turn around me.

so we'll take Wolf's picture here and then we'll drop it

in the thing here, and we'll see. We have wolf room and vertical angle. we're gonna do like a Okay. So we're showing this is the actual wolf room and this is wolf room from below the angle. And we can, this is vertical angle. We can switch it back up this, in this run super quick. And then you can see Wolfram from the top. now we're showing you a coined image with Lores, and you can see that we, we see Wolfram from the top. it correctly guessed, how he would look from actually, I don't know the, the headphones probably not quite here, but I think the cool thing is that we'll see like interpolation video Now, I don't know why I decided to give you like a camera that you pull out out of the blue.

Turn the camera

to a bird's eye view and it took it way.

using Cling behind the scenes is a first and last frame, and

so it gave a first and last frame. Okay. So I have this picture of my cat from this side. And then let's also generate the bird's eye view of this cat. You can see that this is the same kinda steps and from the, from above. And then we're gonna generate the video interpolation. so folks who are just listening again, this is a Qwen image. Edit the new, Laura Fine Tunes for Qwen image edit. And, hugging face has released a bunch of demos and you can use these Lauras to change the camera control and you can also rotate a little bit. So I actually rotate a little bit. so you can probably see the cat from like another angle like this. And I think it's really, really cool for, different reasons. And we can see the video. Look at this. It is beautiful, only like a short video, but still it generates like a little 3D scene type thing and it looks pretty cool. All right, so this is quite image area. You can find it on fowl as well. I think The folks from Foul added all the Lores, so they have nine specialized lores for multiple angles. Next scene and light restoration, camera control and Porter Generation Remove elements, add backgrounds, a bunch of other stuff. remove Element works pretty cool. different remove lighting. The last thing we have to talk about, is Nvidia versus Chrono edit. K Chrono edit is an upscale, is the model that gives you a more beautiful version of your model. This is how it looks. so here's an example I just wanted to show you chron edit is a creative upscale or creative upscale. It means it adds details to the images that you see. and it can do a bunch of stuff like add image. It can turn stick figures into anime as well. change people apparently. but Nvidia is releasing. cool new things and shout out to Nvidia as well. Folks. I think this is it. This has been the show. It's a very interesting week in AI for sure. GBT four, 5.1. I need to keep up with my, numbers is, probably the biggest highlight we had. we had folks, on the show, from 11 Labs. Paul has just joined us to talk about, the real time scribe version of, 11 Labs, which is their a SR model. we had ddu from this DK team talk about, one B lead, obviously with you, Yam, Peleg and Wolfram, and Alex Volkov. We also at Nisten and the LDJ. next week is gonna be a live show from AI engineer called in New York, which I need to get ready for. so we'll see a bunch of stuff. Probably Ryan Carson was gonna join the livestream there, maybe Swix. if you have any questions for any folks in the AI engineer code, there's a bunch of, all of the major labs will be there. We'll try to find some cool guests for the show as well. thank you so much for joining us. If you have missed any part of the show, the show is turned into a podcast and newsletter under ThursdAI news. Feel free to subscribe, share with your friends, and, give us five star ratings on wherever you get your podcast. With this, thank you so much for joining this week's ThursdAI, I hope to see you next week. Bye-bye everyone. Cheers.