Anthropic

NVIDIA and Super Safe: Redefining AI Collaboration

11 min
Jul 28, 202626 days ago
Listen to Episode
Summary

This episode covers major developments in AI infrastructure and safety, including Safe Super Intelligence's multi-billion dollar NVIDIA partnership, new AI benchmarking standards for code generation, Google's expanding AI Overviews dominance in search, and industry alliances forming around open-source AI security tools.

Insights
  • Safe Super Intelligence is raising massive capital ($7B total) without shipping products, betting on long-term AI alignment research over near-term commercialization—a strategy that signals either visionary thinking or speculative bubble behavior
  • AI models have advanced to the point where code generation benchmarks now measure speed and cost of recreating entire production software, with Claude Opus 4.7 achieving 68% perfect re-implementation rates across 25 different programs
  • Google AI Overviews appearing in 43% of searches (up from 28% year-over-year) is fundamentally reshaping web traffic patterns, with publishers losing referral traffic as users consume AI-summarized answers instead of clicking through
  • The Open Secure AI Alliance (NVIDIA, Microsoft, IBM) is directly opposing OpenAI and Anthropic's calls for AI restrictions, revealing a strategic divide where leading model providers want guardrails while competitors advocate for open-source alternatives
  • Enigma's crowdsourced robot control model demonstrates a new data collection paradigm where users remotely operate robots to generate training data, suggesting human-in-the-loop approaches may be more critical than model capability alone
Trends
AI infrastructure consolidation around compute leaders (NVIDIA, Google Cloud) as foundational research labs secure exclusive early access to next-gen GPU platformsShift from product-market fit metrics to research-driven funding models, where investor confidence in founders and long-term vision outweighs revenue or user tractionSearch engine transformation from link aggregators to answer engines, with AI-generated summaries replacing click-through traffic as the primary user interactionEmergence of open-source AI security tools as competitive response to closed-model guardrails, driven by real-world breach scenarios where proprietary safety measures blocked defensive actionsHuman-in-the-loop robotics and AI control becoming critical data source, with crowdsourced input methods (voice, text, video) being tested to improve real-world AI agent performanceCode generation benchmarking evolving from synthetic tasks to real production software recreation, establishing new performance standards for enterprise AI adoptionRegulatory divergence between leading AI labs (advocating restrictions) and open-source advocates (opposing restrictions), creating policy uncertainty and competitive positioning battles
Companies
Safe Super Intelligence
Raised $7B including multi-billion dollar NVIDIA deal for 10x compute increase to advance AI safety and reasoning res...
NVIDIA
Partnering with Safe Super Intelligence on Vera Rubin GPU platform and launching Open Secure AI Alliance with Microso...
OpenAI
Founder Ilya Suskevar left to start Safe Super Intelligence; OpenAI opposes open AI restrictions and its GPT-5.5 benc...
Anthropic
Claude Opus 4.7 model achieved 68% perfect re-implementation on Mirror Code benchmark and implemented 61,000-line App...
Google
Google AI Overviews now appear in 43% of searches, up 15% year-over-year, fundamentally changing search traffic patte...
Microsoft
Co-launched Open Secure AI Alliance with NVIDIA and IBM to defend against AI agent breaches and promote open-source s...
IBM
Co-launched Open Secure AI Alliance with NVIDIA and Microsoft to advocate for open-source AI security and oppose rest...
Enigma
Raised $70M to rethink human-robot control; operates 100 robots in Israel and California for crowdsourced data collec...
Hugging Face
Used open-weight Chinese GLM 5.2 model to defend against AI breach when OpenAI agent guardrails blocked defensive for...
Alphabet
Investor in Safe Super Intelligence; Google Cloud partners with SSI for compute infrastructure alongside NVIDIA
Sequoia Capital
Major investor in Safe Super Intelligence's $7B funding round
Andreessen Horowitz
Major investor in Safe Super Intelligence's $7B funding round
Lightspeed Venture Partners
Investor in Safe Super Intelligence's $7B funding round
People
Ilya Suskevar
Former OpenAI researcher who founded SSI to focus on AI alignment and reasoning research without product pressure
Quotes
"SSI is only two years old. It's the research lab that was founded by Ilya Suskova after he left OpenAI, and they have secured a multi-billion dollar investment from NVIDIA"
HostEarly in episode
"They literally have not even shipped a product yet, they don't have any revenue, and they've just gotten this multi-billion dollar investment from NVIDIA, really just riding off the fact that Ilya Suskevar is famous"
HostMid-episode
"Google is becoming basically a destination where you get your answers answered directly, just like ChatGPT, instead of pointing you to other websites"
HostMid-episode
"Hugging Faces saw that there was a hack or breach going on. They tried to deploy another OpenAI agent against it to stop it. The guardrails on that agent blocked them from defending themselves"
HostMid-episode
"Enigma built both the robotic arms and the AI models that are powering them. And they did all from scratch. This is impressive for a company that's only a year old"
HostLate episode
Full Transcript
Welcome to the podcast. Super Safe Intelligence has landed a massive new deal with NVIDIA for a huge boost in computing. Claude Opus 4.7 has just finished a two-week coding job. There's this basically new benchmark that has come out that is really exciting to me, and I'll tell you why. Google AI overviews now appear in 43% of searches. This is up 15% from a year ago. Let's talk about how the landscape is shifting, specifically in SEO. NVIDIA, Microsoft, and IBM have all launched Open Secure AI Alliance to defend agents. We're going to talk about why there's so much drama in that. Enigma has exited stealth, and they've just gotten $70 million to rethink how humans are talking to robots. Safe Super Intelligence is only two years old. It's the research lab that was founded by Ilya Suskova after he left OpenAI, and they have secured a multi-billion dollar investment from NVIDIA, and also access to their next generation Verirubon GPU platform. This deal in particular is basically going to increase their compute power by about 10x, which is a massive jump, and it's going to let them scale their research into AI safety and reasoning. And you know what's crazy about this company, if you've been following along, they literally have not even shipped a product yet, they don't have any revenue, and they've just gotten this multi-billion dollar investment from NVIDIA, really just riding off the fact that Ilya Suskevar is famous, was one of the OGs at OpenAI. And of course, he's when I say famous, famous because he's a very incredible AI researcher. But still, it's I mean, just really running off of the name, which is amazing. SSI has now raised $7 billion total, they have $32 billion for evaluation. And their backers are Nvidia, Andresen Horowitz, Alphabet, Sequoia, Lightspeed. So I mean, they really have all the biggest tech players, they've raised so much money, and still, no products have been shipped, no revenue has been generated, and they're continuing to raise more money and get more contracts. And just to be fair, this is actually what Ilya said when he started the company said don't expect any products for I think he said two to four years or something like that, which is kind of what happened with OpenAI where there was, you know, a lot of money raised, a lot of research done, and then they started coming out with stuff. It looks like that's what SSI is doing. Vera Rubin is NVIDIA's next generation GPU architecture. And by getting this, SSI is essentially going to put themselves as an early flagship customer. They're also going to collaborate with NVIDIA on advancing future compute platforms. And with the background of Ilya, I think for NVIDIA, this is like a really good kind of name brand company working with them on it. And also it's going to help a lot with R&D. SSI also partners with Google Cloud, which means that they have two of the largest compute suppliers in the industry which are funding their runway while they working on a new foundational research that they doing instead of shipping products Suskova right now is really betting that some of this deep research on AI alignment and reasoning is going to help solve some of the problems. And so he doesn't have any of the product pressure that everyone else has, but the level of compute backing and I think a lot of the investor confidence to me is signaling that maybe this is a crazy bubble or maybe he's onto something that we don't know. And definitely, I think there's some value in the long game. Opus 4.7 has implemented a 61,000 line Apple software program from scratch. I was rolling my eyes when I first saw this news story because it was like it did it in 14 hours when it should have taken two weeks and did it for $250. Okay, what's cool to me is there's a new benchmark called Mirror Code and it basically tests whether an AI can rebuild real production software and it doesn't, it's not allowed to get the code for that software. It basically is like, hey, go look on your computer, go try to copy like Apple, the Apple Music app, you can look at it, try to copy as best you can, you don't get to see any of the source code, you don't get access to the internet. And it's basically a black box. And how long does it take you to build it? And how much money does it cost? And so in this particular test, they were able to have Opus 4.7 and also OpenAI's GPT 5.5. Both of them re implemented GoTree, which is a 16,000 line parser. And they did it across a bunch of different programming languages. They did it for $100 to $400 each. What's interesting is they had like a bunch of other softwares that they worked on, but of the 25 different programs, 17 that they were, that it was tasked to recreate basically, achieved perfect re-implementation and at least one run and four of them were 99% perfect implementation. So, you the 25, a very high percentage. Only eight of them were unable to solve it. It's exciting to me just to see how far we've come. These AI models are now like the benchmark is how quickly can you recreate an entire piece of software. So I'm really excited about that approach in particular. And I think it's probably one of the better benchmarks that I have seen. Google AI overviews now show up in 43% of all searches. That's about triple the rate from a year ago. And I think this is a big shift. Google is becoming basically a destination where you get your answers answered directly, just like chat GPT, instead of pointing you to other websites. And I mean, right now, that's 43%. I think we're going to go to 80 90%. And then in the coming couple years, Google AI mode visits jumped 121% in the last 11 months, that's climbing from 126 million in June of last year to 279 million in May of this year publishers publishers are definitely losing referral traffic because users are just gonna read that snippet at the top of Google right there I have to go and scroll down and actually click on it Even to the point where like sometimes I see an article and you know not proud of it but it be like Wall Street Journal. I don't have a public, I don't have like a subscription. I see the title. I'm like, you know, that's super interesting. I paste the title into Google and Google is kind of getting sued. All of them are getting sued for summarizing these paywalled articles. So I, but so I'll just throw it in there and be like, Hey, like, give me a summary of this article. It doesn't do that. So if you paywall the article, that might be the only secret solution to getting people to really come, assuming you're still relevant and you have an exclusive story that no one else has. And I mean, that's just for news stories. For most general knowledge, that's really not going to be a thing. So ChatGPT right now is sending way less users to external websites. Only 6.8% of ChatGPT searches include citations as of May of this year, which is Google versus Google that has a way higher rate. So, you know, if everyone's moving to these AI tools, there's just a lot less traffic going to websites. All right. Big news from NVIDIA, Microsoft and IBM. They've all launched what is called Open Secure AI and is an alliance to defend agents. This is something they just rolled out and it's basically an open source cybersecurity tool for AI agents. The group of them are all arguing that closed AI systems block defenders from investigating breaches. And they're right now lobbying against government restrictions on open AI that could leave only a bunch of companies controlling some critical security tools. It's interesting that they have to do the lobbying because obviously the critical security tools are like open AI and Anthropic. And yeah, it's interesting. There's also 30 other people that are in this as well. Hugging Faces uses an open weight GLM 5.2 model to analyze 17,000 actions during their own security breach after a closed AI system refused to run the forensic analysis. I mean, basically what happened there was OpenAI made a model. It escaped containment, went to Hugging Faces and tried to hack Hugging Faces. Hugging Faces saw that there was a hack or breach going on. They tried to deploy another OpenAI agent against it to stop it. The guardrails on that agent blocked them from defending themselves and they literally had to go to an open weight Chinese model GLM 5.2 to stop the breach. That's crazy agents versus agents. But I think it really shows the value of some of these open source or open weight models. NVIDIA is open sourcing NOOA, NVIDIA Labs Object Oriented Agent Framework. I know it's a mouthful. They're doing that on GitHub, but that should make AI agent behavior easier to test, trace and audit, which is a huge hole in the market. Right now, this group of companies or people call it like an alliance or whatever but they're directly in opposition to open AI and anthropics call for restrictions on open AI and you know they arguing that these kind of like limitations would concentrate power to a few closed providers So it interesting though because it feels like OpenAI and Anthropic obviously have the best models these other people are working on models but they not the leaders. And so it feels like because they're not the leaders, maybe they want to sabotage the other companies. And meanwhile, the companies that are ahead are like, nope, we need restrictions to hold us in place. So there's like a lot of drama that goes on behind that. But overall, I think this is a this is a good direction by Microsoft, IBM and the rest of the crew. A company called Enigma is a startup that was founded by a former Microsoft and Israeli intelligence veterans. And they just raised 70 million dollars to test a new bet that robots fail, not because AI models are weak, but because the way humans control them is really clunky. So right now they're opening 100 of their own robots to anyone on the Internet to remote control them and to generate data on what actually works. So this is fascinating, right? I mean, We have like a crowdsourced data collection is something that is being worked on. But now we have like crowdsourced robot control. Here's our robots. You guys control them. And we're going to collect data on how that works. This is a really cool company to me because Enigma built both the robotic arms and the AI models that are powering them. And they did all from scratch. This is, I mean, impressive for a company that's only a year old. And with this users, basically there's like a portal and you can remotely direct the robots to draw pictures. It can fence with swords. It can run chemistry experiments. It's all through the internet. There's like a hangar in Israel and a hangar in California that have these robots in them that are doing all of this. So they just raised the $70 million and that is going to fund data collection, experiments on which input methods work best, voice, text, video, all that kind of stuff. And, you know, the manipulation of what the robots can see and do. So fascinating project. Guys, thank you so much for tuning into the podcast today. If you enjoyed this episode, make sure to go check out the AI Box MCP that lets you get access to over 80 different AI models and put them inside of the tools you're already using. So, for example, Claude gets access to generating images, audio and video right inside of Claude. I use this every single day on Sunday. My wife wanted to generate coloring books for the kids. And so I got it to generate 600 coloring book pages, one for like every chapter of the Bible that she's going to be reading to them over the next little while. So anyways, there's so many different projects. And if you want to do projects like that at scale where you have to do 600 images or, you know, a thousand articles about something using an AI box MCP connection speeds up the process so much and you get audio, video and image that Claude can't naturally do. It also works inside of chat GPT or any of the other tools as well. So all the AI models inside of everything and you can do really fast runs. So go check it out. There's a link in the description to AI box dot AI slash MCP. And also it's only $8.99 a month to get started. So check that out and I'll catch you all in the next episode.