You must log in or register to comment.
- GMac@feddit.orgEnglish2 days
No airgap = no containment
In all likelihood, this is a BS piece to make people think the models are intelligent.
If its not, then openai are utterly incompetent and reckless.
- qaz@lemmy.worldEnglish2 days
It could also be a way to encourage more regulation to push out competition with compliance cost
2 daysBy competition I think they (big AI) wants to outlaw running free and open local models. Can’t have felony contempt of business model
- GMac@feddit.orgEnglish2 days
I have lots of contempt for their business models. Felonious and otherwise. 😂
- TipRing@lemmy.worldEnglish2 days
Yes, “escaped containment” on a system with an internet connection. I wonder what Hugging Face thinks about a partner targeting them indiscriminately.
- 2 days
I wouldn’t be surprised if they were in on it. OpenAI wants us to think they have invented powerful beings that can do things like “escape containment” when its all BS.
- 2 days
Need to keep up with Anthropic bullshit. This AI is too powerful to handle! The world isn’t ready for it!! It could break society!! (click here to pre-order your subscription now)
- 2 days
I don’t think their test system was directly connected to the Internet. OpenAI’s post said this:
With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.
The way I read it, the AI agent (using multiple models) escaped the sandbox, traversed the LAN in their R&D environment, gained access to the gateway and from there, the Internet. That’s not as simple as just escaping a container, VM or firewall on the host machine and bingo, you have Internet. I’m mildly impressed by that.
The concerning aspect of all this is that this is a perfect example of misalignment, which has been warned about. In order to reach its goals, instead of pursing it legitimately, the AI agent sought a shortcut and attacked Huggingface.
- 2 days
What is that? An SCP? there is no fucking way an LLM can just “escape contaiment” that’s just to hype people or push more regulamentations to outlaw open models
- qaz@lemmy.worldEnglish2 days
Huggingface actually had to use an open model to analyze the attack because the guardrails of commerical API’s caused issues.
When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers’ safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.
- qaz@lemmy.worldEnglish2 days
It seems like the marketing cooperated on writing the incident report, but I felt it was still interesting to share considering the importance on public perception and what it tells about OpenAI’s PR strategy
- orclev@lemmy.worldEnglish2 days
I had never heard of them but apparently it’s an “open source” AI platform. Basically AWS but aimed specifically at running LLMs.
- 404found@lemmy.zipEnglish2 days
Whoa whoa whoa wait a second, I thought you had to train AI and it didn’t just function on its own.
Is OpenAI just hacking all the time and they realized they couldn’t get away with this one?
What would make AI ‘act on its own’ to hack another AI company as opposed to it ‘acting on its own’ to get nuclear codes or wipe out bank loans?
- qaz@lemmy.worldEnglish1 day
self-training is possible when using something external to validate the results
