blueworld

He’s so full of shit: “The technology is really simple,” he told ABC7. “A car drives by, we take a picture. It’s a static picture of a car, and then we read the license plate. That’s what the technology is – it’s actually not that complicated, it’s pretty simple.”

When in fact these use AI to build what the company calls a “Vehicle Fingerprint” to identify each car by make, model, color, body type, and distinguishing features like bumper stickers, roof racks, dents, decals, and even objects in truck beds. They can track and identify vehicles even when license plates are missing, obscured, or swapped for an 6-12 frame interaction. Moreover it is a database of metadata allowing them to characterize personality, trends, habits, and patterns across 80,000 cameras in 49 states. That’s just the plate readers.

They’ve grown to include ‘first responder’ drones, audio sensors, gunshot detection sensors, and full video surveillance in both the public and private sphere.

And worse is it’s real-time cross-jurisdictional alerting, which has seen multiple false reports.

So it’s damn well not just a static picture of a car, but surveillance capitalism.

You would enjoy a part of this essay talking about the Airline’s experience dealing with automation:

They realised that many pilots flying highly automated jets were becoming what Vanderburgh called “Children of the Magenta Line”.

But you see, we have become what I call children of the magenta. [Note: In modern cockpits, the magenta line is the brightly colored route shown on the flight display, a digital path generated by the navigation computer.] You know, we think we have to have those magenta lines on the map and that magenta V-bar that’s steering us toward that line, or for some reason, the plane won’t fly.

…

That’s why his core message is:

We are not automation managers. We are captains and pilots, and by our aviator skills, we are to ensure the vertical and lateral path of these planes at all times. We will use the wonderful tools of automation that have been provided to us to help us with that task. But when the automation is not maintaining the intended flight path, we will turn it off and maintain the path by our skills.

Automation excels in routine situations, but it can’t improvise when things go wrong. In Vanderburgh’s words:

Automation lacks the ability to create flexible responses to unanticipated changes in flight path requirements. So in these circumstances, a lower level of automation should lower workload and thereby preclude us from becoming task-saturated and losing our situation awareness.

It’s an interesting historical frame of this AI automation downward spiral you describe.

What happened

The intrusion started where AI platforms are uniquely exposed: the data-processing pipeline. A malicious dataset abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration) to run code on a processing worker. From there, the actor escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend.

The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness - used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services. This matches the “agentic attacker” scenario the industry has been forecasting

…

The asymmetry problem

When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers’ safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.

This experience points to a gap worth planning for. We do not know which model powered the attacker’s agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried. The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment. This is not an argument against safety measures on hosted models, and we are sharing this feedback with the providers concerned.

(My emphasis added)