To keep on top of AI, shift from minimising risk to managing it

tech

To keep on top of AI, shift from minimising risk to managing it

EARLIER THIS month, Anthropic released Fable 5, a public version of the more powerful Mythos antificial-intelligence model it had previously withheld. Within days, the American government moved to restrict access by foreign nationals, citing national-security concerns. Anthropic argues that the model’s features were known previously, the risks are being overstated and the response is disproportionate; but since the company is unable to comply with the demand for selective access, it has suspended access for all users.

TL;DR

Outdated regulation is a threat to AI governance.
Anthropic released a powerful AI model, Fable 5.
The US government restricted foreign nationals' access due to national security concerns.
Anthropic believes the risks are overstated and the response is disproportionate.
The company suspended all access because it could not comply with selective access demands.

Continue reading the original article

clem 🤗 reposted

Brian Armstrong

How to keep AI spend flat while token usage grows exponentially: Not with friction and spend alerts. With better defaults, routing, and caching.

Better Defaults (not Usage Caps) – Engineers can choose any model they want, but defaults matter. We’re experimenting with defaulting to open weight models like GLM 5.2 and Kimi 2.7 through our LLM gateway, while still encouraging engineers to choose the right model for the task. 91% of our employees were never hitting their usage caps, so instead of lowering caps and driving up alerts, we're moving to cheaper defaults. Note that code reviews use a diversity of models, so they can check each other's work.

Better Routing – In our custom harnesses, we preprocess prompts and route to the best model for the job, considering cache hits and model pricing. For instance, you may want a frontier model for planning, but not for execution where they can be overkill. Ultimately, humans shouldn't be choosing models - AI can automate this task.

Better Caching – Cache misses are the easiest way to drive your cost up. All of our requests are cache aware, so we’re reusing a warm cache wherever possible. For example, our cache hit rate went from 5% → 60% in LibreChat once properly implemented.

Keep Context Lean – Start fresh sessions when switching tasks. Scope file context narrowly. Disconnect unused tools. Don't just compact. The goal isn't fewer tokens used, it's fewer tokens wasted.

Better Visibility – Our engineers can use as many tokens as they want, from whatever model they want, but we’ve made usage visible – and the more you spend on AI, the more impact we expect.

The goal isn't to suppress usage. It's to build the infrastructure that makes exponential growth sustainable.

Putting this into practice has cut our AI spend nearly in half, while our token usage continues to grow.

Yann LeCun reposted

clem 🤗

It's quite rational to regulate frontier API models, especially to get more transparency for the government, without regulating open-source AI.

Here's why:

1. The most dangerous AI systems right now aren't open models. They're the large frontier LLM APIs distributed through coding tools and assistants, because:

- They're built in secret behind closed doors and stay total black boxes. Zero transparency on what they can or can't do, with "safeguards" that blur everyone's ability to even analyze them.

- They're built and controlled by a few profit-maximizing megacorps, concentrating unprecedented power in very few hands, with every incentive to downplay risks and overstate their safeguards.

- They're distributed to hundreds of millions, maybe billions, of people and trivially easy to run.

2. Open-weight models are orders of magnitude less risky:

- They're not as massively distributed or as easy to use, especially the big ones, as APIs and assistants.

- We (including governments) can quickly and accurately analyze what they're capable of, and for now everyone confirms they're not as good as the APIs at doing bad things.

- They're distributed to everyone, so defenders and law enforcement get as much access as attackers.

The cost-benefit analysis of regulation is completely different too. Regulating frontier APIs is relatively easy and low risk while regulating open source would be much more complex, less efficient, and orders of magnitude more costly.

Regulating frontier APIs would only potentially hurt a few megacorps, if it even hurts them, given all the marketing that it is already generating for them. They can afford armies of lawyers and absorb losing a few billion dollars, especially given they're on track to become some of the most valuable companies in history.

Regulating open source, by contrast, would hurt the very people regulation is supposed to protect: small businesses, startups, researchers, nonprofits, universities, independent developers, and the broader public, while risking killing competition, slowing AI progress, and reducing transparency even more!

Aravind Srinivas

future: continual learning of the model + harness hosted on hardware you own and control, with full access to all sensitive context that never leaves your system

Aravind Srinivas reposted

Kyle Polley

“AI Meltdown” is the scenario where the agent goes off the rails and forgets/ignores all previous instructions and soft guardrails. No prompt injection or malicious actor is needed for this to happen.

At @perplexity_ai we built a suite of tools to prevent AI Meltdown from happening, and we opened sourced it just yesterday! Please use Numbat to secure yourselves from similar self-inflicted attacks

https://t.co/E6Rn70WEkY

Yann LeCun reposted

Christopher Nguyen ⽗

Replying to @ShawnFumo @METR_Evals

The narrative: Blame the Agent, instead of the Agency that told him to “apply all your powers and told to achieve this win.”

Ironically, many forefront members of the AI-Safety community, in their fervor for proof that AI (the weights) have jumped the fence, are actively helping

Yann LeCun reposted

Daniel Jeffries

Our labs keep trying to spin this into a push for broader regulation.

It can and should backfire.

The models are not "going rogue" or acting of their own accord, like they're some Marvel movie evil robot.

People made bad harnesses, told them to hack things and had transparently and objectively bad dev-ops and security practices.

The fact that folks are trying to spin this into a "we need help from the government to regulate everyone" instead of "we should be punished in a narrow way on these specific incidents under existing law" is the real disconnect right now.

Yann LeCun reposted

Yann LeCun

Replying to @willdepue

Using optimization at inference time is a foundational concept of Energy-Based Models (EBM) and Objective-Driven AI architectures (ODAI).

When the variables to be inferred are continuous, it makes sense to use gradient-based optimization.

A good instance of ODAI is world

clem 🤗

Some people are surprised that APIs (aka what Anthropic, OpenAI, and others provide) are treated differently than open weights in the new AI model framework.

I'm not surprised at all, and it's actually very good policy. Let me explain:

Model weights, APIs, and apps are three very different layers of the stack. Treating them the same would be a recipe for bad regulation.

Think about how we handle cars. We don't regulate steel, we crash-test cars. Nobody asks a steel mill to guarantee that nothing dangerous will ever be built with its steel. Obligations sit with the carmaker and rules of the road with the driver, because that's where risk becomes real and where someone can actually act on it.

Model weights are the steel of AI. They're raw research output, closer to science than product: no user, no interface, no deployment. They don't do anything on their own. And because everything else is built on top of them, this is the layer where regulation does the most damage. Restrict weights and you slow down all progress downstream, and you prevent countless positive use cases from ever emerging: the lab fine-tuning an open model for rare diseases, the startup serving a language big providers ignore, the safety researchers who can only audit models because the weights are open. You don't reduce risk, you just kill open source and concentrate power in a few big labs.

APIs are the middle layer, the parts and engine suppliers of AI: a commercial service where a provider serves a model at scale. Here you have a business relationship, terms of service, the ability to monitor for abuse. It makes sense to expect transparency, security standards, and accountability from providers at this layer, because they can actually enforce things.

Apps are the car on the road: where AI meets the real world. A medical assistant, a hiring tool, a companion for kids, a financial advisor. This is where concrete harm can happen, and conveniently, it's where we already have decades of regulation. Health, finance, employment, consumer protection. An AI hiring tool should comply with employment law whether it's powered by an open model, an API, or a spreadsheet.

The principle is simple: regulate at the layer where risk actually materializes and where actors can act on it. Push obligations to the deployment layer, keep the research layer open. We don't regulate steel, we crash-test cars. Well done @realDonaldTrump @DavidSacks @mkratsios47!

Dario Amodei

1/2 Thanks Gavin for an especially thoughtful exchange. I don't usually spend much time on social media but I wanted to engage here because it really brings out the heart of an important conversation.

First, on regulation, I think that “either concentrate it in the hands of a chosen few companies and politicians via regulation or distribute it widely” is a false choice. I know that there’s a sort of Silicon Valley shorthand where regulation = regulatory capture = concentration of power, but I’ve always found this to be an overly simplified picture of the world. Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people. I don’t necessarily agree with that perspective either, rather I think it’s complicated and really depends on what the “regulation” consists of. But in particular I think that those in the “regulation = regulatory capture = concentration of power” frame often underrate the decentralizing power of objective and fair institutional processes. A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative, mob justice. At their best, institutions can vest power in ideas rather than people, and thereby decentralize that power.

This is why Anthropic has always made its policy proposals very carefully. We try very hard to make proposals that disadvantage (slow down) frontier AI companies while *advantaging* smaller competitors. California’s SB53 (which we supported), and even the much-maligned SB 1047 (which we were ambivalent on), completely exempt any company below a certain amount of revenue or model training costs from being covered at all (it was $500M for SB 53, lower for 1047 but we objected to that). More recently the testing process we’ve advocated for at CAISI and the White House involves more rigorous tests for frontier models than off-frontier models — something that differentially advantages challengers. Similarly, the “Pacing the Frontier” letter envisions (or at least Anthropic’s preferred implementation of it envisions) modulating the pace of the very best models while not constraining those who are catching up. This hurts the business interests of the frontier labs and helps challengers, including open-weights!

Overall my view is that AI is *structurally* a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws). Open-weights do help some with this but are nowhere near a sufficient solution because they simply shift the concentration somewhat to those with the most compute and chips (which are roughly the frontier labs plus maybe hardware providers). By contrast I think the right “rules of the road” can simultaneously (a) address AI’s cyber/bio/alignment risks, (b) institutionally constrain the power of the frontier AI companies, and (c) leave room for open-weights models while also addressing the specific risks that they bring.

BTW I do not think that the events of the last few months have “failed to result in [my] preferred regulatory path”. The approach that the Trump administration is reported to be taking — pre-deployment testing for frontier models, and also testing of open-weights models when they get closer to the frontier — is one that I am very supportive of, though of course I have to see the details to be sure. I am also supportive of Demis Hassabis’ ideas around a FINRA-like entity. This contrasts with six months ago when most of the industry was still pushing for preemption of all state regulation and no apparent federal approach either.

Aravind Srinivas

When building AI agents, it is important to still give humans agency to have their hands on the wheel and intervene when necessary.