A lot of interesting discussion after the Anthropic Fable/Mythos 5 model shutdown due to USG Export control restriction:
-
It was apparently a DoP (something like “fix the vulnerabilities in this code” ) rather than some huge Jailbreak as is being made out.
Research leading to U.S. export restrictions
If this is true then it makes no sense - I have used similar prompts to get models to first show me how the code was vulnerable before fixing it. Shutting down frontier models for this is shooting yourself in the foot. -
“Sovereign AI” started being thrown around - the idea that given the capricious Mythos shutdown, every nation should have its own Frontier Models especially a large populous country like India - and the only thing I have to add to it is why not? Building a (near) Frontier Model is actually somewhat easier today than in 22/23.
Consider:
-
There are many trillion token datasets freely available e.g. NVIDIA Nemotron-CC dataset
-
The recipes for training models are published: Just the Chinchilla Paper has a wealth of information Chinchilla paper or Hugging Face wrote a very nice guide Hugging Face training playbook
Start from there. -
If you want to start from a base model there are many great OSS candidates like Google Gemma 4
The hard parts are the upfront funding needed to get the team+hardware together and to keep maintaining it as we go thru rapid AI innovation over the next few years.