The Coming Hackastrophe
For years, cybersecurity experts have been warning about the chaos that highly capable hacking bots could usher in. … Claude Mythos Preview appears to represent not an incremental change but the beginning of a paradigm shift. … Perhaps more concerning than the reported capabilities of Mythos Preview is that other companies are not far behind. (More)
Finding bugs was also hard, so the worst flaws stayed hidden, sometimes for decades. It wasn’t a great system. But the difficulty on both sides created a kind of détente that held. Now, thanks to new A.I. tools, anyone can write code. Soon, bad actors could use those same tools to find out what’s wrong with code. The détente is over. (more)
Use strong passwords that are unique across every site, preferably through a trusted password manager. Better yet, when a site offers a passkey, take it. … For accounts without passkeys, use an authenticator app for two-factor authentication, not text messages. Always keep all your software up to date, and uninstall unnecessary apps. (more)
OK, I’m a few weeks late to this party, but not too late to give many of you news: We may soon face a period (a few years?) of greatly reduced software availability.
For many decades, we have known how to write pretty secure software. It takes a bit longer, and security considerations must be central to early design efforts, but it is possible. However, developers have usually been in too much of a rush to market to do this. So most software systems today are riddled with security holes. What has saved them so far is that it takes humans a lot of work to find and exploit such holes.
However, there now exist powerful AI systems that are far better at finding and using such holes. Soon (within a year or two?) many AI firms will have such tools, and they will spread to be widely available. Yes, such AI systems can also work to patch such holes, but computer security experts tell me that the nature of insecure systems is to make it much easier to find and use than to patch such holes. Attack beats defense.
Software firms would then more eagerly rewrite their code to use more secure designs, and AI could help them to do this. But this takes time, and as there isn’t a lot of secure software out there now, AI hasn’t had big datasets ready to help them learn how to do this well. So it will take some time to replace weak with strong software.
So there may soon be a period, starting within a few years, maybe lasting a few years, when most actual software systems can cheaply be hacked. This will make such software firms vulnerable to ransomware, and make customers wary of using their products. Customers, firms, and App stores, will respond by cutting back on what software systems they offer, and by simplifying them by dropping many features.
As our world has come to rely on software for a great many things, it seems quite concerning that we might soon have to make do with substantially less software. How vulnerable are crucial systems like electricity, cars, traffic lights, voting systems, and payment systems? I don’t think we know. Beware the coming Hackastrophe.
Note: such an event would likely make the public much more willing to regulate AI. And if credit card firms get overwhelmed with false sales, that could make crypto more attractive.
s/reduce/reduced. Our firm is making an emergency plan to keep the business going assuming all Internet-connected PCs may be down for months. We are putting some machines aside offline.
I don't think you are considering the ways in which AI helps defenders as well. Even if they take time to fix the code they can look at patterns of access and monitor vast volumes of traffic to act to block hacks before they can do serious damage. I don't think we fully know who will get the upper hand.
Either way, I don't see how regulating AI would help because the most serious -- and economically impactful -- hacks tend to come from overseas. It's better if our companies get hacked first by a teenager in Indiana looking to replace the webpage with a picture of a penis than Russian ransomware gangs or worse saved to deploy en masse during a geopolitical conflict.
Besides, the process of finding exploits to fix is essentially indistinguishable from trying to exploit them so we need to ensure that our software engineers retain the ability to locate these flaws.
Of course, we might try to regulate access to certain features but -- even if the hackers can't gain unauthorized access -- the problem is that the Chinese and other foreign models just aren't far enough behind for barriers the regulations impose on our cyber security research to be outweighed by any delay in the danger.
---
Could be that people still demand regulation irrationally but I think that misreads the media landscape. People will hear about hacks happening but they won't be able to tie them to particular AI assistants while those same companies will be doing everything they can to highlight all the ways they are working to help make you safer. That plus all the security researchers saying how useful they are in protecting us will likely limit the pressure as a result of cybersecurity issues.
I think the pressure to regulate AI is more likely to come from more prosaic cases where someone with mental illness works with AI to do something harmful (to themselves or others).