Move Fast, Break Laws: AI, Open Source and Devs (Part 1)

Steve Poole

The software development landscape is rapidly changing, with legislation emerging as a key driver of industry trends. As our reliance on software and AI grows, so does our vulnerability to cybercrime, which is now a multi-trillion-dollar problem. This has caught the attention of regulators worldwide. 

This article series explains the various regulatory efforts in play and summarises actions that developers and executives should consider as they prepare for 2025, the year of software legislation.

Part 1 (this article) covers the background, what a software supply chain is and thoughts on AI and open source.

Part 2 will explore how governments are working to create legislation and what the current status is.

Part 3 offers both a Software supply chain and an AI governance & compliance checklists for developers and executives to consider

Part 4 will discuss cybersecurity and incident reporting requirements, examines geopolitical compliance and liability management, and wraps up the series.

There’s a lot to take in. I hope you’re sitting comfortably..

Accountability Cannot be Outsourced.

I am not a lawyer. This document is a technical view of the legislation and regulations being developed or repurposed.  It’s imperative to get your own legal assessment when deciding if these elements apply to your situation. Having said that, some aspects are shared.  The primary one is accountability.  There’s no dodging your responsibilities.  That means wherever you are in the software supply chain, you have responsibilities to those consuming your software and those using it.  Regulations collectively require organisations to assess, monitor, and manage third-party risks, and you’ll have to prove that you did the right thing at the right time.

Blaming others without proper due diligence and safeguards is not a valid defence!


What is a Software Supply Chain?

As a developer, you may assume that a software supply chain is a fancy term for dependency management. However, a software supply chain refers to developing, delivering, and maintaining software, from code creation to deployment. It includes source code, dependencies, CI/CD pipelines, build tools, package repositories, cloud services, and infrastructure. 

For engineering leaders, these are the elements to consider when ensuring security, compliance, and efficiency in software building and deployment. For executives, software supply chains include all the aspects considered in risk management, regulatory compliance, and business continuity.  

When discussing software supply chains, we’re not just discussing the elements you control. Your software supply chains include all the software supply chains for all the software that is a part of yours, from dependencies to tools to platforms.  

Why are Software Supply Chains so important?

When reviewing the ways that the bad actors exploit software, many of the attacks come from vulnerabilities.  That’s weaknesses, bugs and even misguided features.  Another cyberattack weapon: malware can often be found embedded in legitimate downloads and libraries.  Then, there are times when the bad actors create a fake component and trick others into using it.  The software supply chain is the common thread to all these, even to some of the more sledgehammer-style attempts such as Denial of Service attacks.  

From poor software engineering skills that result in code without the required security posture to CI/CD systems that are insecure and are subverted to include malware. In most cases, the root of these attacks can be traced back to some element of behaviour or lack of skill by the organisation producing the software. 

The bad actors exploit these behaviours and insufficient skill sets in many ways, sometimes in incredibly devious and elaborate forms but usually in simple forms.  

Why Now?

The bar for the bad guys is low. Developer community defences are relatively nonexistent, and their attention to the problem is still limited. No wonder cybercrime makes more money than the illicit drug trade. (Cybercrime costs the world around 9 Trillion Dollars annually and is growing rapidly.)

As the final straw in a long line of headline exploits, the Log4Shell exploit demonstrated that the software industry could not police itself sufficiently. Software is far too valuable as an enabler of the modern world to be left undefended. The result is a worldwide effort to create practical and effective incentives to address the problems.  From a government point of view, software supply chains (and hence developers and development organisations) are the foundational element behind the tsunami of attacks we see, and therefore, why they are the target for much regulatory activity. 

What’s the AI angle?

AI is still software and can be exploited in its own way.  AI in the software supply chain (as models, training data, supporting libraries, etc.) is still vulnerable to the same attacks that traditional software is susceptible to.  AI is also weak to new attack styles – poisoned models, for example- and is used to make other attacks more effective.  Related to this is that although the AI technology emerging today might change the world tomorrow, the bad guys already use it to compromise software supply chains. 

Regulatory reviews on the use of AI are resulting in specific legislation. However, once you look past the shiny AI exterior, the interior still consists of software, packages, data, build, test, deployment processes, tools, etc.  

Whether you use AI as a tool in your software development process or as a competitive advantage in your production process, you and your organisation will likely have to consider both AI-specific and software supply chain legislation.  

Open Source Implications

For decades, open-source software (OSS) has been synonymous with innovation. It is often driven by communities with minimal regulatory intervention or consideration. Even so,, Licensing, copyright, IP rights, and other issues can be viewed as a burden for those just trying to help others through their open-source efforts.  

Unfortunately, the landscape is undergoing a dramatic transformation. Critical infrastructure and major industries’ reliance on OSS and high-profile supply chain attacks like SolarWinds and Log4Shell have exposed the vulnerabilities and governance gaps in many open-source projects. 

The rise of AI models trained on open datasets and utilising open-source AI libraries has added to the complexity, value and potential risk many see in consuming open-source technologies.

Everyone acknowledges that open-source projects, in whatever form, are a critical component of the software stacks we rely on. It’s estimated that 90% of modern applications are open-source.  

Governments certainly don’t want to curtail open-source development, but it’s seen as a significant enabler for bad actors. Therefore, governments are beginning to demand greater traceability, provenance, and accountability from free and open-source software. 

Since open source is increasingly perceived as a critical component of the regulated software supply chain rather than a mere free-for-all domain, the aim is to find a balance that has the least effect on the creators while still closing the doors to the bad guys. 

The bottom line?

 The bottom line is that open-source projects will inevitably gain new burdens, and some maintainers will undoubtedly decide to retire their projects. The hope is that the primary burden can be placed on those who consume the open-source software rather than the contributors themselves.  

The proposed changes can be seen as unfavourable, as a big brother play to enforce arbitrary rules on the development community, but that’s not entirely fair. It’s certainly a set of big sticks, but we’re seeing a realisation that, although invisible, software is as important as any other national infrastructure component. Infrastructure that needs to be high-quality and robust. Many of the elements of these regulations are about indirectly improving the software engineering skills of developers and their organisations to achieve the necessary standards. 

How this plays out is a big TBD, but there is no doubt that this is real and already happening.  Those who can reinvent their software creation approach and keep the bureaucracy to a minimum will have a real competitive advantage


Next Time

Read part 2 to understand more about how governments create regulations and what the list looks like. Global efforts to stem the tide of cyberattacks and manage AI usage are giving the software industry a relative tsunami of technical and business challenges to evaluate.

Total
0
Shares
Previous Post

Free Tickets for JCON EUROPE 2025 

Next Post

Move Fast, Break Laws: AI, Open Source and Devs (Part 2)

Related Posts