AI Code Assistants in Practice: Security, Quality and Responsibility

Introduction

Artificial intelligence (AI) is transforming software development at remarkable speed. Large language models (LLMs) such as ChatGPT, Claude, Gemini and others can now generate code across a wide range of programming languages. These tools can accelerate development considerably, reduce repetitive work and support innovation. At the same time, they carry substantial risks that boards must understand and govern.

Recent research reveals both the opportunities and the clear limits and vulnerabilities of AI-generated code. This article summarises the key findings, sets out the conditions for responsible use, and offers concrete recommendations for informed decisions at supervisory and management level. It draws on the analysis “Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis” by Mohammed F. Kharma, Soohyeon Choi, Mohammad Alkhanafseh and David Mohaisen, published on 25 February 2025.

Opportunity and Its Limits

There is no doubt that LLM use already increases productivity in many organisations. Developers can automate routine tasks, build prototypes faster and focus more on conceptual and architectural questions. These efficiency gains come at a cost.

The research shows that code generated by LLMs often contains serious flaws. Security vulnerabilities are widespread, including outdated cryptographic methods, buffer overflows and hard-coded credentials. Quality is inconsistent too: structure, readability and maintainability vary considerably between models and programming languages. No LLM tested could reliably deliver secure, error-free or standards-compliant code without prior review by an experienced developer. The reality is clear. Without human oversight and robust security mechanisms, AI-generated code is not fit for production use.

Key Findings for Executive Teams

One striking finding is that the security and reliability of AI-generated code depend heavily on the programming language used. Python and Java, with their simpler syntax and automatic memory management, tend to produce fewer errors and vulnerabilities. C and C++, by contrast, require manual memory management and pose considerable difficulty for the models. Critical security gaps can arise here that lead to system failures or attacks.

Recurring vulnerabilities appeared across all models. These include insecure cryptographic methods, memory errors, hard-coded passwords and missing input validation, which enables attacks such as SQL injection or cross-site scripting. Even where generated code appears to function, many sections contain so-called “security hotspots”, potential weaknesses that can have serious consequences in production.

Code quality is equally unreliable. Some output is clear and well structured; other code suffers from unnecessary complexity, redundancy or unclear function names. Documentation is often missing too. Without systematic review processes, the risk of technical debt and long-term maintenance problems grows. The point is clear: no LLM tested can be considered secure by default. Human oversight and supplementary review mechanisms are indispensable.

Conditions for Responsible Use

Given these findings, LLM support for code generation should only proceed under clear conditions. Mandatory manual review of all AI-generated code by experienced developers before deployment in production systems is essential. Every piece of code must also be checked automatically with security tools such as SonarQube, to identify known vulnerabilities, quality defects and deviations from standards.

Organisations must consistently enforce current security and coding standards and reject code built on outdated or insecure methods. The choice of programming language matters: LLM-supported code generation is defensible mainly in higher-level languages such as Python or Java, while use in C and C++ should remain limited to clearly defined, tightly controlled scenarios.

Comprehensive staff training is equally important. It should cover how to use AI tools and, just as much, their limits and risks. Governance structures must define who is responsible for reviewing, approving and continuously monitoring AI-generated code. Clear escalation and audit processes are needed too, to allow a rapid response to incidents or discovered vulnerabilities. Regular audits should be mandatory, to verify compliance with all requirements and identify new risks early.

Recommendation for Executive Teams

AI-generated code can only be used safely if technical safeguards and organisational controls are both strictly enforced. Production use should not be approved until it is certain that both human review and automated security analysis are in place. Management should be required to report regularly to the board on the status of all measures described here, before LLM-supported code generation moves beyond pilot or test phases.

Pilot projects in lower-risk areas, with regular evaluation of results, are advisable as a starting point. Where these controls are missing, or where there is uncertainty about whether the standards are met, the risk to the organisation’s security, compliance and reputation remains unacceptably high.

Conclusion

AI-supported software development holds real potential, but it is no substitute for expertise and diligence. Executive teams and those responsible should authorise its use only where strict oversight, consistent review and adherence to established practice are guaranteed, and where staff are prepared to treat AI as support, not as a substitute for professional expertise and accountability.

Source: Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis