Auditing an Existing AI Project: A Complete Technical Checklist

Taking over an AI project developed by another team without first auditing it is like buying a building without a structural survey. According to a 2025 HFS Research study, 83% of large companies identify poor code quality as a major structural cause of technical debt. In projects involving machine learning, the risk is amplified: Google Research has shown that, in a mature ML system, the machine learning code itself sometimes represents only 5% of the codebase. The remaining 95% consists of glue code, data pipelines and infrastructure that is difficult to audit.

Auditing an existing AI project means methodically evaluating code quality, model condition, data pipeline reliability, dependency health and the system's ability to evolve. It is an essential first step before any takeover, change of provider or investment decision.

TL;DR — An AI project audit covers five dimensions: code quality and architecture; ML models and their data; dependencies and infrastructure; security and regulatory compliance; and the organization's ability to maintain the system. This article provides an actionable checklist for each dimension, with practical tools, metrics and warning signs to monitor.

Why an AI-Specific Project Audit Is Essential

ML projects have structurally higher technical debt

An AI project is not conventional software. An empirical study published on arXiv in 2023 found that machine learning projects have twice the rate of self-admitted technical debt, or SATD, of traditional software projects. This higher rate stems from the nature of ML systems: application code is accompanied by data pipelines, model configurations, dependencies between features and training artifacts that create invisible interdependencies.

Google formalized this observation in its seminal 2015 paper, “Hidden Technical Debt in Machine Learning Systems”: ML systems have all the maintenance problems of traditional code, plus a set of ML-specific challenges. These include feature entanglement, where changing one feature affects all the others, hidden feedback loops and dependence on unstable external data.

Warning signs that call for an audit

Several situations justify auditing an AI project before taking it over:

A change of provider: the original team is leaving the project or its contract is ending.
Deteriorating model performance: predictions are getting worse without a clear explanation.
An inability to evolve the system: every change causes cascading regressions.
Technical due diligence: as part of an acquisition, funding round or strategic investment.
Regulatory compliance work: the EU AI Act imposes traceability and explainability requirements on high-risk AI systems.

What a conventional audit does not cover

A traditional code audit examines code quality, security vulnerabilities and architecture. An AI project audit must go further: assessing training reproducibility, training data quality, model drift risk, experiment traceability and data pipeline robustness. Without these additional dimensions, the audit remains blind to the most expensive risks.

Dimension 1: Code Quality and Software Architecture

Static analysis and baseline metrics

The first step is to run the codebase through a static analysis tool. Since version 2025.1, SonarQube has included automatic detection of AI-generated code—a critical point given that 67% of developers say they spend more time debugging AI-generated code than writing it, according to a 2024 Harness survey.

Systematically record the following metrics:

Metric	Warning threshold	Recommended tool
Cyclomatic complexity	> 15 per function	SonarQube, Radon
Test coverage	< 60%	pytest-cov, Istanbul
Code duplication	> 5% of the codebase	SonarQube, jscpd
Critical code smells	> 0	SonarQube, Pylint
Technical debt ratio	> 5% of development time	SonarQube

In 2024, GitClear found that the volume of copied-and-pasted code lines exceeded refactored lines for the first time—a historic reversal directly linked to the widespread adoption of AI coding assistants. During an audit, the proportion of duplicated code is a reliable indicator of the quality of human oversight of the codebase.

Architecture and separation of concerns

A healthy AI codebase clearly separates:

Business code: application logic, APIs and interfaces.
ML code: model training, inference and evaluation.
Data pipelines: ingestion, transformation and validation.
Infrastructure: deployment, monitoring and scaling.

The reality is often different. Glue code—the intermediate layers connecting models, data and infrastructure—frequently accounts for most of the codebase. Identify areas where business and ML logic are intertwined: these are the weak points that will make any changes expensive.

Documentation and readability

Check that the following exist and are up to date:

A current README with installation and deployment instructions.
Docstrings for critical functions: training, preprocessing and inference.
An architecture document describing components and their interactions.
Architecture Decision Records, or ADRs, explaining major technical choices.

A lack of documentation is not necessarily a deal-breaker. What matters is estimating the cost of reconstructing it and including that cost in the takeover budget.

Dimension 2: Models and Data

Inventory the models in production

Collect the following information for every deployed model:

Information	Why it is critical
Model type and framework	Determines the skills required and deployment constraints
Production model version	Makes it possible to trace history and regressions
Date of last training	A model not retrained for six months or more warrants scrutiny
Performance metrics: accuracy, F1 and AUC	Establishes a baseline for measuring deterioration
Training data used	Supports traceability and reproducibility
Hyperparameters	Enables experiment reproducibility
Production inference latency	Affects user experience

Assess model drift risk

Model drift—the gradual deterioration of a model's performance as real-world data changes relative to training data—is the silent risk of production AI projects. According to MLOps practices documented in 2025, drift detection relies on statistical tests, including PSI, the Kolmogorov–Smirnov test and Jensen–Shannon divergence, applied to input feature and prediction distributions.

During the audit, check:

Whether drift monitoring exists: are alerts configured, and at what thresholds?
Retraining frequency: is it automatic, scheduled or nonexistent?
Historical performance metrics: is there a downward trend?
Whether an automated retraining pipeline exists: can the system update itself autonomously?

A complete absence of drift monitoring is a major warning sign. It means nobody knows whether the model has been working correctly since deployment.

Data quality and governance

Data is the fuel for ML models. A serious audit examines:

Provenance and traceability: Where does the training data come from? Is it still accessible? Are the licensing agreements in order?

Data quality: What proportion of values are missing, duplicated or inconsistent? Tools such as Great Expectations or TensorFlow Data Validation can automate these checks.

Bias and representativeness: Does the training data reflect the target population? A model trained on biased data will produce biased results in production, with potentially legal consequences under the AI Act.

Data freshness: Are the data supply pipelines still working? Are external sources stable?

Dimension 3: Dependencies and Technical Infrastructure

Map software dependencies

The Python ML ecosystem evolves at a relentless pace. An 18-month-old AI project may already rely on obsolete versions of TensorFlow, PyTorch, scikit-learn or dozens of supporting libraries. The HFS Research study finds that 50% of companies cite the complexity of integration with existing systems as a major technical debt concern.

Dependency mapping covers:

Direct and transitive dependencies: List every library and its version using pip freeze, poetry.lock or requirements.txt. Identify pinned versus floating versions.

Known vulnerabilities: Run dependencies through a security scanner such as Snyk, safety or pip-audit. Every unpatched CVE is an active security risk.

Version compatibility: Check that ML framework versions are compatible with each other and with the Python version in use. Dependency conflicts are among the most common causes of build failure in ML projects.

Abandoned dependencies: Identify libraries whose last commit was more than a year ago or whose maintainer has stopped working on them. Every abandoned dependency is a time bomb.

Assess deployment infrastructure

A production ML model does not live in a Jupyter notebook. The audit checks:

Deployment environment: is it containerized with Docker or deployed directly to a VM? A non-containerized deployment makes reproducibility uncertain.
Orchestration: Kubernetes, ECS or serverless? Orchestration complexity should be proportionate to the use case.
Infrastructure as code: are environments defined in Terraform, CloudFormation or equivalent files? Can the environment be recreated from scratch?
CI/CD pipelines: do they exist, and do they cover both code AND models? A pipeline that tests only application code without validating models is incomplete.

Manage ML artifacts

Check the existence and condition of:

A model registry, such as MLflow, Weights & Biases or Neptune: are models versioned? Can an earlier version be restored?
A feature store, such as Feast or Tecton: are features centralized and consistent between training and inference? The absence of a feature store is the leading cause of training-serving skew—the silent discrepancy between the data seen during training and the data seen in production.
Experiment tracking: are hyperparameters, metrics and results from previous training runs recorded? Without that history, every retraining effort starts from scratch.

Dimension 4: Security, Compliance and Intellectual Property

An AI-specific security audit

Beyond conventional vulnerabilities such as injection, CSRF and session management, AI projects have specific attack surfaces:

Adversarial attacks: Has the model been tested against malicious inputs designed to deceive it? Image classification, NLP and fraud detection models are particularly vulnerable.

Data poisoning: Are training data pipelines protected against malicious data injection? An attacker who compromises training data compromises the model itself.

Model extraction: Are inference APIs protected against model stealing? Unlimited access to a prediction API makes it possible to reconstruct the model through reverse engineering.

Sensitive data leakage: Can the model memorize and reproduce confidential training data? This is a documented risk for language and generative models.

According to HFS Research, 59% of organizations cite security vulnerabilities as a major concern related to AI adoption, ahead of integration complexity and loss of visibility into model behavior.

Regulatory compliance and the AI Act

The EU AI Act, being phased in since 2024, imposes specific obligations on AI systems according to their risk level. The audit must establish:

System classification: unacceptable, high, limited or minimal risk?
Traceability: can model decisions be explained and audited?
Documentation: is the technical documentation required by the AI Act available?
Human oversight: are human control mechanisms in place for critical decisions?

A high-risk AI system that does not meet these requirements represents a substantial legal and financial risk when taking over a project.

Intellectual property and licenses

Systematically check:

Code ownership: who holds the rights to the codebase? Do contracts with the original developers include an assignment of intellectual property rights?
Open-source licenses: are the libraries compatible with each other and with the intended commercial use? A GPL-licensed dependency in a proprietary product can create a major legal problem.
Training data: are usage rights documented and valid? Under the GDPR and AI Act, traceability of data origins is no longer optional.
Pretrained models: if the project uses foundation models such as GPT, LLaMA or Mistral, do their license terms allow the intended use?

Dimension 5: Organization, Skills and Maintainability

Assess maintenance capacity

Beyond the code, a takeover audit evaluates the organization's ability to maintain and develop the system. HFS Research reports that 80% of companies identify skills shortages as a structural cause of technical debt—a figure that is particularly meaningful in AI, where ML specialists are scarce and expensive.

The key questions are:

Bus factor: how many people understand the system end to end? If the answer is one, the risk is at its highest.
Process documentation: are deployment, retraining and rollback procedures documented?
Team turnover: how much turnover has the project experienced? Every departure without documented handover increases knowledge loss.

MLOps maturity

MLOps maturity is a reliable indicator of the project's future maintainability. Assess it on a four-level scale:

Level	Characteristics	Takeover risk
Level 0 — Manual	Notebook-based training, manual deployment and no model versioning	Very high: an almost complete rebuild
Level 1 — Basic pipeline	Automated training pipeline, partial CI/CD and basic versioning	High: process redesign required
Level 2 — Complete pipeline	Model and code CI/CD, production monitoring and a model registry	Moderate: adaptation and improvement
Level 3 — Mature MLOps	Automated retraining, feature store, drift monitoring and A/B testing	Low: rapid operational takeover

Most AI projects in SMEs and mid-sized companies fall between levels zero and one. Anticipating the cost of improving MLOps maturity is an essential part of the takeover budget.

Assess documentation debt

Documentation debt—the gap between what should be documented and what actually is—is often the most underestimated hidden cost of an AI project takeover. Create an inventory:

System architecture: is there an up-to-date architecture diagram?
Data dictionary: are model features described, including the logic used to calculate them?
Decision history: why this framework? Why this model architecture?
Operational runbooks: what should happen if a pipeline fails, a model deteriorates or a data incident occurs?

Every missing item translates into hours of reverse engineering during the takeover. On a medium-sized project, reconstructing documentation can account for 15–25% of the takeover budget.

The Complete 50-Point Audit Checklist

Here is the consolidated checklist, ready to use when auditing an AI project for takeover:

Code and architecture: 12 points

Cyclomatic complexity below 15 per function.
Test coverage above 60%.
Code duplication below 5%.
No critical code smells.
Clear separation of business code, ML code, pipelines and infrastructure.
An up-to-date README with installation instructions.
Docstrings for critical functions.
Architecture Decision Records available.
Consistent naming conventions.
Explicit error handling, with no silent catches.
Structured logging in place.
Code compiles and tests pass in a clean environment.

Models and data: 12 points

Complete inventory of production models.
Documented performance metrics for each model.
Last training date less than six months ago, or a justification.
Drift monitoring with alerts.
A documented or automated retraining pipeline.
Accessible, versioned training data.
Measured data quality: missing values, duplicates and inconsistencies.
Bias assessed and documented.
Training reproducibility verified.
Hyperparameters and experiment results tracked.
Feature engineering documented.
Inference latency measured and acceptable.

Dependencies and infrastructure: 10 points

A complete dependency list with pinned versions.
No unpatched critical CVEs.
No abandoned dependencies, with the most recent commit less than one year old.
Version compatibility verified.
A containerized environment using Docker.
Infrastructure as code in place.
A CI/CD pipeline covering code AND models.
A working model registry.
Separate development, staging and production environments.
A documented and tested rollback procedure.

Security and compliance: 10 points

Vulnerability scanning completed for code and dependencies.
Protection against adversarial attacks assessed.
Data pipelines protected against poisoning.
Inference API authenticated and rate-limited.
AI Act classification determined.
Traceability of model decisions ensured.
GDPR compliance of training data verified.
Open-source licenses compatible with commercial use.
Intellectual property ownership of the code clarified.
Rights to pretrained models verified.

Organization and maintainability: 6 points

Bus factor greater than one.
Deployment and rollback procedures documented.
MLOps maturity assessed.
Skills required for maintenance identified.
Documentation reconstruction costs estimated.
A transition and knowledge transfer plan established.

Audit Methodology: From Access to the Report

Phase 1 — Scoping and access: one to two days

Before touching the code, secure access and define the scope:

Access to the Git repository, including the complete commit history.
Access to development, staging and production environments.
Access to monitoring and logging tools.
Access to existing documentation, if any.
Identification of technical contacts on the outgoing team.

Scoping also establishes priorities: a pre-acquisition audit does not have the same objectives as an operational takeover audit. Adjust the depth of each dimension accordingly.

Phase 2 — Automated analysis: two to three days

Run automated analysis tools in parallel:

Static analysis: SonarQube or an equivalent across the entire codebase.
Security scanning: Snyk or pip-audit for dependencies and OWASP ZAP for APIs.
Dependency analysis: generate a dependency graph and identify conflicts.
Git metrics: analyze commit history, including frequency, size, contributors and modified code areas.

Git metrics are particularly revealing. A file changed in fifteen commits over a month is probably unstable. An entire module with no commits in six months may be dead code—or code nobody dares to touch.

Phase 3 — In-depth manual analysis: three to five days

Automated analysis is not enough. Human expertise is essential to evaluate:

Overall architectural consistency.
ML code quality: algorithm choices, feature engineering and cross-validation.
The appropriateness of technical choices: framework, infrastructure and pipeline.
Model drift risks and data robustness.
Documentation debt and the cost of reconstructing it.

Arrange question-and-answer sessions with the outgoing team where possible. Every uncertainty left unresolved at this stage will become an unexpected cost during the takeover.

Phase 4 — Report and recommendations: two to three days

The audit report organizes its conclusions around three areas:

A factual assessment: analysis results, measured metrics and areas of compliance or non-compliance.
A risk matrix: each identified risk classified by likelihood and impact, with an estimated remediation cost.
Prioritized recommendations: actions required before takeover, including blockers, within the first 30 days and within the first 90 days.

A complete audit of a medium-sized AI project—20,000–100,000 lines of code and two to five production models—typically takes eight to thirteen business days.

Expensive Mistakes When Taking Over Without an Audit

Underestimating ML technical debt

According to Forrester, 75% of technology decision-makers will face moderate to severe technical debt by 2026. In an AI project, the data-model component amplifies that debt. Taking over without measuring it consistently leads to budget overruns of 40–60% in the first six months—the time spent discovering problems an audit would have uncovered in two weeks.

Ignoring training-serving skew

Training-serving skew—the discrepancy between data seen during training and data processed in production—is the leading cause of silent model deterioration. Without a feature store and systematic validation of input data, the model can produce aberrant results for weeks without anyone noticing. The audit must verify that this risk is addressed.

Neglecting intellectual property

HFS Research reports that 77% of companies cite dependence on systems integrators as a cause of technical debt. If the outgoing provider holds rights to the code, models or data, the takeover can turn into a legal dead end. An IP audit is not an administrative detail: it is a prerequisite.

FAQ

How long does an audit of an existing AI project take?

A complete audit takes eight to thirteen business days for a medium-sized project with 20,000–100,000 lines of code. Duration depends on the number of production models, infrastructure complexity and the quality of existing documentation. A rapid scoping audit for a go/no-go assessment can be completed in three to five days.

Which tools are essential for auditing an AI codebase?

The fundamental tools are SonarQube for static code analysis, Snyk or pip-audit for dependency vulnerabilities, and an OWASP scanner for APIs. On the ML side, MLflow or Weights & Biases help assess model traceability, while Great Expectations or TFDV can audit data quality.

Can you audit an AI project without access to the original team?

Yes, but it costs more. Without a technical contact, allow 30–50% more time for reverse engineering. Automated analysis of code, dependencies and Git metrics remains possible. However, understanding architectural choices and model-specific details requires more manual investigation.

What findings are deal-breakers in a takeover audit?

Three situations justify abandoning a takeover: a complete absence of versioned source code, with no Git repository; dependence on external APIs or data with nontransferable access rights; and a bus factor of zero, meaning nobody is available to explain the system, combined with a complete absence of documentation.

Does the AI Act require a specific audit of AI systems?

The AI Act does not require an “audit” in the strict sense, but it does require technical documentation, risk assessment, decision traceability and human oversight mechanisms for high-risk systems. In practice, a takeover audit covering these dimensions helps verify system compliance and identify the work needed to achieve it.

How do you estimate remediation costs after an audit?

Add together three components: fixing critical security and dependency vulnerabilities; addressing documentation debt, representing 15–25% of the takeover budget; and upgrading the MLOps infrastructure. For a project at MLOps level zero or one, upgrading to a functional level two typically costs 20–40% of the initial development investment.

AI Coder Squad: Audit First, Take Over with Confidence

Taking over an existing AI project without a prior assessment exposes you to additional costs, technical roadblocks and regulatory risks that could have been identified earlier. A structured audit by senior developers familiar with production ML systems turns a blind takeover into a costed action plan.

AI Coder Squad designs custom applications and AI agents for companies that want to move fast without sacrificing quality, with senior developers and an AI-powered approach.

→ Start your project and discover how AI Coder Squad can accelerate your next delivery.