aisentry Audit Report

Overall Score

Initial

Controls Detected

of 61

2660

Files Scanned

2.3s

Recommendations

Category Scores

Prompt Security

28/100

PS-01: Prompt Sanitization
Advanced
PS-02: Rate Limiting
Missing
PS-03: Input Validation
Advanced
PS-04: Output Filtering
Advanced
PS-05: Context Window Protection
Missing
PS-06: Red Team Testing
Missing
PS-07: Prompt Anomaly Detection
Missing
PS-08: System Prompt Protection
Missing

3 Detected 0 Partial 5 Missing

Model Security

25/100

MS-01: Access Control
Missing
MS-02: Model Versioning
Missing
MS-03: Dependency Scanning
Missing
MS-04: API Security
Missing
MS-05: Model Source Verification
Advanced
MS-06: Differential Privacy
Intermediate
MS-07: Model Watermarking
Missing
MS-08: Secure Model Loading
Advanced

3 Detected 0 Partial 5 Missing

Data Privacy

31/100

4 Detected 0 Partial 4 Missing

OWASP LLM Top 10

45/100

OWASP-01: LLM01: Prompt Injection Defense
Advanced
OWASP-02: LLM02: Insecure Output Handling
Intermediate
OWASP-03: LLM03: Training Data Poisoning
Partial
OWASP-04: LLM04: Model Denial of Service
Missing
OWASP-05: LLM05: Supply Chain Vulnerabilities
Intermediate
OWASP-06: LLM06: Sensitive Information Disclosure
Advanced
OWASP-07: LLM07: Insecure Plugin Design
Advanced
OWASP-08: LLM08: Excessive Agency
Partial
OWASP-09: LLM09: Overreliance
Advanced
OWASP-10: LLM10: Model Theft
Missing

6 Detected 2 Partial 2 Missing

Blue Team Operations

21/100

BT-01: Model Monitoring
Advanced
BT-02: Drift Detection
Missing
BT-03: Anomaly Detection
Missing
BT-04: Adversarial Attack Detection
Missing
BT-05: AI Incident Response
Missing
BT-06: Model Drift Monitoring
Missing
BT-07: Data Quality Monitoring
Advanced

2 Detected 0 Partial 5 Missing

AI Governance

0/100

GV-01: Model Explainability
Missing
GV-02: Bias Detection
Missing
GV-03: Model Documentation
Missing
GV-04: Compliance Tracking
Missing
GV-05: Human Oversight
Missing

0 Detected 0 Partial 5 Missing

Supply Chain Security

25/100

SC-01: Dependency Scanning
Missing
SC-02: Model Provenance Tracking
Missing
SC-03: Model Integrity Verification
Advanced

1 Detected 0 Partial 2 Missing

Hallucination Mitigation

35/100

HM-01: RAG Implementation
Advanced
HM-02: Confidence Scoring
Missing
HM-03: Source Attribution
Intermediate
HM-04: Temperature Control
Missing
HM-05: Fact Checking
Intermediate

3 Detected 0 Partial 2 Missing

Ethical AI & Bias

12/100

EA-01: Fairness Metrics
Missing
EA-02: Model Explainability
Intermediate
EA-03: Bias Testing
Missing
EA-04: Model Cards
Missing

1 Detected 0 Partial 3 Missing

Incident Response

0/100

IR-01: Monitoring Integration
Missing
IR-02: Audit Logging
Missing
IR-03: Rollback Capability
Missing

0 Detected 0 Partial 3 Missing

Recommendations

Critical Priority

62 items

Rate Limiting prompt_security

Detection failed: 'bool' object has no attribute 'lower'

Context Window Protection prompt_security

Detection failed: 'bool' object has no attribute 'lower'

Red Team Testing prompt_security

Detection failed: 'ConfigAnalyzer' object has no attribute 'file_exists'

Prompt Anomaly Detection prompt_security

Implement statistical analysis on prompt patterns

Prompt Anomaly Detection prompt_security

Use ML-based anomaly detection for unusual inputs

Prompt Anomaly Detection prompt_security

Set up alerts for prompt anomaly detection

System Prompt Protection prompt_security

Detection failed: 'bool' object has no attribute 'lower'

Access Control model_security

Detection failed: 'bool' object has no attribute 'lower'

Model Versioning model_security

Detection failed: 'bool' object has no attribute 'lower'

Dependency Scanning model_security

Detection failed: 'bool' object has no attribute 'lower'

API Security model_security

Detection failed: 'bool' object has no attribute 'lower'

Model Watermarking model_security

Implement watermarking for model outputs

Model Watermarking model_security

Use cryptographic watermarks for model weights

Model Watermarking model_security

Track watermark verification for model theft detection

Consent Management data_privacy

Detection failed: 'bool' object has no attribute 'lower'

NER PII Detection data_privacy

Use Presidio or SpaCy for NER-based PII detection

NER PII Detection data_privacy

Implement custom NER models for domain-specific PII

NER PII Detection data_privacy

Run PII detection on all inputs and outputs

Data Retention Policy data_privacy

Detection failed: 'bool' object has no attribute 'lower'

GDPR Compliance data_privacy

Detection failed: 'bool' object has no attribute 'lower'

LLM04: Model Denial of Service owasp_llm

Detection failed: 'bool' object has no attribute 'lower'

LLM10: Model Theft owasp_llm

Implement rate limiting on API endpoints

LLM10: Model Theft owasp_llm

Add query logging and anomaly detection

LLM10: Model Theft owasp_llm

Monitor for extraction patterns

Drift Detection blue_team

Implement drift detection with evidently or alibi-detect

Drift Detection blue_team

Monitor input data distribution changes

Drift Detection blue_team

Set up automated alerts for drift events

Anomaly Detection blue_team

Implement anomaly detection on model inputs

Anomaly Detection blue_team

Monitor for unusual query patterns

Anomaly Detection blue_team

Use statistical methods or ML-based detection

Adversarial Attack Detection blue_team

Implement adversarial input detection

Adversarial Attack Detection blue_team

Use adversarial robustness toolkits

Adversarial Attack Detection blue_team

Add input perturbation analysis

AI Incident Response blue_team

Detection failed: 'bool' object has no attribute 'lower'

Model Drift Monitoring blue_team

Use Evidently or alibi-detect for drift monitoring

Model Drift Monitoring blue_team

Set up automated alerts for significant drift

Model Drift Monitoring blue_team

Implement automatic retraining pipelines

Model Explainability governance

Use SHAP or LIME for model explanations

Model Explainability governance

Provide decision explanations in outputs

Model Explainability governance

Implement feature attribution tracking

Bias Detection governance

Use Fairlearn or AIF360 for bias detection

Bias Detection governance

Implement fairness metrics tracking

Bias Detection governance

Test for demographic parity and equalized odds

Model Documentation governance

Detection failed: 'bool' object has no attribute 'lower'

Compliance Tracking governance

Detection failed: 'bool' object has no attribute 'lower'

Human Oversight governance

Detection failed: 'bool' object has no attribute 'lower'

Dependency Scanning supply_chain

Detection failed: 'bool' object has no attribute 'lower'

Model Provenance Tracking supply_chain

Use MLflow, DVC, or Weights & Biases for model tracking

Model Provenance Tracking supply_chain

Implement model versioning with metadata

Model Provenance Tracking supply_chain

Maintain model registry with provenance information

Confidence Scoring hallucination

Detection failed: 'bool' object has no attribute 'lower'

Temperature Control hallucination

Detection failed: 'bool' object has no attribute 'lower'

Fairness Metrics ethical_ai

Use Fairlearn or AIF360 for fairness metrics

Fairness Metrics ethical_ai

Implement demographic parity testing

Fairness Metrics ethical_ai

Monitor fairness metrics in production

Bias Testing ethical_ai

Implement adversarial testing for bias

Bias Testing ethical_ai

Test across demographic groups

Bias Testing ethical_ai

Use TextAttack or CheckList for NLP bias testing

Model Cards ethical_ai

Detection failed: 'ConfigAnalyzer' object has no attribute 'file_exists'

Monitoring Integration incident_response

Detection failed: 'bool' object has no attribute 'lower'

Audit Logging incident_response

Detection failed: 'bool' object has no attribute 'lower'

Rollback Capability incident_response

Detection failed: 'bool' object has no attribute 'lower'