Compliance Analysis - Implementation Plan

---
layout: raw-data.njk
title: Compliance Analysis - Implementation Plan
description: The plan developed to guide this compliance analysis
---

## Implementation Plan

This plan was developed during the planning phase and approved before implementation began.

---

# Plan: AI Guidance Compliance Analysis

## Objective
Analyse how the work documented in the Syncopate Transparency Hub complies with the NZ Public Service Generative AI Guidance, and document this analysis as transparency materials.

## Context

### AI Guidance Framework (from MCP server)
The guidance is built on 5 OECD-aligned principles:
1. **Inclusive, sustainable development** - AI should contribute to inclusive growth
2. **Human-centred values** - Respect rule of law, democratic values, human rights, privacy
3. **Transparency and explainability** - Publicly disclose AI use, development, and effects
4. **Safety and security** - Robust risk management and traceability
5. **Accountability** - Human oversight with appropriate authority at every stage

Key requirements include:
- Publish information about AI use (registers, methodologies, intents)
- Clear processes to respond to requests about AI use
- Human-in-the-loop decision making
- Output verification (truthful, non-harmful, factual, lawful, non-discriminatory)
- Designated responsible officials
- Documentation of data inputs and outputs

### Transparency Hub Content
Three projects documented:
1. **API Standard** - 14,657-word standard synthesized from 5,612 nodes, 280 citations
2. **Identification Management** - 7,140-line consolidation from 30 documents, 415+ citations
3. **AI Guidance Evaluation** - Validation use case (with documented misinterpretation)

Transparency approaches used:
- Citation-based traceability (DocRef)
- Complete audit trails (git history, logs)
- Raw data publication
- Structured methodology documentation
- Human oversight documentation

---

## Deliverables

### 1. First Draft Compliance Analysis
**File**: `ai-guidance/transparency-materials/compliance-analysis-draft-1.md`

Structure:
- Executive Summary
- Methodology
- Analysis by Principle (5 sections with DocRef citations)
  - For each principle: requirement summary, evidence from hub, compliance assessment
- Project-Specific Analysis (API Standard, ID Management, AI Guidance)
- Hub-Wide Analysis (transparency approach as a system)
- Gaps and Recommendations
- Conclusion

### 2. Second Draft Compliance Analysis
**File**: `ai-guidance/transparency-materials/compliance-analysis-draft-2.md`
- Incorporate user feedback and additional context
- Refine assessments and recommendations

### 3. Updated ai-guidance/index.md
Add new section describing:
- This compliance analysis project
- Links to both draft files
- Links to transparency materials (prompt, plan)

### 4. Transparency Materials
**Directory**: `ai-guidance/transparency-materials/compliance-analysis/`
- `prompt.md` - The user's original prompt for this analysis
- `plan.md` - This plan document
- Any search retrieval logs from MCP queries

---

## Implementation Steps

### Step 1: Create transparency materials directory structure
- Create `ai-guidance/transparency-materials/compliance-analysis/`
- Save the original prompt as `prompt.md`
- Save this plan as `plan.md`

### Step 2: Query MCP server for specific guidance citations
Use semantic search to gather DocRef citations for each principle area:
- Transparency requirements
- Accountability requirements
- Human oversight requirements
- Documentation requirements
- Publication requirements

### Step 3: Draft first compliance analysis
Write comprehensive analysis covering:
- Each of the 5 OECD principles with specific DocRef citations
- Evidence from each project demonstrating compliance
- Hub-wide transparency approach assessment
- Identified gaps or areas for improvement

### Step 4: Update ai-guidance/index.md
Add section for this compliance analysis with:
- Description of the analysis purpose
- Links to draft files
- Links to transparency materials

### Step 5: Review with user
Present first draft for feedback and additional context

### Step 6: Write second draft
Incorporate user feedback into refined analysis

---

## Files to Create/Modify

| File | Action |
|------|--------|
| `ai-guidance/transparency-materials/compliance-analysis/prompt.md` | Create |
| `ai-guidance/transparency-materials/compliance-analysis/plan.md` | Create |
| `ai-guidance/transparency-materials/compliance-analysis-draft-1.md` | Create |
| `ai-guidance/transparency-materials/compliance-analysis-draft-2.md` | Create (after feedback) |
| `ai-guidance/index.md` | Modify (add new section) |

---

## Key Compliance Areas to Assess

Based on the AI guidance, the analysis will evaluate:

1. **Transparency** - Is AI use clearly disclosed? Are processes documented?
2. **Accountability** - Is human oversight documented? Are decisions attributable?
3. **Documentation** - Are inputs/outputs recorded? Is methodology clear?
4. **Publication** - Is information publicly accessible? Are registers maintained?
5. **Verification** - Are outputs checked for accuracy? Is quality assurance documented?
6. **Human-in-the-loop** - Is human review integrated throughout?

## Expected Outcome

A thorough, citation-backed analysis demonstrating how the Transparency Hub itself serves as an exemplar of the AI guidance principles it documents - creating a recursive demonstration of good practice.