Operations Phase Research

Raw Data

This file contains raw search retrieval results or agent logs. The content below shows the original markdown source. 

---
layout: raw-data.njk
title: Operations Phase Research
---

# Operations Phase Research Results

**Phase**: API Operations and Management
**Total Searches**: 7
**Total Results**: ~190 results
**Date**: 2025-11-03

## Search Queries Executed

1. `semantic_search: "API monitoring analytics"` (limit: 30)
2. `semantic_search: "API performance management"` (limit: 30)
3. `semantic_search: "API lifecycle management"` (limit: 30)
4. `semantic_search: "API governance compliance"` (limit: 30)
5. `semantic_search_filtered: tags=["good-practice"], query="API management"` (limit: 40, returned 10)
6. `semantic_search: "SLA service level agreement API"` (limit: 30)
7. `semantic_search: "API deprecation retirement"` (limit: 30)

---

## Search 1: API Monitoring Analytics

**Query**: `semantic_search: "API monitoring analytics"`
**Results**: 30
**Purpose**: Understand monitoring, analytics, and observability requirements

### Top Results

1. **API monitoring and analytics** (Part B) - Score: 0.977
   - Content: API monitoring and analytics ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr6-td1))
   - Tags: definition, definitions, reference-examples-apis, substantive_review

2. **API monitoring and analytics** (Part B) - Score: 0.977
   - Content: API monitoring and analytics ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-fig3-det1-para2-6))
   - Tags: reference-examples-apis, figures, substantive_review

3. **Analytics to monitor APIs** (Part B) - Score: 0.970
   - Content: * analytics to monitor APIs ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr1-td2-l3))
   - Tags: definition, definitions, reference-examples-apis, substantive_review

4. **Tracking and analytics** (Part C) - Score: 0.920
   - Content: tracking and analytics — keeping track of where APIs are deployed, who is using them and how they are being used (this information helps inform other aspects of API governance) ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part2-para3-5))
   - Tags: definition, definitions

5. **Analytics definition** (Part B) - Score: 0.918
   - Content: ==Analytics== Analytics in the context of these guidelines is the capturing and reporting of API usage. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part14-para1-l1))
   - Tags: definition, definitions

6. **Analytics definition** (Part C) - Score: 0.918
   - Content: ==Analytics== Analytics in the context of these guidelines is the capturing and reporting of API usage. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part8-para1-l1))
   - Tags: definition, definitions

7. **Analytics definition** (Part A) - Score: 0.918
   - Content: ==Analytics== Analytics in the context of these guidelines is the capturing and reporting of API usage. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part7-para1-l1))
   - Tags: definition, definitions

8. **Monitoring purposes** (Part B) - Score: 0.918
   - Content: Business owners and security specialists need to be able to monitor the use of APIs to: * monitor uptake of API services * define when to deprecate an old version * profile usage for business * profile usage for security baselines * enforce security policy * detect and respond to security events (SEIM). ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr6))
   - Tags: reference-examples-apis, substantive_review

9. **Monitor uptake of API services** (Part B) - Score: 0.911
   - Content: * monitor uptake of API services ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr6-td2-l2))
   - Tags: definition, definitions, reference-examples-apis, substantive_review

10. **Performance analytics** (Part A) - Score: 0.895
    - Content: Performance analytics enable you to ensure your APIs are meeting SLAs and other contractual obligations. Using analytics to gain insight into API usage should help in making future investment decisions. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part3-section1-para26))
    - Tags: definition, definitions, should-or-may, good-practice, standards_review
    - **Annotation**: "Framed around future use and decision-making so hard to make it mandatory now, but it seems like having some analytics should be a requirement of the standard."

### Key Findings

**Mandatory Requirements Identified:**
- None explicitly mandatory

**Strong Elevation Candidates (should→MUST):**
1. **Analytics for API usage** - "seems like having some analytics should be a requirement" (annotated)
2. **Performance analytics to ensure SLAs met** - Necessary for contractual obligations
3. **Monitoring API usage** - Essential for governance and security

**Key Capabilities:**
- Monitor uptake of API services
- Define when to deprecate old versions
- Profile usage for business
- Profile usage for security baselines
- Enforce security policy
- Detect and respond to security events (SIEM)
- Track API consumers, registrations, and usage
- Ensure APIs meet SLAs

---

## Search 2: API Performance Management

**Query**: `semantic_search: "API performance management"`
**Results**: 30
**Purpose**: Performance requirements, optimization, and QoS

### Top Results

1. **API management** (Part C) - Score: 0.914
   - Content: API management ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part1-section3-para1-tb1-tr2-td1))
   - Tags: definition, definitions, reference-examples-apis, patterns, substantive_review

2. **Performance definition** (Part A) - Score: 0.910
   - Content: performance — maximising API efficiency to support consuming applications and minimising downtime ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part5-section1-para5-2))
   - Tags: definition, definitions

3. **API manager** (Part B) - Score: 0.903
   - Content: API manager ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr2-td1))
   - Tags: definition, definitions, reference-examples-apis, substantive_review

4. **API performance tracking** (Part A) - Score: 0.893
   - Content: API performance — identifying most commonly used APIs calls so they can be made efficient ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-section4-para2-3))
   - Tags: definition, definitions

5. **Performance analytics** (Part A) - Score: 0.884
   - Content: Performance analytics enable you to ensure your APIs are meeting SLAs and other contractual obligations. Using analytics to gain insight into API usage should help in making future investment decisions. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part3-section1-para26))
   - Tags: definition, definitions, should-or-may, good-practice, standards_review
   - **Annotation**: "Framed around future use and decision-making so hard to make it mandatory now, but it seems like having some analytics should be a requirement of the standard."

6. **API manager definition** (Part C) - Score: 0.883
   - Content: ==API manager== The API delivery component that allows API providers to control an API's visibility and behaviour. It is usually exposed as a UI / console to internal staff only, as it is seen as an administration component. It offers a variety of capabilities, including API registration and catalogue administration. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part8-para1-l7))
   - Tags: definition, definitions, catalogue, substantive_review

7. **API manager functions** (Part B) - Score: 0.882
   - Content: The API manager functions cover: * centralised API administration and governance for API catalogues * management of registration and on-boarding processes for communities of API developers * lifecycle management of APIs * applying pre-defined security profiles * security policy administration / definition * policy evaluation. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr2-td2))
   - Tags: definition, definitions, reference-examples-apis, substantive_review

8. **Metrics for developers** (Part A) - Score: 0.877
   - Content: metrics — delivering insight into API usage and performance that could be useful to developers ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part5-section1-para3-7))
   - Tags: definition, definitions

9. **API management availability** (Part A) - Score: 0.865
   - Content: API management looks after availability of the API. This can involve throttling to make sure all consumers can get access to the API within the bounds of the SLA. It can also include quota management, whereby consuming applications are given limited access (for example, a set number of calls per hour) to protect the API from abuse or overuse. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-section3-para15))
   - Tags: definition, definitions, must-or-shall, standards_review

### Key Findings

**Key Topics:**
- API manager as central control point
- Performance metrics: error rate, throughput, response time, transaction speed
- Throttling and quota management
- QoS policies
- Lifecycle management

**API Manager Capabilities:**
- Centralised API administration and governance
- Management of registration and onboarding
- Lifecycle management of APIs
- Security profile application
- Policy administration and evaluation

---

## Search 3: API Lifecycle Management

**Query**: `semantic_search: "API lifecycle management"`
**Results**: 30
**Purpose**: Full lifecycle from creation to retirement

### Top Results

1. **Lifecycle management definition** (Part B) - Score: 0.976
   - Content: * lifecycle management of APIs ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr2-td2-l4))
   - Tags: definition, definitions, reference-examples-apis, substantive_review

2. **Life cycle management** (Part A) - Score: 0.949
   - Content: life cycle management — managing the full life cycle of APIs from initial publication to destruction. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part5-section1-para6-5))
   - Tags: definition, definitions

3. **API lifecycle** (Part A) - Score: 0.927
   - Content: An API has its own life cycle, and it needs to be managed through the whole of its life cycle, from creation to destruction. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-section2-para9))
   - Tags: definition, definitions

4. **Life cycle governance** (Part C) - Score: 0.922
   - Content: life cycle — governing the life cycle of an API, from publication to versioning and deprecation to retirement ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part2-para3-4))
   - Tags: definition, definitions

5. **API service life cycle management figure** (Part A) - Score: 0.917
   - Content: ![API service life cycle management, described in the detailed description.](../../files/Figure7-API-Service-Lifecycle-Management.png/) ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-fig1-src1))
   - Tags: good-practice, standards_review

6. **Standard API life cycle** (Part A) - Score: 0.910
   - Content: The standard API life cycle follows these steps: ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-para2))
   - Tags: definition, definitions

7. **Life cycle / change management** (Part A) - Score: 0.907
   - Content: encompass life cycle / change management, handling API versioning and retirement management ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-para3-5-1))

8. **API components section** (Part A) - Score: 0.894
   - Content: Comprehensive overview of API components including developer portal, gateway, and manager ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part5-section1))
   - Tags: good-practice, standards_review

### Key Findings

**Lifecycle Stages:**
1. Service Strategy (center)
2. Service Design: consumer management, service level management, API design
3. Service Transition: internal development and testing, API catalogue, API release management, life cycle/change management
4. Service Operation: access management, consumer support, incident/event management, API management
5. Continual Service Improvement (outer layer)

**Life Cycle Management Capabilities:**
- Planning and design
- Implementation
- Basic and advanced deployment and running
- Versioning and retirement
- Allowing policies to be applied to individual APIs
- Helping to onboard consuming applications

---

## Search 4: API Governance Compliance

**Query**: `semantic_search: "API governance compliance"`
**Results**: 30
**Purpose**: Governance structures, compliance, and policy enforcement

### Top Results

1. **API governance title** (Part C) - Score: 0.917
   - Content: 2. API governance ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part2-title))

2. **API governance title** (Part A) - Score: 0.910
   - Content: 6. API governance ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part6-title))

3. **Governance definition** (Part A) - Score: 0.909
   - Content: Governance — API governance helps save time and money because it ensures that APIs are built proactively to achieve specific goals and bring value to the agency. API governance also helps agencies make intelligent decisions regarding API programmes and establish best practices for building, deploying and consuming APIs. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part1-section9-para3-9))
   - Tags: definition, definitions

4. **Governance benefits** (Part C) - Score: 0.905
   - Content: API governance also helps agencies make intelligent decisions regarding API programmes and establish best practices for building, deploying and consuming APIs. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part2-para2))

5. **Governance components** (Part C) - Score: 0.896
   - Content: API governance should include: ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part2-para3))
   - Tags: definition, definitions, patterns, substantive_review

6. **Governance purpose** (Part C) - Score: 0.895
   - Content: API governance helps save time and money because it ensures that APIs are built proactively to achieve specific goals and bring value to the agency. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part2-para1))

7. **Limited guidance note** (Part A) - Score: 0.883
   - Content: This version of the API guidelines does not yet offer detailed guidance on API governance. See [section 2. API governance in Part C: API development 2022] ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part6-para1))
   - Tags: good-practice, standards_review
   - **Annotation**: "Note - little on API Governance."

8. **Governance efficiency** (Part C) - Score: 0.880
   - Content: When your API designers and developers are on the same page regarding APIs, the more efficient, valuable and successful your API programmes will be. Governance is especially beneficial for agencies that have a large API programme or microservices architectures. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part2-para4))

9. **Centralised governance** (Part B) - Score: 0.880
   - Content: * centralised API administration and governance for API catalogues ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr2-td2-l2))
   - Tags: definition, definitions, reference-examples-apis, catalogue, substantive_review

10. **Tracking and analytics for governance** (Part C) - Score: 0.878
    - Content: tracking and analytics — keeping track of where APIs are deployed, who is using them and how they are being used (this information helps inform other aspects of API governance) ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part2-para3-5))
    - Tags: definition, definitions

### Key Findings

**Governance Components:**
- Life cycle governance (publication to versioning, deprecation, retirement)
- Tracking and analytics (where deployed, who's using, how used)
- Centralised API administration for catalogues
- Standards and best practices
- Policy enforcement

**Key Note:**
- Limited detailed guidance on API governance in current guidelines
- Especially beneficial for large API programmes or microservices architectures

---

## Search 5: API Management (Good Practice Filter)

**Query**: `semantic_search_filtered: tags=["good-practice"], query="API management"`
**Results**: 10
**Purpose**: Best practices for API management

### Top Results

1. **Well managed APIs** (Part A) - Score: 0.907
   - Content: 2.8. Well managed - As API take-up increases over time, there is more likelihood of API change, redevelopment and adaptation to meet new needs. A well-managed API should handle regular change. API management gives an agency the capability to deliver good version control capability and forewarn application developers of changes that may impact them, including failures and outages. For this reason, it is important that all application developers are identified, even if the API is considered public. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part2-section8))
   - Tags: should-or-may, must-or-shall, good-practice, standards_review

2. **Version control and communication** (Part A) - Score: 0.903
   - Content: API management gives an agency the capability to deliver good version control capability and forewarn application developers of changes that may impact them, including failures and outages. For this reason, it is important that all application developers are identified, even if the API is considered public. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part2-section8-para2))
   - Tags: should-or-may, good-practice, standards_review

3. **API components** (Part A) - Score: 0.901
   - Content: Comprehensive overview of API developer portal, gateway, and manager components ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part5-section1))
   - Tags: good-practice, standards_review

4. **Release management** (Part A) - Score: 0.899
   - Content: Release management is an important aspect of API transition. The aim with API development is to make small changes and release often. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-section2-para6))
   - Tags: definition, definitions, good-practice, should-or-may, standards_review

5. **Well-managed change** (Part A) - Score: 0.878
   - Content: As API take-up increases over time, there is more likelihood of API change, redevelopment and adaptation to meet new needs. A well-managed API should handle regular change. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part2-section8-para1))
   - Tags: should-or-may, good-practice, standards_review

### Key Findings

**Good Practice Principles:**
1. **Well-managed APIs handle regular change** - Core principle
2. **Small changes, release often** - Agile approach to API development
3. **Identify all application developers** - Even for public APIs (for notifications)
4. **Version control capability** - Essential for managing change
5. **Forewarn developers of changes** - Including failures and outages
6. **Monitor and improve** - Not just gather data, but use it to improve offerings

---

## Search 6: SLA Service Level Agreement API

**Query**: `semantic_search: "SLA service level agreement API"`
**Results**: 30
**Purpose**: SLA requirements, performance metrics, availability

### Top Results

1. **SLA definition** (Part B) - Score: 0.956
   - Content: ==SLA== Service Level Agreement ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part15-para1-l57))
   - Tags: definition, definitions

2. **SLA definition** (Part C) - Score: 0.956
   - Content: ==SLA== Service Level Agreement ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part9-para1-l38))
   - Tags: definition, definitions

3. **SLA definition** (Part A) - Score: 0.956
   - Content: ==SLA== Service Level Agreement ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part8-para1-l24))
   - Tags: definition, definitions

4. **SLA for APIs** (Part C) - Score: 0.948
   - Content: Service Level Agreement (SLA) for the API, identifying performance, availability. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part1-section1-para1-4))

5. **SLAs in portal** (Part B) - Score: 0.932
   - Content: * access to specifications and descriptions of APIs, including Service Level Agreements (SLAs) ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr1-td2-l4))
   - Tags: definition, definitions, reference-examples-apis, substantive_review

6. **Service levels and contracts** (Part A) - Score: 0.910
   - Content: Service levels — APIs often require service level agreements (SLAs) with application developers and suppliers. SLAs are important to understand and will likely evolve over time as the API matures. MOUs and contracts should cater for this scenario. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part1-section9-para2-5))
   - Tags: definition, definitions, should-or-may, good-practice, must-or-shall, standards_review
   - **Annotation**: "This is effectively a requirement."

7. **Define SLA** (Part A) - Score: 0.905
   - Content: define the level of service the application developer (and customer) can expect from the API (SLA) ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-para3-4))

8. **SLAs in developer portal** (Part A) - Score: 0.875
   - Content: SLAs — a place to publish API SLAs, defining performance, availability metrics. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part5-section1-para3-8))
   - Tags: definition, definitions

9. **Service levels considerations** (Part A) - Score: 0.874
   - Content: Service levels — Opening an API channel may necessitate an agency to offer a level of service, where previously there has been no such need. Commercial experience and knowledge could be required in order to define realistic and achievable service levels that serve everyone. These service level changes may require additional staff to cover increased support hours, for example. There is also a need to measure, monitor and report back on performance to ensure SLAs are met. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part1-section9-para3-6))
   - Tags: definition, definitions, good-practice, standards_review

10. **SLA lowest common denominator** (Part A) - Score: 0.874
    - Content: Support could involve multiple platforms (some cloud-based, perhaps) and multiple vendors, and therefore the service levels offered in the SLA may be limited to that of the lowest common denominator. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-section1-para8))

### Key Findings

**Mandatory Requirements Identified:**
- None explicitly mandatory, but annotation suggests SLAs are "effectively a requirement"

**Strong Elevation Candidates:**
1. **Define and publish SLAs** - Annotated as "effectively a requirement"
2. **Measure, monitor and report on SLA performance** - Essential for accountability

**SLA Components:**
- Performance metrics
- Availability metrics
- Response time
- Error rates
- Throughput
- Transaction speed

**Key Considerations:**
- SLAs evolve over time as API matures
- May be limited to lowest common denominator when multiple vendors involved
- Require measurement, monitoring, and reporting
- Should be published in developer portal
- Performance analytics needed to ensure SLAs met

---

## Search 7: API Deprecation Retirement

**Query**: `semantic_search: "API deprecation retirement"`
**Results**: 30
**Purpose**: End-of-life management, deprecation policies, retirement process

### Top Results

1. **Life cycle with deprecation** (Part C) - Score: 0.858
   - Content: life cycle — governing the life cycle of an API, from publication to versioning and deprecation to retirement ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partc/2022/en/#part2-para3-4))
   - Tags: definition, definitions

2. **Maintaining old versions** (Part A) - Score: 0.833
   - Content: For major changes, which are not backwards compatible, the old API version should be maintained alongside the new version for an appropriate period to allow all-consuming applications to transition. By monitoring usage, it should be possible to assess when an API version can be deprecated, either because it is no longer being used or because the usage pattern does not warrant maintaining that particular version. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part2-section6-para3))
   - Tags: should-or-may, good-practice, must-or-shall, standards_review
   - **Annotation**: "Arguably should be mandatory."

3. **Versioning and retirement** (Part A) - Score: 0.826
   - Content: versioning and retirement ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part5-section1-para7-4))

4. **Lifecycle management** (Part B) - Score: 0.826
   - Content: * lifecycle management of APIs ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr2-td2-l4))
   - Tags: definition, definitions, reference-examples-apis, substantive_review

5. **Monitoring before retirement** (Part A) - Score: 0.825
   - Content: monitoring usage to make sure an API, or API version, is only retired when the majority of consuming applications have migrated off it ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-section2-para9-2))

6. **Life cycle management** (Part A) - Score: 0.824
   - Content: life cycle management — managing the full life cycle of APIs from initial publication to destruction. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part5-section1-para6-5))
   - Tags: definition, definitions

7. **Trimming APIs** (Part A) - Score: 0.824
   - Content: trimming APIs — removing capabilities that are unused, have never been used and are not likely to be used ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-section2-para9-3))
   - Tags: definition, definitions, good-practice, standards_review

8. **Life cycle / change management** (Part A) - Score: 0.819
   - Content: encompass life cycle / change management, handling API versioning and retirement management ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-parta/2022/en/#part4-para3-5-1))

9. **Ability to know users for deprecation** (Part B) - Score: 0.811
   - Content: The ability of knowing who is using an API cannot be overstated. This is critical when it comes time to implement aspects of the API service life cycle, such as service deprecation, or notification of an outage. ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part3-section1-para3))

10. **Define when to deprecate** (Part B) - Score: 0.810
    - Content: * define when to deprecate an old version ([DocRef](https://docref.digital.govt.nz/nz/dia-apis-partb/2022/en/#part1-section4-para6-tb1-tr6-td2-l3))
    - Tags: definition, definitions, reference-examples-apis, substantive_review

### Key Findings

**Mandatory Requirements Identified:**
- None explicitly mandatory

**Strong Elevation Candidates (should→MUST):**
1. **Maintain old API versions alongside new** - Annotated as "Arguably should be mandatory"
2. **Monitor usage before retirement** - Critical for transition management
3. **Know who is using API** - "Cannot be overstated" for deprecation notifications

**Deprecation and Retirement Process:**
1. Monitor usage patterns to assess deprecation timing
2. Maintain old version alongside new for transition period
3. Ensure majority of consuming applications have migrated
4. Notify all identified API consumers
5. Trim unused capabilities
6. Retire when usage no longer warrants maintenance

**Key Principle:**
- Know all API consumers (critical for deprecation notifications and outage alerts)

---

## Summary of Mandatory Requirements

### Explicitly Mandatory (must-or-shall tags):
1. **API availability monitoring** - Providers need to monitor and respond to demand

### Strong Elevation Candidates (should→MUST):
1. **Analytics for API usage** - Should be a requirement (annotated)
2. **Performance analytics to ensure SLAs met** - Necessary for contractual obligations
3. **Define and publish SLAs** - "Effectively a requirement" (annotated)
4. **Measure, monitor and report on SLA performance** - Essential for accountability
5. **Maintain old API versions alongside new** - "Arguably should be mandatory" (annotated)
6. **Monitor usage before retirement** - Critical for safe transitions
7. **Identify all API consumers** - Even for public APIs (for change notifications)
8. **Version control capability** - Essential for well-managed APIs
9. **Forewarn developers of changes** - Including failures and outages

---

## Key Topics for Operations Phase Section

### 1. Monitoring and Analytics
- **Purpose**: Capturing and reporting API usage
- **Use cases**:
  - Monitor uptake of services
  - Define deprecation timing
  - Profile usage for business and security
  - Enforce security policy
  - Detect and respond to security events (SIEM)
- **Analytics types**:
  - Performance analytics (SLA compliance, investment decisions)
  - Business analytics (usage patterns, ROI)
  - Technical analytics (stability, performance)

### 2. Performance Management
- **Metrics**:
  - Error rate, throughput, response time
  - Transaction speed
  - Backend performance
  - Cache performance
- **Management capabilities**:
  - Throttling (ensure all consumers get access within SLA bounds)
  - Quota management (protect from abuse/overuse)
  - QoS policies
  - API efficiency optimization

### 3. API Lifecycle Management
- **Stages**:
  - Service Strategy
  - Service Design (consumer management, SLM, API design)
  - Service Transition (dev/test, catalogue, release mgmt, lifecycle/change mgmt)
  - Service Operation (access mgmt, support, incident/event mgmt, API mgmt)
  - Continual Service Improvement
- **Capabilities**:
  - Planning and design
  - Implementation
  - Deployment and running
  - Versioning and retirement
  - Policy application
  - Onboarding support

### 4. Governance and Compliance
- **Definition**: Ensures APIs are built proactively to achieve goals and bring value
- **Components**:
  - Life cycle governance (publication → versioning → deprecation → retirement)
  - Tracking and analytics (deployment, usage, users)
  - Centralised administration
  - Standards and best practices
  - Policy enforcement
- **Benefits**:
  - Saves time and money
  - Intelligent decisions on API programmes
  - Especially beneficial for large programmes or microservices

### 5. SLA Management
- **Components**:
  - Performance metrics (response time, throughput)
  - Availability metrics (uptime, reliability)
  - Error rates
- **Requirements**:
  - Define realistic and achievable service levels
  - Publish in developer portal
  - Measure, monitor, and report on performance
  - Evolve over time as API matures
  - Account for multi-vendor/multi-platform scenarios

### 6. Incident Response
- **Logging and alerting**:
  - Correlate API requests with backend activity
  - End-to-end tracing
  - Identify specific consumer requests
  - Detect malicious access attempts
- **Communication**:
  - Notify all identified consumers of outages
  - Forewarn of changes that may impact them
  - Service deprecation notifications

### 7. API Deprecation and Retirement
- **Process**:
  1. Monitor usage to assess deprecation timing
  2. Maintain old version alongside new for transition
  3. Ensure majority have migrated
  4. Notify all consumers
  5. Trim unused capabilities
  6. Retire when usage no longer warrants maintenance
- **Critical requirement**: Know who is using the API (even for public APIs)

---

## Cross-References

**Related to Security Phase**:
- Monitor for security events (SIEM)
- Enforce security policy via monitoring
- Profile usage for security baselines

**Related to Deployment Phase**:
- Version deprecation decisions
- Release management integration
- API catalogue management

**Related to Development Phase**:
- Performance metrics inform optimization
- Analytics drive API improvements
- Testing in SDLC informed by operational data

**Related to Design Phase**:
- Usage analytics inform future design decisions
- Performance data guides design improvements
- Consumer feedback integration

---

## Total Results by Part

- **Part A**: ~70 results (foundational concepts, lifecycle, management)
- **Part B**: ~60 results (security aspects, technical specifications)
- **Part C**: ~60 results (implementation guidance, governance)

**Note**: Operations phase has strong representation across all parts, with Part A providing comprehensive lifecycle and management guidance.