Chapter 9 · ProjectVIEW ERP Master SaaS Framework Business continuity is aligned with ISO 22301:2019 and NIST SP 800-34: database RPO of 15 minutes and RTO of 4 hours, full-platform RTO of 8 hours, and failover to a secondary Azure region within 24 hours. Customer data sits on geo-redundant storage (6 copies across 2 Azure regions), backups are integrity-tested, and disaster-recovery drills run from monthly component tests to an annual secondary-region failover. 15 mindatabase RPO 4 h / 8 hdatabase / full-platform RTO 6 copiesgeo-redundant, 2 Azure regions 2 BDon-demand data export 9.1 Overview & Purpose This chapter defines the business-continuity and disaster-recovery (BCDR) obligations governing the Provider’s delivery of the ProjectVIEW ERP Platform. It establishes the backup strategy, replication architecture, failover procedures, recovery-time objectives, data-redundancy commitments, and continuity-planning requirements that ensure Customer Data remains available, durable, and recoverable throughout the Subscription Term. The Provider’s Business Continuity Management System (BCMS) is aligned with ISO 22301:2019 (Security and Resilience — Business Continuity Management Systems) and draws upon the NIST SP 800-34 Rev. 1 (Contingency Planning Guide for Federal Information Systems) and the Microsoft Cloud Enterprise Business Continuity Management (EBCM) Programme. This chapter complements the availability commitments defined in Chapter 3 (Service Level Agreement), the infrastructure-security controls described in Chapter 8 (Information Security & Confidentiality), and the data-protection provisions of Chapter 7 (Data Protection & Privacy). Each Customer’s dedicated private sovereign environment is provisioned with defence-in-depth continuity controls spanning compute, database, storage, and network tiers — ensuring that no single point of failure can result in data loss or extended service interruption beyond the contracted Recovery Point Objective (RPO) and Recovery Time Objective (RTO). 9.2 Scope This chapter covers: (a) Recovery objectives: Contractually binding RPO and RTO commitments for all tiers of the Platform. (b) Backup management: Backup schedules, retention periods, integrity verification, and restoration procedures for virtual machines, databases, and application artefacts. (c) Data redundancy & replication: Azure Storage replication options (LRS, ZRS, GRS), database read-replicas, and geo-redundant failover architecture. (d) Disaster recovery: Failover procedures, secondary-region activation, communication protocols, and service-restoration workflows. (e) Business continuity planning: Business Impact Analysis (BIA), continuity plan maintenance, DR drill schedules, and post-incident review procedures. (f) Data ownership & portability: Customer data-ownership rights, data-export mechanisms, records-management obligations, and e-discovery support. (g) Roles & responsibilities: RACI allocation across Provider and Customer organisations for all BCDR activities. This chapter does not cover application-level security controls (see Chapter 8), personal-data processing (see Chapter 7), or operational-support workflows (see Chapter 4). 9.3 Standards & Framework Alignment The Provider’s BCDR programme is aligned with the following internationally recognised standards and frameworks: Standard / Framework Scope of Alignment Reference ISO 22301:2019 Business Continuity Management Systems — requirements for planning, establishing, implementing, operating, monitoring, reviewing, maintaining and continually improving a BCMS. https://www.iso.org/standard/75106.html ISO 27031:2011 Guidelines for Information and Communication Technology Readiness for Business Continuity (IRBC) — extending ISO 22301 to ICT services. https://www.iso.org/standard/44374.html NIST SP 800-34 Rev. 1 Contingency Planning Guide for Federal Information Systems — BIA methodology, recovery strategies, plan testing, and maintenance. https://csrc.nist.gov/publications/detail/sp/800-34/rev-1/final Azure EBCM Programme Microsoft’s Enterprise Business Continuity Management programme governing Azure platform resilience, availability zones, and geo-redundant infrastructure. https://learn.microsoft.com/en-us/compliance/assurance/assurance-bcdr Azure Well-Architected Framework — Reliability Pillar Design principles for resilient cloud workloads: redundancy, fault isolation, self-healing, and graceful degradation. https://learn.microsoft.com/en-us/azure/well-architected/reliability/ ITIL 4 — Service Continuity Management IT Service Continuity Management (ITSCM) practice ensuring agreed service-continuity levels during and after a disaster. https://www.axelos.com/certifications/itil-service-management 9.4 Recovery Objectives The following contractually binding recovery objectives apply to each Customer’s dedicated private sovereign environment. These objectives define the maximum tolerable data loss (RPO) and the maximum tolerable downtime (RTO) for each tier of the Platform: Recovery Metric Tier Objective Measurement Recovery Point Objective (RPO) Database Tier (SQL Server 2022) 15 minutes Maximum data loss measured from the point of failure to the most recent successful backup or replication checkpoint. Recovery Point Objective (RPO) Virtual Machine Tier 24 hours Maximum data loss for VM-level state (OS, application binaries, configuration). Daily snapshots ensure RPO ≤ 24h. Recovery Point Objective (RPO) File / Blob Storage Tier Near-zero (GRS) Geo-redundant Storage replication provides asynchronous cross-region replication with near-zero RPO under normal conditions. Recovery Time Objective (RTO) Database Tier (SQL Server 2022) 4 hours Maximum elapsed time from incident declaration to restored database availability, including failover to read-replica if provisioned. Recovery Time Objective (RTO) Full Platform (End-to-End) 8 hours Maximum elapsed time from incident declaration to full Platform operational status, including compute, database, networking, and application verification. Recovery Time Objective (RTO) Secondary Region Failover 24 hours Maximum elapsed time for complete failover to secondary Azure region when primary region is declared unrecoverable. Where the Customer has elected the Enhanced DR Option (secondary server with read-only database replicas and high-frequency replication at up to 15-minute intervals), the database-tier RTO is reduced to 1 hour as the secondary VM assumes the primary role with minimal data and time loss. The Enhanced DR Option is subject to additional subscription fees as set out in the applicable Commercial Contract. Azure SQL Database Business Continuity Overview 9.5 Backup Management The Provider maintains a comprehensive backup regime covering all components of each Customer’s dedicated private sovereign environment. Backup operations are performed using Azure Backup and native SQL Server backup capabilities. 9.5.1 Backup Schedule & Retention Component Backup Type Frequency Retention Storage Redundancy SQL Server 2022 Databases Full database backup Daily 7 days (default); extended retention available on request Geo-Redundant Storage (GRS) SQL Server 2022 Databases Transaction log backup Every 15 minutes (Enhanced DR) 72 hours rolling Geo-Redundant Storage (GRS) Virtual Machines (Windows Server 2022) Azure Backup VM snapshot Daily 14 days (default); customisable up to 99 years via Azure policy Geo-Redundant Storage (GRS) Application Artefacts & Configuration Git repository (Master + Client branches) Continuous (on every commit) Indefinite (full version history) Azure DevOps geo-redundant storage Blob / File Storage Azure Storage replication Continuous (synchronous within region) As per Customer retention policy GRS (6 copies across 2 regions) Azure Key Vault Secrets & Keys Azure-managed replication Continuous Indefinite (soft-delete + purge protection enabled) Geo-redundant by default Customised higher-frequency backups may be provisioned upon Customer request subject to additional subscription fees. All backup frequencies and retention periods are documented in the applicable Backup Schedule Addendum annexed to the Commercial Contract. Azure Backup Overview SQL Server Backup to Azure 9.5.2 Backup Integrity Verification The Provider shall perform the following backup-integrity verification procedures: (a) Automated checksum validation: Every backup operation generates a cryptographic checksum (SHA-256). The checksum is verified automatically upon completion and logged in the backup audit trail. (b) Monthly restore test: At least once per calendar month, the Provider shall perform a test restoration of a randomly selected database backup to an isolated environment, verifying data completeness, referential integrity, and application-layer functionality. (c) Quarterly full-stack restore drill: At least once per calendar quarter, the Provider shall perform a complete platform restoration (VM + database + application + configuration) to an isolated environment, simulating a full disaster-recovery scenario. Results are documented in the DR Drill Report. (d) Backup monitoring & alerting: Azure Monitor alerts are configured to notify the Provider’s Cloud Operations team within 15 minutes of any backup failure, timeout, or integrity-check failure. Failed backups trigger an automatic retry within 1 hour. 9.6 Data Redundancy & Replication Each Customer’s dedicated private sovereign environment leverages Azure’s multi-layered data-redundancy architecture to protect against hardware failure, facility failure, and regional disaster: Replication Option Copies Regions Durability Use Case Locally Redundant Storage (LRS) 3 copies 1 region, 1 facility 11 nines (99.999999999%) Default for non-critical temporary storage within a single data-centre. Zone-Redundant Storage (ZRS) 3 copies 1 region, 2–3 availability zones 12 nines Protection against single availability-zone failure within a region. Used for application configuration stores. Geo-Redundant Storage (GRS) 6 copies 2 regions (primary + secondary, hundreds of km apart) 16 nines Default for all Customer Data. Provides cross-region durability with automatic failover capability. Read-Access Geo-Redundant Storage (RA-GRS) 6 copies 2 regions with read access to secondary 16 nines Available upon Customer election for read-only access to replicated data during primary-region outage. By default, each Customer’s Azure Storage account is provisioned with Geo-Redundant Storage (GRS), maintaining six copies of Customer Data — three within the primary Azure region and three within a geographically paired secondary region. In the event of a primary-region failure, Azure Storage automatically fails over to the secondary region, ensuring data durability without manual intervention. Azure Storage Redundancy 9.6.1 Database Replication SQL Server 2022 databases within each Customer’s environment support the following replication configurations: (a) Standard configuration: Daily full backups with 7-day retention, stored in GRS. Transaction logs are backed up according to the configured schedule (default: hourly; Enhanced DR: every 15 minutes). (b) Enhanced DR configuration (optional): A secondary SQL Server instance is provisioned with read-only replicas of all Customer databases. High-frequency log shipping (up to 15-minute intervals) ensures near-real-time data synchronisation. In the event of a primary-server failure, the secondary server automatically assumes the primary role, providing continuity with minimal data loss (RPO ≤ 15 minutes) and rapid recovery (RTO ≤ 1 hour). (c) Data residency compliance: Data replication is constrained to the Customer’s selected geographic region unless the Customer explicitly authorises cross-region replication in writing. Data cannot be transmitted outside the selected geographic area without Customer consent, in compliance with GDPR Article 44 and UAE PDPL Article 22 cross-border transfer requirements (see Chapter 7, §7.14). 9.7 Disaster Recovery Procedures The Provider maintains documented disaster-recovery procedures for each tier of the Platform. These procedures are tested at least quarterly and updated following any material change to the Platform architecture, Azure service capabilities, or Customer environment configuration. 9.7.1 Incident Classification & Escalation Disaster-recovery events are classified according to the following severity taxonomy: DR Severity Description Example Scenarios Escalation DR-1: Critical Complete loss of primary environment — compute, database, and/or networking fully unavailable. No workaround exists. Azure region-wide outage; primary VM and all backups corrupted; data-centre destruction. Immediate: CTO + Cloud Ops Lead + Customer Account Manager. Customer notified within 30 minutes. DR-2: Major Partial loss of a critical tier — service is degraded but partially functional, or a single-tier failure with workaround available. Primary database server failure (read-replica available); VM failure (backup restorable); storage account degradation. Within 1 hour: Cloud Ops Lead + DBA. Customer notified within 2 hours. DR-3: Moderate Non-critical component failure or performance degradation that does not impact core transactional processing. Backup job failure; monitoring agent offline; secondary-region replication lag exceeding threshold. Within 4 hours: Cloud Ops team. Customer notified in next scheduled status update. DR-4: Minor Precautionary alert or near-miss event requiring investigation but no immediate service impact. Azure health advisory for planned maintenance; storage latency spike resolved automatically; failed DR drill component. Logged and investigated. Customer notified only if escalated to DR-3 or above. 9.7.2 Failover Procedures The following failover procedures are executed in sequence upon declaration of a DR-1 or DR-2 event: Phase Activity Owner Target Duration Success Criteria 1. Detection Azure Monitor / SIEM alert received and acknowledged by Cloud Operations on-call engineer. Cloud Ops (On-Call) < 15 minutes Alert acknowledged; incident ticket created; severity classified. 2. Assessment Determine scope of failure: affected tier(s), data integrity status, availability of read-replicas and recent backups. Cloud Ops Lead + DBA < 30 minutes Root-cause hypothesis documented; recovery path selected. 3. Customer Notification Notify Customer Account Manager and designated Customer contacts via agreed communication channel. Account Manager < 30 min (DR-1) < 2 hr (DR-2) Customer acknowledges receipt; Incident Bridge convened for DR-1. 4. Database Recovery Activate read-replica failover (Enhanced DR) or restore most recent database backup to replacement VM. DBA + Cloud Ops < 1 hr (Enhanced DR) < 4 hr (Standard) Database online; referential-integrity check passed; RPO verified. 5. Compute Recovery Restore VM from Azure Backup snapshot or provision new VM from baseline image + apply latest configuration. Cloud Ops < 4 hours VM operational; application services responding; health-check passed. 6. Application Verification Execute automated smoke tests and manual validation of core transactional workflows (login, data entry, reporting). QA / Cloud Ops < 1 hour All critical paths pass; no data corruption detected. 7. DNS / Network Cutover Update DNS records and/or Azure Traffic Manager to route traffic to restored environment. Verify SSL/TLS certificates. Cloud Ops < 30 minutes End-users can connect; HTTPS verified; latency within SLA. 8. Service Restoration Declare service restored. Resume SLA clock. Begin post-incident monitoring (heightened alerting for 72 hours). Cloud Ops Lead N/A Service restored; SLA metrics resume; monitoring confirmed. 9. Post-Incident Review Conduct Root Cause Analysis (RCA) within 5 Business Days. Publish Post-Incident Report (PIR) to Customer. CTO + Cloud Ops Lead 5 Business Days PIR delivered; corrective actions assigned; BCDR plan updated. 9.7.3 Secondary-Region Failover Where the primary Azure region is declared unrecoverable by Microsoft or by the Provider’s CTO, the Provider shall execute a full secondary-region failover to the geographically paired Azure region. This procedure includes: (a) GRS failover initiation: Azure Storage automatically fails over to the secondary region. RA-GRS accounts provide immediate read access to replicated data. (b) VM re-provisioning: New virtual machines are provisioned in the secondary region from the most recent Azure Backup vault (cross-region restore enabled). (c) Database restoration: SQL Server databases are restored from geo-redundant backups or, where Enhanced DR is provisioned, from the secondary read-replica server in the paired region. (d) DNS reconfiguration: Azure Traffic Manager or DNS records are updated to route all Customer traffic to the secondary-region endpoints. (e) Validation & declaration: Full application-verification suite is executed prior to declaring service restored from the secondary region. The target RTO for a complete secondary-region failover is 24 hours from the point of primary-region declaration. The Customer shall be kept informed at minimum every 2 hours during the failover procedure via the agreed communication channel. Azure Cross-Region Disaster Recovery 9.8 Business Continuity Planning 9.8.1 Business Impact Analysis (BIA) The Provider maintains a Business Impact Analysis for the ProjectVIEW ERP Platform that identifies critical business processes, maps their dependencies on Platform components, and quantifies the impact of service disruption. The BIA is reviewed and updated: (a) At least annually as part of the BCMS management review cycle. (b) Following any material change to the Platform architecture, Azure region configuration, or Customer deployment topology. (c) Following any DR-1 or DR-2 incident that reveals gaps in the existing BIA. The BIA informs the recovery-objective calibration (RPO/RTO), backup-frequency decisions, and replication-architecture design for each Customer environment. 9.8.2 Business Continuity Plan (BCP) The Provider’s Business Continuity Plan documents the procedures, roles, communication protocols, and resource allocations required to maintain or restore Platform services during a disruption. The BCP includes: (a) Crisis Management Team (CMT) composition: CTO (Incident Commander), Cloud Operations Lead, DBA Lead, Customer Account Manager, and Communications Lead. (b) Communication plan: Predefined templates for Customer notification (initial alert, progress update, resolution notice, PIR distribution), internal escalation matrices, and vendor-contact registers (Microsoft Azure Support, domain registrars, SSL providers). (c) Resource inventory: Documented inventory of all Azure resources per Customer environment (VMs, SQL instances, storage accounts, networking components, Key Vault secrets), maintained in the CMDB and validated quarterly. (d) Alternate-site procedures: Step-by-step procedures for activating the secondary Azure region, including pre-provisioned infrastructure templates (Azure Resource Manager / Bicep) and automated deployment scripts. (e) Supplier continuity: Assessment of critical third-party dependencies (Microsoft Azure SLA commitments, domain registrars, certificate authorities) and contingency measures for each. 9.8.3 DR Drill Programme The Provider shall conduct regular DR drills to validate the effectiveness of the BCDR plan and ensure operational readiness: Drill Type Frequency Scope Participants Output Tabletop Exercise Quarterly Walk-through of a hypothetical DR scenario with the CMT to validate decision-making, communication, and escalation procedures. CMT + Customer representative (optional) Tabletop Exercise Report with findings and corrective actions. Component Recovery Test Monthly Isolated restore of a single backup component (database, VM, or storage) to verify recoverability. Cloud Ops + DBA Recovery Test Log with RPO/RTO achieved vs. target. Full-Stack DR Drill Semi-annually End-to-end platform recovery in an isolated environment simulating a DR-1 event, including database failover, VM restoration, and application verification. CMT + QA + DBA + Cloud Ops DR Drill Report with measured RPO/RTO, deviations, and improvement actions. Secondary-Region Failover Drill Annually Complete failover to the paired secondary Azure region with live traffic redirection (during a pre-agreed maintenance window). CMT + Customer + Cloud Ops Regional Failover Report with measured RTO and Customer sign-off. All drill results are documented, retained for a minimum of three (3) years, and made available to the Customer upon written request. Where a drill reveals a deviation from contracted RPO/RTO objectives, the Provider shall implement corrective actions within thirty (30) calendar days and repeat the relevant drill to confirm remediation. 9.9 Customer Data Ownership & Portability The Customer retains sole and exclusive ownership of all Customer Data processed, stored, or generated within the Platform. The Provider does not inspect, approve, monitor, or claim any ownership right over Customer Data at any time, whether during the Subscription Term or following its expiration or termination. Microsoft, as the underlying infrastructure provider, does not know what kind of data Customers choose to store in Azure and does not claim data ownership over Customer information entered into Azure services. 9.9.1 Data Export & Portability The Provider shall make Customer Data available for export in the following manner: (a) On-demand export: The Customer may request a full or partial export of Customer Data at any time during the Subscription Term. The Provider shall deliver the export within two (2) Business Days of receiving a valid written request, in a machine-readable, industry-standard format (SQL backup file, CSV, JSON, or XML as agreed). (b) Termination export: Upon expiration or termination of the Framework, the Provider shall make all Customer Data available for export within two (2) Business Days of the Customer’s written request, in accordance with the data-return provisions of Chapter 7, §7.13 (Data Return on Termination). The overall export window shall remain open for a minimum of sixty (60) calendar days from the effective date of termination. (c) Audit & compliance export: The Customer may request audit logs, access logs, and change-tracking records for compliance purposes. Such exports are provided within two (2) Business Days of the written request. 9.9.2 Records Management The Customer is responsible for identifying its own record-retention requirements in accordance with applicable law and industry regulation. The Provider supports Customer records-management obligations by: (a) Configurable retention policies: The Platform supports configurable data-retention rules at the application level, allowing the Customer to define retention periods for different record categories. (b) Audit-trail preservation: Azure Storage retention policies and SQL Server change tracking provide an immutable audit trail for all data modifications. Audit records are retained for a minimum period agreed in the Commercial Contract (default: seven years). (c) Export for external archival: The Customer may export data and audit reports for external archival at any time during the Subscription Term. The Provider shall facilitate bulk export in standard formats upon written request. 9.9.3 Electronic Discovery (e-Discovery) The Customer is responsible for complying with e-discovery requirements in its use of the Platform. The Provider shall support Customer e-discovery obligations by: (a) Data preservation holds: Upon written request from the Customer citing a litigation-hold or regulatory-investigation requirement, the Provider shall place a preservation hold on specified Customer Data, suspending all automated deletion and retention-policy enforcement for the affected data sets. (b) Data export for e-discovery: The Provider shall export and deliver the specified data sets in a format suitable for e-discovery review (including metadata and audit trails) within two (2) Business Days of the preservation-hold request. (c) Chain-of-custody documentation: The Provider shall provide chain-of-custody documentation for all e-discovery exports, certifying the completeness and integrity of the exported data. 9.10 Azure Platform Resilience The Provider leverages the following Azure-native resilience capabilities within each Customer’s dedicated private sovereign environment: (a) Availability Zones: Where available in the Customer’s selected Azure region, critical resources (VMs, managed disks, load balancers) are deployed across multiple Availability Zones to protect against single-zone failure. (b) Azure Resource Health: Monitors VM health and checks whether resources are running as expected, detecting known Azure platform problems that affect existing resources. Alerts are routed to the Cloud Operations team via Azure Monitor. (c) Azure Backup Vault: Centralised backup management with cross-region restore capability, soft-delete protection (14 days), and multi-user authorisation for critical backup-deletion operations. (d) Azure Resource Locks: CanNotDelete and ReadOnly locks are applied to all production resources to prevent accidental or malicious deletion or modification of critical infrastructure components. (e) Azure DDoS Protection: Standard-tier DDoS protection is enabled on all Customer virtual networks, providing automatic attack mitigation with real-time telemetry and post-attack reporting. (f) Azure Key Vault: Encryption keys, secrets, and certificates are stored in Azure Key Vault with soft-delete and purge-protection enabled, ensuring cryptographic material is recoverable even in the event of accidental deletion. Azure Availability Zones Overview Azure Resource Health Overview Azure Backup Architecture 9.11 Customer Responsibilities The Customer acknowledges and agrees to the following responsibilities in support of the BCDR programme: (a) Contact designation: The Customer shall designate and maintain at least two (2) authorised contacts for DR-related communications, including after-hours emergency contact details. Contact details shall be updated within five (5) Business Days of any personnel change. (b) DR drill participation: The Customer shall make reasonable efforts to participate in annual Secondary-Region Failover Drills and quarterly Tabletop Exercises, providing feedback on communication effectiveness and business-impact accuracy. (c) Retention-policy specification: The Customer shall specify its data-retention requirements in writing during onboarding and update them as needed. The Provider shall configure the Platform retention policies accordingly. (d) Data-export validation: Following any data-export operation (on-demand or termination), the Customer shall verify the completeness and integrity of the exported data within the agreed validation window. Failure to raise a data-completeness objection within the validation window shall constitute acceptance of the export. (e) End-user continuity planning: The Customer is responsible for its own internal business-continuity planning, including end-user communication, manual-process fallbacks, and internal-systems dependencies that are outside the scope of the Platform. 9.12 Continuous Improvement The Provider’s BCDR programme is subject to a continuous-improvement cycle aligned with the Plan-Do-Check-Act (PDCA) methodology prescribed by ISO 22301:2019: (a) Plan: Annual BIA review, recovery-objective calibration, and BCDR plan update based on changes to the Platform, Azure capabilities, and Customer requirements. (b) Do: Execution of backup schedules, replication configurations, and DR drill programme as defined in this chapter. (c) Check: Monitoring of backup success rates, DR drill results (measured RPO/RTO vs. target), post-incident reviews, and Customer feedback. (d) Act: Implementation of corrective actions, architecture improvements, and procedure refinements identified through the Check phase. All improvements are tracked in the BCDR Improvement Register and reported in the quarterly Service Review meeting (see Chapter 4, §4.12). 9.13 Lifetime Software Warranty The Provider warrants that the Platform shall, for the entire duration of the Subscription Term and any renewal thereof (the “ Warranty Period ”): (a) Conformance: perform materially in accordance with the specifications, functionality, and service levels described in this Framework and the applicable Commercial Contract; (b) Defect Remediation: be free from material defects that prevent the Customer from using the Platform for its intended business purposes. Any confirmed defect shall be remediated by the Provider at no additional cost to the Customer within the applicable SLA response and resolution targets defined in Chapter 3; (c) Version Currency: remain on a currently supported and maintained technology stack. The Provider shall proactively upgrade all underlying COTS components (database engine, application framework, operating system, middleware) to vendor-supported versions within the release cycle defined in Section 4.7; (d) Security Maintenance: receive continuous security patching throughout the Warranty Period, including Critical and High vulnerability remediation within the timelines specified in Chapter 8; (e) No Degradation: maintain performance characteristics that meet or exceed the SLO and recovery-objective targets (RPO/RTO) defined in this Framework. The Provider shall not introduce changes that materially degrade Platform performance without Customer approval. This warranty is a lifetime warranty co-extensive with the Customer’s active Subscription. It does not expire, reduce in scope, or require separate renewal. The warranty survives version transitions under Chapter 10 and applies equally to the successor version from its General Availability date. Exclusions: The warranty does not cover defects arising from (i) Customer modifications to the Platform outside the Provider’s change management process; (ii) use of the Platform in a manner inconsistent with its documentation; (iii) third-party software or hardware not supplied or approved by the Provider; or (iv) Force Majeure Events. 9.14 Source Code Escrow To provide the Customer with long-term continuity assurance, the Provider shall maintain a source code escrow arrangement as follows: 9.14.1 Escrow Deposit (a) Escrow Agent: The Provider shall appoint a reputable, independent third-party escrow agent (the “Escrow Agent”) to hold the Platform’s source code. The identity of the Escrow Agent shall be disclosed to the Customer upon request. (b) Deposit Contents: The escrow deposit shall include: (i) the complete source code of the Platform; (ii) build scripts and compilation instructions; (iii) database schemas and migration scripts; (iv) third-party dependency manifests and licence documentation; (v) system architecture documentation; and (vi) deployment procedures sufficient to enable a competent software professional to compile, deploy, and operate the Platform. (c) Update Frequency: The escrow deposit shall be updated within thirty (30) days of each major version release (Upgrade) and at least once per calendar quarter for incremental releases (Updates). (d) Verification: The Escrow Agent shall perform an annual verification of the deposit to confirm completeness and buildability. A summary verification certificate shall be provided to the Customer upon request. 9.14.2 Release Conditions The Escrow Agent shall release the escrowed source code to the Customer upon the occurrence of any of the following release conditions: (a) Insolvency: The Provider enters into liquidation, administration, receivership, or any analogous insolvency proceeding in any jurisdiction; (b) Cessation of Business: The Provider ceases to carry on business or substantially all of its business; (c) Material Breach: The Provider commits a material breach of this Framework that remains unremedied for sixty (60) days following written notice from the Customer; (d) Discontinuation: The Provider permanently discontinues the Platform without providing a commercially reasonable successor product or transition path. 9.14.3 Customer Rights Upon Release Upon release of the escrowed source code, the Customer shall be granted a non-exclusive, non-transferable, irrevocable licence to use, compile, modify, and deploy the source code solely for the purpose of continuing to operate the Platform for the Customer’s own internal business purposes. This licence shall not include the right to redistribute, sublicense, or commercially exploit the source code. 9.14.4 Costs The costs of establishing and maintaining the escrow arrangement, including the Escrow Agent’s fees, deposit updates, and annual verification, shall be borne by the Customer. The Provider shall facilitate the initial set-up with the Escrow Agent and shall co ‑ operate with all deposit and verification activities, but the Customer shall be solely responsible for the Escrow Agent’s fees and any associated costs throughout the Subscription Term. 9.15 Roles & Responsibilities (RACI Matrix) The following RACI matrix allocates responsibilities for key BCDR activities across Provider and Customer roles: Activity CTO Cloud Ops Lead DBA Lead Account Mgr Customer IT Lead Customer CISO BIA review & update A R C I C C BCP maintenance A R C I I I Backup schedule configuration I R R I C I Backup integrity verification I R R I I I DR drill planning & execution A R R C C C DR drill reporting A R C R I I Incident detection & classification I R R I I I Incident communication to Customer I I I R I I Database failover execution I C R I I I VM restoration execution I R C I I I Secondary-region failover A R R C I I Post-incident review (PIR) A R R C C C Data-export fulfilment I R R C R I e-Discovery hold management A R R C R C Retention-policy configuration I C R I R C BCDR improvement tracking A R C I I I Warranty defect remediation A R R C C I Escrow deposit & update A R C I I I Escrow verification (annual) I C I I I I Escrow release adjudication C I I I R C Legend: R = Responsible (executes the activity) · A = Accountable (ultimate decision authority) · C = Consulted (provides input before decision) · I = Informed (notified after decision). 9.16 Process Register The following process register catalogues all formal processes defined in this chapter. Each process is assigned a unique identifier, trigger condition, responsible handler, accountable approver, expected outcome, and cross-reference to the governing section. Process ID Process Name Trigger Handler Approver Outcome Section Ref. P-9.01 BIA Review Annual management review cycle or material architecture change Cloud Ops Lead CTO Updated BIA document with revised impact ratings and recovery-priority rankings. §9.8.1 P-9.02 BCP Update BIA revision, post-incident review findings, or Azure capability change Cloud Ops Lead CTO Updated Business Continuity Plan approved and distributed to CMT. §9.8.2 P-9.03 Backup Configuration New Customer onboarding, environment provisioning, or backup-policy change request DBA + Cloud Ops Cloud Ops Lead Backup schedules, retention periods, and storage redundancy configured and documented. §9.5.1 P-9.04 Backup Integrity Verification Completion of each backup operation; scheduled monthly/quarterly drill Cloud Ops + DBA Cloud Ops Lead Checksum validated; restore test passed; results logged in backup audit trail. §9.5.2 P-9.05 Backup Failure Remediation Azure Monitor alert for backup failure, timeout, or integrity-check failure Cloud Ops (On-Call) Cloud Ops Lead Automatic retry within 1 hour; manual investigation if retry fails; root-cause documented. §9.5.2 P-9.06 DR Incident Detection Azure Monitor / SIEM alert indicating potential service disruption Cloud Ops (On-Call) Cloud Ops Lead Alert acknowledged within 15 min; incident ticket created; severity classified (DR-1 to DR-4). §9.7.1 P-9.07 DR Incident Assessment DR incident detected and classified as DR-1 or DR-2 Cloud Ops Lead + DBA CTO Root-cause hypothesis documented; recovery path selected; CMT convened for DR-1. §9.7.2 P-9.08 Customer DR Notification DR incident classified as DR-1 or DR-2 Account Manager Cloud Ops Lead Customer notified within 30 min (DR-1) or 2 hr (DR-2); Incident Bridge convened for DR-1. §9.7.2 P-9.09 Database Failover DR-1 or DR-2 incident affecting database tier DBA Lead CTO Database online on read-replica or restored from backup; referential- integrity check passed. §9.7.2 P-9.10 VM Restoration DR-1 or DR-2 incident affecting compute tier Cloud Ops Cloud Ops Lead VM restored from Azure Backup snapshot; application services responding; health-check passed. §9.7.2 P-9.11 Application Verification Database and/or VM restoration completed QA / Cloud Ops Cloud Ops Lead Automated smoke tests and manual validation passed; no data corruption detected. §9.7.2 P-9.12 Service Restoration Declaration All recovery phases completed and verified Cloud Ops Lead CTO Service declared restored; SLA clock resumes; heightened monitoring for 72 hours. §9.7.2 P-9.13 Post-Incident Review DR-1 or DR-2 incident resolved Cloud Ops Lead CTO Post-Incident Report (PIR) with RCA, corrective actions, and BCDR plan updates; delivered to Customer within 5 Business Days. §9.7.2 P-9.14 Secondary-Region Failover Primary Azure region declared unrecoverable Cloud Ops + DBA CTO Full platform operational in secondary region within 24 hours; Customer sign-off obtained. §9.7.3 P-9.15 DR Drill Execution Scheduled drill (monthly/quarterly/semi-annual/annual per §9.8.3) Cloud Ops + DBA + QA CTO Drill completed; RPO/RTO measured against targets; Drill Report published. §9.8.3 P-9.16 Data Export Fulfilment Customer written request for on-demand or termination data export Cloud Ops + DBA Account Manager Customer Data exported in agreed format within 2 Business Days; chain-of-custody documented. §9.9.1 P-9.17 e-Discovery Hold Customer written request citing litigation hold or regulatory investigation DBA + Cloud Ops CTO Preservation hold applied; automated deletion suspended; affected data sets exported within 2 Business Days. §9.9.3 P-9.18 BCDR Improvement Tracking DR drill deviation, PIR finding, or BIA gap identified Cloud Ops Lead CTO Corrective action implemented within 30 days; follow-up drill confirms remediation; tracked in Improvement Register. §9.12 P-9.19 Warranty Defect Remediation Customer reports confirmed material defect Provider (Engineering) CTO Defect remediated at no cost within SLA targets; fix deployed via standard release process. §9.13, §3 P-9.20 Escrow Deposit Update Major version release or quarterly release cycle Provider (Engineering) CTO Updated source code, build scripts, schemas, and documentation deposited with Escrow Agent. §9.14.1 P-9.21 Escrow Verification Annual verification cycle Escrow Agent CTO Verification certificate confirming deposit completeness and buildability; copy to Customer on request. §9.14.1 P-9.22 Escrow Release Release condition triggered (insolvency, cessation, material breach, discontinuation) Escrow Agent Customer Source code released to Customer under irrevocable licence; Customer may compile and operate Platform. §9.14.2 9.17 Process Flow Schematic The following nine-stage lifecycle illustrates the end-to-end flow of business-continuity and disaster-recovery management within the Provider’s BCDR programme: ① BIA & Risk Assessment → ② BCP & DR Plan Development → ③ Backup & Replication Configuration ↓ ④ Monitoring & Detection → ⑤ Incident Classification & Escalation → ⑥ Failover & Recovery Execution ↓ ⑦ Service Restoration & Verification → ⑧ Post-Incident Review & PIR → ⑨ Continuous Improvement & Drill Programme Stage ① BIA & Risk Assessment: Annual Business Impact Analysis identifies critical processes, maps Platform dependencies, and quantifies disruption impact. Recovery-priority rankings and RPO/RTO targets are calibrated based on BIA outputs (P-9.01). Stage ② BCP & DR Plan Development: The Business Continuity Plan and DR procedures are developed or updated based on BIA findings, incorporating CMT composition, communication templates, resource inventories, and secondary-region activation procedures (P-9.02). Stage ③ Backup & Replication Configuration: Backup schedules, retention policies, and replication architecture (LRS/ZRS/GRS/RA-GRS) are configured for each Customer environment. Enhanced DR read-replicas are provisioned where elected (P-9.03). Stage ④ Monitoring & Detection: Azure Monitor, SIEM, and Resource Health continuously monitor all Platform components. Backup completion and integrity are verified automatically with alerting on failure (P-9.04, P-9.05, P-9.06). Stage ⑤ Incident Classification & Escalation: Detected incidents are classified (DR-1 to DR-4) and escalated per the severity taxonomy. Customer notification timelines are triggered based on severity level (P-9.07, P-9.08). Stage ⑥ Failover & Recovery Execution: Database failover, VM restoration, and application verification are executed in sequence per the failover procedure table. Secondary-region failover is invoked for DR-1 events where the primary region is unrecoverable (P-9.09 through P-9.11, P-9.14). Stage ⑦ Service Restoration & Verification: Service is declared restored upon completion of all recovery phases. SLA clock resumes and heightened monitoring is maintained for 72 hours. DNS/network cutover is verified (P-9.12). Stage ⑧ Post-Incident Review & PIR: Root Cause Analysis is conducted within 5 Business Days. The Post-Incident Report is delivered to the Customer with corrective actions, BCDR plan updates, and lessons learned (P-9.13). Stage ⑨ Continuous Improvement & Drill Programme: DR drills (tabletop, component, full-stack, regional) are executed per the scheduled programme. Deviations from RPO/RTO targets trigger corrective actions tracked in the Improvement Register. The PDCA cycle loops back to Stage ① (P-9.15, P-9.18). ← PreviousInformation Security & ConfidentialityNext →End-of-Life, Version Transition & Subscription Cessation See how ProjectVIEW ERP runs in a dedicated cloud environment for your projects. Book a Demo →
Chapter 9 · ProjectVIEW ERP Master SaaS Framework Business continuity is aligned with ISO 22301:2019 and NIST SP 800-34: database RPO of 15 minutes and RTO of 4 hours, full-platform RTO of 8 hours, and failover to a secondary Azure region within 24 hours. Customer data sits on geo-redundant storage (6 copies across 2 Azure regions), backups are integrity-tested, and disaster-recovery drills run from monthly component tests to an annual secondary-region failover. 15 mindatabase RPO 4 h / 8 hdatabase / full-platform RTO 6 copiesgeo-redundant, 2 Azure regions 2 BDon-demand data export 9.1 Overview & Purpose This chapter defines the business-continuity and disaster-recovery (BCDR) obligations governing the Provider’s delivery of the ProjectVIEW ERP Platform. It establishes the backup strategy, replication architecture, failover procedures, recovery-time objectives, data-redundancy commitments, and continuity-planning requirements that ensure Customer Data remains available, durable, and recoverable throughout the Subscription Term. The Provider’s Business Continuity Management System (BCMS) is aligned with ISO 22301:2019 (Security and Resilience — Business Continuity Management Systems) and draws upon the NIST SP 800-34 Rev. 1 (Contingency Planning Guide for Federal Information Systems) and the Microsoft Cloud Enterprise Business Continuity Management (EBCM) Programme. This chapter complements the availability commitments defined in Chapter 3 (Service Level Agreement), the infrastructure-security controls described in Chapter 8 (Information Security & Confidentiality), and the data-protection provisions of Chapter 7 (Data Protection & Privacy). Each Customer’s dedicated private sovereign environment is provisioned with defence-in-depth continuity controls spanning compute, database, storage, and network tiers — ensuring that no single point of failure can result in data loss or extended service interruption beyond the contracted Recovery Point Objective (RPO) and Recovery Time Objective (RTO). 9.2 Scope This chapter covers: (a) Recovery objectives: Contractually binding RPO and RTO commitments for all tiers of the Platform. (b) Backup management: Backup schedules, retention periods, integrity verification, and restoration procedures for virtual machines, databases, and application artefacts. (c) Data redundancy & replication: Azure Storage replication options (LRS, ZRS, GRS), database read-replicas, and geo-redundant failover architecture. (d) Disaster recovery: Failover procedures, secondary-region activation, communication protocols, and service-restoration workflows. (e) Business continuity planning: Business Impact Analysis (BIA), continuity plan maintenance, DR drill schedules, and post-incident review procedures. (f) Data ownership & portability: Customer data-ownership rights, data-export mechanisms, records-management obligations, and e-discovery support. (g) Roles & responsibilities: RACI allocation across Provider and Customer organisations for all BCDR activities. This chapter does not cover application-level security controls (see Chapter 8), personal-data processing (see Chapter 7), or operational-support workflows (see Chapter 4). 9.3 Standards & Framework Alignment The Provider’s BCDR programme is aligned with the following internationally recognised standards and frameworks: Standard / Framework Scope of Alignment Reference ISO 22301:2019 Business Continuity Management Systems — requirements for planning, establishing, implementing, operating, monitoring, reviewing, maintaining and continually improving a BCMS. https://www.iso.org/standard/75106.html ISO 27031:2011 Guidelines for Information and Communication Technology Readiness for Business Continuity (IRBC) — extending ISO 22301 to ICT services. https://www.iso.org/standard/44374.html NIST SP 800-34 Rev. 1 Contingency Planning Guide for Federal Information Systems — BIA methodology, recovery strategies, plan testing, and maintenance. https://csrc.nist.gov/publications/detail/sp/800-34/rev-1/final Azure EBCM Programme Microsoft’s Enterprise Business Continuity Management programme governing Azure platform resilience, availability zones, and geo-redundant infrastructure. https://learn.microsoft.com/en-us/compliance/assurance/assurance-bcdr Azure Well-Architected Framework — Reliability Pillar Design principles for resilient cloud workloads: redundancy, fault isolation, self-healing, and graceful degradation. https://learn.microsoft.com/en-us/azure/well-architected/reliability/ ITIL 4 — Service Continuity Management IT Service Continuity Management (ITSCM) practice ensuring agreed service-continuity levels during and after a disaster. https://www.axelos.com/certifications/itil-service-management 9.4 Recovery Objectives The following contractually binding recovery objectives apply to each Customer’s dedicated private sovereign environment. These objectives define the maximum tolerable data loss (RPO) and the maximum tolerable downtime (RTO) for each tier of the Platform: Recovery Metric Tier Objective Measurement Recovery Point Objective (RPO) Database Tier (SQL Server 2022) 15 minutes Maximum data loss measured from the point of failure to the most recent successful backup or replication checkpoint. Recovery Point Objective (RPO) Virtual Machine Tier 24 hours Maximum data loss for VM-level state (OS, application binaries, configuration). Daily snapshots ensure RPO ≤ 24h. Recovery Point Objective (RPO) File / Blob Storage Tier Near-zero (GRS) Geo-redundant Storage replication provides asynchronous cross-region replication with near-zero RPO under normal conditions. Recovery Time Objective (RTO) Database Tier (SQL Server 2022) 4 hours Maximum elapsed time from incident declaration to restored database availability, including failover to read-replica if provisioned. Recovery Time Objective (RTO) Full Platform (End-to-End) 8 hours Maximum elapsed time from incident declaration to full Platform operational status, including compute, database, networking, and application verification. Recovery Time Objective (RTO) Secondary Region Failover 24 hours Maximum elapsed time for complete failover to secondary Azure region when primary region is declared unrecoverable. Where the Customer has elected the Enhanced DR Option (secondary server with read-only database replicas and high-frequency replication at up to 15-minute intervals), the database-tier RTO is reduced to 1 hour as the secondary VM assumes the primary role with minimal data and time loss. The Enhanced DR Option is subject to additional subscription fees as set out in the applicable Commercial Contract. Azure SQL Database Business Continuity Overview 9.5 Backup Management The Provider maintains a comprehensive backup regime covering all components of each Customer’s dedicated private sovereign environment. Backup operations are performed using Azure Backup and native SQL Server backup capabilities. 9.5.1 Backup Schedule & Retention Component Backup Type Frequency Retention Storage Redundancy SQL Server 2022 Databases Full database backup Daily 7 days (default); extended retention available on request Geo-Redundant Storage (GRS) SQL Server 2022 Databases Transaction log backup Every 15 minutes (Enhanced DR) 72 hours rolling Geo-Redundant Storage (GRS) Virtual Machines (Windows Server 2022) Azure Backup VM snapshot Daily 14 days (default); customisable up to 99 years via Azure policy Geo-Redundant Storage (GRS) Application Artefacts & Configuration Git repository (Master + Client branches) Continuous (on every commit) Indefinite (full version history) Azure DevOps geo-redundant storage Blob / File Storage Azure Storage replication Continuous (synchronous within region) As per Customer retention policy GRS (6 copies across 2 regions) Azure Key Vault Secrets & Keys Azure-managed replication Continuous Indefinite (soft-delete + purge protection enabled) Geo-redundant by default Customised higher-frequency backups may be provisioned upon Customer request subject to additional subscription fees. All backup frequencies and retention periods are documented in the applicable Backup Schedule Addendum annexed to the Commercial Contract. Azure Backup Overview SQL Server Backup to Azure 9.5.2 Backup Integrity Verification The Provider shall perform the following backup-integrity verification procedures: (a) Automated checksum validation: Every backup operation generates a cryptographic checksum (SHA-256). The checksum is verified automatically upon completion and logged in the backup audit trail. (b) Monthly restore test: At least once per calendar month, the Provider shall perform a test restoration of a randomly selected database backup to an isolated environment, verifying data completeness, referential integrity, and application-layer functionality. (c) Quarterly full-stack restore drill: At least once per calendar quarter, the Provider shall perform a complete platform restoration (VM + database + application + configuration) to an isolated environment, simulating a full disaster-recovery scenario. Results are documented in the DR Drill Report. (d) Backup monitoring & alerting: Azure Monitor alerts are configured to notify the Provider’s Cloud Operations team within 15 minutes of any backup failure, timeout, or integrity-check failure. Failed backups trigger an automatic retry within 1 hour. 9.6 Data Redundancy & Replication Each Customer’s dedicated private sovereign environment leverages Azure’s multi-layered data-redundancy architecture to protect against hardware failure, facility failure, and regional disaster: Replication Option Copies Regions Durability Use Case Locally Redundant Storage (LRS) 3 copies 1 region, 1 facility 11 nines (99.999999999%) Default for non-critical temporary storage within a single data-centre. Zone-Redundant Storage (ZRS) 3 copies 1 region, 2–3 availability zones 12 nines Protection against single availability-zone failure within a region. Used for application configuration stores. Geo-Redundant Storage (GRS) 6 copies 2 regions (primary + secondary, hundreds of km apart) 16 nines Default for all Customer Data. Provides cross-region durability with automatic failover capability. Read-Access Geo-Redundant Storage (RA-GRS) 6 copies 2 regions with read access to secondary 16 nines Available upon Customer election for read-only access to replicated data during primary-region outage. By default, each Customer’s Azure Storage account is provisioned with Geo-Redundant Storage (GRS), maintaining six copies of Customer Data — three within the primary Azure region and three within a geographically paired secondary region. In the event of a primary-region failure, Azure Storage automatically fails over to the secondary region, ensuring data durability without manual intervention. Azure Storage Redundancy 9.6.1 Database Replication SQL Server 2022 databases within each Customer’s environment support the following replication configurations: (a) Standard configuration: Daily full backups with 7-day retention, stored in GRS. Transaction logs are backed up according to the configured schedule (default: hourly; Enhanced DR: every 15 minutes). (b) Enhanced DR configuration (optional): A secondary SQL Server instance is provisioned with read-only replicas of all Customer databases. High-frequency log shipping (up to 15-minute intervals) ensures near-real-time data synchronisation. In the event of a primary-server failure, the secondary server automatically assumes the primary role, providing continuity with minimal data loss (RPO ≤ 15 minutes) and rapid recovery (RTO ≤ 1 hour). (c) Data residency compliance: Data replication is constrained to the Customer’s selected geographic region unless the Customer explicitly authorises cross-region replication in writing. Data cannot be transmitted outside the selected geographic area without Customer consent, in compliance with GDPR Article 44 and UAE PDPL Article 22 cross-border transfer requirements (see Chapter 7, §7.14). 9.7 Disaster Recovery Procedures The Provider maintains documented disaster-recovery procedures for each tier of the Platform. These procedures are tested at least quarterly and updated following any material change to the Platform architecture, Azure service capabilities, or Customer environment configuration. 9.7.1 Incident Classification & Escalation Disaster-recovery events are classified according to the following severity taxonomy: DR Severity Description Example Scenarios Escalation DR-1: Critical Complete loss of primary environment — compute, database, and/or networking fully unavailable. No workaround exists. Azure region-wide outage; primary VM and all backups corrupted; data-centre destruction. Immediate: CTO + Cloud Ops Lead + Customer Account Manager. Customer notified within 30 minutes. DR-2: Major Partial loss of a critical tier — service is degraded but partially functional, or a single-tier failure with workaround available. Primary database server failure (read-replica available); VM failure (backup restorable); storage account degradation. Within 1 hour: Cloud Ops Lead + DBA. Customer notified within 2 hours. DR-3: Moderate Non-critical component failure or performance degradation that does not impact core transactional processing. Backup job failure; monitoring agent offline; secondary-region replication lag exceeding threshold. Within 4 hours: Cloud Ops team. Customer notified in next scheduled status update. DR-4: Minor Precautionary alert or near-miss event requiring investigation but no immediate service impact. Azure health advisory for planned maintenance; storage latency spike resolved automatically; failed DR drill component. Logged and investigated. Customer notified only if escalated to DR-3 or above. 9.7.2 Failover Procedures The following failover procedures are executed in sequence upon declaration of a DR-1 or DR-2 event: Phase Activity Owner Target Duration Success Criteria 1. Detection Azure Monitor / SIEM alert received and acknowledged by Cloud Operations on-call engineer. Cloud Ops (On-Call) < 15 minutes Alert acknowledged; incident ticket created; severity classified. 2. Assessment Determine scope of failure: affected tier(s), data integrity status, availability of read-replicas and recent backups. Cloud Ops Lead + DBA < 30 minutes Root-cause hypothesis documented; recovery path selected. 3. Customer Notification Notify Customer Account Manager and designated Customer contacts via agreed communication channel. Account Manager < 30 min (DR-1) < 2 hr (DR-2) Customer acknowledges receipt; Incident Bridge convened for DR-1. 4. Database Recovery Activate read-replica failover (Enhanced DR) or restore most recent database backup to replacement VM. DBA + Cloud Ops < 1 hr (Enhanced DR) < 4 hr (Standard) Database online; referential-integrity check passed; RPO verified. 5. Compute Recovery Restore VM from Azure Backup snapshot or provision new VM from baseline image + apply latest configuration. Cloud Ops < 4 hours VM operational; application services responding; health-check passed. 6. Application Verification Execute automated smoke tests and manual validation of core transactional workflows (login, data entry, reporting). QA / Cloud Ops < 1 hour All critical paths pass; no data corruption detected. 7. DNS / Network Cutover Update DNS records and/or Azure Traffic Manager to route traffic to restored environment. Verify SSL/TLS certificates. Cloud Ops < 30 minutes End-users can connect; HTTPS verified; latency within SLA. 8. Service Restoration Declare service restored. Resume SLA clock. Begin post-incident monitoring (heightened alerting for 72 hours). Cloud Ops Lead N/A Service restored; SLA metrics resume; monitoring confirmed. 9. Post-Incident Review Conduct Root Cause Analysis (RCA) within 5 Business Days. Publish Post-Incident Report (PIR) to Customer. CTO + Cloud Ops Lead 5 Business Days PIR delivered; corrective actions assigned; BCDR plan updated. 9.7.3 Secondary-Region Failover Where the primary Azure region is declared unrecoverable by Microsoft or by the Provider’s CTO, the Provider shall execute a full secondary-region failover to the geographically paired Azure region. This procedure includes: (a) GRS failover initiation: Azure Storage automatically fails over to the secondary region. RA-GRS accounts provide immediate read access to replicated data. (b) VM re-provisioning: New virtual machines are provisioned in the secondary region from the most recent Azure Backup vault (cross-region restore enabled). (c) Database restoration: SQL Server databases are restored from geo-redundant backups or, where Enhanced DR is provisioned, from the secondary read-replica server in the paired region. (d) DNS reconfiguration: Azure Traffic Manager or DNS records are updated to route all Customer traffic to the secondary-region endpoints. (e) Validation & declaration: Full application-verification suite is executed prior to declaring service restored from the secondary region. The target RTO for a complete secondary-region failover is 24 hours from the point of primary-region declaration. The Customer shall be kept informed at minimum every 2 hours during the failover procedure via the agreed communication channel. Azure Cross-Region Disaster Recovery 9.8 Business Continuity Planning 9.8.1 Business Impact Analysis (BIA) The Provider maintains a Business Impact Analysis for the ProjectVIEW ERP Platform that identifies critical business processes, maps their dependencies on Platform components, and quantifies the impact of service disruption. The BIA is reviewed and updated: (a) At least annually as part of the BCMS management review cycle. (b) Following any material change to the Platform architecture, Azure region configuration, or Customer deployment topology. (c) Following any DR-1 or DR-2 incident that reveals gaps in the existing BIA. The BIA informs the recovery-objective calibration (RPO/RTO), backup-frequency decisions, and replication-architecture design for each Customer environment. 9.8.2 Business Continuity Plan (BCP) The Provider’s Business Continuity Plan documents the procedures, roles, communication protocols, and resource allocations required to maintain or restore Platform services during a disruption. The BCP includes: (a) Crisis Management Team (CMT) composition: CTO (Incident Commander), Cloud Operations Lead, DBA Lead, Customer Account Manager, and Communications Lead. (b) Communication plan: Predefined templates for Customer notification (initial alert, progress update, resolution notice, PIR distribution), internal escalation matrices, and vendor-contact registers (Microsoft Azure Support, domain registrars, SSL providers). (c) Resource inventory: Documented inventory of all Azure resources per Customer environment (VMs, SQL instances, storage accounts, networking components, Key Vault secrets), maintained in the CMDB and validated quarterly. (d) Alternate-site procedures: Step-by-step procedures for activating the secondary Azure region, including pre-provisioned infrastructure templates (Azure Resource Manager / Bicep) and automated deployment scripts. (e) Supplier continuity: Assessment of critical third-party dependencies (Microsoft Azure SLA commitments, domain registrars, certificate authorities) and contingency measures for each. 9.8.3 DR Drill Programme The Provider shall conduct regular DR drills to validate the effectiveness of the BCDR plan and ensure operational readiness: Drill Type Frequency Scope Participants Output Tabletop Exercise Quarterly Walk-through of a hypothetical DR scenario with the CMT to validate decision-making, communication, and escalation procedures. CMT + Customer representative (optional) Tabletop Exercise Report with findings and corrective actions. Component Recovery Test Monthly Isolated restore of a single backup component (database, VM, or storage) to verify recoverability. Cloud Ops + DBA Recovery Test Log with RPO/RTO achieved vs. target. Full-Stack DR Drill Semi-annually End-to-end platform recovery in an isolated environment simulating a DR-1 event, including database failover, VM restoration, and application verification. CMT + QA + DBA + Cloud Ops DR Drill Report with measured RPO/RTO, deviations, and improvement actions. Secondary-Region Failover Drill Annually Complete failover to the paired secondary Azure region with live traffic redirection (during a pre-agreed maintenance window). CMT + Customer + Cloud Ops Regional Failover Report with measured RTO and Customer sign-off. All drill results are documented, retained for a minimum of three (3) years, and made available to the Customer upon written request. Where a drill reveals a deviation from contracted RPO/RTO objectives, the Provider shall implement corrective actions within thirty (30) calendar days and repeat the relevant drill to confirm remediation. 9.9 Customer Data Ownership & Portability The Customer retains sole and exclusive ownership of all Customer Data processed, stored, or generated within the Platform. The Provider does not inspect, approve, monitor, or claim any ownership right over Customer Data at any time, whether during the Subscription Term or following its expiration or termination. Microsoft, as the underlying infrastructure provider, does not know what kind of data Customers choose to store in Azure and does not claim data ownership over Customer information entered into Azure services. 9.9.1 Data Export & Portability The Provider shall make Customer Data available for export in the following manner: (a) On-demand export: The Customer may request a full or partial export of Customer Data at any time during the Subscription Term. The Provider shall deliver the export within two (2) Business Days of receiving a valid written request, in a machine-readable, industry-standard format (SQL backup file, CSV, JSON, or XML as agreed). (b) Termination export: Upon expiration or termination of the Framework, the Provider shall make all Customer Data available for export within two (2) Business Days of the Customer’s written request, in accordance with the data-return provisions of Chapter 7, §7.13 (Data Return on Termination). The overall export window shall remain open for a minimum of sixty (60) calendar days from the effective date of termination. (c) Audit & compliance export: The Customer may request audit logs, access logs, and change-tracking records for compliance purposes. Such exports are provided within two (2) Business Days of the written request. 9.9.2 Records Management The Customer is responsible for identifying its own record-retention requirements in accordance with applicable law and industry regulation. The Provider supports Customer records-management obligations by: (a) Configurable retention policies: The Platform supports configurable data-retention rules at the application level, allowing the Customer to define retention periods for different record categories. (b) Audit-trail preservation: Azure Storage retention policies and SQL Server change tracking provide an immutable audit trail for all data modifications. Audit records are retained for a minimum period agreed in the Commercial Contract (default: seven years). (c) Export for external archival: The Customer may export data and audit reports for external archival at any time during the Subscription Term. The Provider shall facilitate bulk export in standard formats upon written request. 9.9.3 Electronic Discovery (e-Discovery) The Customer is responsible for complying with e-discovery requirements in its use of the Platform. The Provider shall support Customer e-discovery obligations by: (a) Data preservation holds: Upon written request from the Customer citing a litigation-hold or regulatory-investigation requirement, the Provider shall place a preservation hold on specified Customer Data, suspending all automated deletion and retention-policy enforcement for the affected data sets. (b) Data export for e-discovery: The Provider shall export and deliver the specified data sets in a format suitable for e-discovery review (including metadata and audit trails) within two (2) Business Days of the preservation-hold request. (c) Chain-of-custody documentation: The Provider shall provide chain-of-custody documentation for all e-discovery exports, certifying the completeness and integrity of the exported data. 9.10 Azure Platform Resilience The Provider leverages the following Azure-native resilience capabilities within each Customer’s dedicated private sovereign environment: (a) Availability Zones: Where available in the Customer’s selected Azure region, critical resources (VMs, managed disks, load balancers) are deployed across multiple Availability Zones to protect against single-zone failure. (b) Azure Resource Health: Monitors VM health and checks whether resources are running as expected, detecting known Azure platform problems that affect existing resources. Alerts are routed to the Cloud Operations team via Azure Monitor. (c) Azure Backup Vault: Centralised backup management with cross-region restore capability, soft-delete protection (14 days), and multi-user authorisation for critical backup-deletion operations. (d) Azure Resource Locks: CanNotDelete and ReadOnly locks are applied to all production resources to prevent accidental or malicious deletion or modification of critical infrastructure components. (e) Azure DDoS Protection: Standard-tier DDoS protection is enabled on all Customer virtual networks, providing automatic attack mitigation with real-time telemetry and post-attack reporting. (f) Azure Key Vault: Encryption keys, secrets, and certificates are stored in Azure Key Vault with soft-delete and purge-protection enabled, ensuring cryptographic material is recoverable even in the event of accidental deletion. Azure Availability Zones Overview Azure Resource Health Overview Azure Backup Architecture 9.11 Customer Responsibilities The Customer acknowledges and agrees to the following responsibilities in support of the BCDR programme: (a) Contact designation: The Customer shall designate and maintain at least two (2) authorised contacts for DR-related communications, including after-hours emergency contact details. Contact details shall be updated within five (5) Business Days of any personnel change. (b) DR drill participation: The Customer shall make reasonable efforts to participate in annual Secondary-Region Failover Drills and quarterly Tabletop Exercises, providing feedback on communication effectiveness and business-impact accuracy. (c) Retention-policy specification: The Customer shall specify its data-retention requirements in writing during onboarding and update them as needed. The Provider shall configure the Platform retention policies accordingly. (d) Data-export validation: Following any data-export operation (on-demand or termination), the Customer shall verify the completeness and integrity of the exported data within the agreed validation window. Failure to raise a data-completeness objection within the validation window shall constitute acceptance of the export. (e) End-user continuity planning: The Customer is responsible for its own internal business-continuity planning, including end-user communication, manual-process fallbacks, and internal-systems dependencies that are outside the scope of the Platform. 9.12 Continuous Improvement The Provider’s BCDR programme is subject to a continuous-improvement cycle aligned with the Plan-Do-Check-Act (PDCA) methodology prescribed by ISO 22301:2019: (a) Plan: Annual BIA review, recovery-objective calibration, and BCDR plan update based on changes to the Platform, Azure capabilities, and Customer requirements. (b) Do: Execution of backup schedules, replication configurations, and DR drill programme as defined in this chapter. (c) Check: Monitoring of backup success rates, DR drill results (measured RPO/RTO vs. target), post-incident reviews, and Customer feedback. (d) Act: Implementation of corrective actions, architecture improvements, and procedure refinements identified through the Check phase. All improvements are tracked in the BCDR Improvement Register and reported in the quarterly Service Review meeting (see Chapter 4, §4.12). 9.13 Lifetime Software Warranty The Provider warrants that the Platform shall, for the entire duration of the Subscription Term and any renewal thereof (the “ Warranty Period ”): (a) Conformance: perform materially in accordance with the specifications, functionality, and service levels described in this Framework and the applicable Commercial Contract; (b) Defect Remediation: be free from material defects that prevent the Customer from using the Platform for its intended business purposes. Any confirmed defect shall be remediated by the Provider at no additional cost to the Customer within the applicable SLA response and resolution targets defined in Chapter 3; (c) Version Currency: remain on a currently supported and maintained technology stack. The Provider shall proactively upgrade all underlying COTS components (database engine, application framework, operating system, middleware) to vendor-supported versions within the release cycle defined in Section 4.7; (d) Security Maintenance: receive continuous security patching throughout the Warranty Period, including Critical and High vulnerability remediation within the timelines specified in Chapter 8; (e) No Degradation: maintain performance characteristics that meet or exceed the SLO and recovery-objective targets (RPO/RTO) defined in this Framework. The Provider shall not introduce changes that materially degrade Platform performance without Customer approval. This warranty is a lifetime warranty co-extensive with the Customer’s active Subscription. It does not expire, reduce in scope, or require separate renewal. The warranty survives version transitions under Chapter 10 and applies equally to the successor version from its General Availability date. Exclusions: The warranty does not cover defects arising from (i) Customer modifications to the Platform outside the Provider’s change management process; (ii) use of the Platform in a manner inconsistent with its documentation; (iii) third-party software or hardware not supplied or approved by the Provider; or (iv) Force Majeure Events. 9.14 Source Code Escrow To provide the Customer with long-term continuity assurance, the Provider shall maintain a source code escrow arrangement as follows: 9.14.1 Escrow Deposit (a) Escrow Agent: The Provider shall appoint a reputable, independent third-party escrow agent (the “Escrow Agent”) to hold the Platform’s source code. The identity of the Escrow Agent shall be disclosed to the Customer upon request. (b) Deposit Contents: The escrow deposit shall include: (i) the complete source code of the Platform; (ii) build scripts and compilation instructions; (iii) database schemas and migration scripts; (iv) third-party dependency manifests and licence documentation; (v) system architecture documentation; and (vi) deployment procedures sufficient to enable a competent software professional to compile, deploy, and operate the Platform. (c) Update Frequency: The escrow deposit shall be updated within thirty (30) days of each major version release (Upgrade) and at least once per calendar quarter for incremental releases (Updates). (d) Verification: The Escrow Agent shall perform an annual verification of the deposit to confirm completeness and buildability. A summary verification certificate shall be provided to the Customer upon request. 9.14.2 Release Conditions The Escrow Agent shall release the escrowed source code to the Customer upon the occurrence of any of the following release conditions: (a) Insolvency: The Provider enters into liquidation, administration, receivership, or any analogous insolvency proceeding in any jurisdiction; (b) Cessation of Business: The Provider ceases to carry on business or substantially all of its business; (c) Material Breach: The Provider commits a material breach of this Framework that remains unremedied for sixty (60) days following written notice from the Customer; (d) Discontinuation: The Provider permanently discontinues the Platform without providing a commercially reasonable successor product or transition path. 9.14.3 Customer Rights Upon Release Upon release of the escrowed source code, the Customer shall be granted a non-exclusive, non-transferable, irrevocable licence to use, compile, modify, and deploy the source code solely for the purpose of continuing to operate the Platform for the Customer’s own internal business purposes. This licence shall not include the right to redistribute, sublicense, or commercially exploit the source code. 9.14.4 Costs The costs of establishing and maintaining the escrow arrangement, including the Escrow Agent’s fees, deposit updates, and annual verification, shall be borne by the Customer. The Provider shall facilitate the initial set-up with the Escrow Agent and shall co ‑ operate with all deposit and verification activities, but the Customer shall be solely responsible for the Escrow Agent’s fees and any associated costs throughout the Subscription Term. 9.15 Roles & Responsibilities (RACI Matrix) The following RACI matrix allocates responsibilities for key BCDR activities across Provider and Customer roles: Activity CTO Cloud Ops Lead DBA Lead Account Mgr Customer IT Lead Customer CISO BIA review & update A R C I C C BCP maintenance A R C I I I Backup schedule configuration I R R I C I Backup integrity verification I R R I I I DR drill planning & execution A R R C C C DR drill reporting A R C R I I Incident detection & classification I R R I I I Incident communication to Customer I I I R I I Database failover execution I C R I I I VM restoration execution I R C I I I Secondary-region failover A R R C I I Post-incident review (PIR) A R R C C C Data-export fulfilment I R R C R I e-Discovery hold management A R R C R C Retention-policy configuration I C R I R C BCDR improvement tracking A R C I I I Warranty defect remediation A R R C C I Escrow deposit & update A R C I I I Escrow verification (annual) I C I I I I Escrow release adjudication C I I I R C Legend: R = Responsible (executes the activity) · A = Accountable (ultimate decision authority) · C = Consulted (provides input before decision) · I = Informed (notified after decision). 9.16 Process Register The following process register catalogues all formal processes defined in this chapter. Each process is assigned a unique identifier, trigger condition, responsible handler, accountable approver, expected outcome, and cross-reference to the governing section. Process ID Process Name Trigger Handler Approver Outcome Section Ref. P-9.01 BIA Review Annual management review cycle or material architecture change Cloud Ops Lead CTO Updated BIA document with revised impact ratings and recovery-priority rankings. §9.8.1 P-9.02 BCP Update BIA revision, post-incident review findings, or Azure capability change Cloud Ops Lead CTO Updated Business Continuity Plan approved and distributed to CMT. §9.8.2 P-9.03 Backup Configuration New Customer onboarding, environment provisioning, or backup-policy change request DBA + Cloud Ops Cloud Ops Lead Backup schedules, retention periods, and storage redundancy configured and documented. §9.5.1 P-9.04 Backup Integrity Verification Completion of each backup operation; scheduled monthly/quarterly drill Cloud Ops + DBA Cloud Ops Lead Checksum validated; restore test passed; results logged in backup audit trail. §9.5.2 P-9.05 Backup Failure Remediation Azure Monitor alert for backup failure, timeout, or integrity-check failure Cloud Ops (On-Call) Cloud Ops Lead Automatic retry within 1 hour; manual investigation if retry fails; root-cause documented. §9.5.2 P-9.06 DR Incident Detection Azure Monitor / SIEM alert indicating potential service disruption Cloud Ops (On-Call) Cloud Ops Lead Alert acknowledged within 15 min; incident ticket created; severity classified (DR-1 to DR-4). §9.7.1 P-9.07 DR Incident Assessment DR incident detected and classified as DR-1 or DR-2 Cloud Ops Lead + DBA CTO Root-cause hypothesis documented; recovery path selected; CMT convened for DR-1. §9.7.2 P-9.08 Customer DR Notification DR incident classified as DR-1 or DR-2 Account Manager Cloud Ops Lead Customer notified within 30 min (DR-1) or 2 hr (DR-2); Incident Bridge convened for DR-1. §9.7.2 P-9.09 Database Failover DR-1 or DR-2 incident affecting database tier DBA Lead CTO Database online on read-replica or restored from backup; referential- integrity check passed. §9.7.2 P-9.10 VM Restoration DR-1 or DR-2 incident affecting compute tier Cloud Ops Cloud Ops Lead VM restored from Azure Backup snapshot; application services responding; health-check passed. §9.7.2 P-9.11 Application Verification Database and/or VM restoration completed QA / Cloud Ops Cloud Ops Lead Automated smoke tests and manual validation passed; no data corruption detected. §9.7.2 P-9.12 Service Restoration Declaration All recovery phases completed and verified Cloud Ops Lead CTO Service declared restored; SLA clock resumes; heightened monitoring for 72 hours. §9.7.2 P-9.13 Post-Incident Review DR-1 or DR-2 incident resolved Cloud Ops Lead CTO Post-Incident Report (PIR) with RCA, corrective actions, and BCDR plan updates; delivered to Customer within 5 Business Days. §9.7.2 P-9.14 Secondary-Region Failover Primary Azure region declared unrecoverable Cloud Ops + DBA CTO Full platform operational in secondary region within 24 hours; Customer sign-off obtained. §9.7.3 P-9.15 DR Drill Execution Scheduled drill (monthly/quarterly/semi-annual/annual per §9.8.3) Cloud Ops + DBA + QA CTO Drill completed; RPO/RTO measured against targets; Drill Report published. §9.8.3 P-9.16 Data Export Fulfilment Customer written request for on-demand or termination data export Cloud Ops + DBA Account Manager Customer Data exported in agreed format within 2 Business Days; chain-of-custody documented. §9.9.1 P-9.17 e-Discovery Hold Customer written request citing litigation hold or regulatory investigation DBA + Cloud Ops CTO Preservation hold applied; automated deletion suspended; affected data sets exported within 2 Business Days. §9.9.3 P-9.18 BCDR Improvement Tracking DR drill deviation, PIR finding, or BIA gap identified Cloud Ops Lead CTO Corrective action implemented within 30 days; follow-up drill confirms remediation; tracked in Improvement Register. §9.12 P-9.19 Warranty Defect Remediation Customer reports confirmed material defect Provider (Engineering) CTO Defect remediated at no cost within SLA targets; fix deployed via standard release process. §9.13, §3 P-9.20 Escrow Deposit Update Major version release or quarterly release cycle Provider (Engineering) CTO Updated source code, build scripts, schemas, and documentation deposited with Escrow Agent. §9.14.1 P-9.21 Escrow Verification Annual verification cycle Escrow Agent CTO Verification certificate confirming deposit completeness and buildability; copy to Customer on request. §9.14.1 P-9.22 Escrow Release Release condition triggered (insolvency, cessation, material breach, discontinuation) Escrow Agent Customer Source code released to Customer under irrevocable licence; Customer may compile and operate Platform. §9.14.2 9.17 Process Flow Schematic The following nine-stage lifecycle illustrates the end-to-end flow of business-continuity and disaster-recovery management within the Provider’s BCDR programme: ① BIA & Risk Assessment → ② BCP & DR Plan Development → ③ Backup & Replication Configuration ↓ ④ Monitoring & Detection → ⑤ Incident Classification & Escalation → ⑥ Failover & Recovery Execution ↓ ⑦ Service Restoration & Verification → ⑧ Post-Incident Review & PIR → ⑨ Continuous Improvement & Drill Programme Stage ① BIA & Risk Assessment: Annual Business Impact Analysis identifies critical processes, maps Platform dependencies, and quantifies disruption impact. Recovery-priority rankings and RPO/RTO targets are calibrated based on BIA outputs (P-9.01). Stage ② BCP & DR Plan Development: The Business Continuity Plan and DR procedures are developed or updated based on BIA findings, incorporating CMT composition, communication templates, resource inventories, and secondary-region activation procedures (P-9.02). Stage ③ Backup & Replication Configuration: Backup schedules, retention policies, and replication architecture (LRS/ZRS/GRS/RA-GRS) are configured for each Customer environment. Enhanced DR read-replicas are provisioned where elected (P-9.03). Stage ④ Monitoring & Detection: Azure Monitor, SIEM, and Resource Health continuously monitor all Platform components. Backup completion and integrity are verified automatically with alerting on failure (P-9.04, P-9.05, P-9.06). Stage ⑤ Incident Classification & Escalation: Detected incidents are classified (DR-1 to DR-4) and escalated per the severity taxonomy. Customer notification timelines are triggered based on severity level (P-9.07, P-9.08). Stage ⑥ Failover & Recovery Execution: Database failover, VM restoration, and application verification are executed in sequence per the failover procedure table. Secondary-region failover is invoked for DR-1 events where the primary region is unrecoverable (P-9.09 through P-9.11, P-9.14). Stage ⑦ Service Restoration & Verification: Service is declared restored upon completion of all recovery phases. SLA clock resumes and heightened monitoring is maintained for 72 hours. DNS/network cutover is verified (P-9.12). Stage ⑧ Post-Incident Review & PIR: Root Cause Analysis is conducted within 5 Business Days. The Post-Incident Report is delivered to the Customer with corrective actions, BCDR plan updates, and lessons learned (P-9.13). Stage ⑨ Continuous Improvement & Drill Programme: DR drills (tabletop, component, full-stack, regional) are executed per the scheduled programme. Deviations from RPO/RTO targets trigger corrective actions tracked in the Improvement Register. The PDCA cycle loops back to Stage ① (P-9.15, P-9.18). ← PreviousInformation Security & ConfidentialityNext →End-of-Life, Version Transition & Subscription Cessation See how ProjectVIEW ERP runs in a dedicated cloud environment for your projects. Book a Demo →