Skip to content
felicity.lois// Cloud & devops
Back to Portfolio

Multi-Cloud Security Posture Management

Microsoft Defender for Cloud at Scale

Project Overview

Designed and implemented a comprehensive multi-cloud security solution using Microsoft Defender for Cloud for a Latin American multinational retail conglomerate with operations spanning department stores, home improvement centers, supermarkets, banking, and e-commerce across seven countries. The organization operates a complex, multi-cloud infrastructure with over 2,000 Google Cloud Platform (GCP) projects, hundreds of Google Kubernetes Engine (GKE) clusters, extensive GCP-managed databases, and supplementary workloads on AWS and Azure. The engagement focused on establishing unified security posture management and threat protection across all three major cloud providers (GCP, AWS, Azure) through Microsoft Defender for Cloud, with a primary emphasis on the massive GCP estate. Due to the unprecedented scale of GCP projects and limitations in Azure's native API handling for large multi-cloud environments, custom Python automation was developed to programmatically manage Defender for Cloud configurations, onboard GCP projects at scale, apply intelligent tagging strategies, and dynamically enable or disable Defender plans based on workload types and business requirements.

Problems

  • Fragmented security visibility across 2,000+ GCP projects, AWS accounts, and Azure subscriptions made it impossible to maintain a unified security posture or detect threats consistently.
  • Manual onboarding at scale was infeasible: Connecting thousands of GCP projects to Microsoft Defender for Cloud through the Azure portal would have taken weeks or months and been error-prone.
  • Azure API limitations for managing multi-cloud connectors at this scale resulted in frequent timeouts, throttling, and failures when attempting bulk operations through native tooling.
  • No standardized security policies or compliance monitoring across cloud platforms, creating regulatory risk and inconsistent security controls.
  • Cost inefficiency: Enabling all Defender plans (Servers, Containers, Databases, Storage, etc.) across all 2,000+ GCP projects would have resulted in excessive licensing costs for workloads that didn't require advanced protection.
  • Lack of automation for tagging, grouping, and managing Defender plan enablement/disablement meant any configuration change would require manual intervention at massive scale.
  • Operational overhead: Security and cloud operations teams lacked tools to manage Defender configurations programmatically, making ongoing maintenance unsustainable.

Goals

  • Establish unified multi-cloud security visibility across GCP (primary), AWS, and Azure through Microsoft Defender for Cloud, providing centralized threat detection, security recommendations, and compliance monitoring.
  • Automate onboarding of 2,000+ GCP projects to Defender for Cloud programmatically, overcoming Azure API limitations through custom Python automation with retry logic and batch processing.
  • Implement intelligent tagging and grouping of GCP projects based on workload types (GKE clusters, databases, compute instances, storage) to enable selective plan enablement and cost optimization.
  • Dynamically enable/disable Defender plans at scale based on project groupings and business requirements, ensuring high-risk workloads (production GKE, databases) receive advanced protection while minimizing costs for dev/test environments.
  • Ensure reliability and resilience of automation through error handling, state tracking, and idempotent operations to handle Azure API failures gracefully.
  • Enable compliance monitoring against industry standards (CIS, ISO 27001, PCI-DSS) across all cloud platforms for regulatory reporting and audit readiness.
  • Reduce operational overhead for security and cloud teams by providing self-service tools and automated workflows for Defender for Cloud lifecycle management.

Role & Responsibilities

  • Led the end-to-end design and implementation of Microsoft Defender for Cloud across GCP (2,000+ projects), AWS, and Azure, including architecture design, connector setup, and policy configuration.
  • Developed custom Python automation framework to programmatically onboard GCP projects to Defender for Cloud, manage connector lifecycle, apply tags, and enable/disable Defender plans at scale.
  • Designed tagging and grouping strategy for GCP projects based on workload characteristics (GKE, databases, compute, storage) to enable cost-optimized, risk-based plan enablement.
  • Implemented robust error handling and retry logic to overcome Azure API throttling, timeouts, and failures when managing thousands of multi-cloud connectors.
  • Configured Defender plans (Servers, Containers/GKE, Databases, Storage, Key Vault, Resource Manager) selectively across project groups, balancing security requirements with cost constraints.
  • Established compliance frameworks (CIS, ISO 27001) within Defender for Cloud and configured automated compliance assessments across all connected cloud environments.
  • Collaborated with security, cloud operations, and engineering teams to define security requirements, plan enablement criteria, and operational workflows for ongoing Defender management.
  • Provided training and documentation for internal teams on using Defender for Cloud dashboards, investigating security alerts, and leveraging automation tools for self-service operations.

Architecture & Deployment Flow

  • Microsoft Defender for Cloud serves as the centralized security command center for GCP (primary), AWS, and Azure environments.
  • GCP Security Command Center Integration: Defender for Cloud integrates with Google Cloud's Security Command Center via connectors, ingesting findings, vulnerabilities, and misconfigurations from 2,000+ GCP projects.
  • AWS Security Hub Integration: Defender for Cloud connects to AWS Security Hub for threat detection and compliance monitoring across AWS accounts.
  • Azure-Native Integration: Defender for Cloud natively monitors Azure subscriptions for VMs, storage, databases, Kubernetes (AKS), and other resources.
  • Custom Python automation queries Google Cloud Resource Manager API to retrieve all GCP projects, classifies them by workload type, and creates Defender connectors in batches using Azure SDK.
  • Python script maintains state tracking (SQLite/PostgreSQL) for connector creation status with retry logic and exponential backoff for API failures.
  • Tag-based Defender plan management: Python automation queries connectors by tags and enables/disables specific Defender plans based on workload classification and environment.
  • Compliance standards (CIS, ISO 27001) configured in Defender for Cloud with automated assessments running continuously across all GCP projects.
  • Security alerts forwarded to SIEM (Azure Sentinel) and ticketing systems (ServiceNow, Jira) for incident response workflows.

Key Implementation Details

Multi-Account AWS Organization

Built using Azure SDK for Python (azure-mgmt-security, azure-identity) for Defender management and Google Cloud SDK for GCP project discovery. Key capabilities include bulk connector onboarding with batch processing, tag management, plan lifecycle management with dry-run support, error handling with exponential backoff retry logic, and idempotent operations that can be safely re-run. The framework uses SQLite or PostgreSQL for state tracking and resume capability, with detailed logging and reporting for all operations.

ECS Fargate & Containerization

Azure's multi-cloud connector APIs have aggressive rate limits (~10-20 requests/second) and bulk operations caused timeouts and '429 Too Many Requests' errors. Solutions implemented include exponential backoff retry, batch processing of 50-100 connectors at a time, parallelization with throttling using concurrent threads with rate-limiting semaphore, state persistence to enable resume from last successful batch, and health checks to validate connector status after creation and detect silent failures.

CI/CD Pipeline

High-risk production workloads (GKE, production databases, production VMs) have all relevant Defender plans enabled. Staging environments have Containers and Databases plans enabled with Servers plan disabled to reduce costs. Dev/test environments only have basic CSPM (free) enabled with all paid plans disabled. Defender for Storage is enabled only for projects with sensitive data (PII, financial records). This strategy resulted in an estimated 40-60% reduction in Defender licensing costs compared to blanket enablement across all 2,000+ projects.

Security & Compliance

Defender for Containers protects GKE clusters with runtime threat detection, vulnerability scanning, and Kubernetes admission control, integrated with Google Container Analysis API for container image scanning and alerting on suspicious pod behavior, privilege escalation, and crypto-mining activity. Defender for Databases monitors Cloud SQL (MySQL, PostgreSQL, SQL Server), Spanner, and open-source databases, detecting SQL injection attempts, anomalous access patterns, and credential theft, with vulnerability assessments and compliance checks for encryption and access controls.

Results & Impact

  • Successfully onboarded 2,000+ GCP projects to Microsoft Defender for Cloud programmatically, achieving full visibility and threat protection across the organization's primary cloud platform.
  • Reduced onboarding time from weeks to days through custom Python automation, overcoming Azure API limitations and eliminating manual connector creation.
  • Achieved 40-60% cost optimization by implementing intelligent, tag-based Defender plan enablement, ensuring high-risk workloads received advanced protection while minimizing costs for dev/test environments.
  • Established unified multi-cloud security posture across GCP, AWS, and Azure, enabling centralized threat detection, compliance monitoring, and security recommendation management from a single platform.
  • Improved operational efficiency by providing security and cloud operations teams with automated, self-service tools for managing Defender configurations at scale, reducing manual effort and operational overhead.
  • Enhanced compliance readiness through automated assessments against CIS, ISO 27001, and custom regulatory frameworks, simplifying audit preparation and regulatory reporting.
  • Built resilient, production-grade automation with robust error handling, retry logic, state tracking, and idempotent operations to reliably manage thousands of cloud resources despite API limitations.

Technology Stack

Security Platform

  • Microsoft Defender for Cloud
  • Google Cloud Security Command Center
  • AWS Security Hub

Automation & Orchestration

  • Python 3.9+
  • Azure SDK for Python
  • Google Cloud SDK for Python
  • SQLite
  • PostgreSQL

Compliance & Reporting

  • CIS Google Cloud Platform Foundation Benchmark
  • ISO 27001

Defender Plans Configured

  • Defender for Servers
  • Defender for Containers
  • Defender for Databases
  • Defender for Storage
  • Defender for Resource Manager

CI/CD & Deployment

  • Google Cloud Platform (2
  • 000+ projects)
  • GKE
  • Cloud SQL
  • Compute Engine
  • Cloud Storage
  • AWS
  • Azure

Integration & Alerting

  • Azure Sentinel
  • ServiceNow
  • Jira