Governance Framework

┌─────────────────────────────────────────────────────────────────────────────┐
│                    DATA GOVERNANCE FRAMEWORK                                  │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  1. ACCESS CONTROL                                                  │    │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐             │    │
│  │  │ Lake         │  │ IAM Policies │  │ SCPs         │             │    │
│  │  │ Formation    │  │              │  │ (Org-wide)   │             │    │
│  │  └──────────────┘  └──────────────┘  └──────────────┘             │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  2. DATA CLASSIFICATION                                             │    │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐             │    │
│  │  │ Macie        │  │ Custom Tags  │  │ Glue         │             │    │
│  │  │ (PII Detect) │  │              │  │ Classifiers  │             │    │
│  │  └──────────────┘  └──────────────┘  └──────────────┘             │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  3. COMPLIANCE MONITORING                                           │    │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐             │    │
│  │  │ AWS Config   │  │ CloudTrail   │  │ Security     │             │    │
│  │  │ Rules        │  │              │  │ Hub          │             │    │
│  │  └──────────────┘  └──────────────┘  └──────────────┘             │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  4. AUDIT & LINEAGE                                                 │    │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐             │    │
│  │  │ CloudTrail   │  │ Glue Lineage │  │ CloudWatch   │             │    │
│  │  │ (API Audit)  │  │              │  │ Logs         │             │    │
│  │  └──────────────┘  └──────────────┘  └──────────────┘             │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────────────────┘

Amazon Macie

import boto3

macie = boto3.client('macie2')

# Enable Macie
macie.enable_macie(
    findingPublishingFrequency='FIFTEEN_MINUTES',
    status='ENABLED'
)

# Create classification job
response = macie.create_classification_job(
    jobType='ONE_TIME',
    name='s3-data-classification',
    s3JobDefinition={
        'bucketDefinitions': [
            {
                'accountId': '123456789012',
                'buckets': ['data-lake-raw', 'data-lake-processed']
            }
        ],
        'scopingRule={
            'excludes': {
                'and': [{
                    'simpleCriterion': {
                        'comparator': 'EQ',
                        'key': 'OBJECT_EXTENSION',
                        'values': ['.log', '.tmp']
                    }
                }]
            }
        }'
    },
    schedulingFrequency='DAILY'
)

# Get findings
findings = macie.list_findings(
    findingCriteria={
        'criterion': {
            'classificationDetails.result.status': {
                'eq': ['COMPLETE']
            }
        }
    }
)

AWS Config Rules

import boto3

config = boto3.client('config')

# Check S3 bucket encryption
config.put_config_rule(
    ConfigRule={
        'ConfigRuleName': 's3-bucket-server-side-encryption-enabled',
        'Source': {
            'Owner': 'AWS',
            'SourceIdentifier': 'S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED'
        },
        'Scope': {
            'ComplianceResourceTypes': ['AWS::S3::Bucket']
        }
    }
)

# Custom rule: Check data classification tags
config.put_config_rule(
    ConfigRule={
        'ConfigRuleName': 'data-classification-tag-exists',
        'Source': {
            'Owner': 'CUSTOM_LAMBDA',
            'SourceIdentifier': 'arn:aws:lambda:us-east-1:123456789012:function:check-classification-tag',
            'SourceDetails': [
                {
                    'EventSource': 'aws.config',
                    'MessageType': 'ConfigurationItemChangeNotification'
                }
            ]
        },
        'Scope': {
            'ComplianceResourceTypes': ['AWS::S3::Bucket']
        }
    }
)

Interview Q&A

Q1: What is data governance?

Answer: Data governance is the framework of policies, processes, and standards that ensure data is managed securely, compliantly, and effectively across an organization.

Q2: How does Macie help with governance?

Answer: Macie automatically discovers and classifies sensitive data (PII, financial) in S3, providing visibility into what sensitive data you have and where it's stored.

Q3: What are SCPs in the governance context?

Answer: Service Control Policies are organization-wide guardrails that restrict which AWS services and actions accounts can use, enforcing governance at scale.

Summary

• Access Control: Lake Formation, IAM policies, SCPs
• Classification: Macie for sensitive data discovery
• Compliance: AWS Config rules for resource compliance
• Audit: CloudTrail for API audit trails
• Lineage: Glue Lineage for data flow tracking