π‘οΈ Lake Formation Deep Dive
Master Lake Formation fine-grained permissions and row/column level security.
Module: AWS Data Engineering β’ Topic 45 of 65 β’ Premium Content
Lake Formation Permission Model
Architecture Diagram
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β LAKE FORMATION PERMISSIONS β
β β
β Permission Levels: β
β β’ Database: CREATE, ALTER, DROP β
β β’ Table: SELECT, INSERT, DELETE, ALTER, DROP β
β β’ Column: SELECT (specific columns) β
β β’ Row: SELECT (with filters) β
β β
β Cross-Account Sharing: β
β β’ Register S3 locations β
β β’ Grant cross-account table access β
β β’ Fine-grained column permissions β
β β’ Centralized governance β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Permission Examples
import boto3
lakeformation = boto3.client('lakeformation')
# Grant table-level permissions
lakeformation.grant_permissions(
Principal={'DataLakePrincipalIdentifier': 'arn:aws:iam::123456789012:role/AnalystRole'},
Resource={'Table': {'DatabaseName': 'db', 'Name': 'sales'}},
Permissions=['SELECT']
)
# Grant column-level permissions
lakeformation.grant_permissions(
Principal={'DataLakePrincipalIdentifier': 'arn:aws:iam::123456789012:role/AnalystRole'},
Resource={
'TableWithColumns': {
'DatabaseName': 'db', 'Name': 'customers',
'ColumnNames': ['id', 'name', 'email'],
'ColumnWildcard': {'ExcludedColumnNames': ['ssn', 'credit_card']}
}
},
Permissions=['SELECT']
)
Interview Q&A
Q1: How does Lake Formation differ from IAM?
Answer: Lake Formation provides data-level permissions (table, column, row). IAM provides AWS resource-level permissions. Both work together.
Q2: What is the Data Lake Administrator?
Answer: An IAM user/role with permissions to manage Lake Formation settings, grant/revoke permissions, and register locations.
Q3: How do you handle row-level security?
Answer: Use tag-based access control or row-level filters in Lake Formation to restrict data visibility per user/group.
Summary
- Permissions: Database, Table, Column, Row level
- Cross-Account: Share tables and columns across accounts
- Integration: Works with Athena, Spectrum, Glue, EMR
- Governance: Centralized permission management
- Security: Fine-grained access control with audit trails