📊 Amazon QuickSight

Master QuickSight SPICE engine, ML-powered insights, dashboards, and embedded analytics.

Module: AWS Data Engineering • Topic 13 of 65 • Premium Content

QuickSight Architecture

Architecture Diagram

┌─────────────────────────────────────────────────────────────────────────────┐
│                    AMAZON QUICKSIGHT ARCHITECTURE                             │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  DATA SOURCES                                                       │    │
│  │  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐          │    │
│  │  │ Redshift │  │    S3    │  │  RDS/    │  │  Athena  │          │    │
│  │  │          │  │ (SPICE)  │  │  Aurora   │  │          │          │    │
│  │  └────┬─────┘  └────┬─────┘  └────┬─────┘  └────┬─────┘          │    │
│  └───────┼──────────────┼──────────────┼──────────────┼───────────────┘    │
│          ▼              ▼              ▼              ▼                     │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  QUICKSIGHT SERVICE                                                  │    │
│  │                                                                     │    │
│  │  ┌───────────────────────────────────────────────────────────────┐  │    │
│  │  │  SPICE ENGINE (In-memory)                                     │  │    │
│  │  │                                                               │  │    │
│  │  │  • Columnar storage                                           │  │    │
│  │  │  • Data compression (10:1 typical)                            │  │    │
│  │  │  • Parallel query execution                                    │  │    │
│  │  │  • Auto-refresh capability                                     │  │    │
│  │  │  • 1 GB per SPICE session                                     │  │    │
│  │  └───────────────────────────────────────────────────────────────┘  │    │
│  │                                                                     │    │
│  │  ┌───────────────────────────────────────────────────────────────┐  │    │
│  │  │  ML INSIGHTS                                                  │  │    │
│  │  │                                                               │  │    │
│  │  │  • Forecasting (time series)                                  │  │    │
│  │  │  • Anomaly detection                                          │  │    │
│  │  │  • Auto-narratives                                            │  │    │
│  │  │  • Q&A (natural language)                                     │  │    │
│  │  └───────────────────────────────────────────────────────────────┘  │    │
│  │                                                                     │    │
│  │  ┌───────────────────────────────────────────────────────────────┐  │    │
│  │  │  VISUALIZATION                                                │  │    │
│  │  │                                                               │  │    │
│  │  │  • 25+ chart types                                            │  │    │
│  │  │  • Interactive dashboards                                     │  │    │
│  │  │  • Mobile responsive                                          │  │    │
│  │  │  • Custom templates                                            │  │    │
│  │  └───────────────────────────────────────────────────────────────┘  │    │
│  └─────────────────────────────┬───────────────────────────────────────┘    │
│                                │                                           │
│                                ▼                                           │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  DELIVERY                                                           │    │
│  │  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐          │    │
│  │  │ Web App  │  │  Mobile  │  │ Embedded │  │ API      │          │    │
│  │  │          │  │          │  │          │  │          │          │    │
│  │  └──────────┘  └──────────┘  └──────────┘  └──────────┘          │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────────────────┘

SPICE Engine

Feature	Description
In-Memory	Columnar storage in memory
Compression	10:1 typical compression
Incremental Refresh	Only changed data imported
Data Preparation	Calculated fields, joins
Row-Level Security	Filter data per user
容量	1 GB per SPICE session, can purchase additional

SPICE Configuration

import boto3

quicksight = boto3.client('quicksight')

# Create SPICE dataset from S3
response = quicksight.create_data_set(
    AwsAccountId='123456789012',
    DataSetId='sales-data',
    Name='Sales Data',
    ImportMode='SPICE',
    PhysicalTableMap={
        'sales_table': {
            'S3Source': {
                'DataSourceArn': 'arn:aws:quicksight:us-east-1:123456789012:datasource/s3-datasource',
                'InputColumns': [
                    {'Name': 'sale_id', 'Type': 'INTEGER'},
                    {'Name': 'customer_id', 'Type': 'STRING'},
                    {'Name': 'amount', 'Type': 'DECIMAL'},
                    {'Name': 'sale_date', 'Type': 'DATETIME'}
                ],
                'UploadSettings': {
                    'Format': 'PARQUET',
                    'StartFromRow': 1,
                    'ContainsHeader': True
                }
            }
        }
    },
    Permissions=[
        {
            'Principal': 'arn:aws:quicksight:us-east-1:123456789012:group/default/analysts',
            'Actions': [
                'quicksight:DescribeDataSet',
                'quicksight:DescribeDataSetPermissions',
                'quicksight:PassDataSet',
                'quicksight:DescribeIngestion',
                'quicksight:ListIngestions'
            ]
        }
    ],
    Tags=[{'Key': 'Team', 'Value': 'analytics'}]
)

# Refresh SPICE dataset
quicksight.create_ingestion(
    AwsAccountId='123456789012',
    DataSetId='sales-data',
    IngestionId='manual-refresh-001'
)

# Schedule automatic refresh
quicksight.update_data_set(
    AwsAccountId='123456789012',
    DataSetId='sales-data',
    RefreshSchedule={
        'Frequency': 'DAILY',
        'Timezone': 'America/New_York',
        'RefreshWindowSchedule': {
            'TimeOfDay': '02:00',
            'DayOfWeek': 'MONDAY'
        }
    }
)

QuickSight ML Insights

Architecture Diagram

┌─────────────────────────────────────────────────────────────────────────────┐
│                    QUICKSIGHT ML INSIGHTS                                    │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  FORECASTING                                                        │    │
│  │                                                                     │    │
│  │  ┌───────────────────────────────────────────────────────────────┐  │    │
│  │  │  Historical Data ──► ML Model ──► Forecast                    │  │    │
│  │  │                                                               │  │    │
│  │  │  • Time series forecasting                                    │  │    │
│  │  │  • Automatic granularity detection                            │  │    │
│  │  │  • Confidence intervals                                       │  │    │
│  │  │  • Up to 5 forecast horizons                                  │  │    │
│  │  │  • Min 100 data points recommended                            │  │    │
│  │  └───────────────────────────────────────────────────────────────┘  │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  ANOMALY DETECTION                                                   │    │
│  │                                                                     │    │
│  │  ┌───────────────────────────────────────────────────────────────┐  │    │
│  │  │  Data Stream ──► ML Model ──► Anomalies                       │  │    │
│  │  │                                                               │  │    │
│  │  │  • Univariate and multivariate                                │  │    │
│  │  │  • Root cause analysis                                        │  │    │
│  │  │  • Contributing factors                                       │  │    │
│  │  │  • Sensitivity tuning                                         │  │    │
│  │  └───────────────────────────────────────────────────────────────┘  │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  AUTO-NARRATIVES                                                     │    │
│  │                                                                     │    │
│  │  Automatically generates natural language insights:                  │    │
│  │  • "Sales increased by 15% compared to last month"                  │    │
│  │  • "Top contributor to decline is Region X"                         │    │
│  │  • "Peak sales occur on weekends"                                   │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────────────────┘

Embedded Analytics

# Generate embed URL for dashboard
response = quicksight.generate_embed_url_for_registered_user(
    AwsAccountId='123456789012',
    UserArn='arn:aws:quicksight:us-east-1:123456789012:default/user/analyst-user',
    ExperienceConfiguration={
        'Dashboard': {
            'InitialDashboardId': 'dashboard-12345'
        }
    },
    SessionLifetimeInMinutes=600
)

embed_url = response['EmbedUrl']

ℹ️

Pro Tip: Use Row-Level Security (RLS) to filter data per user without creating separate datasets. Map users to rules via dataset or tag-based approach.

Interview Questions & Answers

Q1: What is SPICE and why is it important?

Answer: SPICE (Super-fast, In-memory, Parallel, Calculation, Engine) is QuickSight's in-memory engine. Benefits:

Fast query performance (sub-second)
10:1 compression ratio
Automatic refresh scheduling
Reduces load on source databases

Q2: How does QuickSight handle row-level security?

Answer:

Dataset-based RLS: Map users to filter rules via dataset
Tag-based RLS: Use tags from IAM or QuickSight group
Rules: Define column-value filters per user/group
Apply: Filters applied automatically at query time

Q3: What is the difference between SPICE and Direct Query?

Answer:

SPICE: Data imported to in-memory, fast queries, needs refresh
Direct Query: Queries sent directly to source, real-time, slower

Use SPICE for frequently accessed dashboards. Use Direct Query for large datasets or real-time requirements.

Q4: How do QuickSight ML Insights work?

Answer:

Forecasting: Uses time series ML models to predict future values
Anomaly Detection: Statistical models identify unusual patterns
Auto-Narratives: NLG generates natural language summaries
No ML expertise required, built into QuickSight

Q5: What are the pricing tiers for QuickSight?

Answer:

Author: $9/month (creation, publishing)
Reader: $0.30/session (viewing dashboards)
SPICE: $0.25/GB/month (additional capacity)
Enterprise edition adds: RLS, encryption, VPC connectivity

Summary

QuickSight is AWS's fully managed BI service. Key takeaways:

SPICE: In-memory engine for fast queries
ML Insights: Forecasting, anomaly detection, auto-narratives
Embedded: Embed dashboards in applications
Security: Row-level security for data access control
Pricing: Pay-per-session for readers, monthly for authors
Integration: Native integration with all AWS data services

Amazon QuickSight for Data Engineers

📊 Amazon QuickSight

QuickSight Architecture

SPICE Engine

SPICE Configuration

QuickSight ML Insights

Embedded Analytics

Interview Questions & Answers

Q1: What is SPICE and why is it important?

Q2: How does QuickSight handle row-level security?

Q3: What is the difference between SPICE and Direct Query?

Q4: How do QuickSight ML Insights work?

Q5: What are the pricing tiers for QuickSight?

Summary