Data Sharing: Secure Sharing, Marketplace & Direct Shares
Architecture Diagram 1: Data Sharing Architecture
┌─────────────────────────────────────────────────────────────────────────────┐
│ SNOWFLAKE DATA SHARING ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ PROVIDER ACCOUNT │
│ ══════════════════ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Database: analytics_prod │ │
│ │ Schema: shared_data │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ Shared Objects: │ │ │
│ │ │ • sales_data (Table) - 100M rows │ │ │
│ │ │ • customer_dim (Table) - 10M rows │ │ │
│ │ │ • sales_summary (View) - Aggregated data │ │ │
│ │ │ • product_lookup (Table) - Reference data │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Sharing Configuration: │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ Share Name: analytics_share │ │ │
│ │ │ Type: SECURE │ │ │
│ │ │ Accounts: [consumer_account_1, consumer_account_2] │ │ │
│ │ │ Region: Global (cross-region enabled) │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ │ Secure Share │
│ │ (Zero-copy, real-time) │
│ ▼ │
│ CONSUMER ACCOUNT (Account A) │
│ ════════════════════════════ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Database: analytics_share (Shared Database) │ │
│ │ Schema: shared_data │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ Read-Only Objects (Live from Provider): │ │ │
│ │ │ • sales_data (Table) - Same data as provider │ │ │
│ │ │ • customer_dim (Table) - Same data as provider │ │ │
│ │ │ • sales_summary (View) - Same logic as provider │ │ │
│ │ │ • product_lookup (Table) - Same data as provider │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Consumer Capabilities: │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ ✓ Query shared data using standard SQL │ │ │
│ │ │ ✓ Join with local tables │ │ │
│ │ │ ✓ Create views on shared tables │ │ │
│ │ │ ✓ Use in stored procedures and queries │ │ │
│ │ │ ✗ Cannot modify shared data (INSERT/UPDATE/DELETE) │ │ │
│ │ │ ✗ Cannot share with other accounts │ │ │
│ │ │ ✗ Cannot grant object-level privileges │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ │ Consumer can also be │
│ │ a provider to others │
│ ▼ │
│ CONSUMER ACCOUNT (Account B) - Receives from Account A │
│ ═════════════════════════════════════════════════════════ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Database: analytics_share_b │ │
│ │ Schema: shared_data │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ Inherits from Account A: │ │ │
│ │ │ • sales_data │ │ │
│ │ │ • customer_dim │ │ │
│ │ │ • sales_summary │ │ │
│ │ │ │ │ │
│ │ │ Local additions: │ │ │
│ │ │ • local_enrichments (Table) │ │ │
│ │ │ • combined_analytics (View) │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ SHARING CHAIN: │
│ ═══════════════ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Provider (Original) ──▶ Consumer A ──▶ Consumer B ──▶ ... │ │
│ │ │ │
│ │ Note: Sharing chains are limited to prevent circular dependencies │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Architecture Diagram 2: Data Marketplace Architecture
┌─────────────────────────────────────────────────────────────────────────────┐
│ SNOWFLAKE DATA MARKETPLACE ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ SNOWFLAKE DATA MARKETPLACE │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ │ │ │
│ │ │ Marketplace Portal: │ │ │
│ │ │ ┌────────────────────────────────────────────────────────┐ │ │ │
│ │ │ │ Browse │ Search │ Filter │ Request │ Get │ │ │ │
│ │ │ │ Listings │ Data │ Categories│ Access │ Data │ │ │ │
│ │ │ └────────────────────────────────────────────────────────┘ │ │ │
│ │ │ │ │ │
│ │ │ Data Categories: │ │ │
│ │ │ ┌────────────────────────────────────────────────────────┐ │ │ │
│ │ │ │ • Financial Data (Market data, fundamentals) │ │ │ │
│ │ │ │ • Healthcare (Clinical trials, patient data) │ │ │ │
│ │ │ │ • Weather (Forecasts, historical data) │ │ │ │
│ │ │ │ • Geospatial (Mapping, location data) │ │ │ │
│ │ │ │ • Demographics (Population, consumer data) │ │ │ │
│ │ │ │ • IoT (Sensor data, device telemetry) │ │ │ │
│ │ │ └────────────────────────────────────────────────────────┘ │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ Listing Types: │ │ │
│ │ │ │ │ │
│ │ │ 1. FREE LISTING │ │ │
│ │ │ • No cost to consumer │ │ │
│ │ │ • Instant access │ │ │
│ │ │ • Limited data coverage │ │ │
│ │ │ │ │ │
│ │ │ 2. PAID LISTING (Consumption-based) │ │ │
│ │ │ • Pay per query/row consumed │ │ │
│ │ │ • Usage tracked by Snowflake │ │ │
│ │ │ • Billing through Snowflake │ │ │
│ │ │ │ │ │
│ │ │ 3. PAID LISTING (Subscription) │ │ │
│ │ │ • Monthly/annual subscription fee │ │ │
│ │ │ • Unlimited queries within subscription │ │ │
│ │ │ • Predictable costs │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ │ One-click access │
│ ▼ │
│ CONSUMER EXPERIENCE: │
│ ════════════════════ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Step 1: Browse Marketplace │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ Search: "weather forecast data" │ │ │
│ │ │ Results: 15 listings found │ │ │
│ │ │ │ │ │
│ │ │ ┌────────────────────────────────────────────────────────┐ │ │ │
│ │ │ │ Listing: Global Weather Forecasts │ │ │ │
│ │ │ │ Provider: WeatherCorp │ │ │ │
│ │ │ │ Type: Free │ │ │ │
│ │ │ │ Data: Daily forecasts, 5-year history │ │ │ │
│ │ │ │ Coverage: Global │ │ │ │
│ │ │ │ [Get Data] [View Details] [Contact Provider] │ │ │ │
│ │ │ └────────────────────────────────────────────────────────┘ │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Step 2: Request Access │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ • Click "Get Data" │ │ │
│ │ │ • Accept terms of service │ │ │
│ │ │ • Specify database name │ │ │
│ │ │ • Access granted automatically (for free listings) │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Step 3: Query Data │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ SELECT * FROM weather_forecasts │ │ │
│ │ │ WHERE city = 'New York' │ │ │
│ │ │ AND forecast_date = CURRENT_DATE(); │ │ │
│ │ │ │ │ │
│ │ │ -- Data appears as local table, but lives in provider │ │ │
│ │ │ -- Queries execute in provider's warehouse │ │ │
│ │ │ -- No data transfer or copying required │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Architecture Diagram 3: Data Exchange Architecture
┌─────────────────────────────────────────────────────────────────────────────┐
│ DATA EXCHANGE ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ DATA EXCHANGE HUB │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ │ │ │
│ │ │ Exchange Types: │ │ │
│ │ │ │ │ │
│ │ │ 1. PRIVATE EXCHANGE │ │ │
│ │ │ ┌────────────────────────────────────────────────────────┐ │ │ │
│ │ │ │ • Invite-only participants │ │ │ │
│ │ │ │ • Controlled by exchange administrator │ │ │ │
│ │ │ │ • Custom data sharing agreements │ │ │ │
│ │ │ │ • Suitable for partner ecosystems │ │ │ │
│ │ │ └────────────────────────────────────────────────────────┘ │ │ │
│ │ │ │ │ │
│ │ │ 2. SECURE EXCHANGE │ │ │
│ │ │ ┌────────────────────────────────────────────────────────┐ │ │ │
│ │ │ │ • Verified participants only │ │ │ │
│ │ │ │ • Snowflake-managed access control │ │ │ │
│ │ │ │ • Standard sharing policies │ │ │ │
│ │ │ │ • Suitable for industry consortiums │ │ │ │
│ │ │ └────────────────────────────────────────────────────────┘ │ │ │
│ │ │ │ │ │
│ │ │ 3. PUBLIC EXCHANGE (Marketplace) │ │ │
│ │ │ ┌────────────────────────────────────────────────────────┐ │ │ │
│ │ │ │ • Open to all Snowflake accounts │ │ │ │
│ │ │ │ • Listed on Data Marketplace │ │ │ │
│ │ │ │ • Standard terms of service │ │ │ │
│ │ │ │ • Suitable for commercial data providers │ │ │ │
│ │ │ └────────────────────────────────────────────────────────┘ │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ │ Data Flow │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ PROVIDERS CONSUMERS │ │
│ │ ════════════ ═══════════ │ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ Provider A │ │ Consumer 1 │ │ │
│ │ │ (Financial) │──────┐ │ (Bank A) │ │ │
│ │ └──────────────┘ │ └──────────────┘ │ │
│ │ │ ▲ │ │
│ │ ┌──────────────┐ │ │ │ │
│ │ │ Provider B │──────┼────────────┼──────── │ │
│ │ │ (Healthcare) │ │ │ │ │
│ │ └──────────────┘ │ │ │ │
│ │ ▼ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ Provider C │ │ Exchange Hub │ │ │
│ │ │ (Weather) │──▶│ (Manages │ │ │
│ │ └──────────────┘ │ shares) │ │ │
│ │ └──────────────┘ │ │
│ │ │ │ │ │
│ │ ▼ ▼ │ │
│ │ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ Consumer 2 │ │ Consumer 3 │ │ │
│ │ │ (Bank B) │ │ (Insurance) │ │ │
│ │ └──────────────┘ └──────────────┘ │ │
│ │ │ │
│ │ Exchange Administration: │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ • Manage participants and roles │ │ │
│ │ │ • Monitor data usage and billing │ │ │
│ │ │ • Enforce sharing policies │ │ │
│ │ │ • Resolve disputes and issues │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Secure Data Sharing enables sharing live data between Snowflake accounts without copying or transferring files. Data remains in the provider's account and is accessed via a share object, ensuring a single source of truth with zero data duplication.
A Data Exchange is a Snowflake Marketplace listing owned by a provider, enabling consumers to browse, request, and access shared datasets. Exchanges support both free and paid listings with automatic metering and billing.
Data sharing is ideal for: data products monetization, supply chain partners, multi-tenant analytics, and cross-organization reporting. Use Data Exchange for controlled distribution with governance policies.
- Zero-copy sharing: Data stays in provider account; no duplication
- Real-time: Always current data; no batch transfers
- Governance: Provider controls access; consumer cannot modify shared data
- Marketplace: Buy/sell data products with automatic billing
- Cross-cloud: Share across AWS/Azure/GCP using Snowflake's federation
Detailed Explanation
Secure Data Sharing Fundamentals
Snowflake's secure data sharing enables real-time data sharing between Snowflake accounts without data movement or copying. Unlike traditional ETL processes that extract, transform, and load data between systems, Snowflake sharing uses a metadata-based approach where consumers access the same underlying data as providers. This ensures data consistency, eliminates synchronization overhead, and provides immediate availability of updates.
The sharing mechanism leverages Snowflake's micro-partition architecture. When data is shared, consumers receive access to the same micro-partitions that the provider owns. No data is copied or transferred; instead, Snowflake's cloud services layer manages access control and query routing. This approach provides several advantages: zero data latency, no storage duplication, automatic updates, and consistent data across all consumers.
Shared data is read-only by design. Consumers cannot modify, insert, or delete data from shared tables. This constraint ensures data integrity and prevents conflicts between providers and consumers. Consumers can create local copies of shared data if they need to modify it, using CREATE TABLE ... CLONE or INSERT ... SELECT patterns.
Data Marketplace: Discovering and Accessing Data
The Snowflake Data Marketplace is a curated ecosystem where providers publish data products and consumers discover and access them. The marketplace supports various listing types, from free datasets to premium paid content with consumption-based or subscription pricing. This platform enables organizations to monetize their data assets and access high-quality external data without complex integration projects.
Marketplace listings include detailed metadata describing data content, update frequency, coverage, and pricing. Consumers can browse categories, search for specific data types, and request access directly from the marketplace interface. Free listings provide instant access, while paid listings may require provider approval or credit card information.
Once access is granted, marketplace data appears as local tables in the consumer's Snowflake account. Queries execute against the provider's warehouse, ensuring consistent performance and availability. Consumers can join marketplace data with their local tables, create views, and use the data in analytics workflows without any data movement or transformation.
Data Exchange: Multi-Party Data Collaboration
Data exchanges enable controlled data sharing environments where multiple participants share data under agreed-upon terms and conditions. Exchanges can be private (invite-only), secure (verified participants), or public (marketplace listings). They provide governance, billing, and access control mechanisms for complex data sharing scenarios.
In a data exchange, an administrator manages participant enrollment, defines sharing policies, and monitors data usage. Providers publish data products to the exchange, and consumers request access based on their needs. The exchange infrastructure handles billing, usage tracking, and compliance reporting, simplifying multi-party data collaboration.
Exchanges are particularly useful for industry consortia, partner ecosystems, and data monetization scenarios. They enable organizations to share data securely while maintaining control over access and usage. The exchange administrator can enforce data quality standards, usage limitations, and sharing restrictions to ensure compliance with regulations and business requirements.
Billing and Usage Tracking
Data sharing in Snowflake involves billing considerations for both providers and consumers. For paid listings, providers set pricing based on consumption metrics (queries executed, rows scanned, or bytes processed). Snowflake tracks usage and handles billing through the consumer's account, eliminating the need for separate invoicing arrangements.
Providers can monitor data usage through the ACCOUNT_USAGE.SHARING_HISTORY view, which provides detailed information about which consumers are accessing their data, query volumes, and resource consumption. This visibility enables providers to optimize their data products, understand customer behavior, and set appropriate pricing.
Consumers can track their data sharing costs through warehouse metering and query history views. Since queries against shared data execute in the provider's warehouse, consumers may not see direct compute costs, but they should monitor query performance and consider the value of external data in their analytics workflows.
Best Practices for Data Sharing
Effective data sharing requires careful planning and governance. Providers should document data schemas, update frequencies, and quality metrics to help consumers understand the data. They should implement monitoring to track usage patterns and identify potential issues.
Consumers should evaluate data quality before integrating external data into critical workflows. They should implement data validation checks, monitor freshness, and understand the provider's update schedule. It's also important to consider the cost-benefit tradeoff between external data and internal data collection.
Key Concepts Table
| Sharing Type | Mechanism | Latency | Cost Model |
|---|---|---|---|
| Direct Share | Metadata pointer | Real-time | Free (compute) |
| Marketplace | Curated listing | Real-time | Free/Paid |
| Data Exchange | Governed collaboration | Real-time | Varies |
| Listing Type | Access | Billing | Use Case |
|---|---|---|---|
| Free | Instant | None | Discovery, evaluation |
| Paid (Consumption) | Approval required | Per query/row | Variable workloads |
| Paid (Subscription) | Approval required | Monthly fee | Predictable workloads |
| Sharing Constraint | Description | Rationale |
|---|---|---|
| Read-only access | No DML on shared data | Data integrity |
| No cross-account sharing | Cannot re-share without provider approval | Control |
| No object grants | Cannot grant privileges on shared objects | Security |
Code Examples
-- Example 1: Create a share
CREATE SHARE analytics_share
COMMENT = 'Share for analytics data';
-- Example 2: Add objects to share
GRANT USAGE ON DATABASE analytics_prod TO SHARE analytics_share;
GRANT USAGE ON SCHEMA analytics_prod.shared_data TO SHARE analytics_share;
GRANT SELECT ON TABLE analytics_prod.shared_data.sales_data TO SHARE analytics_share;
GRANT SELECT ON TABLE analytics_prod.shared_data.customer_dim TO SHARE analytics_share;
-- Example 3: Add consumer accounts to share
ALTER SHARE analytics_share ADD ACCOUNTS = consumer_account_1, consumer_account_2;
-- Example 4: Consumer creates database from share
CREATE DATABASE analytics_share
FROM SHARE provider_account.analytics_share;
-- Example 5: Query shared data
SELECT
s.product,
s.region,
SUM(s.amount) as total_sales,
c.customer_segment
FROM analytics_share.shared_data.sales_data s
JOIN analytics_share.shared_data.customer_dim c
ON s.customer_id = c.customer_id
GROUP BY 1, 2, 4;
-- Example 6: Create view on shared data
CREATE VIEW local_analytics AS
SELECT
s.*,
l.local_enrichment_column
FROM analytics_share.shared_data.sales_data s
LEFT JOIN local_table l
ON s.id = l.id;
-- Example 7: Monitor sharing usage
SELECT
share_name,
consumer_account,
consumer_region,
queries_executed,
bytes_scanned,
last_query_time
FROM TABLE(INFORMATION_SCHEMA.SHARING_HISTORY(
SHARE_NAME => 'analytics_share',
START_TIME => DATEADD(day, -30, CURRENT_TIMESTAMP())
))
ORDER BY last_query_time DESC;
-- Example 8: Create data exchange
CREATE DATA EXCHANGE partner_exchange
TYPE = PRIVATE
COMMENT = 'Private exchange for partner data';
-- Example 9: Add participants to exchange
ALTER DATA EXCHANGE partner_exchange ADD PARTICIPANT
PARTICIPANT_TYPE = PROVIDER
ACCOUNT_NAME = partner_account;
ALTER DATA EXCHANGE partner_exchange ADD PARTICIPANT
PARTICIPANT_TYPE = CONSUMER
ACCOUNT_NAME = customer_account;
-- Example 10: List available shares
SHOW SHARES LIKE 'analytics_%';
-- List shares from specific provider
SHOW SHARES IN ACCOUNT provider_account;
-- Get share details
DESC SHARE analytics_share;
Performance Metrics
| Metric | Target | Warning | Critical |
|---|---|---|---|
| Data Latency | Real-time | < 1 min | > 5 min |
| Query Performance | Same as local | 10-20% slower | > 20% slower |
| Availability | 99.9% | 99.5% | < 99.5% |
| Billing Accuracy | 100% | 99.9% | < 99.9% |
Best Practices
-
Document data schemas: Provide comprehensive documentation for shared data including schemas, data types, update frequencies, and quality metrics.
-
Monitor usage patterns: Track which consumers are accessing your data, query volumes, and resource consumption to optimize data products.
-
Implement data quality checks: Validate shared data regularly and communicate quality issues proactively to consumers.
-
Set appropriate pricing: Consider the value of your data, update frequency, and consumer usage patterns when setting pricing.
-
Use appropriate listing types: Choose free listings for discovery, consumption-based for variable workloads, and subscriptions for predictable workloads.
-
Monitor query performance: Ensure shared data queries execute efficiently. Consider clustering and optimization for frequently accessed data.
-
Implement access controls: Use Snowflake's sharing features to control who can access your data and under what conditions.
-
Plan for scale: Design shared data structures to handle growth in data volume and consumer demand.
-
Communicate updates: Notify consumers of data updates, schema changes, or service interruptions.
-
Review sharing agreements: Regularly review sharing terms, pricing, and usage to ensure they remain appropriate for all parties.
See Also
- PySpark Iceberg - Iceberg data sharing patterns
- Delta Lake on Databricks - Delta Lake sharing model
- Data Warehouse Concepts - Data warehouse design principles