CREATE EXTERNAL VOLUME

Mounts external object storage (Alibaba Cloud OSS, Tencent Cloud COS, Amazon S3) and creates an External Volume object in Lakehouse.

Prerequisites

Before creating an External Volume, you need to create the corresponding Storage Connection:

-- Create an Alibaba Cloud OSS storage connection CREATE STORAGE CONNECTION IF NOT EXISTS oss_conn TYPE oss ENDPOINT = 'oss-cn-hangzhou.aliyuncs.com' ACCESS_ID = 'your_access_key_id' ACCESS_KEY = 'your_access_key_secret';

Syntax

CREATE EXTERNAL VOLUME [IF NOT EXISTS] [schema_name.]<volume_name> LOCATION '<storage_url>' USING CONNECTION <connection_name> DIRECTORY = ( enable = { true | false }, auto_refresh = { true | false } ) RECURSIVE = { true | false };

Parameters

ParameterDescription
IF NOT EXISTSIf the Volume already exists, skip without error
schema_nameName of the owning schema; current schema is used if omitted
volume_nameVolume name, must be unique within the same schema
LOCATIONObject storage path, format: oss://bucket_name/path/, cos://bucket_name/path/, s3://bucket_name/path/
USING CONNECTIONThe Storage Connection name to reference
DIRECTORY.enableWhether to enable the directory feature; recommended to set to true
DIRECTORY.auto_refreshWhether to automatically refresh file metadata
RECURSIVEWhether to recursively scan subdirectories

Examples

  1. Mount an Alibaba Cloud OSS bucket:

CREATE EXTERNAL VOLUME my_oss_vol LOCATION 'oss://mcp-data-hangzhou/test/' USING CONNECTION oss_conn DIRECTORY = (enable = true, auto_refresh = true) RECURSIVE = true;

  1. Mount a Tencent Cloud COS bucket:

CREATE EXTERNAL VOLUME my_cos_vol LOCATION 'cos://my-bucket-1234567890/data/' USING CONNECTION cos_conn DIRECTORY = (enable = true, auto_refresh = true) RECURSIVE = true;

  1. Mount an Amazon S3 bucket:

CREATE EXTERNAL VOLUME my_s3_vol LOCATION 's3://my-s3-bucket/data/' USING CONNECTION s3_conn DIRECTORY = (enable = true, auto_refresh = true) RECURSIVE = true;

  1. Create under a specific schema with IF NOT EXISTS:

CREATE EXTERNAL VOLUME IF NOT EXISTS my_schema.my_oss_vol LOCATION 'oss://mcp-data-hangzhou/test/' USING CONNECTION oss_conn DIRECTORY = (enable = true, auto_refresh = true) RECURSIVE = true;

Notes

  • External Volumes only store path metadata; actual data is stored in the external cloud storage, so there are no additional storage costs on the Lakehouse side
  • Cross-cloud creation is not supported: Alibaba Cloud instances can only create OSS Connections, Tencent Cloud instances can only create COS Connections, etc.
  • Deleting an External Volume does not delete the actual files in the external storage

Required Privileges

PrivilegeDescription
CREATE VOLUMECreate a Volume under the current schema