Catalog Connection stores the authentication information needed to access external metadata services (Hive Metastore, Databricks Unity Catalog, Iceberg REST Catalog, etc.) and is used by External Catalogs and External Schemas.
Supported Types
Type Value
Data Source
Authentication
HMS
Apache Hive
Storage key (+ optional Kerberos)
databricks
Databricks Unity Catalog
OAuth M2M (CLIENT_ID / CLIENT_SECRET)
ICEBERG_REST
Generic Iceberg REST Catalog (including Snowflake Open Catalog)
No auth or OAuth
⚠️ Note: Databricks storage (S3/OSS/COS) must be on the same cloud platform as the Lakehouse instance.
HMS (Hive Metastore)
Syntax
CREATE CATALOG CONNECTION [IF NOT EXISTS] <connection_name>
TYPE HMS
HIVE_METASTORE_URIS = '<thrift://host:port>'
STORAGE_CONNECTION = '<storage_connection_name>'
[AUTH_TYPE = 'kerberos'
KERBEROS_CLIENT_PRINCIPAL = '<principal>'
KERBEROS_SERVICE_PRINCIPAL = '<principal>'
KERBEROS_KRB5_CONFIG_PATH = '<volume_path>'
KERBEROS_KEYTAB_PATH = '<volume_path>'];
Parameter Description
HIVE_METASTORE_URIS: The Hive Metastore service address, in the format thrift://host:9083. The port is typically 9083.
STORAGE_CONNECTION: The name of an existing storage connection, used to read Hive data files (OSS/COS/S3/HDFS).
AUTH_TYPE: Authentication type. Defaults to no authentication if omitted. Set to 'kerberos' to enable Kerberos authentication.
Kerberos parameters are only required when AUTH_TYPE = 'kerberos'. The config file and keytab file must be uploaded to User Volume in advance using the PUT command.
Examples
Hive ON OSS (Alibaba Cloud)
-- Step 1: Create storage connection
CREATE STORAGE CONNECTION IF NOT EXISTS oss_conn
TYPE OSS
ACCESS_ID = 'LTAIxxxxxxxxxxxx'
ACCESS_KEY = 'T8Gexxxxxxmtxxxxxx'
ENDPOINT = 'oss-cn-hangzhou-internal.aliyuncs.com';
-- Step 2: Create Catalog Connection
-- Ensure the server where HMS is located has network connectivity to Lakehouse. Refer to: Create Alibaba Cloud Endpoint Service
CREATE CATALOG CONNECTION IF NOT EXISTS hive_oss_conn
TYPE HMS
HIVE_METASTORE_URIS = 'thrift://192.168.1.100:9083'
STORAGE_CONNECTION = 'oss_conn';
Hive ON COS (Tencent Cloud)
CREATE STORAGE CONNECTION IF NOT EXISTS cos_conn
TYPE COS
ACCESS_KEY = '<access_key>'
SECRET_KEY = '<secret_key>'
REGION = 'ap-shanghai'
APP_ID = '1310000503';
CREATE CATALOG CONNECTION IF NOT EXISTS hive_cos_conn
TYPE HMS
HIVE_METASTORE_URIS = 'thrift://192.168.1.100:9083'
STORAGE_CONNECTION = 'cos_conn';
Hive ON S3 (AWS)
CREATE STORAGE CONNECTION IF NOT EXISTS s3_conn
TYPE S3
ACCESS_KEY = 'AKIAQNBSBP6EIJE33***'
SECRET_KEY = '7kfheDrmq***'
ENDPOINT = 's3.cn-north-1.amazonaws.com.cn'
REGION = 'cn-north-1';
CREATE CATALOG CONNECTION IF NOT EXISTS hive_s3_conn
TYPE HMS
HIVE_METASTORE_URIS = 'thrift://192.168.1.100:9083'
STORAGE_CONNECTION = 's3_conn';
Hive + Kerberos Authentication
-- Upload authentication files to User Volume first
PUT '/etc/krb5.conf' TO USER VOLUME FILE 'krb5.conf';
PUT '/path/to/hive.keytab' TO USER VOLUME FILE 'hive.keytab';
CREATE CATALOG CONNECTION IF NOT EXISTS hive_kerberos_conn
TYPE HMS
HIVE_METASTORE_URIS = 'thrift://your-hms-host:9083'
STORAGE_CONNECTION = 'oss_conn'
AUTH_TYPE = 'kerberos'
KERBEROS_CLIENT_PRINCIPAL = 'hive/localhost@YOUR-REALM.COM'
KERBEROS_SERVICE_PRINCIPAL = 'hive/localhost@YOUR-REALM.COM'
KERBEROS_KRB5_CONFIG_PATH = 'volume:user//~/krb5.conf'
KERBEROS_KEYTAB_PATH = 'volume:user//~/hive.keytab';
HOST: The Databricks workspace URL, in the format https://dbc-xxxxx.cloud.databricks.com.
CLIENT_ID / CLIENT_SECRET: OAuth M2M authentication credentials. Obtain these by creating a Service Principal in the Databricks console. Refer to the Databricks OAuth M2M documentation.
ACCESS_REGION: The region where the Databricks workspace is located, such as us-west-2.
Databricks Pre-configuration
Create a Service Principal and obtain the CLIENT_ID and CLIENT_SECRET.
Enable External Data Access on the Metastore.
Grant permissions to the Service Principal:
GRANT EXTERNAL USE SCHEMA ON SCHEMA <catalog>.<schema> TO `<service_principal_id>`;
Example
CREATE CATALOG CONNECTION IF NOT EXISTS my_databricks_conn
TYPE databricks
HOST = 'https://dbc-12345678-9abc.cloud.databricks.com'
CLIENT_ID = '12345678-9abc-def0-1234-56789abcdef0'
CLIENT_SECRET = 'abcdef1234567890abcdef1234567890'
ACCESS_REGION = 'us-west-2';
Verify the connection:
CREATE EXTERNAL CATALOG my_databricks_catalog
CONNECTION my_databricks_conn
OPTIONS ('catalog' = 'main');
SHOW SCHEMAS IN my_databricks_catalog;
Iceberg REST Catalog
The Iceberg REST protocol is an open standard. Any service compatible with this protocol (self-hosted Iceberg REST services, Polaris, Dremio, Snowflake Open Catalog, etc.) uses the same TYPE ICEBERG_REST. The difference is whether OAuth authentication is required.
-- List all connections
SHOW CONNECTIONS;
-- View connection details
DESC CONNECTION my_conn;
-- Drop a connection (drop any dependent External Catalogs first)
DROP CONNECTION my_conn;
