Hive与Iceberg版本
ㅤ | ㅤ | ㅤ |
ㅤ | ㅤ | ㅤ |
ㅤ | ㅤ | ㅤ |
开启Hive支持Iceberg
下载jar包
Hive操作Iceberg格式表
Hive操作iceberg支持多种Catalog元数据管理方式:Hive、Hadoop、第三方、自定义
To globally register different catalogs, set the following Hadoop configurations:
Config Key | Description |
iceberg.catalog.<catalog_name>.type | type of catalog: hive, hadoop, or left unset if using a custom catalog |
iceberg.catalog.<catalog_name>.catalog-impl | catalog implementation, must not be null if type is empty |
iceberg.catalog.<catalog_name>.<key> | any config key and value pairs for the catalog |
Here are some examples using Hive CLI:
Register a
HiveCatalog
called another_hive
:SET iceberg.catalog.another_hive.type=hive; SET iceberg.catalog.another_hive.uri=thrift://example.com:9083; SET iceberg.catalog.another_hive.clients=10; SET iceberg.catalog.another_hive.warehouse=hdfs://example.com:8020/warehouse;、
Register a
HadoopCatalog
called hadoop
:SET iceberg.catalog.hadoop.type=hadoop; SET iceberg.catalog.hadoop.warehouse=hdfs://example.com:8020/warehouse;
Register an AWS
GlueCatalog
called glue
:SET iceberg.catalog.glue.catalog-impl=org.apache.iceberg.aws.GlueCatalog; SET iceberg.catalog.glue.warehouse=s3://my-bucket/my/key/prefix; SET iceberg.catalog.glue.lock-impl=org.apache.iceberg.aws.glue.DynamoLockManager; SET iceberg.catalog.glue.lock.table=myGlueLockTable;