关于Cloudera Hadoop的安装(CentOS)- 官文标注(待完成)

CDH 5 and Cloudera Manager 5 Requirements and Supported Versions

In an enterprise data hub, Cloudera Manager and CDH interact with several products such as Apache Accumulo, Apache Impala (incubating), Hue, Cloudera Search, and Cloudera Navigator. This guide provides information about which major and minor release version of a product is supported with which release version of CDH and Cloudera Manager.

Compatibility across different release versions of Cloudera Manager and CDH must be taken into account, especially when carrying out install/upgrade procedures.

当安装、升级时,尤其要考虑Cloudera Manager和CDH版本的兼容性。

JDK compatibility also varies across different Cloudera Manager and CDH versions. Certain versions of CDH 5 are compatible with both JDK 7 and JDK 8. In such cases, ensure all your services are deployed on the same major version. For example, you should not run Hadoop on JDK 7 while running Sqoop on JDK 8. Additionally, since Cloudera does not support mixed environments, all nodes in your cluster must be running the same major JDK version. Cloudera only supports JDKs provided by Oracle.

Cloudera Hadoop集群所有节点必须要使用同一版本的JDK。官方建议安装Oracle JDK,不支持混合版本环境的JDK。

After installing each entity, upgrade to the latest patch version and apply any other appropriate updates. An available update might be specific to the operating system on which it is installed. For example, if you are using CentOS in your environment, you could choose 6 as the major version and 4 as the minor version to indicate that you are using CentOS 6.4. After installing this operating system, apply all relevant CentOS 6.4 upgrades and patches. In some cases, such as some browsers, a minor version might not be listed.

Each product matrix contains at least a subset of the following fields:
  • Feature: This column lists notable new features that have been included in a particular release. For products/releases that do not have this column, refer the respective Release Notes for detailed information.
  • Lowest supported Cloudera Manager version: Specifies the earliest version of Cloudera Manager that supports a product release version.
  • Lowest supported CDH version: Specifies the earliest version of CDH that supports a product release version. The Cloudera Search and Impala matrices are an exception to this since each release is only compatible with a specific CDH release.
  • Lowest supported Impala version: This column might not apply to all products, for example, Cloudera Search.
  • Lowest supported Search version: This column might not apply to all products, for example, Cloudera Impala.
  • Integrated into CDH: This column specifies whether a particular release is shipped with CDH or available independently. This field may not apply to all products, for example, Cloudera Navigator.

Continue reading:

  • CDH Requirements for Cloudera Manager
  • CDH and Cloudera Manager Supported Operating Systems
  • Filesystem Requirements
  • CDH and Cloudera Manager Supported Databases
  • CDH and Cloudera Manager Supported JDK Versions
  • Cloudera Manager Supported Browsers
  • CDH and Cloudera Manager Supported Transport Layer Security Versions
  • Cloudera Manager Resource Requirements
  • CDH and Cloudera Manager Networking and Security Requirements
  • Apache Accumulo
  • Apache Impala (incubating)
  • Apache Kafka
  • Apache Kudu
  • Cloudera Navigator
  • Cloudera Navigator Encryption
  • Apache Sentry
  • Apache Spark
  • EMC Isilon
  • Cloudera Manager Backup and Disaster Recovery
  • Supported Configurations with Virtualization and Cloud Platforms

CDH Requirements for Cloudera Manager

The Cloudera Manager minor version must always be equal to or greater than the CDH minor version. Older versions of Cloudera Manager might not support features in newer versions of CDH. For example, to upgrade to CDH 5.7.1 you must first upgrade to Cloudera Manager 5.7.0.

Cloudera Manager的主版本号和次版本号要和CDH的主次版本号一致或者比CDH的次版本号稍大。比如,CDH为5.7.1那么Cloudera Manager至少为CM5.7.0(CM5.7.1、CM5.7.2都可以使用)

CDH and Cloudera Manager Supported Operating Systems

CDH 5.x provides 64-bit packages for select versions of RHEL-compatible, SLES, Ubuntu, and Debian operating systems.

  • CDH and Cloudera Manager 5.11.x Supported Operating Systems
  • CDH and Cloudera Manager 5.10.x Supported Operating Systems
  • CDH and Cloudera Manager 5.9.x Supported Operating Systems
  • CDH and Cloudera Manager 5.8.x Supported Operating Systems
  • CDH and Cloudera Manager 5.7.x Supported Operating Systems
  • CDH and Cloudera Manager 5.6.x Supported Operating Systems
  • CDH and Cloudera Manager 5.5.x Supported Operating Systems
  • CDH and Cloudera Manager 5.4.x Supported Operating Systems
  • CDH and Cloudera Manager 5.3.x Supported Operating Systems
  • CDH and Cloudera Manager 5.2.x Supported Operating Systems
  • CDH and Cloudera Manager 5.1.x Supported Operating Systems
  • CDH and Cloudera Manager 5.0.x Supported Operating Systems

每个版本的CDH和CM都有自己的相关说明文档,写的很详细。

Important: In order to be covered by Cloudera Support:
  • All CDH hosts in a logical cluster must run on the same major OS release.                    操作系统主版本号必须一致
  • Cloudera temporarily allows a mixed OS configuration during an OS upgrade project.  在操作系统升级时,Cloudera允许混合版本操作系统间的配置。
  • Cloudera Manager must run on the same OS release as one of the CDH clusters it manages.CM也必须和CDH安装在相同版本的操作系统上。

Cloudera recommends running the same minor release cross-cluster. However, the risk caused by running different minor OS releases is considered lower than the risk of running different major OS releases.

Gateway hosts may use RHEL/Centos 7.2, subject to some restrictions. See Operating System Support for Gateway Hosts (CDH 5.11 and higher only)

Other disclaimers:
  • RHEL / CentOS / OEL 6.9 is supported in C5.11.0 with a Known Issue.
  • RHEL / CentOS / OEL 7.0 is not supported. 不支持CentOS7.0
  • Red Hat only supports specific upgrades from RHEL 6 to 7. Contact your OS vendor and review What are the supported use cases for upgrading to RHEL 7?
  • SLES hosts running Cloudera Manager agents must use SLES SDK 11 SP1.
  • Cloudera does not support CDH cluster deployments in Docker containers.
  • Cloudera Enterprise (without Cloudera Navigator) is supported on platforms with Security-Enhanced Linux (SELinux) enabled.
Important: Cloudera is not responsible for policy support nor policy enforcement. If you experience issues with SELinux, contact your OS support provider.

CDH and Cloudera Manager 5.11.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible

RHEL / CentOS

Max SE Linux support: 7.2

7.3, 7.2, 7.1

6.9 (with Known Issue) 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Enterprise Linux (OEL)

7.3, 7.2, 7.1 (UEK default)

6.8, 6.7, 6.6 (UEK R3)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP2, 12 SP1

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

16.04 LTS (Xenial)

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.2, 8.4 (Jessie)

7.0, 7.1, 7.8 (Wheezy)

Operating System Support for Gateway Hosts (CDH 5.11 and higher only)

A Gateway host that is dedicated to running Cloudera Data Science Workbench can use RHEL/CentOS 7.2 even if the remaining hosts in your cluster are running any of the other supported operating systems. All hosts must run the same version of the Oracle JDK.

CDH and Cloudera Manager 5.10.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.7

7.3, 7.2, 7.1

6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Enterprise Linux (OEL)

7.3, 7.2, 7.1 (UEK default)

6.8, 6.7, 6.6 (UEK R3)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP1

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.4, 8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.9.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.7

7.2, 7.1

6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Enterprise Linux (OEL)

7.2, 7.1 (UEK default)

6.8, 6.7, 6.6 (UEK R3)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP1

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.4, 8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.8.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.6

7.2, 7.1

6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Enterprise Linux (OEL)

7.2, 7.1 (UEK default)

6.7, 6.6 (UEK R3)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.7.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.6

7.2, 7.1

6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Enterprise Linux (OEL)

7.2, 7.1 (UEK default)

6.7, 6.6 (UEK R3)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.6.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.6

7.1

6.7, 6.6, 6.5, 6.4

5.10, 5.7

Oracle Enterprise Linux (OEL)

7.1 (UEK default)

6.7, 6.6 (UEK R3)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.10, 5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.5.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.6

7.1

6.7, 6.6, 6.5, 6.4

5.10, 5.7

Oracle Enterprise Linux (OEL)

7.1 (UEK default)

6.7, 6.6 (UEK R3)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.10, 5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.4.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.5

6.6, 6.5, 6.4

5.10, 5.7

Oracle Enterprise Linux (OEL)

6.6 (UEK R3)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.10, 5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.3.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.4

6.5, 6.4

5.7

Oracle Enterprise Linux (OEL)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.2.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.4

6.5, 6.4

5.7

Oracle Enterprise Linux (OEL)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.1.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.4

6.5, 6.4

5.7

Oracle Enterprise Linux (OEL)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.0.x Supported Operating Systems

Operating System Version
Red Hat Enterprise Linux-compatible
RHEL / CentOS

Max SE Linux support: 6.6

6.5, 6.4

5.7

Oracle Enterprise Linux (OEL)

6.5 (UEK R2, UEK R3)

6.4 (UEK R2)

5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

Filesystem Requirements

Supported Filesystems

The Hadoop Distributed File System (HDFS) is designed to run on top of an underlying filesystem in an operating system. Cloudera recommends that you use either of the following filesystems tested on thesupported operating systems:

  • ext3: This is the most tested underlying filesystem for HDFS.
  • ext4: This scalable extension of ext3 is supported in more recent Linux releases.Important: Cloudera does not support in-place upgrades from ext3 to ext4. Cloudera recommends that you format disks as ext4 before using them as data directories.
  • XFS: This is the default filesystem in RHEL 7.

File Access Time

Linux filesystems keep metadata that record when each file was accessed. This means that even reads result in a write to the disk. To speed up file reads, Cloudera recommends that you disable this option, called atime, using the mount option in /etc/fstab:

/dev/sdb1 /data1 ext4 defaults,noatime 0

Apply the change without rebooting:

mount -o remount /data1

Filesystem Mount Options

The filesystem mount options have a sync option that allows you to write synchronously.

However, using the sync option will lead to poor performance for services that write data to disks, such as HDFS, YARN, Kafka and Kudu. In CDH most writes are already replicated. Therefore, having synchronous writes to disks is unnecessary, expensive, and not worth the added safety it provides.

CDH and Cloudera Manager Supported Databases

Cloudera Manager and CDH come packaged with an embedded PostgreSQL database, but it is recommended that you configure your cluster with custom external databases, especially in production.

In most cases (but not all), Cloudera supports versions of MariaDB, MySQL and PostgreSQL that are native to each supported Linux distribution.

After installing a database, upgrade to the latest patch and apply appropriate updates. Available updates may be specific to the operating system on which it is installed.

Notes:
  • Use UTF8 encoding for all custom databases.
  • Cloudera Manager installation fails if GTID-based replication is enabled in MySQL.
  • Hue requires the default MySQL/MariaDB version (if used) of the operating system on which it is installed. See Hue Databases.
  • Both the Community and Enterprise versions of MySQL are supported, as well as MySQL configured by the AWS RDS service.
Important: When you restart processes, the configuration for each of the services is redeployed using information saved in the Cloudera Manager database. If this information is not available, your cluster does not start or function correctly. You must schedule and maintain regular backups of the Cloudera Manager database to recover the cluster in the event of the loss of this database.
 MySQL Support across CM/CDH 5 Releases
MySQL C5.11 C5.10 C5.9 C5.8 C5.7 C5.6 C5.5 C5.4 C5.3 C5.2 C5.1 C5.0
5.1
5.5
5.6  
5.7                    
 MariaDB Support across CM/CDH 5 Releases
MariaDB C5.11 C5.10 C5.9 C5.8 C5.7 C5.6 C5.5 C5.4 C5.3 C5.2 C5.1 C5.0
5.5          
10.0                  
 PostgreSQL Support across CM/CDH 5 Releases
Pgres C5.11 C5.10 C5.9 C5.8 C5.7 C5.6 C5.5 C5.4 C5.3 C5.2 C5.1 C5.0
8.1
8.3
8.4
9.1
9.2  
9.3    
9.4          
9.5                      
 Oracle Support across CM/CDH 5 Releases
Oracle C5.11 C5.10 C5.9 C5.8 C5.7 C5.6 C5.5 C5.4 C5.3 C5.2 C5.1 C5.0
11gr2
12c            
12cr1                      

CDH and Cloudera Manager Supported JDK Versions

Only 64 bit JDKs from Oracle are supported. Oracle JDK 7 is supported across all versions of Cloudera Manager 5 and CDH 5. Oracle JDK 8 is supported in C5.3.x and higher.

A supported minor JDK release will remain supported throughout a Cloudera major release lifecycle, from the time of its addition forward, unless specifically excluded. Warning: JDK 1.8u40 and JDK 1.8u60 are excluded from support. Also, the  Oozie Web Console returns 500 error when Oozie server runs on JDK 8u75 or higher.
Running CDH nodes within the same cluster on different JDK releases is not supported. JDK release across a cluster needs to match the patch level.
  • All nodes in your cluster must run the same Oracle JDK version.
  • All services must be deployed on the same Oracle JDK version.
The Cloudera Manager repository is packaged with Oracle JDK 1.7.0_67 (for  example) and can be automatically installed during a new installation or an upgrade. Note:   =  Minimum required JDK version.   =  Recommended JDK versions.
JDK 7 Versions Supported across CM/CDH 5 Releases
JDK 7 C5.11 C5.10 C5.9 C5.8 C5.7 C5.6 C5.5 C5.4 C5.3 C5.2 C5.1 C5.0
1.7u55  
1.7u67  
1.7u75        
1.7u80          
JDK 8 Versions Supported across CM/CDH 5 Releases
JDK 8 C5.11 C5.10 C5.9 C5.8 C5.7 C5.6 C5.5 C5.4 C5.3 C5.2 C5.1 C5.0
1.8u31      
1.8u74              
1.8u91                
1.8u102                  
1.8u111                    
1.8u121                      

Cloudera Manager Supported Browsers

Hue

Hue works with the two most recent  LTS (long term support) or  ESR (extended support release) browsers. Cookies and JavaScript must be on.
  • Chrome: Version history
  • Firefox: Version history
  • Internet Explorer: Version history
  • Safari (Mac only): Version history
Hue can display in older, and other, browsers, but you might not have access to all of its features. Important: To see all icons in the Hue Web UI, users with IE and HTTPS must add a Load Balancer.

Cloudera Manager

The Cloudera Manager Admin Console, which you use to install, configure, manage, and monitor services, supports the latest version of the following browsers:
  • Mozilla Firefox
  • Google Chrome
  • Internet Explorer
  • Safari.

CDH and Cloudera Manager Supported Transport Layer Security Versions

The following components are supported by the indicated versions of Transport Layer Security (TLS):

Components Supported by TLS

Component

Role Name Port Version
Cloudera Manager Cloudera Manager Server   7182 TLS 1.2
Cloudera Manager Cloudera Manager Server   7183 TLS 1.2
Flume     9099 TLS 1.2
Flume   Avro Source/Sink   TLS 1.2
Flume   Flume HTTP Source/Sink   TLS 1.2
HBase Master HBase Master Web UI Port 60010 TLS 1.2
HDFS NameNode Secure NameNode Web UI Port 50470 TLS 1.2
HDFS Secondary NameNode Secure Secondary NameNode Web UI Port 50495 TLS 1.2
HDFS HttpFS REST Port 14000 TLS 1.1, TLS 1.2
Hive HiveServer2 HiveServer2 Port 10000 TLS 1.2
Hue Hue Server Hue HTTP Port 8888 TLS 1.2
Impala Impala Daemon Impala Daemon Beeswax Port 21000 TLS 1.2
Impala Impala Daemon Impala Daemon HiveServer2 Port 21050 TLS 1.2
Impala Impala Daemon Impala Daemon Backend Port 22000 TLS 1.2
Impala Impala StateStore StateStore Service Port 24000 TLS 1.2
Impala Impala Daemon Impala Daemon HTTP Server Port 25000 TLS 1.2
Impala Impala StateStore StateStore HTTP Server Port 25010 TLS 1.2
Impala Impala Catalog Server Catalog Server HTTP Server Port 25020 TLS 1.2
Impala Impala Catalog Server Catalog Server Service Port 26000 TLS 1.2
Oozie Oozie Server Oozie HTTPS Port 11443 TLS 1.1, TLS 1.2
Solr Solr Server Solr HTTP Port 8983 TLS 1.1, TLS 1.2
Solr Solr Server Solr HTTPS Port 8985 TLS 1.1, TLS 1.2
Spark History Server   18080 TLS 1.2
YARN ResourceManager ResourceManager Web Application HTTP Port 8090 TLS 1.2
YARN JobHistory Server MRv1 JobHistory Web Application HTTP Port 19890 TLS 1.2

Cloudera Manager Resource Requirements

Cloudera Manager requires the following resources:
  • Disk Space
    • Cloudera Manager Server
      • 5 GB on the partition hosting /var.
      • 500 MB on the partition hosting /usr.
      • For parcels, the space required depends on the number of parcels you download to the Cloudera Manager Server and distribute to Agent hosts. You can download multiple parcels of the same product, of different versions and different builds. If you are managing multiple clusters, only one parcel of a product/version/build/distribution is downloaded on the Cloudera Manager Server—not one per cluster. In the local parcel repository on the Cloudera Manager Server, the approximate sizes of the various parcels are as follows:
        • CDH 5 (which includes Impala and Search) - 1.5 GB per parcel (packed), 2 GB per parcel (unpacked)
        • Impala - 200 MB per parcel
        • Cloudera Search - 400 MB per parcel
    • Cloudera Management Service -The Host Monitor and Service Monitor databases are stored on the partition hosting /var. Ensure that you have at least 20 GB available on this partition.
    • Agents - On Agent hosts, each unpacked parcel requires about three times the space of the downloaded parcel on the Cloudera Manager Server. By default, unpacked parcels are located in/opt/cloudera/parcels.
  • RAM - 4 GB is recommended for most cases and is required when using Oracle databases. 2 GB might be sufficient for non-Oracle deployments with fewer than 100 hosts. However, to run the Cloudera Manager Server on a machine with 2 GB of RAM, you must tune down its maximum heap size (by modifying -Xmx in /etc/default/cloudera-scm-server). Otherwise the kernel might kill the Server for consuming too much RAM.
  • Python - Cloudera Manager requires Python 2.4 or higher (but is not compatible with Python 3.0 or higher). Hue in CDH 5 and package installs of CDH 5 require Python 2.6 or 2.7. All supported operating systems include Python version 2.4 or higher. Cloudera Manager is compatible with Python 2.4 through the latest version of Python 2.x. Cloudera Manager does not support Python 3.0 and higher.
  • Perl - Cloudera Manager requires perl.
  • python-psycopg2 package - Cloudera Manager 5.8 and higher has a new dependency on the packagepython-psycopg2. This package is not available in standard SLES 11 and SLES 12 repositories. You need to add this repository or install it manually to any machine that runs the Cloudera Manager Agent before you install or upgrade Cloudera Manager.

    If the Cloudera Manager Server and Agent run on the same host, install the Cloudera Manager Server first and then add the python-psycopg2 repository or package. After adding the repository or package, install the Cloudera Manager Agent.

    Download the python-psycopg2 repository or package from the following URL by selecting the correct SLES version: http://software.opensuse.org/download.html?project=server%3Adatabase%3Apostgresql&package=python-psycopg2

    You can add and install the repository manually or grab the package directly.

    To add the repository and manually install it, you need the URL to the repository. You can find the URL for your operating system version on the download page.

    For example, to add and manually install the  python-psycopg2 repository for SLES 11 SP4, run the following commands:
    zypper addrepo http://download.opensuse.org/repositories/server:database:postgresql/SLE_11_SP4/server:database:postgresql.repo
    zypper refresh
    zypper install python-psycopg2

    Alternatively, you can grab the package directly from the download page. Select the python-psycopg2-2.6.2-.x86_64.rpm file for your operating system version.

CDH and Cloudera Manager Networking and Security Requirements

The hosts in a Cloudera Manager deployment must satisfy the following networking and security requirements:

  • Networking Protocols Support

    CDH requires IPv4. IPv6 is not supported and must be disabled.

    See also Configuring Network Names.

  • Multihoming Support

    – Multihoming CDH or Cloudera Manager is not supported outside specifically certified Cloudera partner appliances. Cloudera finds that current Hadoop architectures combined with modern network infrastructures and security practices remove the need for multihoming. Multihoming, however, is beneficial internally in appliance form factors to take advantage of high-bandwidth InfiniBand interconnects.

    Although some subareas of the product may work with unsupported custom multihoming configurations, there are known issues with multihoming. In addition, unknown issues may arise because multihoming is not covered by our test matrix outside the Cloudera-certified partner appliances.

  • Entropy

    Data at rest encryption requires sufficient entropy to ensure randomness.

    See Entropy Requirements.

  • Cluster hosts must have a working network name resolution system and correctly formatted /etc/hostsfile. All cluster hosts must have properly configured forward and reverse host resolution through DNS. The /etc/hosts files must:
    • Contain consistent information about hostnames and IP addresses across all hosts
    • Not contain uppercase hostnames
    • Not contain duplicate IP addresses

    Cluster hosts must not use aliases, either in /etc/hosts or in configuring DNS. A properly formatted/etc/hosts file should be similar to the following example:

    127.0.0.1 localhost.localdomain localhost
    192.168.1.1 cluster-01.example.com cluster-01
    192.168.1.2 cluster-02.example.com cluster-02
    192.168.1.3 cluster-03.example.com cluster-03 
  • In most cases, the Cloudera Manager Server must have SSH access to the cluster hosts when you run the installation or upgrade wizard. You must log in using a root account or an account that has password-less sudo permission. For authentication during the installation and upgrade procedures, you must either enter the password or upload a public and private key pair for the root or sudo user account. If you want to use a public and private key pair, the public key must be installed on the cluster hosts before you use Cloudera Manager.

    Cloudera Manager uses SSH only during the initial install or upgrade. Once the cluster is set up, you can disable root SSH access or change the root password. Cloudera Manager does not save SSH credentials, and all credential information is discarded when the installation is complete.

  • If single user mode is not enabled, the Cloudera Manager Agent runs as root so that it can make sure the required directories are created and that processes and files are owned by the appropriate user (for example, the hdfs and mapred users).
  • No blocking is done by Security-Enhanced Linux (SELinux).Note: Cloudera Enterprise is supported on platforms with Security-Enhanced Linux (SELinux) enabled. However, Cloudera does not support use of SELinux with Cloudera Navigator. Cloudera is not responsible for policy support nor policy enforcement. If you experience issues with SELinux, contact your OS provider.
  • No blocking by iptables or firewalls; port 7180 must be open because it is used to access Cloudera Manager after installation. Cloudera Manager communicates using specific ports, which must be open.
  • For RHEL and CentOS, the /etc/sysconfig/network file on each host must contain the hostname you have just set (or verified) for that host.
  • Cloudera Manager and CDH use several user accounts and groups to complete their tasks. The set of user accounts and groups varies according to the components you choose to install. Do not delete these accounts or groups and do not modify their permissions and rights. Ensure that no existing systems prevent these accounts and groups from functioning. For example, if you have scripts that delete user accounts not in a whitelist, add these accounts to the list of permitted accounts. Cloudera Manager, CDH, and managed services create and use the following accounts and groups:
Users and Groups

Component (Version)

Unix User ID Groups Notes
Cloudera Manager (all versions) cloudera-scm cloudera-scm Cloudera Manager processes such as the Cloudera Manager Server and the monitoring roles run as this user.
The Cloudera Manager keytab file must be named  cmf.keytab since that name is hard-coded in Cloudera Manager. Note: Applicable to clusters managed by Cloudera Manager only.
Apache Accumulo (Accumulo 1.4.3 and higher) accumulo accumulo Accumulo processes run as this user.
Apache Avro   No special users.
Apache Flume (CDH 4, CDH 5) flume flume The sink that writes to HDFS as this user must have write privileges.
Apache HBase (CDH 4, CDH 5) hbase hbase The Master and the RegionServer processes run as this user.
HDFS (CDH 4, CDH 5) hdfs hdfs, hadoop The NameNode and DataNodes run as this user, and the HDFS root directory as well as the directories used for edit logs should be owned by it.
Apache Hive (CDH 4, CDH 5) hive hive

The HiveServer2 process and the Hive Metastore processes run as this user.

A user must be defined for Hive access to its Metastore DB (for example, MySQL or Postgres) but it can be any identifier and does not correspond to a Unix uid. This isjavax.jdo.option.ConnectionUserName in hive-site.xml.

Apache HCatalog (CDH 4.2 and higher, CDH 5) hive hive

The WebHCat service (for REST access to Hive functionality) runs as the hive user.

HttpFS (CDH 4, CDH 5) httpfs httpfs

The HttpFS service runs as this user. SeeHttpFS Security Configuration for instructions on how to generate the merged httpfs-http.keytab file.

Hue (CDH 4, CDH 5) hue hue

Hue services run as this user.

Hue Load Balancer (Cloudera Manager 5.5 and higher) apache apache The Hue Load balancer has a dependency on the apache2 package that uses the apache user name. Cloudera Manager does not run processes using this user ID.
Cloudera Impala (CDH 4.1 and higher, CDH 5) impala impala, hive Impala services run as this user.
Apache Kafka (Cloudera Distribution of Kafka 1.2.0) kafka kafka Kafka services run as this user.
Java KeyStore KMS (CDH 5.2.1 and higher) kms kms The Java KeyStore KMS service runs as this user.
Key Trustee KMS (CDH 5.3 and higher) kms kms The Key Trustee KMS service runs as this user.
Key Trustee Server (CDH 5.4 and higher) keytrustee keytrustee The Key Trustee Server service runs as this user.
Kudu kudu kudu Kudu services run as this user.
Llama (CDH 5) llama llama Llama runs as this user.
Apache Mahout   No special users.
MapReduce (CDH 4, CDH 5) mapred mapred, hadoop Without Kerberos, the JobTracker and tasks run as this user. The LinuxTaskController binary is owned by this user for Kerberos.
Apache Oozie (CDH 4, CDH 5) oozie oozie The Oozie service runs as this user.
Parquet   No special users.
Apache Pig   No special users.
Cloudera Search (CDH 4.3 and higher, CDH 5) solr solr The Solr processes run as this user.
Apache Spark (CDH 5) spark spark The Spark History Server process runs as this user.
Apache Sentry (CDH 5.1 and higher) sentry sentry The Sentry service runs as this user.
Apache Sqoop (CDH 4, CDH 5) sqoop sqoop This user is only for the Sqoop1 Metastore, a configuration option that is not recommended.
Apache Sqoop2 (CDH 4.2 and higher, CDH 5) sqoop2 sqoop, sqoop2 The Sqoop2 service runs as this user.
Apache Whirr   No special users.
YARN (CDH 4, CDH 5) yarn yarn, hadoop Without Kerberos, all YARN services and applications run as this user. The LinuxContainerExecutor binary is owned by this user for Kerberos.
Apache ZooKeeper (CDH 4, CDH 5) zookeeper zookeeper The ZooKeeper processes run as this user. It is not configurable.

Product Compatibility Matrix for Apache Accumulo

This matrix contains compatibility information across versions of Apache Accumulo, and CDH and Cloudera Manager. For detailed information on each release, see  Apache Accumulo documentation.
Product Lowest supported Cloudera Manager Version Lowest supported CDH Version Lowest supported Impala Version Lowest supported Search Version Integrated into CDH
Accumulo 1.7.2 Cloudera Manager 5.0.0 CDH 5.5.0 Not Supported Not Supported No
Accumulo 1.6.0 Cloudera Manager 5.0.0 CDH 4.6.0 - 4.x.x, CDH 5.1.0 Not Supported Not Supported No
Accumulo 1.4.4 Cloudera Manager 5.0.0 CDH 4.5.0 (Not for use with CDH 5) Not Supported Not Supported No
Accumulo 1.4.3 Cloudera Manager 5.0.0 CDH 4.3.0 (Not for use with CDH 5) Not Supported Not Supported No

Product Compatibility Matrix for Apache Impala (incubating)

This matrix contains compatibility information across versions of Impala and CDH/Cloudera Manager. For detailed information on each release, see the Impala documentation.

Product Supported Cloudera Manager Versions Supported CDH Versions Integrated into CDH
Impala 2.8.x (plus additional patches) for CDH 5.11.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.11.x CDH 5.11.x
Impala 2.8.x (plus a few minor patches) for CDH 5.10.x; although version string still says Impala 2.7

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.10.x CDH 5.10.x
Impala 2.7.x for CDH 5.9.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.9.x CDH 5.9.x
Impala 2.6.x for CDH 5.8.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.8.x CDH 5.8.x
Impala 2.5.x for CDH 5.7.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.7.x CDH 5.7.x
Impala 2.4.x for CDH 5.6.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.6.x CDH 5.6.x
Impala 2.3.x for CDH 5.5.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.5.x CDH 5.5.x
Impala 2.2.x for CDH 5.4.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.4.x CDH 5.4.x
Impala 2.2.0

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.4.0 CDH 5.4.0
Impala 2.1.8

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.10 CDH 5.3.10
Impala 2.1.7

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.9 CDH 5.3.9
Impala 2.1.6

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.8 CDH 5.3.8
Impala 2.1.5

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.6 CDH 5.3.6
Impala 2.1.4

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.4 and CDH 5.3.5 CDH 5.3.4 and CDH 5.3.5
Impala 2.1.3

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.3 CDH 5.3.3
Impala 2.1.2

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.2 CDH 5.3.2
Impala 2.1.1

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.1 CDH 5.3.1
Impala 2.1.0

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.0 CDH 5.3.0
Impala 2.0.5

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.6 CDH 5.2.6
Impala 2.0.4

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.5 CDH 5.2.5
Impala 2.0.3

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.4 CDH 5.2.4
Impala 2.0.2

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.3 CDH 5.2.3
Impala 2.0.1

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.1 CDH 5.2.1
Impala 2.0.0

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.0 CDH 5.2.0
Impala 1.4.4

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.1.5 CDH 5.1.5
Impala 1.4.3

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.1.4 CDH 5.1.4
Impala 1.4.2

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.1.3 CDH 5.1.3
Impala 1.4.1

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.1.2 CDH 5.1.2
Impala 1.4.0

Cloudera Manager 5.0.0 - 5.x.x; Recommended: Cloudera Manager 5.1.0

CDH 5.1.0 CDH 5.1.0
Impala 1.3.3 Cloudera Manager 5.0.0 - 5.x.x CDH 5.0.5 CDH 5.0.5
Impala 1.3.2 Cloudera Manager 5.0.0 - 5.x.x CDH 5.0.4 CDH 5.0.4
Impala 1.3.1 Cloudera Manager 5.0.0 - 5.x.x CDH 5.0.1 - 5.0.x CDH 5.0.1
Impala 1.3.0 Cloudera Manager 5.0.0 - 5.x.x CDH 5.0.0 CDH 5.0.0

Product Compatibility Matrix for Cloudera Distribution of Apache Kafka

Cloudera Distribution of Apache Kafka is currently distributed as a package and in a parcel that is independent of the CDH parcel. The parcel integrates with Cloudera Manager using a Custom Service Descriptor (CSD).

For the latest documentation, see Kafka Documentation.

Product Feature Lowest Supported Cloudera Manager Version Supported CDH Versions Integrated into CDH
Kafka 2.1.x Sentry authorization Cloudera Manager 5.9.x CDH 5.9.x and higher No
Kafka 2.0.x Enhanced security Cloudera Manager 5.5.3 CDH 5.4.x and higher No
Kafka 1.4.x Distributed both as package and parcel Cloudera Manager 5.2.x CDH 5.4.x, 5.5.x, 5.6.x No
Kafka 1.3.x Includes Kafka Monitoring Cloudera Manager 5.2.x CDH 5.4.x, 5.5.x, 5.6.x No
Kafka 1.2.x   Cloudera Manager 5.2.x CDH 5.4.x, 5.5.x, 5.6.x No

Product Compatibility Matrix for Apache Kudu

Apache Kudu is currently distributed as a package and in a parcel that is independent of the CDH parcel. The parcel integrates with Cloudera Manager using a Custom Service Descriptor (CSD).

For the latest documentation, see the Kudu Documentation.

Product Feature Lowest Supported Cloudera Manager Version Supported CDH Versions Integrated into CDH
Kudu 1.3.0 / CDH 5.11.x Available as a CSD within Cloudera Manager. Cloudera Manager 5.11.0 CDH 5.11.x No
Kudu 1.2.0 / CDH 5.10.x Distributed both as package and parcel.

Kudu's first generally available (GA) release that is supported on Cloudera Manager and CDH 5.10.x clusters.

Cloudera Manager 5.10.0 CDH 5.10.x No
Kudu 1.1.x (and lower) - Beta Public beta releases that are not commercially supported on CDH clusters. N/A N/A No

Product Compatibility Matrix for Cloudera Navigator

This matrix contains compatibility information across versions of Cloudera Navigator, Cloudera Manager, and CDH. For detailed information on each release, see  Cloudera Navigator documentation. Note:

Cloudera Navigator requires HiveServer2 for complete governance Hive queries. Cloudera Navigator does not capture audit events for queries that are run on HiveServer1/Hive CLI, and lineage is not captured for certain types of operations that are run on HiveServer1.

HiveServer1 and Hive CLI are deprecated in CDH 5. If you use Cloudera Navigator to capture auditing, lineage, and metadata for Hive operations, upgrade to HiveServer2 if you have not done so already.

Product Feature Lowest supported Cloudera Manager Version Lowest supported CDH Version Lowest supported Impala Version Lowest supported Search Version
Cloudera Navigator 2.10.x Auditing, Metadata, Analytics, and Security 5.11.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • HiveServer2 - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark - supported starting in 5.11.0
Impala 1.2.1 with CDH 4.4.0 CDH 5.4.0
Cloudera Navigator 2.9.x Auditing, Metadata, Analytics, and Security 5.10.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • HiveServer2 - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark (available but unsupported) - 5.4.0
Impala 1.2.1 with CDH 4.4.0 CDH 5.4.0
Cloudera Navigator 2.8.x Auditing, Metadata, Analytics, and Security 5.9.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • HiveServer2 - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark (available but unsupported) - 5.4.0
Impala 1.2.1 with CDH 4.4.0 CDH 5.4.0
Cloudera Navigator 2.7.x Auditing, Metadata, Analytics, and Security 5.8.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark (available but unsupported)- 5.4.0
Impala 1.2.1 with CDH 4.4.0 CDH 5.4.0
Cloudera Navigator 2.6.x Auditing, Metadata, Analytics, and Security 5.7.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark (available but unsupported)- 5.4.0
Impala 1.2.1 with CDH 4.4.0 CDH 5.4.0
Cloudera Navigator 2.5.x Auditing, Metadata, Analytics, and Security 5.6.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Impala, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala and Spark (available but unsupported)- 5.4.0
Impala 1.2.1 with CDH 4.4.0 CDH 5.4.0
Cloudera Navigator 2.4.x Auditing, Metadata, Analytics, and Security 5.5.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Impala, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala and Spark (available but unsupported)- 5.4.0
Impala 1.2.1 with CDH 4.4.0 CDH 5.4.0
Cloudera Navigator 2.3.x Auditing, Metadata, and Security 5.4.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Impala, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala and Spark (available but unsupported)- 5.4.0
Impala 1.2.1 with CDH 4.4.0 CDH 5.4.0
Cloudera Navigator 2.2.x Auditing, Metadata, and Security 5.3.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Oozie, MapReduce, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
Impala 1.2.1 with CDH 4.4.0 Not Supported
Cloudera Navigator 2.1.x Auditing, Metadata, and Security 5.2.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Oozie, MapReduce, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
Impala 1.2.1 with CDH 4.4.0 Not Supported
Cloudera Navigator 2.0.1 Auditing, Metadata, and Security 5.1.2
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Oozie, MapReduce, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
Impala 1.2.1 with CDH 4.4.0 Not Supported
Cloudera Navigator 2.0.0 Auditing, Metadata, and Security 5.1.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Oozie, MapReduce, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
Impala 1.2.1 with CDH 4.4.0 Not Supported
Cloudera Navigator 1.2.x Auditing 5.0.0
  • HDFS, HBase - 4.0.0
  • Hue - 4.2.0
  • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
Impala 1.2.1 with CDH 4.4.0 Not Supported
Metadata (2.0 beta 2) 5.0.0
  • HDFS, Hive, Oozie, MapReduce, Sqoop 1 - 4.4.0
  • Pig - 4.6.0
Not Supported Not Supported

Cloudera Navigator Supported Databases

Cloudera Navigator supports the following databases:
  • Maria DB 5.5, 10
  • MySQL 5.1, 5.5, 5.6, 5.7
  • Oracle 11gR2 and 12c
  • PostgreSQL 8.1, 8.3, 8.4, 9.1, 9.2, 9.3, 9.4

Cloudera Navigator Supported Browsers

The Cloudera Navigator UI supports the following browsers:
  • Mozilla Firefox 24 and higher
  • Google Chrome 36 and higher
  • Internet Explorer 11
  • Safari 5 and higher

Cloudera Navigator Supported CDH and Managed Service Versions

This section describes the CDH and managed service versions supported by the Cloudera Navigator auditing and metadata features.

Cloudera Navigator Auditing
This section describes the audited operations and service versions supported by Cloudera Navigator auditing.
Component Operations (For details, see Cloudera Navigator Auditing). Minimum Supported Service Version
HDFS
  • Operations that access or modify a file's or directory's data or metadata
  • Operations denied due to lack of privileges
CDH 4.0.0
HBase
  • In CDH versions less than 4.2.0, for grant and revoke operations, the operation in log events is ADMIN
  • In simple authentication mode, if the HBase Secure RPC Engine property is false (the default), the username in log events isUNKNOWN. To see a meaningful username:
    1. Click the HBase service.
    2. Click the Configuration tab.
    3. Select Service-wide > Security.
    4. Set the HBase Secure RPC Engine property to true.
    5. Save the change and restart the service.
CDH 4.0.0
Hive
  • Operations (except grant, revoke, and metadata access only) sent to HiveServer2
  • Operations denied due to lack of privileges
Limitations:
  • Actions taken against Hive using the Hive CLI are not audited. Therefore if you have enabled auditing you should disable the Hive CLI to prevent actions against Hive that are not audited.
  • In simple authentication mode, the username in log events is the username passed in the HiveServer2 connect command. If you do not pass a username in the connect command, the username is log events is anonymous.
CDH 4.2.0, CDH 4.4.0 for operations denied due to lack of privileges.
Hue
  • Operations (except grant, revoke, and metadata access only) sent through the Beeswax Server
CDH 4.4.0
  • User operations such as log in, log out, add and remove user, add and remove LDAP group, add and remove user from LDAP group
CDH 5.5.0
Impala
  • Queries denied due to lack of privileges
  • Queries that pass analysis
Impala 1.2.1 with CDH 4.4.0
Navigator Metadata Server
  • Viewing and changing audit reports
  • Viewing and changing authorization configurations
  • Viewing and changing metadata
  • Viewing and changing policies
  • Viewing and changing saved searches
Cloudera Navigator 2.3
Sentry
  • Operations sent to the HiveServer2 and Hive Metastore Server roles and Impala service
  • Adding and deleting roles, assigning roles to groups and removing roles from groups, creating and deleting privileges, granting and revoking privileges
  • Operations denied due to lack of privileges
You do not directly configure the Sentry service for auditing. Instead, when you configure the Hive and Impala services for auditing, grant, revoke, and metadata operations appear in the Hive or Impala service audit logs.
CDH 5.1.0
Solr
  • Index creation and deletion
  • Schema and configuration file modification
  • Index, service, document tag access
CDH 5.4.0
Cloudera Navigator Metadata
This section describes the CDH and managed service versions supported by the Cloudera Navigator metadata feature.
Component Minimum Supported Version
HDFS. However, federated HDFS is not supported. CDH 4.4.0
Hive CDH 4.4.0
Impala CDH 5.4.0
MapReduce CDH 4.4.0
Oozie. Supported actions:
  • 2.4 - map-reduce, pig, hive, hive2, sqoop
  • 2.3 and lower - map-reduce, pig, hive, sqoop
CDH 4.4.0
Pig CDH 4.6.0
Spark CDH 5.4.0
Sqoop 1. All Cloudera connectors are supported. CDH 4.4.0
YARN CDH 5.0.0

Product Compatibility Matrix for Cloudera Navigator Encryption

Cloudera Navigator encryption comprises several components. See Cloudera Navigator Data Encryption Overview for more information on the components, concepts, and architecture involved in encrypting data.

Although the version numbers differ, Cloudera Navigator encryption component releases are generally coordinated with Cloudera Enterprise. See the following table for information on which component versions correspond with a given Cloudera Enterprise release:

Cloudera Navigator Encryption Components Release Versions
Cloudera Enterprise Release Cloudera Navigator Key Trustee Server Version Key Trustee KMS Version Cloudera Navigator Key HSM Version Cloudera Navigator Encrypt Version
5.11.x 5.11.x 5.11.x 1.9.x 3.12.x
5.10.x 5.10.x 5.10.x None 3.11.x
5.9.x 5.9.x 5.9.x 1.8.x None
5.8.x 5.8.x 5.8.x 1.7.x 3.10.x
5.7.x 5.7.x 5.7.x 1.6.x 3.9.x
5.6.x None None None None
5.5.x 5.5.x 5.5.x 1.5.x 3.8.x
5.4.x 5.4.x 5.4.x 1.4.x 3.7.x
5.3.x 3.8.x 5.3.x 1.3.x 3.6.x

See below for the individual compatibility matrices for each component:

  • Cloudera Navigator Key Trustee Server
  • Key Trustee KMS
  • Cloudera Navigator Key HSM
  • Cloudera Navigator Encrypt

Cloudera Navigator Key Trustee Server

Because of a change in the ports used by Key Trustee Server, Navigator Encrypt versions lower than 3.7 and Key Trustee KMS versions lower than 5.4 are not supported in Key Trustee Server 5.4 and higher.

Cloudera Navigator Key Trustee Server Compatibility Matrix
Cloudera Navigator Key Trustee Server Version Supported Operating Systems Lowest Supported Cloudera Manager Version Lowest Supported Cloudera Navigator Key HSM Versions Supported Key Trustee KMS Versions Supported Cloudera Navigator Encrypt Versions
5.11.x
  • RHEL and CentOS:6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2, 7.3
  • Oracle Enterprise Linux: 6.7, 6.8, 7.1, 7.2, 7.3
5.11.x 1.3.x 5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x, 5.11.x 3.7.x, 3.8.x, 3.9.x, 3.10.x, 3.11.x, 3.12.x
5.10.x
  • RHEL and CentOS:6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2, 7.3
  • Oracle Enterprise Linux: 6.7, 6.8, 7.1, 7.2, 7.3
5.10.x 1.3.x 5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x 3.7.x, 3.8.x, 3.9.x, 3.10.x, 3.11.x, 3.12.x
5.9.x
  • RHEL and CentOS:6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2
  • Oracle Enterprise Linux: 6.7, 6.8, 7.1, 7.2
5.9.x 1.3.x 5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x 3.7.x, 3.8.x, 3.9.x, 3.10.x, 3.11.x, 3.12.x
5.8.x
  • RHEL and CentOS:6.4, 6.5, 6.6, 6.7, 7.1, 7.2
  • Oracle Enterprise Linux: 6.7, 7.1, 7.2
5.8.x 1.3.x 5.4.x, 5.5.x, 5.7.x, 5.8.x 3.7.x, 3.8.x, 3.9.x, 3.10.x, 3.11.x, 3.12.x
5.7.x
  • RHEL and CentOS:6.4, 6.5, 6.6, 6.7, 7.1, 7.2
  • Oracle Enterprise Linux: 6.7, 7.1, 7.2
5.7.x 1.3.x 5.4.x, 5.5.x, 5.7.x 3.7.x, 3.8.x, 3.9.x, 3.10.x, 3.11.x, 3.12.x
5.5.x
  • RHEL and CentOS:6.4, 6.5, 6.6, 6.7, 7.1
  • Oracle Enterprise Linux: 6.7, 7.1, 7.2
5.5.x 1.3.x 5.4.x, 5.5.x, 5.7.x 3.7.x, 3.8.x, 3.9.x, 3.10.x, 3.11.x, 3.12.x
5.4.x
  • RHEL and CentOS:6.4, 6.5, 6.6
5.4.x 1.3.x 5.4.x, 5.5.x, 5.7.x 3.7.x, 3.8.x, 3.9.x, 3.10.x, 3.11.x, 3.12.x
3.8.x
  • RHEL and CentOS:6.4, 6.5
Not supported 1.3.x 5.3.x 3.6.x

Key Trustee KMS

Key Trustee KMS Compatibility Matrix
Key Trustee KMS Version Supported Operating Systems Supported Key Trustee Server Versions Lowest Supported Cloudera Manager Version Supported CDH Versions
5.11.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2, 7.3
  • Oracle Enterprise Linux: 5.7, 5.10, 5.11, 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2, 7.3
  • SLES: 11 SP2, 11 SP3, 11 SP4, 12 SP1, 12 SP2
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 12.04, 14.04, 16.04
5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x, 5.11.x 5.11.x 5.3.x, 5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x, 5.11.x
5.10.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2, 7.3
  • Oracle Enterprise Linux: 5.7, 5.10, 5.11, 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2, 7.3
  • SLES: 11 SP2, 11 SP3, 11 SP4, 12 SP1
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 12.04, 14.04
5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x 5.10.x 5.3.x, 5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x
5.9.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2
  • Oracle Enterprise Linux: 5.7, 5.10, 5.11, 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2
  • SLES: 11 SP2, 11 SP3, 11 SP4, 12 SP1
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 12.04, 14.04
5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x 5.9.x 5.3.x, 5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x
5.8.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 7.1, 7.2
  • Oracle Enterprise Linux: 5.7, 5.10, 5.11, 6.4, 6.5, 6.6, 6.7, 7.1, 7.2
  • SLES: 11 SP2, 11 SP3, 11 SP4
  • Debian: 7.1, 7.8, 8.2
  • Ubuntu: 12.04, 14.04
5.4.x, 5.5.x, 5.7.x, 5.8.x 5.8.x 5.3.x, 5.4.x, 5.5.x, 5.7.x, 5.8.x
5.7.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 7.1, 7.2
  • Oracle Enterprise Linux: 5.7, 5.10, 5.11, 6.4, 6.5, 6.6, 6.7, 7.1, 7.2
  • SLES: 11 SP2, 11 SP3, 11 SP4
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04
5.4.x, 5.5.x, 5.7.x 5.7.x 5.3.x, 5.4.x, 5.5.x, 5.7.x
5.5.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 7.1
  • Oracle Enterprise Linux: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 7.1
  • SLES: 11 SP2, 11 SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
5.4.x, 5.5.x 5.5.x 5.3.x, 5.4.x, 5.5.x
5.4.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6
  • Oracle Enterprise Linux: 5.7, 5.10, 6.4, 6.5, 6.6
  • SLES: 11 SP2, 11 SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
5.4.x 5.4.x 5.3.x, 5.4.x
5.3.x
  • RHEL and CentOS: 5.7, 6.4, 6.5
  • Oracle Enterprise Linux: 5.7, 6.4, 6.5
  • SLES: 11 SP2, 11 SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
3.8.x 5.3.x 5.3.x

Cloudera Navigator Key HSM

Cloudera Navigator Key HSM must be installed on the same host as Key Trustee Server. Although Key HSM is compatible across all versions of Key Trustee Server, Cloudera strongly recommends also upgrading Key HSM after you upgrade Key Trustee Server. See Installing Cloudera Navigator Key HSM and Upgrading Cloudera Navigator Key HSM for more information.

Cloudera Navigator Key HSM Compatibility Matrix
Cloudera Navigator Key HSM Version Supported Operating Systems Lowest Supported Key Trustee Server Version
1.9.x
  • RHEL and CentOS: 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2
3.8.x
1.8.x
  • RHEL and CentOS: 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2
3.8.x
1.7.x
  • RHEL and CentOS: 6.4, 6.5, 6.6, 6.7, 7.1, 7.2
3.8.x
1.6.x
  • RHEL and CentOS: 6.4, 6.5, 6.6, 6.7, 7.1, 7.2
3.8.x
1.5.x
  • RHEL and CentOS: 6.4, 6.5, 6.6, 6.7, 7.1
3.8.x
1.4.x
  • RHEL and CentOS: 6.4, 6.5, 6.6
3.8.x
1.3.x
  • RHEL and CentOS: 6.4, 6.5
3.8.x

Cloudera Navigator Encrypt

Cloudera Navigator Encrypt Compatibility Matrix
Cloudera Navigator Encrypt Version Supported Operating Systems Supported Key Trustee Server Versions
3.12.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2
  • Oracle Enterprise Linux:6.4, 6.5, 6.6, 6.7, 7.1
  • SLES: 11 SP2, 11 SP3, 11 SP4, 12 SP1
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3, 16.04
5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x, 5.11.x
3.11.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2
  • Oracle Enterprise Linux:6.4, 6.5, 6.6, 6.7, 7.1
  • SLES: 11 SP2, 11 SP3, 11 SP4, 12 SP1
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3
5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x, 5.11.x
3.10.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 6.8, 7.1, 7.2
  • Oracle Enterprise Linux:6.4, 6.5, 6.6, 6.7, 7.1
  • SLES: 11 SP2, 11 SP3, 11 SP4
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3
5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x, 5.11.x
3.9.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 7.1, 7.2
  • Oracle Enterprise Linux:6.4, 6.5, 6.6, 6.7, 7.1
  • SLES: 11 SP2, 11 SP3, 11 SP4
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3
5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x, 5.11.x
3.8.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6, 6.7, 7.1
  • Oracle Enterprise Linux:6.4, 6.5, 6.6, 6.7, 7.1
  • SLES: 11 SP2, 11 SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x, 5.11.x
3.7.x
  • RHEL and CentOS: 5.7, 5.10, 6.4, 6.5, 6.6
  • Oracle Enterprise Linux:6.4, 6.5, 6.6
  • SLES: 11 SP2, 11 SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
5.4.x, 5.5.x, 5.7.x, 5.8.x, 5.9.x, 5.10.x, 5.11.x
3.6.x
  • RHEL and CentOS: 5.7, 6.4, 6.5
  • Oracle Enterprise Linux:6.4, 6.5
  • SLES: 11 SP2, 11 SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
3.8.x

Product Compatibility Matrix for Apache Sentry

Sentry enables role-based, fine-grained authorization for HiveServer2 and provides classic database-style authorization for Hive, Cloudera Impala and Cloudera Search. You can use either the Sentry service (introduced in Cloudera Manager 5.1.0 and CDH 5.1.0) or the policy file approach to secure your data.

Note: It's possible for a single cluster to use both the Sentry service (for Hive and Impala) and Sentry policy files (for Solr).
Product Feature Lowest supported Cloudera Manager Version Lowest supported CDH Version Lowest supported Impala Version Lowest supported Search Version Integrated into CDH
Apache Sentry Sentry Service Cloudera Manager 5.1.0 CDH 5.1.0 Impala 1.4.0 for CDH 5 Search for CDH 5.8.0 Yes
Policy File Cloudera Manager 4.7.0 CDH 4.3.0 Impala 1.2.1 Search 1.1.0 Yes; Starting CDH 4.4.0

Support for Other Filesystems

Filesystems Lowest Release Supported
Amazon S3 (s3a) Apache Sentry for CDH 5.8
Amazon RDS Apache Sentry for CDH 5.9

Product Compatibility Matrix for Apache Spark

Spark is a fast, general engine for large-scale data processing. For installation and configuration instructions, see Spark Installation. To see new features introduced with each release, refer to the CDH 5 Release Notes.

Since Spark has been integrated into the CDH package, its compatibility with Cloudera Manager and CDH depends on the CDH 5.x.x release it is shipped with.

Product Lowest Supported Cloudera Manager Version Lowest Supported CDH Version Lowest supported Impala Version Lowest supported Search Version Integrated into CDH
Apache Spark 1.6.x Cloudera Manager 5.7.x CDH 5.7.x Not Supported Not Supported CDH 5.7.0
Apache Spark 1.5.x Cloudera Manager 5.5.x and Cloudera Manager 5.6.x CDH 5.5.x and CDH 5.6.x Not Supported Not Supported CDH 5.5.0 and CDH 5.6.0
Apache Spark 1.3.x Cloudera Manager 5.4.x CDH 5.4.x Not Supported Not Supported CDH 5.4.0
Apache Spark 1.2.x Cloudera Manager 5.3.x CDH 5.3.x Not Supported Not Supported CDH 5.3.0
Apache Spark 1.1.x Cloudera Manager 5.2.x CDH 5.2.x Not Supported Not Supported CDH 5.2.0
Apache Spark 1.0.x Cloudera Manager 5.1.x CDH 5.1.x Not Supported Not Supported CDH 5.1.0
Apache Spark 0.9.x Cloudera Manager 5.0.x CDH 5.0.x Not Supported Not Supported CDH 5.0.0

Product Compatibility for EMC Isilon

For compatibility with Cloudera Manager Replication and Snapshot features, see Product Compatibility Matrix for Backup and Disaster Recovery. For information on using Isilon with CDH, see Using CDH with Isilon Storage.

CDH Version Isilon OneFS Version
5.11
  • 8.0.0.4
  • 8.0.0.x releases that are higher than 8.0.0.4.
5.10
  • 8.0.0.3 (with patch 184679)Note: The patch fixes a bug that prevented high availability services from working correctly in a Kerberized environment.
  • 8.0.0.4
  • 8.0.0.x releases that are higher than 8.0.0.4.
5.9
  • 8.0.0.2
  • 8.0.0.3Note: There is a bug in versions 8.0.0.1, 8.0.0.2 and 8.0.0.3 that prevents high availability services from working correctly in a Kerberized environment.
  • 8.0.0.4
  • 8.0.0.x releases that are higher than 8.0.0.4.
5.8 8.0.0.x
5.4.4 and higher 7.2.0.3 and higher

Cloudera recommends 7.2.1.1

Product Compatibility Matrix for Backup and Disaster Recovery

This matrix contains compatibility information across features of Cloudera Manager Backup and Disaster Recovery and CDH and Cloudera Manager.
Product Feature Lowest supported Cloudera Manager Version Lowest supported CDH Version Supported Services
Backup & Disaster Recovery Replication Cloudera Manager 4.5.0 CDH 4.0.1 HDFS, Hive, Impala
Replication to and from Amazon S3 Cloudera Manager 5.9.0 CDH 5.9.0 HDFS, Hive, Impala
Snapshots Cloudera Manager 5.0.0 CDH 5.0.0 HDFS, Hive, Impala
Snapshots fromIsilon storage Not Supported Not Supported HDFS, Hive, Impala
The tables below list the supported and unsupported replication scenarios:

Supported Replication Scenarios

In Cloudera Manager 5, replication is supported between CDH 5 or CDH 4 clusters. The following tables describe support for HDFS and Hive/Impala replication.

Service Source Destination
  Cloudera Manager Version CDH Version Comment Cloudera Manager Version CDH Version Comment
HDFS, Hive 4 4   5 4  
HDFS, Hive 4 4.4 or higher   5 5  
HDFS, Hive 5 5 TLS/SSL enabled on Hadoop services 5 5 TLS/SSL enabled on Hadoop services
HDFS, Hive 5 5 TLS/SSL enabled on Hadoop services 5 5 TLS/SSL notenabled on Hadoop services
HDFS, Hive 5 5.1 TLS/SSL enabled on Hadoop services and YARN 5 4 or 5  
HDFS, Hive 5 4   4.7.3 or higher 4  
HDFS, Hive 5 4   5 4  
HDFS, Hive 5 5   5 5  
HDFS, Hive 5 5   5 4.4 or higher  
HDFS, Hive 5.7 5.7, with Isilon storage See Supported Replication Scenarios for Clusters using Isilon Storage. 5.7 5.7, with Isilon storage See Supported Replication Scenarios for Clusters using Isilon Storage.
HDFS, Hive 5.7 5.7 5.7 5.7, with Isilon storage
HDFS, Hive 5.7 5.7, with Isilon storage 5.7 5.7
HDFS, Hive 5.8 5.8, with or without Isilon Storage
  • Kerberos can be enabled
  • SeeSupported Replication Scenarios for Clusters using Isilon Storage.
5.8 5.8, with or without Isilon Storage
  • Kerberos can be enabled
  • SeeSupported Replication Scenarios for Clusters using Isilon Storage.

Unsupported Replication Scenarios

Note: If you are using Isilon storage for CDH, see  Supported Replication Scenarios for Clusters using Isilon Storage.
Service Source Destination
  Cloudera Manager Version CDH Version Comment Cloudera Manager Version CDH Version Comment
Any 4 or 5 4 or 5 Kerberos enabled. 4 or 5 4 or 5 Kerberos not enabled
Any 4 or 5 4 or 5 Kerberos not enabled. 4 or 5 4 or 5 Kerberos enabled
HDFS, Hive 4 or 5 4 Where the replicated data includes a directory that contains a large number of files or subdirectories (several hundred thousand entries), causing out-of-memory errors.

To work around this issue, follow this procedure.

4 or 5 5  
Hive 4 or 5 4 Replicate HDFS Files is disabled. 4 or 5 4 or 5 Over-the-wire encryption is enabled.
Hive 4 or 5 4 Replication can fail if the NameNode fails over during replication. 4 or 5 5, with high availability enabled Replication can fail if the NameNode fails over during replication.
Hive 4 or 5 4 The clusters use different Kerberos realms. 4 or 5 5 An older JDK is deployed. (Upgrade the CDH 4 cluster to use JDK 7 or JDK6u34 to work around this issue.)
Any 4 or 5 4 SSL enabled on Hadoop services. 4 or 5 4 or 5  
Hive 4 or 5 4.2 or higher If the Hive schema contain views. 4 or 5 4  
HDFS 4 or 5 4, with high availability enabled Replications fail if NameNode failover occurs during replication. 4 or 5 5, without high availability Replications fail if NameNode failover occurs during replication.
HDFS 4 or 5 4 or 5 Over the wire encryption is enabled. 4 or 5 4  
HDFS 4 or 5 5 Clusters where there are URL-encoding characters such as % in file and directory names. 4 or 5 4  
Hive 4 or 5 4 or 5 Over the wire encryption is enabled andReplicate HDFS Files is enabled. 4 or 5 4  
Hive 4 or 5 4 or 5 From one cluster to the same cluster. 4 or 5 4 or 5 From one cluster to the same cluster.
HDFS, Hive 4 or 5 5 Where the replicated data includes a directory that contains a large number of files or subdirectories (several hundred thousand entries), causing out-of-memory errors.

To work around this issue, follow this procedure.

4 or 5 4  
HDFS 4 or 5 5 The clusters use different Kerberos realms. 4 or 5 4 An older JDK is deployed. (Upgrade the CDH 4 cluster to use JDK 7 or JDK6u34 to work around this issue.)
Hive 4 or 5 5 Replicate HDFS Files is enabled and the clusters use different Kerberos realms. 4 or 5 4 An older JDK is deployed. (Upgrade the CDH 4 cluster to use JDK 7 or JDK6u34 to work around this issue.)
Any 4 or 5 5 SSL enabled on Hadoop services and YARN. 4 or 5 4 or 5  
Any 4 or 5 5 SSL enabled on Hadoop services. 4 or 5 4  
HDFS 4 or 5 5, with high availability enabled Replications fail if NameNode failover occurs during replication. 4 or 5 4, without high availability Replications fail if NameNode failover occurs during replication.
HDFS, Hive 5 5   4 4  
Hive 5.2 5.2 or lower Replication of Impala UDFs is skipped. 4 or 5 4 or 5  

Supported Replication Scenarios for Clusters using Isilon Storage

Note the following when scheduling replication jobs for clusters that use Isilon storage:
  • As of CDH 5.8 and higher, Replication is supported for clusters using Kerberos and Isilon storage on the source or destination cluster, or both. See Configuring Replication with Kerberos and Isilon. Replication between clusters using Isilon storage and Kerberos is not supported in CDH 5.7.
  • Make sure that the hdfs user is a superuser in the Isilon system. If you specify alternate users with theRun As option when creating replication schedules, those users must also be superusers.
  • Cloudera recommends that you use the Isilon root user for replication jobs. (Specify root in the Run Asfield when creating replication schedules.)
  • Select the Skip checksum checks property when creating replication schedules.
  • Clusters that use Isilon storage do not support snapshots. Snapshots are used to ensure data consistency during replications in scenarios where the source files are being modified. Therefore, when replicating from an Isilon cluster, Cloudera recommends that you do not replicate Hive tables or HDFS files that could be modified before the replication completes.

See Using CDH with Isilon Storage.

Supported Configurations with Virtualization and Cloud Platforms

This section lists supported cloud and virtualization platforms for deploying Cloudera software. The linked reference architectures are not replacements for statements of support above, but are guides to assist with deployment and sizing options.

Amazon Web Services

For information on deploying Cloudera software on a Amazon Web Services (AWS) cloud infrastructure, see the Cloudera Enterprise Reference Architecture for AWS Deployments.

Google Cloud Platform

For information on deploying Cloudera software on a Google Cloud Platform infrastructure, see theCloudera Enterprise Reference Architecture for Google Cloud Platform Deployments.

Microsoft Azure

For information on deploying Cloudera software on a Microsoft Azure cloud infrastructure, see theCloudera Enterprise Reference Architecture for Azure Deployments.

VMware

For information on deploying Cloudera software on a VMware-based infrastructure, see the Reference architecture for deploying on VMware.

Recommendation when deploying on VMware in the current release:

你可能感兴趣的:(Hadoop,Linux,hadoop,cloudera,安装)