Hortonworks Data Platform: HDP 3.0

HDP provides the basis for supporting GPUs in Apache Hadoop clusters, enhancing the performance of computations required for Data Science and AI use cases. It enables GPU pooling for sharing of GPU resources with more workloads for cost effectiveness. It also supports GPU isolation, which dedicates a GPU to an application so that no other application has access to that GPU.

HDP includes a containerized TensorFlow tech preview which combined with GPU pooling delivers easier designing, building and training for deep learning models.

HDP, gives you the freedom to deploy big data workloads in hybrid and multi-cloud environments without vendor lock-in to a particular cloud architecture. Customers are able to seamlessly create and manage big data clusters in any cloud setting. HDP is cloud agnostic and automates provisioning to simplify big data deployments while optimizing the use of cloud resources.

Cloud storage support to store endless amounts of data in its native format including:

• Microsoft ADLS
• WASB
• AWS S3
• And Google Cloud Storage

Cloudbreak provides easy provisioning of clusters in the cloud by deploying HDP to your cloud provider of choice

HDP includes improved query performance to focus on faster queries. Hive LLAP, the fastest Apache Hive engine, runs in a multi-tenant environment without causing resource competition. This integration drastically speeds up queries commonly used in Business Intelligence scenarios, such as join and aggregation queries. In addition to query optimization, Hive also allows the creation of resource pools, for fine-grained resource allocations.

HDP enables ACID transactions by default making it easier to updates in Hive tables and support GDPR requirements. Hive, as a real-time database, eliminates the performance gap between low latency and high throughput workloads to process more data at a faster rate.

HDP continues to provide comprehensive security and governance. HDP’s security is integrated in layers and includes features for authentication, authorization, accountability, and data protection. The integration of security and governance allows security professionals to set classification-based security policies. In addition, data governance tools empower organizations to apply consistent data classification across the data ecosystem.

Additional features allow the auditing of events to get more fine-grained and detailed, making it easier for auditors to do their job. Auditors and users can see full chain of custody as the data moves through the ecosystem. Tag propagation to allow auditors and users to see where the data is going across the enterprise and to retain context of data that is sensitive. Time base polices allow temporary access to a given user.