Hadoop Open Platform-as-a-Service (Hops)

    Gautier Berthou (KTH/SICS, Stockholm)

    Hadoop is the de-facto standard for Big Data, that is, large scale data processing and storage. Hadoop is a relatively young platform that still suffer from fundamental scalability issues: all the file system metatdata must be stored in the memory heap of one unique node, the name node, and all the scheduling ecisions are taken by one unique node, the scheduler. At KTH/SICS, we are developing Hadoop Open Platform-as-a-Service (Hops), a new distribution of Apache Hadoop with scalable, highly available, customizable metadata and scheduling. In this talk, we will investigate the architectural challenges in designing and implementing distributed meta-data for both HDFS and YARN.

    adatbányászat szeminárium
    csütörtök, 2015, március 26 - 10:15
    Tavaszi félév