SC15 Austin, TX

Consistent Hashing Distance Metrics for Large-Scale Object Storage


Authors: Philip Carns (Argonne National Laboratory), Kevin Harms (Argonne National Laboratory), John Jenkins (Argonne National Laboratory), Misbah Mubarak (Argonne National Laboratory), Robert B. Ross (Argonne National Laboratory), Christopher Carothers (Rensselaer Polytechnic Institute)

Abstract: Large-scale storage systems often use object-level replication to protect against server failures. This calls for the development of efficient algorithms to map replicas to servers at scale. We introduce a modular, open-source consistent hashing library, called libch-placement, and use it to investigate design trade-offs in CPU efficiency, replica declustering, and object sharding for replicated object placement. We identify a multiring hashing configuration that strikes a balance between all three properties. We also demonstrate that large-scale object placement can be calculated at an aggregate rate of tens of billions of objects per second with 8,192 servers. These technologies can be leveraged by large-scale storage systems to improve performance, improve reliability, and manage data placement with per-object granularity.

Poster: pdf
Two-page extended abstract: pdf


Poster Index