LinePillar FS

LinePillar FS is a company-level storage solution designed to satisfy HPC, AI LLM, big data analytics and other scenarios. With high throughput, short latency and expandable flexibility as its core goals, it aims to satisfy massive amount of data storage, high concurrent access and complex computing tasks.

Characteristics

Parallel Structural Design: 

Based on scattered Metadata managing and data fragmenting technologies, the product supports parallel R&W with multiple nodes, breaking through the single node efficiency in traditional file systems, achieving linear scale-out. It can be expanded horizontally to thousands of nodes, reaching a EB-level storage scale.

High-Efficient IO Optimisation: 

Using self-adapting data distribution strategy, intelligent cache mechanism and RDMA network acceleration technique, LinePillar FS elevates the processing efficiency of small files and sequential read-and-write capacity of large files, being suitable for AI training, scientific calculations and other high IO scenarios.

High Reliability & Fault-Tolerance: 

The product supports multiple data redundant mechanisms such as Erasure Coding and Replication, meanwhile, it can automatically detect breakdowns and recovers data quickly. It provides 99.999% utility for the system and ensures the continuity of tasks. Its exclusive separate-area defence mechanism enables a 1-Tb scale data transfer to be done within 30 minutes. Plus, the recovery speed is more than 10 times that of the traditional RAID technology, while allowing four-disk fault-tolerance.

Cross-Platform Compatibility: 

Compatible with mainstream operating systems (Linux/Windows), virtual environment and Cloud-Native ecosystem (Kubernetes CSI port), it also works seamlessly with Public Cloud, Private Cloud and Hybrid Cloud structures while supporting POSIX, NFS, S3 and other access protocols.

The product adopts an exclusive design of data structure, storing Metadata and datas into the nodes, achieving complete sharing of all the datas. This eliminates data traffic flows running east to west and simplifies the overall structure, enhancing efficiency of the entire data platform. 

Values

High Efficient Concurrent R&W Abilities:

It allows thousands of computing nodes accessing simultaneously, increases IO throughput. The product uses design of linking the client to Metadata and data services, combines RDMA Zero-copy transfer technology and bypasses traditional FUSE kernel restriction, achieving latency in microseconds and simplified data redundancy.

Linear Scale-out & Optimisation:

Supporting horizontal scale-out, hence linear increase in capacity and efficiency as number of nodes increases, fulfilling needs of boom in tasks.

High Utility & Breaktown Tolerance:

Tasks are ensured running 24/7 with multiple copies or cross-node data mirroring eliminating the breakdown risk with single node. Non-stop maintenance and balance in dynamic load, avoiding loss in storage data due to hardware RAID damage and crash in nodes.