MLPerf Storage v3.0 adds an S3 data layer to the existing POSIX framework, alongside new KV Cache and vector database workloads, closely aligning benchmark criteria with modern AI data system requirements.
Developed by MLCommons, the MLPerf Storage benchmark originally evaluated storage performance for machine learning training workloads including Unet3D, Cosmoflow and Resnet50, utilizing NVIDIA A100 and H100 GPU hardware. The v3.0 release features a comprehensive test restructuring, centering on four core workload scenarios.
The updated testing framework includes AI training based on Unet3D and RetinaNet, which measures maximum accelerator scalability while maintaining over 90% utilization, simulated on NVIDIA B200 or AMD MI355 GPUs. It also covers checkpointing for Llama 3-style models ranging from 8B to 1.25T parameters, prioritizing low latency for 10 rounds of iterative writes and reads. Additionally, the suite incorporates vector database testing on a 1‑million, 1536‑dimension Milvus dataset to maximize query throughput while tracking latency and recall rates, plus KV cache testing to gauge maximum concurrent conversation support capacity.
Brian Belgodere, Co-Chair of the MLPerf Storage Working Group, stated the new test portfolio enriches benchmark coverage for mainstream AI inference storage demands. The updated suite balances training‑focused training and checkpointing tests with inference‑oriented KV cache and vector database evaluations. By breaking down integrated AI systems into targeted storage workload assessments, including checkpointing, KV caching and vector query processing, the benchmark enables industry practitioners to better optimize AI infrastructure configurations and eliminate storage performance bottlenecks.
The newly added S3 object storage layer supports training, checkpointing and partial vector database tests, with roughly one‑sixth of all v3.0 submissions adopting S3‑based storage access.
Curtis Anderson, another Working Group Co-Chair, noted that as AI context scales to trillions of tokens, object storage is emerging as a viable, even preferable alternative to traditional file‑system storage. Native S3 support in v3.0 provides transparent performance data to guide enterprise storage deployment decisions.
A total of 19 vendors submitted results for this benchmark cycle. Notably, major legacy enterprise storage players including DDN, Dell, Huawei, IBM, NetApp, VAST Data and WEKA were absent from the submission list. MLPerf officials commented that traditional NAS vendors have limited participation, with most established incumbents sitting out the latest round.
Given the diversified AI storage landscape, v3.0 does not publish a single overall ranking. Instead, it delivers independent results for the four workload categories with multi‑dimensional metrics. The complete result dataset spans around 145 rows and 55 columns, totaling nearly 8,000 data cells, with 12 dedicated columns exclusively for KV cache performance metrics. These cover throughput, read/write bandwidth and P95 latency under both storage‑only and storage+memory architectures for 8B and 70B Llama model workloads.
MLCommons also recommends standardized performance evaluation based on per‑rack density and per‑watt energy efficiency to measure overall infrastructure operational effectiveness.
The updated benchmark suite features highly complex evaluation logic, allowing vendors to selectively submit results for targeted workloads. For instance, only NewFW, Samsung, Suzhou Zishan and TTA submitted vector database test results.
Everpure reported top-tier performance for its FlashBlade//EXA across checkpointing and KV cache tests for 405B and 1.25T parameter models. Deployed on 30 data nodes to support 1,024 simulated accelerators, the platform achieved 877.52 GiB/s write bandwidth (17.74s completion) and 588.28 GiB/s read bandwidth (28.99s completion) for 1.25T model checkpointing. Its performance scaled linearly from 327.65 GiB/s write throughput on 10 nodes to the peak 30‑node result.
Supplemental data shows on-premises checkpoint write submissions delivered a median energy efficiency of 14 GB/s per watt, with a peak of 201 GB/s per watt. Meanwhile, UNet3D read tests recorded a median efficiency of 34 GB/s per watt and a maximum of 277 GB/s per watt.
Beijing Qianxing Jietong Technology Co., Ltd.
Sandy Yang/Global Strategy Director
WhatsApp / WeChat: +86 13426366826
Email: yangyd@qianxingdata.com
Website: www.qianxingdata.com/www.storagesserver.com
Business Focus:
ICT Product Distribution/System Integration & Services/Infrastructure Solutions
With 20+ years of IT distribution experience, we partner with leading global brands to deliver reliable products and professional services.
“Using Technology to Build an Intelligent World”Your Trusted ICT Product Service Provider!
Το μήνυμά σας πρέπει να αποτελείται από 20-3.000 χαρακτήρες!
Greek
