Google Cloud TPU sets 1 million images-per-second record in MLPerf Inference benchmarks

[Google Cloud] Results from the new MLPerf Inference v0.5 benchmarks, which cover image classification, object detection and language translation, are now available, according to Google Cloud. Google Cloud said it submitted results for all three datacenter-class benchmarks using Cloud TPU v3 devices and demonstrated near-linear scalability up to a record 1 million images processed per second on ResNet-50 v1.5 using 32 Cloud TPU v3 devices. According to Google Cloud, Cloud TPUs offer bfloat16 floating-point numerics and let training and serving run on the same hardware platform, avoiding quantization, recalibration or retraining and the associated accuracy loss and engineering effort; Google Translate can push new models to production within hours of model validation, it said. Google Cloud said Cloud TPU v3 devices are publicly available to Google Cloud customers in beta and are also used across large-scale Google products including Google Search.