{"id":401161,"date":"2020-12-16T12:03:03","date_gmt":"2020-12-16T17:03:03","guid":{"rendered":"http:\/\/www.marketnewsdesk.com\/?p=401161"},"modified":"2020-12-16T12:03:03","modified_gmt":"2020-12-16T17:03:03","slug":"octoml-announces-faster-machine-learning-model-inferencing-on-apples-new-m1-than-apples-core-ml-4-provides","status":"publish","type":"post","link":"https:\/\/www.marketnewsdesk.com\/index.php\/octoml-announces-faster-machine-learning-model-inferencing-on-apples-new-m1-than-apples-core-ml-4-provides\/","title":{"rendered":"OctoML Announces Faster Machine Learning Model Inferencing on Apple\u2019s New M1 Than Apple\u2019s Core ML 4 Provides"},"content":{"rendered":"
\n

SEATTLE, Dec. 16, 2020 (GLOBE NEWSWIRE) — OctoML, the MLOps automation company for superior model performance, portability and productivity, today demonstrated better model performance on Apple\u2019s M1 chip than Apple\u2019s core inferencing engine.\u00a0 OctoML\u2019s results showcased lower model latency than any of Apple\u2019s own developed software, ranging from a 30% improvement against Apple\u2019s latest Core ML 4 inference engine to a 13x improvement on Apple\u2019s standard Core ML 3. All comparisons were based on the BERT-base model, a common deep learning model used widely for natural language processing tasks, and conducted on both the Mac Mini CPU and GPU.<\/p>\n

\u201cApple is great at showcasing their newest products for the most cutting-edge ML uses,\u201d said Luis Ceze, co-founder and CEO of OctoML. \u201cBut in practice, machine learning engineers can struggle to achieve good performance and may spend months trying to manually debug issues. In contrast, OctoML\u2019s work shows how an automated model optimization service can effortlessly add new hardware and immediately deliver superior model performance.\u201d<\/p>\n

Apple\u2019s latest Core ML 4 resulted in 139 milliseconds of latency on the CPU and 59 milliseconds in latency on GPU. In contrast, OctoML\u2019s work delivered model latency of 108 milliseconds on the CPU and 42 milliseconds on the GPU. These performance improvements represent a 22% improvement on the CPU and nearly 30% improvement on the GPU and are especially notable because they were produced automatically and only weeks after Apple\u2019s public launch of the M1 chip.<\/p>\n

Other performance comparisons included Keras with MLCompute and TensorFlow with Graphdef. For Keras, the Apple M1\u2019s latency on CPU was 579 milliseconds and on GPU was 1,767 milliseconds. For TensorFlow, the M1 demonstrated 512 milliseconds of latency on the CPU and 543 milliseconds on the GPU.<\/p>\n

\u201cHow did we get better results than Apple\u2019s Core ML 4 in just a few weeks? Two reasons,\u201d said Bing Xu, Principal Engineer at OctoML. \u201cFirst, the Apache TVM compiler uses a machine learning-based auto-scheduler to search out CPU and GPU code optimization. Second, the Apache TVM compiler is able to automatically fuse qualified subgraphs and directly generate code. Even the best engineers can\u2019t anticipate the interactions between model architectures, computation workloads and hardware target resource availability.\u201d<\/p>\n

Additional Resources<\/p>\n