A low-latency and fault-tolerant framework for distributed and deep neural networks over the cloud-to-things continuum

Author / Creator

AEiC 2021 - 25th Ada-Europe International Conference on Reliable Software Technologies (2021)

Conferences

AEiC 2021 - 25th Ada-Europe International Conference on Reliable Software Technologies Technical Session 6: Emerging applications with reliability requirements (2021)

Available as

Online

Summary

The current dependency of Artificial Intelligence (AI) systems on Cloud computing implies higher transmission latency and bandwidth consumption. Moreover, it challenges the real-time monitoring of ...

The current dependency of Artificial Intelligence (AI) systems on Cloud computing implies higher transmission latency and bandwidth consumption. Moreover, it challenges the real-time monitoring of physical objects, e.g., the Internet of Things (IoT). Edge systems bring computing closer to end devices and support time-sensitive applications. However, Edge systems struggle with state-of-the-art Deep Neural Networks (DNN) due to computational resource limitations. In this proposal, an introduced framework combines the Edge-Cloud architecture concept with BranchyNet advantages to support fault-tolerant and low-latency AI predictions. The implementation and evaluation of this framework allow assessing the benefits of running Distributed DNN (DDNN) in the Cloud-to-Things continuum. Compared to a Cloud-only deployment, the results obtained show an improvement of 45.34% in the response time. Furthermore, this proposal presents an extension for Kafka-ML that reduces rigidness over the Cloud-to-Things continuum managing and deploying DDNN.

Show More