Today, resource restrictions are of utmost importance. Energy consumption in particular receives considerable attention. Machine learning is put to good use in order to save energy. Google considers the application of DeepMindâs machine learning to its data centres to be its most important application. The energy used for cooling could be reduced by up to 40 percent through machine learning. Machine learning algorithms themselves are enhanced for low energy demands. One of the invited talks at the International Conference on Machine learning (ICML) 2018 is by Max Welling and has the title Intelligence per Kilowatt-hour. His statement supports our approach to join embedded systems and machine learning research: "The next battleground in AI might well be a race for the most energy efficient combination of hardware and algorithms." CRC 876 is already participating in exactly this race. Ultimately, for the ease of model selection and the understandability and trustworthiness of the models, there could be a kind of wash paper with characteristics of resource demands, quality and privacy guarantees. Then, users may configure their learning systems like we select our household appliances according to their brochures.
The topic of energy efficiency starts with measuring energy consumption. Measuring the true energy consumption directly is difficult because of noise sampling and the need to use minimal resources for the sensing itself. The hardware-integrated sensing instruments are often not precise enough for determining the energy consumed by software running on an embedded system. Hence, an easy-to-use system for direct energy sensing and an energy model for ARM processors based on linear regression have been developed in the platform project A4. The Internet of Things (IoT) and communication networks in general connect small devices. Extremely restricted are the ultra-low-power devices that are used in logistics, e.g., devices attached to a container. Based on reliable measurements, energy harvesting of devices with photovoltaic elements can be realised such that the operating time is enhanced. In a collaboration with Luca Benini, the project succeeded in the even more demanding task of energy measurement for ultra-low power-devices. The analysis of energy consumption of main memory processes in the algorithm project A2 and that of high-performance computing in the particle physics project C5 complete the energy modelling.
Minimising the energy of machine learning processes requires the analysis of the algorithms and the models of learning so that statistical guarantees can be given. Exponential families are a model of learning that covers many learning tasks, e.g., the estimation of probability density as it is used by, e.g., topic models, or the prediction of the maximum likely state as it is used by, e.g., naive Bayes or conditional random fields. A detailed analysis of the resource demands of exponential families has been achieved in the data analysis for embedded systems project A1. Integer Markov random fields are almost as expressible as are real-valued ones, but can be executed on an ultra-low-power device. Work on decision trees and feed-forward networks has already started.
Reducing the memory demand of storage and analysis of data has been investigated by the algorithm project A2 and the particle physics project C5. In particular for genomic data, a task-specific compression could successfully reduce the storage need of a database. On the level of programming languages and operating systems, the resource utilisation project A3 reduces the memory footprint. Moreover, the dynamic sharing of memory has been optimised. Regarding models of learning, the reduction of the memory demand has been investigated for the exponential families. The key is the compression through regularisation and reparametrisation which exploits redundancies in the true parameters. For exponential families, A1 has investigated several ways of decreasing memory demands. This work will be continued in the next funding phase.
Our CRC 876 takes into account the execution of learning programs and learned models not only regarding time complexity, but also regarding real-time. Since many embedded systems are integrated into large products that interact with the physical world, timeliness is an important issue. If the results are delivered too late, they may have become useless. As a result, real-time guarantees are needed for such systems. To efficiently utilise the available resources, e.g., processing power, memory, and accelerators, with respect to response time, energy consumption, and power dissipation, different scheduling algorithms and resource management strategies have been developed in different projects. Our CRC 876 has made significant contributions towards realistic scheduling models and analyses for advanced embedded systems. Some of our results have answered several long-standing open problems. The data analysis project A1 has started to look at deadline misses in real-time systems. The algorithm project A2 explores the impact of different scheduling strategies for processing queries in databases and energy efficiency of different resource management strategies in main-memory databases. The resource utilisation project A3 optimises the parallel execution of R programs with scheduling for heterogeneous runtimes as they are found both in servers and mobile embedded systems. The optimisation framework creates a scheduling plan based on runtime predictions of jobs. The platform project A4 studies scheduling strategies to improve the quality of transmission and the energy efficiency of Long Term Evolution (LTE) devices. The nano-object detection project B2 focuses on real-time scheduling, discussing partitioned scheduling in multiprocessor systems and task partitioning for self-suspending tasks.
The last resource to be presented here is communication. This is particularly important for all processes on mobile devices used in the IoT, logistics as well as traffic forecasting and management. Hence, project platform A4 and traffic B4 were intensively investigating and modeling the communication demands of various scenarios. In his PhD thesis, Christoph Ide has investigated resource-efficient communication in vehicular environments. The view of cars as mobile sensors in upcoming 5G networks makes up for context-aware communication. Communication-efficient machine learning from sensor data has been investigated in the industrial production project B3. The method was general enough to be applied to traffic data, as well.