Resource-aware Machine Learning - 4th International Summer School 2017

TU Dortmund, Germany, 25.09. - 28.09.2017

Main Navigation

Lectures Summer School 2017

Welcome-breakfast and introduction to resource-aware machine learning

Schedule: Monday, 25.09., 09.00-10.30

Lecturer: Katharina Morik

Start the summer school on resource-aware machine learning together with your fellow participants and our welcome-breakfast.

During the breakfast, listen to the introduction about hot topics on resource-awareness: Where are the limitations, being it due to huge amounts of data, high-dimensionality or restrictions imposed by small devices? What about the four Vs of data analysis: Velocity, Variety, Volume and Value.

Field-programmable technology and machine learning

Schedule: Monday, 25.09, 11.00-12.30

Lecturer: Wayne Luk

This lecture covers recent research on field-programmable technology and machine learning. It describes how field-programmable technology can accelerate machine learning applications, and how machine learning techniques can enhance the performance of systems based on field-programmable technology. The proposed approaches will be illustrated by several examples including hyperspectral image classification, and Sequential Monte Carlo designs for air traffic management. The potential of the proposed approaches will also be presented.

Deep Learning

Schedule: Monday, 25.09, 16.00-17.30

Lecturer: Sebastian Buschjäger

With the rise of Big Data and cheaply available computation power, Neural Networks have exceeded state-of-the art results in many practical areas such as speech recognition or image classification. Even though Neural Networks have been around for over 70 years, this success is quite recent and somewhat sudden. Reasons for this success lie not only in more data and in more computation power available, but also in a more engineering-style approach towards Neural Networks in general. In this lecture, we want to discuss this recent rise of Neural Networks as part of the Deep Learning approach. The first half of the talk will cover the basics about (Convolutional) Neural Networks, such as Backpropagation, the effect of different activation functions and general model architecture. The second half of the talk will focus on recent results in stochastic gradient descent optimization and their impact in Neural Network training.

Deep Learning on FPGAs

Schedule: Monday, 25.09, 18.00-19.00

Lecturer: Sebastian Buschjäger

Deep Learning is one of today's state-of-the art tool for image classification and one of the driving force for autonomous driving (no pun intended). For image classification we usually see deep models with up to 128 layers and more than billions of parameters. Today's approaches for training and deploying these large models are mostly based on energy hungry GPUs and thus are not suitable for embedded devices found in e.g. cars. In this talk we will explore the possibilities of learning and using large Neural Networks on small, embedded devices. A specific focus will be laid on Field-Programmable Gate Arrays (FPGAs) which offer a fast, but less energy demanding computation architecture for Deep Learning. I will cover the basics of FPGAs and discuss some specialized Deep Learning approaches designed for FPGAs. Additionally, I will lay out some recent results in model pre-processing which help to further reduce computational need during model execution.

Generalised LDPC architectures for high-throughput FPGA realisation

Schedule: Monday, 25.09., 14.00-15.30

Lecturer: Rob Maunder

In this lecture, I will provide an tutorial on LDPC codes and their FPGA implementation. I will present a survey of 140 published designs and use this to draw conclusions about the fundamental trade-offs in the FPGA implementation of LDPC decoders. This survey will highlight the need for new designs having much greater flexibility than ever before. Finally, I will present a generalised LDPC architecture, which can flexibly support a wide variety of LDPC code designs with a high throughput.

A Tale of a Quest for Business Intelligence from Social Data

Schedule: Tuesday, 26.09., 09.00-10.30

Lecturer: Rakesh Agrawal

I present the story of a research expedition (code-named WaveFour) into building an enterprise-scale, real-time business intelligence system over social data. I discuss what drove us to undertake this journey and the system prototype we built. I also describe the investigation we carried out to assess the overlap between Google and Bing search results and whether including social data in the mix can produce different and useful results. I conclude with lessons learned and future directions.

Probabilistic Graphical Models

Schedule: Tuesday, 26.09., 11.00-15.30

Lecturer: Nico Piatkowski

Small, wearable, and resource-constrained devices are ubiquitous---they accompany and support us in many personal and professional settings. Uncertainty is inherent in almost all real-world situations since many important aspects of a system may only be partially observed or cannot be observed at all. Probabilistic graphical models allow us to analyze such uncertainties via stochastic dependencies between specific parts of a system, they handle rare and unexpected cases, and may be estimated from incomplete data.

In this lecture, we focus on probabilistic models on ultra-low-power microcontroller units. We will see why a certain class of probability distributions, namely exponential family models, is especially well-suited for resource-constrained systems. We derive ab initio a sub-class of exponential family models, which is restricted to integer parameters and allows for probabilistic inference with integer-only arithmetic. Theoretical and empirical results show, that this technique strongly reduces the memory and computation time requirements for weak computational systems.

On the one hand, restricting the parameter space to the natural numbers, phrased as a special kind of regularization, excludes an uncountable infinite set of models from our consideration. On the other hand, overflows in the underlying data types force us to propose a new approximate inference technique, which is based on the bit-lengths of intermediate results, and is related to the sum-product algorithm. By analyzing these different types of error, we draw conclusions about which kinds of probability mass functions admit a lossless representation by this sub-class of exponential family models.

Finally, we investigate how the integer-restriction increases the size of models that can be applied on real-world ultra-low-power systems.

Graph Streaming

Schedule: Tuesday, 26.09., 16.00-17.30

Lecturer: Chris Schwiegelshohn

Consider a sequence of edge insertions (and possibly deletions) to a graph. In the streaming model we try to (approximately) solve some problem on the graph while storing as little space as possible. In this talk, we will give an overview on the basic aims, challenges, and limitations of the graph streaming model. We will further present two crown jewels of the field:

1. A 2+ε approximation algorithm for weighted matching in insertion-only streams (Paz and Schwartzman, SODA 2017)

2. Connectivity testing in dynamic streams (Ahn, Guha, and McGregor, SODA 2012)

Presentation of student grant awardees

Schedule: Tuesday, 26.09., 18.00-19.00

Presenters will be announced after awardees for the student grants are selected.

Programming Energy-Harvesting-Based Sensor Nodes

Schedule: Wednesday, 27.09., 09.00-10.30

Lecturer: Olaf Spinczyk

Energy-harvesting based computer systems will enable various interesting use cases, especially in the context of the Industry 4.0 and Internet of Things visions. They turn almost arbitrary objects into smart networked sensors with minimal maintenance effort and at moderate costs.

However, programming these ultra-low-power systems is a challenge. Besides harsh resource constraints in terms of RAM, FLASH, communication capabilities, and CPU power, developers have to deal with a varying amount of available energy. In fact energy management dominates the whole hardware/software design process.

Taking the PhyNode platform and its operating system KratOS as a running example, the lecture will address energy models and their creation, energy management, energy-aware operating systems, and energy-aware communication.

In doing so, the lecture will also prepare the participants for the practical exercises in the following sessions.

Ultra low power learning

Schedule: Wednesday, 27.09., 11.00-19.00 and Thursday, 28.09. 09.00-10.30

Lecturer: Motjaba Masoudinejad

A world full of sensors leads to two main developments: Gigantic centralized services, gathering and analyzing data, and highly distributed data generation. The latter demands respecting a multitude of restricted resources: Computational power, limited communication speed and reliablity, often limited energy. The PhyNodes are embedded computing and sensing platforms developed at the collaborative research center for as a large scale testbed for the Internet of Things.

Tractable Probabilistic Graphical Models

Schedule: Thursday, 28.09., 11.00-12.30

Lecturer: Kristian Kersting

Probabilistic models have had broad impact in machine learning, both in research and industry. Unfortunately, inference in unrestricted probabilistic models is often intractable. Motivated by the importance of efficient inference for large-scale applications, the research focus has shifted to very large probabilistic models that provide enough symmetries so that inference is feasible or even to model that are tractable despite having high treewidth. Example are relational models with lifted inference and sum-product networks. In this tutorial I shall introduce and review some of these developments with an emphasis on distributions beyond multinomials and Gaussian.

Mobility Models

Schedule: Thursday, 28.09., 14.00-15.30

Presenter: Thomas Liebig

In street-based transportation science, vehicle volume estimation receives lots of attention, as it provides important insights to several applications, such as situation-aware trip planning, attraction ranking and traffic control systems. In practice, empirical measurements are sparse due to budget limitations and constrained mounting options. Therefore, estimation of vehicle quantity is required to perform mobility analysis at unobserved locations. Accurate vehicle mobility analysis is difficult to achieve due to non-random path selection of the individual persons (resulting from motivated movement behaviour). This causes the vehicle volumes to distribute non-uniformly among the traffic network. Existing approaches (vehicle simulations and data mining methods) are hard to adjust to sensor measurements or require more expensive input data (e.g. high fidelity street networks plans or total number of vehicles in a city) and are, thus, unfeasible. In order to achieve a mobility model that encodes vehicle volumes accurately, we discuss two methods which overcome the limitations of existing methods. These two methods incorporate topological information and episodic sensor readings, as well as prior knowledge on movement preferences and movement patterns.