Aalto  

Faster Queues for Big Data (FQ4BD)

Site of research: Aalto University School of Electrical Engineering
Department of Communications and Networking
Duration: 24 months (1.1.2016-31.12.2017)
Funded by: Academy of Finland

In recently emerging computing paradigms, three distinct features can be identified: (i) more detailed in- formation is available per job (e.g., server-specific service times), (ii) this information must be utilized in real-time, and (iii) the systems must scale (cf. massive data centers). Also heterogeneity in types of jobs and servers is increasing. This project focuses on the performance analysis and optimization of the parallel computing schemes such as MapReduce. Vast amounts of digital information are frequently processed, the so-called big data, and our aim is to develop theoretical understanding on how such stochastic systems can be analyzed and optimized. The new computing paradigms, where even the notion of a job is far more complex than it used to be, require the development of new stochastic tools that faciliate the analysis. In particular, we do not study, e.g., any particular data mining problem or algorithm, but seek a better understanding of processing concurrent (sets of) jobs efficiently in order to meet often strict deadlines, thereby advancing the field as a whole.

Keywords

Large-scale Server Systems, Data centers, Cloud computing, MapReduce, Big Data
historical development.
Fig. 1: Evolution of stochastic models and their ICT applications.

Research visits

People

Publications:

  1. E. Hyytiä and R. Righter, Evaluating Rare Events in Mission Critical Dispatching Systems, in 30th International Teletraffic Congress (ITC'30), 2018, Vienna, Austria.
  2. S. G. Samúelsson and E. Hyytiä, Applying Reinforcement Learning to Basic Routing Problem, in 13th International Conference on Queueing Theory and Network Applications (QTNA2018), 2018, Tsukuba, Japan.
  3. E. Hyytiä, D. Down, P. Lassila and S. Aalto, Dynamic Control of Running Servers, in 19th International GI/ITG Conference on ``Measurement, Modelling and Evaluation of Computing Systems'' (MMB), vol. LNCS 10740, pp. 127-141, Springer Verlag, 2018.
  4. K. Gardner, M. Harchol-Balter, E. Hyytiä and R. Righter, Scheduling for Efficiency and Fairness in Systems with Redundancy, Performance Evaluation, 2017.
  5. E. Hyytiä, R. Righter, J. Virtamo and L. Viitasaari, Value (generating) functions for the MX/G/1 queue, in 29th International Teletraffic Congress (ITC'29), 2017, Genoa, Italy.
  6. X. Wu, P. Loiseau and E. Hyytiä, Towards Designing Cost-Optimal Policies to Utilize IaaS Clouds under Online Learning, in The International Conference on Cloud and Autonomic Computing (ICCAC 2017), 2017.
  7. E. Hyytiä, R. Righter, S. G. Samúelsson, Beyond the shortest queue routing with heterogeneous servers and general cost function, ValueTools'17 (December 2017, Venice, Italy).
  8. J. Doncel, S. Aalto and U. Ayesta, Economies of scale in parallel-server systems, IEEE International Conference on Computer Communications (IEEE Infocom), Atlanta, GA, USA, May 2017.
  9. E. Hyytiä, R. Righter, O. Bilenne and X. Wu, Dispatching fixed-sized jobs with multiple deadlines to parallel heterogeneous servers, Performance Evaluation, 2017.
    The extended abstract of this paper appears in ValueTools'16 proceedings (October 2016, Taormina, Italy).
  10. K. Gardner, S. Zbarsky, S. Doroudi, M. Harchol-Balter, E. Hyytiä and A. Scheller-Wolf, Queueing with Redundant Requests: Exact Analysis, Queueing Systems, vol. 83, pp. 227-259, 2016. DOI
  11. E. Hyytiä and R. Righter, Routing Jobs with Deadlines to Heterogeneous Parallel Servers, Operations Research Letters, vol. 44, no. 4, pp. 507-513, 2016. DOI
  12. E. Hyytiä and S. Aalto, On Round-Robin Routing Policy with FCFS and LCFS Scheduling, Performance Evaluation, vol. 97, pp. 83-103, 2016. DOI
  13. E. Hyytiä, R. Righter and J. Virtamo, Meeting Soft Deadlines in Single- and Multi-Server Systems, in 28th International Teletraffic Congress (ITC'28), 2016, Würzburg, Germany. DOI

Related projects: