Posts

Showing posts with the label performance monitoring

The Linux Scheduler: A Decade of Wasted Cores

Authors: Jean-Pierre Lozi, Baptiste Lepers, Justin Funston, Fabien Gaud, Vivien Quema, Alexandra Fedorova Venue: EuroSys 2016 Before diving into this paper, it's worth mentioning that this paper is presented more like a technical report rather than a research paper. Its clearly very important work, but the authors present software "bugs" within Linux, and fixes for those bugs. The paper is not trying to establish something completely new (other than a set of tools), but rather analysis and fixes. . . . The paper presents findings that the Linux CFS scheduler breaks a fundamental invariant: make sure that ready threads are scheduled if cores are idle. Due to increased complexity within the scheduler to deal with multiprocessors and NUMA domains, the scheduler has issues which prevent this invariant from being met. The CFS scheduler relies on a hierarchy of "scheduling groups" of cores and NUMA domains (scheduling domains). As an aside, it is not clear if th...

PADDLE: Performance Analysis using a Data-Driven Learning Environment

Authors: Jayaraman Thiagarajan, Rushil Anirdh, Bhavya Kaikhura, Nikhil Jain, Tanzima Islam, Abhinav Bhatele, Jae-Seung Yeom, Todd Gamblin Venue: I EEE International Parallel and Distributed Processing Symposium (IPDPS) In the scope of HPC, machine learning is gaining increased traction to add in performance analysis and tuning. However, this approach includes a pipeline of data collection, data pre-processing, various machine learning algorithm testing, tuning, and then finally trying to understand the model. The paper states that while this process is repetitive, rarely can insights be reused from one domain to another. To address this void, the propose PADDLE. PADDLE has three key steps: deep feature extraction, model design, and visualization. The first step allows users to throw extensive amounts of data at the problem, and an automated solutions determines the key inputs, mapping them to a new feature space. The next step in paddle automatically tests a number of machine lea...

Seer: Leveraging Big Data to Navigate the Complexity of Performance Debugging in Cloud Microservices

Authors: Yu Gan, Yanqi Zhang, Kelvin Hu, Dailun Cheng, Yuan He, Meghna Pancholi, Christina Delimitrou Venue: ASPLOS 2018 Seer presents a framework to diagnose and avoid QoS violations in real-time. The motivation, design, and experimental framework in this paper are some of the best and most through I have seen in my recent reading. The work begins by discussing the microservice design of cloud providers. Such frameworks have numerous layers of abstraction, are often written in multiple programming languages, and have complex (and changing) dependency graphs. A performance bug in one microservice can cause QoS in many others, and diagnosing the root cause can be difficult. The work then builds a complex data collection framework which uses RPC-level and perf counters. When perf counters aren't available, the system uses microbenchmarks to diagnose the bottleneck. This area is particularly complex, and the authors even note that their system is similar to Dapper and Zipkin wh...