Institute of Computer Languages
Compilers and Languages Group

Talks 2011 - Matthias Hauswirth

Der Arbeitsbereich für Programmiersprachen und Übersetzer am Institut für Computersprachen lädt zu folgender 3-teiliger Vortragsreihe ein:

Ass.Prof. Dr. Matthias Hauswirth

University of Lugano, Switzerland


Pitfalls in System Performance Evaluation

Datum: Mittwoch, 25. Mai 2011
Zeit: 9:00 - 11:00
Ort: TU Wien, Bibliothek E185.1, Argentinierstraße 8, 4. Stock (Mitte)

Datum: Donnerstag, 26. Mai 2011
Zeit: 8:00 - 10:00
Ort: TU Wien, Seminarraum Argentinierstr., Argentinierstr. 8, EG
(Eingang Paniglgasse)

Datum: Freitag, 27. Mai 2011
Zeit: 9:00 - 11:00
Ort: TU Wien, Seminarraum Argentinierstr., Argentinierstr. 8, EG
(Eingang Paniglgasse)


In the last few years, researchers have identified disturbing flaws in the way that experiments are performed in computer science. For example, in the area of performance evaluation of computer systems, our measurements on one system are rarely reproducible on another. As hardware and software grow more complex, this problem just gets worse.

Bad evaluations misdirect research and curtail creativity. A poorly performed but successfully published evaluation can encourage fruitless investigation of a flawed idea, while publication of flawed observation can discourage further exploration of an important area of research.

In this mini-lecture we distill not only our own experience in performance evaluation but also the insight gained through our involvement in the "Evaluate Collaboratory" ( ), an open group of researchers with an interest in improving the state of practice in experimental evaluation of software and systems.

This mini-lecture covers topics including the selection of benchmarks, performance metrics, measurement bias and accuracy, statistics, reproducibility, negative results, and observational studies. The lecture includes discussions of recent research results on these topics. Besides the above topics, the lecture can include discussions of topics and issues proposed by the participants. The goal of this mini-lecture is to enable the participants to learn from past mistakes of others and to avoid common pitfalls in their own future performance evaluations.

This lecture series is particularly devoted to PhD students and advanced master students.

Kurzbiographie von Matthias Hauswirth

Matthias Hauswirth is an assistant professor at the Faculty of Informatics at the University of Lugano (Switzerland), where he leads the Software and Programmer Efficiency (Sape) research group. He is interested in performance measurement, understanding, and optimization. Matthias received his PhD from the University of Colorado at Boulder.
( /

The Sape Research Group at the University of Lugano is recruiting PhD students for the ParaBoost project (" for research on concurrent speculative execution in virtual machines.

Sie möchten auf diese Vortragsreihe durch Aushang hinweisen? Eine druckfertige Einladung im pdf-Format dafür finden Sie hier.

   About Us
      Talks 2017
      Talks 2016
      Talks 2015
      Talks 2014
      Talks 2013
      Talks 2012
      Talks 2011
      Talks 2010
      Talks 2009
      Talks 2008
      Talks 2007
      Talks 2006
      Talks 2005
      Talks 2004
Fast Access:
Previous Talk
Next Talk
Faculty of Informatics
Vienna University of Technology
top | HTML 4.01 | Datenschutzerklärung | last update: 2018-05-25 (Webmaster)