A.o. Univ. Prof. Dr. Dipl.-Ing. eva Kühn
TU Wien

Thomas Scheller

An Automated Measurement Method for the Usability of APIs and its Application in the Area of Middleware


Dissertation Thesis, TU-Vienna, 2014


Usability is an important quality factor for every kind of software product. Usable software increases productivity and reduces cost and errors, while software that is difficult to use causes worry and frustration, and discourages further use. Because of that, usability tests have become an integral part in the development of software products that provide any kind of user interface. For the special area of application programming interfaces (APIs), this is unfortunately not yet the case. APIs are used by developers to access the functionality of external software libraries, which they want to utilize when writing their own software. A large and rapidly growing number of such APIs is available for a variety of tasks, e.g. for logging, database access or unit testing. So, many developers are using APIs on a daily basis, which makes their usability an important attribute.


While many methods have already been introduced for measuring usability, like Heuristic Evaluation and Thinking Aloud, most of these methods are themselves not easy to use: They require experienced evaluators, and the results also strongly depend on their opinions. Many methods require an already fully functional product, and a usability lab with sufficient equipment is needed for running user tests. This makes tests cost- and time-expensive.


The goal of this thesis is to treat these problems, by creating an automated and objective API usability measurement method. It is based on evaluating the API structure as well as API usage examples. By providing an API developer with fast initial response to the designs of his/her APIs, it can help to avoid costly changes in late development stages. Further, by calculating comparable usability values it allows to objectively compare the usability of different APIs. Because no experienced evaluators are needed, the method can also be used by developers that are inexperienced in this area, or cannot afford costly usability tests. All this makes the method a useful addition to existing usability measurement methods.


In this thesis, we conduct a comprehensive literature review of existing usability evaluation methods as well as automated software metrics, to show that such a method does not yet exist. We then conduct two usability studies to find factors influencing usability that can be measured automatically. We also review existing studies, guidelines and reports about API usability problems to find potential usability factors. From the resulting data, we extract measurable concepts and properties, using statistical evaluation methods. We then construct an automated usability measurement method based on these properties, called the "API Concepts Framework". Finally, we apply the framework to APIs in the area of communication middleware, and present a middleware that is optimized for usability. We show that the results provided by the framework are valid and provide useful information for rating, comparing and improving and API 's usability.



top | XHtml 1.0 strict | last update: Jun 2015