Revisiting Exception Handling Practices with Exception Flow Analysis

Abstract

Modern programming languages, such as Java and C#, typically provide features that handle exceptions. These features separate error-handling code from regular source code and aim to assist in the practice of software comprehension and maintenance. Having acknowledged the advantages of exception handling features, their misuse can still cause reliability degradation or even catastrophic software failures. Prior studies on exception handling aim to understand the practices of exception handling in its different components, such as the origin of the exceptions and the handling code of the exceptions. Yet, the observed findings were scattered and diverse. In this paper, to complement prior research findings on exception handling, we study its features by enriching the knowledge of handling code with a flow analysis of exceptions. Our case study is conducted with over 10K exception handling blocks, and over 77K related exception flows from 16 open-source Java and C# (.NET) libraries and applications. Our case study results show that each try block has up to 12 possible potentially recoverable yet propagated exceptions. More importantly, 22% of the distinct possible exceptions can be traced back to multiple methods (average of 1.39 and max of 34). Such results highlight the additional challenge of composing quality exception handling code. To make it worse, we confirm that there is a lack of documentation of the possible exceptions and their sources. However, such critical information can be identified by exception flow analysis on well- documented API calls (e.g., JRE and .NET documentation). Finally, we observe different strategies in exception handling code between Java and C#. Our findings highlight the opportunities of leveraging automated software analysis to assist in exception handling practices and signify the need of more further in-depth studies on exception handling practice.

Read more about our paper: Paper

If you use our data or code, please cite our paper. BibTex

Evaluated Subject Projects

The subject projects are 16 open-source softwares. To reproduce our study, the releases we analysed can be found in the links below.

Exception Flow Analyzer

Our analyzer parse try-catch blocks and traverse call graphs of the subject projects. We use .NET Compiler Platform (“Roslyn”) for C# and Eclipse Java development tools (JDT) for Java.

The source code and binary files of the two source-code analysis tools we developed are available below.

Java Binary and Source
C# Binary and Source

Data Output and Visualizations

The Summary data and Full data can be visible (and also downloaded as CSV file) using the following Tableau Public Visualizations. To Download, click the bottom right button (“Download”) in the Tableau frame. Choose Data option then you can choose Summary or Full Data in the pop-up window that will come up.

The full data sources can also be downloaded here: Exported Data sources

Exception flows data

Statistical tests

We performed Wilcoxon Rank Sum test to examine if there exists statistically significant difference (i.e., p-value < 0.05) on the exception handling actions between Java and C#, between specific and overcatch strategies, and between the most common exception types and the general behaviour. A p-value < 0.05 means that the difference is likely not by chance.

Wilcoxon Rank Sum results file

More on .NET and Java Exception Handling

To understand more about Exception Handling, check .Net and Java documentation.

Chapter 11. Exceptions - Java SE Specification

Handling and Throwing Exceptions - .NET Framework Documentation

Exception Hierarchy - .NET Framework Documentation