Papers

A Look into Programmers’ Heads (TSE)

Abstract: Program comprehension is an important, but hard to measure cognitive process. This makes it difficult to provide suitable programming languages, tools, or coding conventions to support developers in their everyday work. Here, we explore whether functional magnetic resonance imaging (fMRI) is feasible for soundly measuring program comprehension. To this end, we observed 17 participants inside an fMRI scanner while they were comprehending source code. The results show a clear, distinct activation of five brain regions, which are related to working memory, attention, and language processing, which all fit well to our understanding of program comprehension. Furthermore, we found reduced activity in the default mode network, indicating the cognitive effort necessary for program comprehension. We also observed that familiarity with Java as underlying programming language reduced cognitive effort during program comprehension. To gain confidence in the results and the method, we replicated the study with 11 new participants and largely confirmed our findings. Our results encourage us and, hopefully, others to use fMRI to observe programmers and, in the long run, answer questions, such as: How should we train programmers? Can we train someone to become an excellent programmer? How effective are new languages and tools for program comprehension?

Resources

Simultaneous Measurement of Program Comprehension with fMRI and Eye Tracking: A Case Study (ESEM 18)

Background Researchers have recently started using functional magnetic resonance imaging (fMRI) to validate decades-old programcomprehension models. While fMRI helps us to understand neuronal correlates of cognitive processes during program comprehension, its comparatively low temporal resolution (i.e., seconds) cannot capture the fast cognitive subprocesses (i.e., milli seconds).
Aims To increase the explanatory power of fMRI measurement of programmers, we are exploring the feasibility of adding simultaneous eye tracking to the fMRI measurement. By observing programmers with two complementary methods, we aim at obtaining a more holistic understanding of program comprehension.
Method We conducted a controlled fMRI experiment of 22 student participants with simultaneous eye tracking.
Results We could successfully capture fMRI and eye-tracking data, although with some limitations, including spatial imprecision and a negligible drift. The biggest issue that we experienced is the partial loss of data, such that for only 10 participants, we could collect a complete set of high-precision eye-tracking data. Since some participants of fMRI studies show excessive head motion, the proportion of full and high-quality data on fMRI and eye tracking is rather low. Still, the remaining data allowed us to confrm our prior hypothesis of semantic recall during program comprehension, which was not possible with fMRI alone.
Conclusions Simultaneous measurement of program comprehension with fMRI and eye tracking is feasible and promising. By adding simultaneous eye tracking to our fMRI study framework, we can conduct more fne-grained fMRI analyses, which in turn helps us to understand programmer behavior better.

Resources

Shorter Identifier Names Take Longer To Comprehend (SANER 17)

Abstract

Developers spend the majority of their time comprehending code, a process in which identifier names play a key role. Although many identifier naming styles exist, they often lack an empirical basis and it is not quite clear whether short or long identifier names facilitate comprehension. In this paper, we investigate the effect of different identifier naming styles (letters, abbreviations, words) on program comprehension, and whether these effects arise because of their length or their semantics. We conducted an experimental study with 72 professional C# developers, who looked for defects in source-code snippets. We used a within-subjects design, such that each developer saw all three versions of identifier naming styles and we measured the time it took them to find a defect. We found that words lead to, on average, 19% faster comprehension speed compared to letters and abbreviations, but we did not find a significant difference in speed between letters and abbreviations. The results of our study suggest that defects in code are more difficult to detect when code contains only letters and abbreviations. Words as identifier names facilitate program comprehension and can help to save costs and improve software quality.

Resources

Measuring Neural Efficiency of Program Comprehension (ESEC/FSE 17)

Abstract

Most modern software programs cannot be understood in their entirety by a single programmer. Instead, programmers must rely on a set of cognitive processes that aid in seeking, filtering, and shaping relevant information for a given programming task. Several theories have been proposed to explain these processes, such as beacons, for locating relevant code, and plans, for encoding cognitive models. However, these theories are decades old and lack validation with modern cognitive-neuroscience methods. In this paper, we report on a study using functional magnetic resonance imaging (fMRI) with 11 participants who performed program comprehension tasks. We manipulated experimental conditions related to beacons and layout to isolate specific cognitive processes related to bottom-up comprehension and comprehension based on semantic cues. We found evidence of semantic chunking during bottom-up comprehension and lower activation of brain areas during comprehension based on semantic cues, confirming that beacons ease comprehension.

Resources

Empirical Research of Program Comprehension