Where User Experience and Software Engineering Meet

Andrew Ko

In method, I'm an HCI (human-computer interaction) researcher, bit in practice, I'm an SE (software engineering) researcher. In HCI, how do we get the right design, in SE, how do we get the design right. Question is how do we merge these.

Why is software evolution difficult?

From study at Microsoft, observed 25 hours of coding and bug fixing, took 357 pages of (handwritten) notes, logged 4231 events in a spreadsheet. each some piece of information a developer or manager looking for at the one time.

Looked also at reasons why people switched tasks, such as face-to-face meeting, email, all sorts of reasons. Found that work is fragmented. People are interrupted every 5-10 minutes. Only that amount of time to focus. Also, blocked about every 10 minutes. They need info that they can't get themselves.

22 information needs. 5 least often satisfied:

• what code caused behavior (36%)
• what code caused program state (615)
• what is program suppose to do (15%)
• in what situations does failure occur (41%)
• why was code implemented this way (44%)

One reason why it's hard to fix bugs is because answering some of these questions requires getting people together for meetings to make decisions, meetings where stakeholders must be involved.

Software development is tacit. Plans and specifications are unwritten, developers have to communicate (a lot) to make progress. It takes time to coordinate communications. Can't make people coordinate better just with tools. Software quality depends a lot on quality of communication. Communication and cognition are inherently faulty and unreliable .

How can users help software evolution

Looked at bug reports (more than 500K) in the Mozilla community. Did quantitative analysis the characterize report trends, and qualitative analysis to explain resolution trends. About 45% or reports submitted by "reporters" the type of user that submits the majority of community bug reports, are duplicates. Most "reporter" reports are not fixed. Many also are "as designed" or not reproducible.

The percentage of bugs entered by "reporters" has gone down over time. Most "reporter" reports were tech support for power users' tinkering. They also rarely gave enough useful information to reproduce problems. Also entered problems that had already been solved.

Open bug reporting is useful, but lots of overhead to process bad reports, only a skewed subset of users report bugs, users who report bugs are bad at providing useful information, and text isn't precise enough to express useful information.

The challenge is to enable users to submit bugs that are precise, structured, aggregatable, with not training.