"Why bad bugs hit good people", Nick Arnott

A reflection on the mortifying iOS 8.0.1 update that Apple had to withdraw after it broke critical functionality, including phone service, on the iPhone 6 and iPhone 6 Plus. Arnott is not writing as an Apple insider, but rather as a quality assurance (QA) lead at another company who knows how hard the job is.

If you've never written software, you might wonder why it's so hard to avoid bugs, or software errors. Bugs typically arise as unexpected side effects of otherwise reasonable-sounding changes to important functionality. The problem is that the engineer doesn't have a good enough mental model of the project. Sometimes that's because she simply has an incorrect understanding of it, but it's far more likely that the project is simply too big for anyone to keep all the details in her head at one time.

How do you find bugs? Well, you have to put the software (and/or hardware) through its paces, which is the job of QA. To do QA properly, you have to try out all the edge cases: you have to do the nutty or extraordinarily dumb things that some ordinary users do. In my experience, QA never has enough time to do edge-case testing well. Even if they got the time, users are amazingly creative and keep coming up with new (strange) ways of (mis)using the product.

Arnott's pretty sympathetic to the Apple engineering and QA staffs, and I'm inclined to agree with him. If you want to know why you should too, read his piece.

(I completely understand if you don't feel like giving Apple the benefit of the doubt. If my car stalled after I drove away from the mechanic, I wouldn't feel too sympathetic toward him. Having written code for a living, though, I know how problems can slip by, especially in a deadline crunch.)