The talk by 2D Boy developer “Ron Carmel” creator of World of Goo was probably one of my most enjoyable presentations of the Agile conference, however it was probably the least focused on actual “Agile” and more on user experience testing.
The main focus on their discussion was really user experience testing. What that means is observing and reporting on the users interaction with the software, or in this case “World of Goo” video game, and how the user responds to the interaction. This is something that I think happens a lot of time after a product has already been shipped. We may beta test it, or get a few “UI specialists” to run through experience testing, but do we ever really pilot play test during the Software Development process?
The answer is usually no. We do not and probably should. The reasons I believe is that we would be able to find more than just “bugs” but "defects” in the users actual experience. The code may work fine but if the user experience is difficult, confusing, frustrating or anything other than positive we can witness these as they occur and apply those adjustments into the code during the development process and not after shipment.
Now unfortunately there is not always a way to have people off the street come and test your user experience during development, mostly for legal/security reasons. However there is probably many people in the company who could fill that role of an untrained user to help test the experience. Having someone go through and run your iterations worth of code when it is near complete(at the story level) to see if they understand or the deliverables are clear can be very valuable feedback tool.
I am very curious thou to successful user experience testing in Agile development shops that focus on web services and how they achieve great usability, any comments?