Yes, when I was consulting last year with a well-known New York-based usability consultancy, we used this technique to great effect on e-commerce sites. I think it’s because everybody, more or less, has shopped at some point in their lives and relevant tasks aren’t too difficult to shape on the fly.

The real skill rests with a moderator who can ask the right questions during the interview and use that data to suggest the right tasks. You certainly get more natural behavior because the user is focusing on the things HE/SHE find relevant or important.

Downside is that it can be hard to shape those tasks; the technique may not work for more directed applications such as specialized software; it’s tough to assign statistical rigor; and it takes time to craft the tasks. You also run the risk of the user veering well away from the site or issue the client is interested in. In those cases, the art of “gentle steering without leading” comes into play.

Yell at me when you’re in Dallas next! Best regards to you & Christine.