Science is about empirical evidence, not survey data. To call comments on drugs from self-selecting users in uncontrolled conditions "evidence" is like calling creationism "science" or calling homeopathy "medicine."

And my point is that double blind studies ARE the best mechanism to ensure user experience is captured in the least biased way possible.

Jon, self-selecting users contributing feedback in an uncontrolled setting are not providing evidence. "Anecdotal evidence" is not evidence.

Ian- Yes. Precisely.

Got an emailed comment from a reader who wishes to remain anonymous but gave me permission to share her experience.

My vendor rant relates to a salesperson’s response to my request for a quote for a reduced number of simultaneous users. I explained that we had to economize and cut back and his response was “Is there anything we can do, such as offer you XXX, along with your current XXX subscription, for no additional cost? I am trying to get creative and would be open to any suggestions or ideas.”

My (internal) response was, How about giving me the info I requested, not suggesting we pay the same amount and also receive a product we don’t want? His creativity left a lot to be desired. While he promised me the quote that same day, I had to send another email a few days later requesting the quote again which I didn’t receive for a week after my original request.

...

This same vendor once opened up access (unbeknownst to me) to one of their products for our entire campus for a month and then had the nerve to quote me usage stats as an indicator of interest then insisting we should purchase it. Boy that salesman...got an earful from me.

I didn't know you'd decided to make PubGet free to libraries, Ian! That's spectacular! The trick with vendors of information resources is that sometimes there really isn't an appropriate product available from a competitor.

I absolutely agree, Ian. (Just spent a few minutes looking for pricing info on http://pubget.com, but only to give you a hard time.) ;)

Hi Jason-

It isn't my list- the list is made by Library Journal.

Absolutely, though: Nominate her Kraftiness for 2010!

Also annoying: IntenseDebate doesn't seem to realize that davidlrothman on FF is me. :(

Thanks, Rachel! (Rachel commented on this post in FriendFeed to see if IntenseDebate would see it and import it here. Cool, eh?)