This is a wonderful statement about personal libraries and why we have books, by Nassim Nicholas Taleb, in his book The Black Swan. He's discussing Umberto Eco's attitude towards the collecting of books. I have definitely dealt with a few of the first type of visitor to my library.
"The writer Umberto Eco belongs to that small class of scholars who are encyclopedic, insightful, and non dull. He is the owner of a large personal lib ray (containing thirty thousand books), and separated visitors into two categories: those who react with 'Wow! Signore professore dottore Eco, what a library you have! How many of these books have you read?' and the others--a very small minority--who get the point that a private library is not an ego-boosting appendage but a research tool. Read books are far less valuable than unread ones. The library should contain as much of what you do not know as your financial means, mortgage rates, and the the currently tight real-estate market allow you to put there. You will accumulate more knowledge and more books as you grow older, and the growing number of unread books on the shelves will look at you menacingly. Indeed, the more you know, the larger the rows of unread books. Let us call this collection of unread books an antilibrary."
-Nassim Nicholas Taleb, The Black Swan