Filling your bookshelves

In my dream home there is a room dedicated to bookshelves. It doesn't have to necessarily be a library only -- maybe a library/music room. Or a library/living room. Or a library/dining room. Or a library/office (I'm big on multi-tasking rooms. My dream home is also small enough to clean easily). The walls in this dream room are top-to-bottom book shelves. There's probably at least one window, and depending on it's partner function, there's probably a desk, piano, or sideboard taking up a small part of one wall. In this room, it's (relatively) easy to put out one's books. You can organize it however you want -- or not -- and in my mind the choices are easier. Because you know what's hard? Trying to organize your books when you have half of them in one room, half in another, and a small percentage in the random built-in next to the fireplace. (so...I guess that's more like 48%, 48%, 4%).

What complicates things, I'm ashamed to admit, is also a bit of "what-will-people-think-itis." In other words: what books do I want front and center, because I'm cool if people know I read that? What do these books say about me? What do I want them to say?
Which is ridiculous.
So, I'm going pretty general. Nice looking hardback series and cookbooks on the built-in (which is next to the kitchen too). Fiction in the front room, non-fiction in the office/guest room. There may be exceptions to the fiction/non-fiction rule, but that's how we're organizing at the moment. Although I'd love to have some of hubby's cool history books in the front room...there's just not space for everything! And I've got to have at least some kind of system, as loose as this one is.
How do you decide where books go?