I'm not sure how we're supposed to get that "let's" is implied since "요" form is just the simple present.
Actually in 해요체 there is no conjugation distinction between the present forms for the declarative, interrogative, and propositive moods so the propositive mood that implies "let's" is actually an appropriate mood to use when translating (though albeit it's not the only possible mood here).
it is a form of imperative, but does not include the speaker...
i would translate it as "speak quietly in the library!"
I talked about this sentence with a native speaker, and she said that this is more likely a statement that "(we) speak quietly in the library", or imperative as you said, but in this case, an exclamation mark has to follow.
If this is imperative then why doesn't it end in 하세요 or 합시다? I answered "Speaking quietly in the library" and I feel that it should have been correct.
Shouldn't 말하다 and 얘기하다 be used interchangably? I'm just curious, because I usually use 말하다 when I talk to Koreans. And they didn't say I was wrong... Please correct me.