Very common to say 'peace and quiet' as thought it were a single thing in itself and not two separate things. To be "at peace" suggests being dead as in RIP (Rest In Peace).
"I just want to be at peace" sounds like you are wishing for inner peace in general and whatever is going on in the outside world has no effect on you. The Finnish sentence is more about wanting to left alone to experience the quiet and solitude of the surrounding world to find that peace. A fine distinction I know, but an important one. :)