It can be, yes. A closet built into the wall is called a 벽장 but no one is that pedantic.
Can this be used for the other usage of wardrobe, meaning your complete set of clothes, or is it just the container that you put clothes in?
It refers to the container. The 欌 (장) in 옷장 is the same as in 책장, bookcase/bookshelf
Definitely lookup wardrobe on wiktionary to get a better idea. It's an archaic/bougey distinction that can be made against closet but most people today would regard them as synonymous. Wardrobes are theoretically clothing cabinets that aren't part of the floor plan like a closet. My grandparents had one and they are nice in a studio apartment if you can afford to buy one. I just had a wheeled hanger bar apparatus with no enclosure when I was in a studio. Synonyms for wardrobe (movable furniture for storing clothes): armoir, dresser; cupboard (UK); closet (regional US), press (Irish & Scots), shrank