Luke is their child, but he's an adult - if he was an actual child, Vader would have much less issue taking from Padmé's closet.

It also depended on what was in there.

"Her's are not the only clothing here. There is likely trunks of fabric as well." Look he knew how she packed.