Preheat an oven to 180c (350f) and prepare a baking sheet with parchment paper.
Into a large mixing bowl, add the room temperature butter, vanilla extract, kosher salt, and sugar. Mix with an electric hand mixer until well combined and creamy
Add the egg yolk and mix until the mixture is smooth.
Add the flour, and mix the dough until it is just combined and there are no lumps of flour visible. The dough might look crumbly, but it will come together beautifully.
Using an ice cream scooper, scoop out about 1 tablespoon of the cookie dough and roll them into balls. Using your thumb or a teaspoon, make indents on each cookie.
Bake the cookies in a 180c (350f) preheated oven for 10 minutes. Don't overbake these cookies so they will stay soft!
When you take the cookies out of the oven, immediately reshape the cookies using a spoon or your thumb.
Let the cookies cool to room temperature and make the icing: in a small mixing bowl, add the powdered sugar, milk, and vanilla extract. Mix with a spoon or a mini spatula until it's silky smooth.
Transfer the icing to a piping bag and cut off the tip. Fill each cookie with the icing, and top them with powdered sugar.