Preheat oven to 350°F (175°C). Line a muffin tin with cupcake liners.
In a large bowl, whisk together flour, sugar, baking powder, and salt.
In another bowl, mix melted butter, eggs, vanilla extract, and milk. Combine wet and dry ingredients until just mixed.
In a small bowl, mix brown sugar, cinnamon, and melted butter for the swirl.
Fill each cupcake liner halfway with batter. Add a spoonful of cinnamon mixture and swirl with a toothpick. Top with more batter.
Bake for 20 minutes or until a toothpick comes out clean. Let cool.
For the glaze, mix powdered sugar, milk, and vanilla extract until smooth. Drizzle over cooled cupcakes.