Preheat the oven to 350°F (175°C). Line a cupcake pan with paper liners.
In a large bowl, whisk together the flour, sugar, cocoa powder, baking powder, baking soda, and salt.
Add the milk, vegetable oil, and vanilla extract to the dry ingredients and mix until well combined.
Carefully add the boiling water to the batter and mix until smooth.
Fill the cupcake liners about two-thirds full with the batter.
Bake for 18-20 minutes, or until a toothpick inserted into the center comes out clean. Allow to cool completely.
For the cream filling, beat the butter until light and fluffy. Add the marshmallow creme, powdered sugar, and vanilla extract, and beat until smooth.
Once the cupcakes are cool, use a small knife to cut a hole in the center of each cupcake. Fill with the cream mixture.