Preheat oven to 350°F (175°C). Line baking sheets with parchment paper.
In a bowl, whisk together flour, cocoa powder, baking soda, baking powder, and salt.
In another bowl, beat butter and sugar until fluffy. Add egg and vanilla, mix well.
Gradually add dry ingredients to wet ingredients, alternating with milk. Mix until smooth.
Drop batter by spoonfuls onto prepared baking sheets. Bake for 10 minutes. Cool completely.
For filling, beat butter until creamy. Add powdered sugar and vanilla, mix well. Fold in marshmallow creme.
Spread filling on flat side of one cookie, top with another cookie. Repeat with remaining cookies.