Preheat your oven to 200°C (400°F). Roll out the puff pastry sheets and cut them into equal rectangles. Place them on a baking sheet lined with parchment paper.
Prick the pastry with a fork to prevent it from puffing up too much. Bake for 15-20 minutes until golden brown. Let them cool completely.
In a saucepan, heat the milk until it starts to simmer. In a bowl, whisk together the sugar, egg yolks, and cornstarch until smooth.
Slowly pour the hot milk into the egg mixture, whisking constantly. Return the mixture to the saucepan and cook over medium heat, stirring constantly, until it thickens.
Remove from heat and stir in the vanilla extract and butter. Let the pastry cream cool completely.
Assemble the mille-feuille by layering the puff pastry and pastry cream. Start with a layer of pastry, then cream, and repeat. Finish with a layer of pastry on top.
Dust the top with powdered sugar or drizzle with icing. Chill in the fridge for at least an hour before serving.