Heat the oven to 350°F and arrange the rack in the middle. Line muffin pans with cupcake liners. Sift together dry ingredients in a bowl and set aside.
Whisk together pumpkin and milk in a separate bowl and set aside. In the bowl of a stand mixer fitted with a paddle attachment, beat butter and sugar on high speed until light and fluffy, about 5 minutes.
Add eggs one at a time, beating and then scraping the bowl down after each addition.
Add vanilla and beat until smooth.
Add dry ingredients in three batches, alternating with pumpkin-milk mixture, mixing on low speed, and scraping the bowl down between additions. When all ingredients have been added, mix batter 30 seconds on medium-high speed until uniformly combined.Fill the cupcake liners 3/4 full with batter and bake until a tester inserted in cupcake centers comes out clean, about 18 to 20 minutes.
Remove cupcakes from the oven and allow to cool 5 minutes before removing from the pans.
Let cool completely on a rack, decorate with icing, and serve.