Put flour, butter, granulated sugar, and salt in the bowl of a stand mixer.
Add the zest of 1 lemon and rosemary.
Mix on low speed, then increase to medium and mix until dough is no longer crumbly and just comes together.
Form dough into a disk and chill 30 minutes. On a lightly floured work surface, roll out dough until 1/2 in. thick.
Cut dough into circles with a 1 1/2-in. cookie cutter. Reroll scraps as needed. Arrange on baking sheets 1 in. apart and chill 15 minutes.
Bake until light golden brown, 12 to 15 minutes.
Mix powdered sugar, lemon juice, and the remaining zest until smooth. Spoon 1/2 tsp. glaze over each cooled cookie, spreading with the back of the spoon; then press 1 rosemary sprig into center of each cookie.
