Whisk together flour, sugar, and salt in a large bowl; with a pastry cutter, chop in the butter until the mixture resembles coarse crumbs. Stir the rum extract and almond extract into the water in a small bowl.
Mix into the dry ingredients, a little bit at a time, until the mixture holds together in a ball when you squeeze it.
Place the dough onto a floured work surface, and sprinkle with flour; roll out into a sheet 1/4 inch thick.
Cut into shapes with floured cookie cutters, and place 1 inch apart on ungreased baking sheets. Cover with a layer of plastic wrap, and refrigerate for 30 minutes.
Preheat oven to 325 degrees F (165 degrees C).
Bake the cookies just until the edges are lightly browned, 15 to 18 minutes. Allow to cool on sheets for 2 minutes before removing to wire racks to finish cooling. Cool completely before decorating.
For frosting, whisk the confectioners' sugar and milk in a bowl until smooth. If desired, divide frosting into small bowls, and tint each bowl a desired shade with food coloring. Frost fully cooled cookies, and sprinkle with edible glitter before the frosting sets.
