Remove zest from oranges with a vegetable peeler and put zest on a baking sheet (save fruit for another use).
Bake until zest dries, starts to curl up, and begins to brown, 20 to 25 minutes.
Let cool.
Put all ingredients except sugar in a 1-qt. glass jar and seal tightly.
Let sit in a cool, dark place for 2 weeks.
Strain liquid through cheesecloth into a 1 1/2- to 2-qt. glass jar (save flavorings) and set aside. Put flavorings in a small saucepan with 2 cups water. Cover and bring to a boil, then reduce heat and simmer 10 minutes. Set aside.
Cook sugar in a small saucepan over medium-high heat, lifting and shaking pan occasionally, until sugar caramelizes and turns deep amber, 8 to 10 minutes.
Pour caramel into liquid in pan (it will bubble furiously and may firm up). If needed, return pan to medium heat and cook, stirring, for a few minutes until caramel melts again.
Set pan in a bowl of ice water until liquid is cold, about 10 minutes.
Pour into jar with first infusion. Seal and let stand in a dark place 5 days.
Strain final mixture through cheesecloth into a glass measuring cup and discard flavorings. Divide bitters into small jars and seal tightly.
