Arrange celery stalks next to each other in the bottom of a heavy roasting pan.
Place 1 onion quarter, 1 lemon quarter, and 1 sprig rosemary in the cavity of each hen. Rub outside of hens with 1 tablespoon olive oil; sprinkle with salt and lemon pepper. Arrange hens atop celery; sprinkle with tarragon, basil, thyme, and chopped rosemary. Scatter garlic cloves and 1 chopped onion around hens.
Bake in the preheated oven for 25 minutes. Reduce heat to 350 degrees F (175 degrees C).
Whisk wine, broth, and 3 tablespoons olive oil together in a bowl; pour over hens.
Cook hens until no longer pink at the bone and the juices run clear, about 50 more minutes, basting every 15 minutes. An instant-read thermometer inserted near the bone should read 165 degrees F (74 degrees C). Turn off oven and let hens rest in the oven for 10 more minutes.
