Place baguette slices on a rimmed baking sheet and toast, 8-10 minutes.
Let cool and break into pieces. Meanwhile, whisk vinegar, lemon and orange zests, ginger, and pepper in a large bowl; season with salt and whisk in oil.
Using a sharp knife, cut all peel and white pith from oranges; discard. Working over bowl with dressing, cut between membranes to release segments into bowl; discard membranes.
Add fennel, fennel fronds, mustard greens, and croutons to bowl; toss to combine.