Brush 3 large baking sheets generously with some oil.
With a mandoline or other manual slicer cut the potatoes crosswise into 1/8-inch thick slices. Arrange slices in 1 layer on a baking sheet and brush with some remaining oil.
Sprinkle potatoes with rosemary, salt and pepper. Repeat these steps for remaining 2 baking sheets.
Bake slices in middle of oven until golden, 15 to 20 minutes. With a metal spatula immediately transfer to a rack to cool completely.
Top potato chips with 1 teaspoon each of caviar and creme fraiche.
