Place potatoes in a single layer in center of a large piece of heavy-duty aluminum foil.
Drizzle with olive oil; sprinkle with salt and pepper. Bring up foil sides over potatoes; double fold top and side edges to seal, making 1 large packet.
Grill potatoes, in foil packet, covered with grill lid, 15 minutes on each side.
Remove packet from grill. Carefully open packet, using tongs, and let potatoes cool 5 minutes.
Whisk together mayonnaise and next 4 ingredients in a large bowl; add potatoes, tossing gently to coat. Stir in onion, blue cheese, and bacon.
*Balsamic vinegar may be substituted but will darken the color of the dressing.