Add potatoes and cook until tender but still firm, about 15 minutes.
Drain and set in the refrigerator to cool.
Place eggs in a saucepan and cover with cold water. Bring water to a boil and immediately remove from heat. Cover and let eggs stand in hot water for 10 to 12 minutes.
Remove from hot water, cool, peel and chop.
Place bacon in a large, deep skillet. Cook over medium high heat until evenly brown.
Drain, crumble and set aside.
Chop the cooled potatoes, leaving skin on.
Add to a large bowl, along with the eggs, bacon, onion and celery.
Add mayonnaise, salt and pepper to taste. Chill for an hour before serving.