Mix brown and white sugar with the milk in large pot; bring mixture to a boil. Stir in peanut butter, reduce heat to medium and bring the mixture back to boil (stirring constantly).
Remove the pot from heat when a drop of the mixture forms a ball in a glass of cold water.
Stir margarine and vanilla into the mixture; stir vigorously until the fudge hardens. ( Always stir in same direction.)