Mix evaporated milk and sugar together in a large, heavy-bottomed pot over medium-high heat. Bring mixture to a boil, stirring constantly, until syrup reaches 234 degrees F (110 degrees C), 5 to 7 minutes. A teaspoon of hot syrup dropped into a glass of cold water should form a soft ball of candy.
Remove from heat and immediately stir in peanut butter, butter, and vanilla extract. Stir constantly until fudge is smooth and thick.