Pour milk into a bowl; add pudding mix and beat continually until the powder is completely dissolved and the mixture begins to thicken, about 2 minutes. Beat melted peanut butter into the pudding mixture.
Chill pudding mixture in refrigerator until slightly thickened, about 3 minutes.
Spread about 1 heaping tablespoon pudding mix onto each graham cracker square; sandwich squares together to get 16 pudding-filled squares. Arrange onto a baking sheet and cover lightly with plastic wrap.
Chill in freezer until the pudding is firm, at least 1 hour. Store in a resealable plastic bag in the freezer.
