Combine brown sugar, white sugar, butter, corn syrup, cream, milk and vanilla in a medium saucepan over medium heat.
Heat, stirring occasionally, to 250 to 265 degrees F (121 to 129 degrees C), or until a small amount of syrup dropped into cold water forms a rigid ball.