Combine cream cheese and milk in a medium bowl; beat at medium speed of an electric mixer until smooth. Stir in marmalade and ginger.
Beat whipping cream at high speed of an electric mixer until foamy; gradually add powdered sugar, beating until stiff peaks form. Gently fold whipped cream into cream cheese mixture.