Whisk together first 3 ingredients in a large heavy saucepan. Gradually whisk in milk and half-and-half. Cook over medium heat, stirring constantly, 8 to 10 minutes or until mixture thickens slightly.
Remove from heat.
Whisk egg yolk until slightly thickened. Gradually whisk about 1 cup hot cream mixture into yolk.
Add yolk mixture to remaining cream mixture, whisking constantly.
Pour mixture through a fine wire-mesh strainer into a bowl, discarding solids. Cool 1 hour, stirring occasionally.
Place plastic wrap directly on cream mixture; chill 8 to 24 hours.
Pour mixture into freezer container of a 1 1/2-qt. electric ice-cream maker, and freeze according to manufacturer's instructions; stir in Key lime zest, Key lime juice, and crushed graham crackers halfway through freezing.
Let stand at room temperature 5 to 10 minutes before serving.
*Granulated sugar may be substituted.
Note: We tested with Whey Low 100% All Natural Granular Sweetener for Ice Cream. Get 20% off any time you order from wheylow.com. Coupon code: Southrn