Mix Base:
In a medium saucepan, whisk together milk, sugar, and cocoa powder. Heat over medium heat, stirring frequently, until sugar dissolves and mixture is warm (do not boil).
Add Cream and Vanilla:
Remove from heat. Stir in heavy cream and vanilla extract until well combined. Let cool slightly.
Chill Mixture:
Cover and refrigerate the mixture for at least 2 hours, or until completely chilled.
Churn Ice Cream:
Pour the chilled mixture into an ice cream maker and churn according to the manufacturer’s instructions, usually 20–25 minutes, until soft-serve consistency is reached.
Add Mix-Ins:
During the last 5 minutes of churning, fold in marshmallows, nuts, and chocolate chunks.
Freeze to Set:
Transfer ice cream to a freezer-safe container and freeze for at least 4 hours, or until firm.
Serve:
Scoop and serve with extra chocolate or marshmallows if desired.