Spray 24 mini muffin cups with cooking spray; set aside.
Beat together butter, sugar, and brown sugar until smooth.
Add peanut butter and mix just until blended.
Add egg and vanilla and mix well.
In a medium bowl, whisk together flour, baking powder, baking soda, and salt.
Gradually add flour mixture to wet mixture, just until blended.
Roll out your cookie dough into balls. {The size of each ball will depend on how big you want your cookie cups. I did about 1-inch ones, but next time I will probably do 2-inch ones.}
Place one ball into each of the 24 muffin cups.
Bake for 12-15 minutes, or until lightly golden.
After baking, use the end of a wooden spoon and press down the center of each cookie to make an indentation for the caramel.
Let cool completely.
Filling
In a medium, microwavable bowl, microwave caramels and water uncovered on high for 1 minute.
Stir and continue to microwave in 15-second increments until caramels are melted and can be stirred smooth.
Stir in milk.
Fill each cookie cup with about 1 tablespoon of melted caramel.