In a large pot, cover eggs with cold water by 2 inches and bring to a boil over high. Boil 1 minute, remove from heat, cover, and let sit 10 minutes. Transfer eggs to a colander and run cold water over them until cool enough to handle. Peel, then slice each egg in half lengthwise. Remove yolks with a spoon and place in a small bowl.
Mash yolks with a fork. Add mayonnaise, celery, Old Bay, and mustard and mash into a smooth paste. Gradually add milk, stirring, until yolk mixture has a creamy consistency. Season mixture with pepper. With a spoon, fill each egg white half with about 1 tablespoon yolk mixture. Top each with a celery leaf before serving.