A drift "punch" is misleadingly named; it is not used as a punch in the traditional sense of the term. A drift punch, or drift pin, is used as an aid in aligning bolt or rivet holes prior to inserting a fastener or in blacksmithing to enlarge a hole.
A drift punch is constructed as a tapered rod, with the hammer acting on the large end of the taper. The tapered end of a drift punch is placed into the hole and then driven into the hole. As it is driven in, the taper forces the two components into alignment or force the material to displace (if the metal is hot or soft), allowing for easy insertion of the fastener. Unlike most punches, force is never (and should never be) applied to the tip, or end of a drift pin.