Mine the Scrap is a data driven process that designs new structures algorithmically generated from existing scrap. Using computer vision and construction automation, we address the pressing need to convert waste into resource. The installation allows one to experience an algorithm at work. The project transforms irregular, non-uniform stocks of construction scrap into new forms, finding beauty and intricacy in neglected waste. Instead of pulverizing and chemically aggregating these disparate scraps in energy-intensive processes, Mine the Scrap finds the unique best use of each piece in a new structure through a process of scanning and classification. Combining the logic of the quilt with customized shape and pattern detection, Mine the Scrap scans the material and generates an inventory of different forms. The software encapsulates this process, allowing the user to proceed from scrap scanning through design and construction.