Deep Learning Methods of Cross-Modal Tasks for Conceptual Design of Product Shapes

Abstract

Conceptual design is the foundational phase in design that transforms vague design problems into low-fidelity concepts and prototypes by exploring, creating, and integrating ideas. Product shape design is critical at this stage, yet two main challenges arise when applying deep learning (DL) methods: (1) design data exists in multiple modalities, and (2) creativity demands are increasing. Recent advances in cross-modal deep learning (DLCMT), which enables transformation between design modalities, offer new opportunities for AI to support product shape design. This paper systematically reviews 50 studies (from an initial pool of 1341) on retrieval, generation, and manipulation methods in three DLCMT categories—text-to-3D shape, text-to-sketch, and sketch-to-3D shape—drawn from fields like computer graphics, computer vision, and engineering design. The review examines state-of-the-art DLCMT methods relevant to product shape design and identifies key challenges, such as the need to consider engineering performance early in the design phase. Finally, potential solutions and research questions are proposed to guide future data-driven conceptual design research.