Over the past years, a growing trend of utilizing camera-equipped drones for periodical building facade inspection has emerged. Building façade anomalies, such as cracks and erosion, can be detected through analyzing drone-captured video, photographs, and infrared images. Such anomalies are known to have an impact on various building performance aspects, e.g., thermal, energy, moisture control issues. Current research efforts mainly focus on computational image processing methods to recognize certain types of facade anomalies. However, there is a lack of research on mapping detected anomalies into the building model and managing lifecycle inspection information. This paper aims to propose a systematic process for detecting and managing building façade anomalies based on drone-collected images. An overall data structure, data flow, and related processing techniques within this systematic process are defined. The proposed systematic process will support the facade anomaly detection and building maintenance decision-making.