Semantic Scale Space: A Framework for Controllable Image Abstraction

Image abstraction, a fundamental component of non-photorealistic rendering (NPR), aims to simplify photographs into stylized depictions while preserving perceptually important structures. A central difficulty is selectivity: removing fine textures while preserving semantically meaningful boundaries. Existing approaches often expose only a few entangled controls, so smoothing strength and structural scale cannot be adjusted independently, which limits intuitive user control.We propose the Semantic Scale Space (SSS), a framework that organizes abstraction on two decoupled axes, abstraction strength and semantic granularity. SSS externalizes the stopping criteria by using a controllable semantic boundary detector to specify which structures act as barriers to smoothing, independently of how strongly homogeneous regions are simplified. We instantiate SSS with Adaptive Granularity Scheduling Smoothing (AGSS), which combines a donor-gated diffusion operator with a fine-to-coarse granularity schedule, and we introduce an effect-matched evaluation protocol based on a Region Homogeneity Index that compares methods at matched smoothing levels. On SBD and DIV2K, AGSS achieves higher boundary preservation and lower geometric drift than strong baselines at the same degree of smoothing, and a user study shows that its abstractions are consistently preferred in downstream NPR pipelines. These results demonstrate that SSS and AGSS provide practical, controllable image abstraction for modern creative applications.