In this talk, we propose a method for designing loss functions based on component trees that can be optimized by gradient descent algorithms and are therefore usable in conjunction with recent machine learning approaches such as neural networks. The nodes of this tree are the connected components of the upper level sets of an image and the leaves represent the regional maxima (or regional minima if the dual tree is considered) of the image, i.e., connected sets of bright pixels surrounded by darker pixels. The proposed loss function is thus defined at the level of connected components rather than at the level of individual pixels, which allows for the optimization of higher semantic level quantities such as topological features. We show how the altitudes associated with the nodes of such hierarchical image representations can be differentiated with respect to the values of the image pixels.This property is used to design a generic loss function that can select or discard image maxima based on various attributes, such as extinction values based on the contrast or the size of the maxima. The possibilities of the proposed method are demonstrated on simulated and real image filtering.