I really hope this is in the correct forum. I’m new to photo editing and I don’t know if I’m missing something.
When I use a selective color adjustment above my other layers, any areas I’ve used the brush tool to paint on seem unaffected such as around the sides, the bones, and man’s shadow. I’m using Affinity Photo 2.
I’ve never used Affinity but in Photoshop the Selective Color Adjustment only effects a single specific color. As you can see below, I have red selected, after the adjustment it only effected the red areas. Maybe try a different adjustment type?
Again, I’m not familiar with Affinity nor do I know what you are trying to achieve.