We investigate the dynamics of a driven optical parametric oscillator under the injection of orbital angular momentum. The injected mode is adiabatically driven through arbitrary transformations on the Poincaré sphere of first-order paraxial beams. As a result, the down-converted beam conjugated to the seed is shown to follow a path imposed by a nontrivial symmetry on the Poincaré sphere. This symmetry allows controllable distinguishability between the spatial modes of the down-converted beams. In this Letter, we provide convincing experimental evidence of this effect.