We investigate the orbital angular momentum mixing in frequency up-conversion with structured light beams. Phase and polarization vortex beams are coupled in noncollinear second harmonic generation under type-II phase matching. The spatial properties of the second harmonic beam are shaped by the combined features of the input beams. This effect creates a spin-orbit crosstalk allowing for angular momentum transfer between the two degrees of freedom. The information transfer between photonic degrees of freedom encoded on different wavelengths can be useful for connecting different physical platforms in quantum information protocols.