Optical damage of congruent lithium niobate (LiNbO3) is a long-known problem. A popular method to limit this damage is doping LN with magnesium. Here, the dielectric permittivity of pure and magnesium-doped lithium niobate was characterized depending on the crystal orientation over a broad frequency range: 1 mHz to 0.75 PHz (λ=300 nm), using impedance measurements, quasi-optical free-space characterization, THz time domain spectroscopy (THz-TDS) and ellipsometry. Three different frequency ranges, separated by well identified resonances, are observed: low frequency 'piezoelectric' response, intermediate frequency 'ionic' response and high frequency 'electronic' response. Doping with magnesium influences neither the electronic nor the piezoelectric contribution to the dielectric permittivity, whereas the ionic contributions is strongly affected due to the changes in the defect structure.