The observed variability from the first discovered accreting millisecond pulsar SAX J1808.4-3658 can be explained if the X-ray emission is produced by Comptonization in a hot slab of Thomson optical depth τes ∼ 1 at the neutron star surface. We construct a detailed model of the X-ray production accounting for the Doppler boosting, relativistic aberration and gravitational light bending in the Schwarzschild metric. We show that the black body radiation is strongly beamed along the normal to the slab (a "pencil"-like emission pattern), while the Comptonized luminosity has a broader angular distribution peaking at about 60° from the slab normal (a "fan"-like pattern). Our model reproduces well the pulse profiles at different energies simultaneously, corresponding phase lags, as well as the time-averaged spectrum. The observed soft phase lags result from different radiation patterns of the two main emission components. We determine the radius of the compact object to be R ∼ 11 km if its mass M = 1.6 M ⊙, while for M = 1.4 M⊙ the best-fitting radius is ∼ 8.5 km. The lower limit on the inclination of the system is 65°. We present also simple analytical formulae for computing the light curves and oscillation amplitudes expected from hot spots in X-ray bursters and millisecond pulsars.