The spatial range of a species habitat is generally determined by the ability of the species to cope with biotic and abiotic variables that vary in space. Therefore, the species range is itself an evolvable property. Indeed, environmental gradients permit a mode of evolution in which range expansion and adaptation go hand in hand. This process can contribute to rapid evolution of drug resistant bacteria and viruses, because drug concentrations in humans and livestock treated with antibiotics are far from uniform. Here, we use a minimal stochastic model of discrete, interacting organisms evolving in continuous space to study how the rate of adaptation of a quantitative trait depends on the steepness of the gradient and various population parameters. We discuss analytical results for the mean-field limit as well as extensive stochastic simulations. These simulations were performed using an exact, event-driven simulation scheme that can deal with continuous time-, density- and coordinate-dependent reaction rates and could be used for a wide variety of stochastic systems. The results reveal two qualitative regimes. If the gradient is shallow, the rate of adaptation is limited by dispersion and increases linearly with the gradient slope. If the gradient is steep, the adaptation rate is limited by mutation. In this regime, the mean-field result is highly misleading: it predicts that the adaptation rate continues to increase with the gradient slope, whereas stochastic simulations show that it in fact decreases with the square root of the slope. This discrepancy underscores the importance of discreteness and stochasticity even at high population densities; mean-field results, including those routinely used in quantitative genetics, should be interpreted with care.