Visual sensitivity is comprehensively described by the contrast sensitivity function (CSF), but current routine clinical care does not include its assessment because of the time-consuming need to estimate thresholds for a large number of spatial frequencies. The quick CSF method, however, dramatically reduces testing times by using a Bayesian information maximization rule. We evaluate the test-retest variability of a tablet-based quick CSF implementation in a study with 100 subjects who repeatedly assessed their vision with and without optical correction. We first discuss two commonly used measures of repeatability, intraclass correlation and the Bland-Altman Coefficient of Repeatability, and show that they are vulnerable to artifacts. Instead, we propose to formulate precision as an information retrieval task: from all repeat test scores, can we retrieve a certain individual based on their first test score? We then use rank-based analyses such as mean average precision as a better measure to compare different test metrics, and show that the highest test-retest precision is achieved using a summary statistic, the area under the log CSF (AULCSF). This demonstrates the benefit of assessment of the whole CSF compared to sensitivity at individual spatial frequencies only. AULCSF also yields best discrimination performance (99.2%) between measurements that were taken with and without glasses, respectively, even better than CSF Acuity. The tablet-based quick CSF thus enables the rapid and reliable home monitoring of visual function, which has the potential to improve early diagnosis and treatment of ophthalmic pathologies such as diabetic retinopathy or age-related macular degeneration.