Cytosolic proteins can be selectively delivered to lysosomes for degradation through a type of autophagy known as chaperone-mediated autophagy (CMA). CMA contributes to intracellular quality control and to the cellular response to stress. Compromised CMA has been described in aging and in different age-related disorders. CMA substrates cross the lysosomal membrane through a translocation complex; consequently, changes in the properties of the lysosomal membrane should have a marked impact on CMA activity. In this work, we have analyzed the impact that dietary intake of lipids has on CMA activity. We have found that chronic exposure to a high-fat diet or acute exposure to a cholesterol-enriched diet both have an inhibitory effect on CMA. Lysosomes from livers of lipid-challenged mice had a marked decrease in the levels of the CMA receptor, the lysosome-associated membrane protein type 2A, because of loss of its stability at the lysosomal membrane. This accelerated degradation of lysosome-associated membrane protein type 2A, also described as the mechanism that determines the decline in CMA activity with age, results from its increased mobilization to specific lipid regions at the lysosomal membrane. Comparative lipidomic analyses revealed qualitative and quantitative changes in the lipid composition of the lysosomal membrane of the lipid-challenged animals that resemble those observed with age. Our findings identify a previously unknown negative impact of high dietary lipid intake on CMA and underscore the importance of diet composition on CMA malfunction in aging.