Synaptic vesicle recycling involves AP-2/clathrin-mediated endocytosis, but it is not known whether the endosomal pathway is also required. Mice deficient in the tissue-specific AP-1-sigma1B complex have impaired synaptic vesicle recycling in hippocampal synapses. The ubiquitously expressed AP-1-sigma1A complex mediates protein sorting between the trans-Golgi network and early endosomes. Vertebrates express three sigma1 subunit isoforms: A, B and C. The expressions of sigma1A and sigma1B are highest in the brain. Synaptic vesicle reformation in cultured neurons from sigma1B-deficient mice is reduced upon stimulation, and large endosomal intermediates accumulate. The sigma1B-deficient mice have reduced motor coordination and severely impaired long-term spatial memory. These data reveal a molecular mechanism for a severe human X-chromosome-linked mental retardation.