How a developing nervous system discovers meaning in complex sensory inputs has typically been examined separately for each sensory modality. Even as studies have uncovered modality-specific strategies, it remains unclear whether common principles underlie such discovery. Here, we pursue the thesis that the detection and exploitation of temporal regularities may provide a unifying mechanism for sensory organization across modalities. We synthesize research spanning neurophysiology and cognitive neuroscience and incorporate results from theoretical computer science. This integration supports the conclusion that time may be the fundamental dimension along which the brain organizes its sensorium and that the computational complexity of this problem is rendered tractable by ecologically appropriate heuristics. This proposal suggests the centrality of temporal processing in perceptual development, with implications for studies of typical and atypical development, clinical populations, and computational modeling.