Since I love social puzzles, I was pleased to discover the book The Problem of Ritual Efficiency:
Ritual has come to be thought of in popular discourse as a kind of action that is ineffective, superficial, and/or purely formal, and this view is the unexamined premise behind much of ritual studies. This attitude explains why …we “know it when we see it” – and what we know to be rituals when we see them are acts that are apparently non rational, in which the means do not seem proportionate to the ends, the intended objects of human action are non empirical beings, or the theories of efficacy that ostensibly explain the ritual acts are inconsistent with modern, scientific paradigms. This reaction is similar to what an archaeologist does when he discovers a structure whose purpose is unclear – he calls it a temple. … The notion that ritual is ineffective is false. … Shamatic rituals heal, legal rituals ratify, political rituals unify, and religious rituals sanctify. Rituals transform sick persons into healthy ones, public spaces into prohibited sanctuary, citizens into presidents, princesses into queens … One of our most important tasks as scholars is to explain how rituals accomplish these things. (pp.6-7)
I bought and read that book, and have also been reading The Creation of Inequality, which emphasizes the centrality of rituals to foragers:
Cosmology, religion, and the arts were crucial to hunters and gatherers. … The lessons of myth were passed on audio visually. Performances combining art, music, and dance fixed in memory the myth and its moral lessons. … We doubt that art, music, and dance arose independently. More than likely the evolved as a package that committed sacred lore to memory more effectively than any lecture. … The archeological data suggest … that the use of the arts increased as larger social units appeared, because each moiety, clan, section, or subsection had its own body of sacred lore to commit to memory. … Dancing, drinking, and singing for days, as some tribes did, opened a window in the the spirit world and thereby confirmed it existence. (pp. 62-63)
Also, reading a BBS article on “ritual behavior”, I came across this comment by Bjorn Merker:
Literal duplication … lies at the very heart of ritual. The need to remember and reproduce essentially arbitrary details on an obligatory basis burdens behavior with a handicap, and the ability to sustain that burden is proof of capacity, and hence tends to impress. … There is reason to believe that humans by nature are carriers of ritual culture in the sense just defined. We, in contrast to chimpanzees – indeed, alone among all the primates but like many songbirds and the humpback whale – are in possession of a neural mechanism that allows us to duplicate with our voice that which we have heard with our ears. Most mammals, who excel at learning in other respects, are incapable of doing so; yet we humans do so with every song we know how to sing and with every word we know how to pronounce. … Such a perspective helps us understand why ritual form marks human culture not only in domains touched by precautionary concerns, but in well nigh every area of human pursuit. (p.624)
While much about ritual remains puzzling, one thing seems clear: the essential human difference, the one that has let us conquer the Earth, was an ability to accumulate useful innovations via culture. And this primarily required a good ritual sense, i.e., a good way to watch and copy procedures like starting a fire, shaping a knife, etc. Having language and big neighbor-friendly social groups helped, but were less essential.
The fact that language require good vocal ritual skills suggests better ritual abilities appeared before full recursive language. Since they were very social, pre-language but post-ritual humans probably filled much of their social lives with complex rituals, showing off their abilities to precisely execute complex procedures, and showing loyalty via doing their group’s procedures. Music, dance, and other such ritual habits continued after full language.
Language let us better express and enforce complex social norms, and helped us gain a more conscious and flexible understanding and control of our procedures. But we still have poor introspective access to our pre-language systems, and so still don’t know a lot about why we do which procedures when, and why they make us feel good or bad.
When humans believe in hidden spirits who take an interest in whether they follow social norms, hard-to-understand rituals offer a natural place to locate their connection to spirits. Humans can believe that spirits watch their rituals, and then respond by making them feel good or bad. This helps us understand why religions emphasize rituals.
Added: This poll has a majority favoring culture coming before language.
People generally use and pass on culture without understanding why it works. There's a great paper on this that traces the evolution of the America felling axe, the rudder, and pregnancy food taboos with this model in mind.
Rarely do we understand all the functions of things, including rituals and institutions. That's why it's so hard to design societies (or, really, anything) from scratch. Design failures make themselves known; design "fit" is invisible.
The riddle of ritual is a paradox and can only be found in something simpler. Perhaps a purpose or a principle.