Though it is enshrined in the Constitution, the grand jury is one of the least respected institutions in American criminal justice today. Scholars regard the grand jury just as doctors regard the appendix: an organic part of our constitutional makeup, but not of much use. While scholars have proposed reforms, most of them seem only loosely related to the fundamental purpose of the grand jury. In an era of plea bargains, the grand jury can serve a crucial role in insuring popular legitimacy in the criminal justice system. In light of the criticism, however, the grand jury seems to be failing in that role. This Article theorizes that, as the United States has become more diverse, the grand jury has lost its role as “the voice of the community.” Since a grand jury functions by majority vote and is drawn from the entire jurisdiction, the grand jury has lost its role as a countermajoritarian force of the local community against central authority. Ironically, the problem may have developed from efforts to insure diverse representation in criminal justice through unthinking adoption of the principle that trial juries should be drawn from panels representing a “fair cross-section of the community.” As the grand jury has become a microcosm of the broader melting pot, each community’s voice has been lost amid a cacophony of voices from other communities within the same jurisdiction. This has harmed citizens in poor and minority communities where legitimacy issues are most salient. No jurisdiction is just one community, and no grand jury can serve its purpose of representing a community if it is drawn from all communities. Grand juries should be reconstituted so that each grand jury represents an actual community of people who are likely to share common concerns about local issues of criminal justice.