Colorado rang in 2014 by becoming the first state to sell marijuana to its residents for recreational use. Along with Washington, Colorado legalized recreational marijuana use by state referendum in the 2012 election. Despite the fact that marijuana use is still illegal under federal law, the feds have decided to allow these two states to experiment with legalized marijuana as long as they follow certain guidelines. 

Along with the usual barrage of tired stoner jokes, there has been some handwringing over the possible consequences of legalized pot use. Among them, the potential health risks, the possibility of more marijuana users, and, in particular, increased use among children. But rarely do those concerned about the consequences of legalized marijuana confront the consequences of our current marijuana prohibition regime, under which some groups of Americans are swiftly labeled criminals and set on a path of being marginalized by society for what the elite and well-off usually end up recalling as foolish but ultimately harmless experimentiation with drugs. 

Consider that at least the last three American presidents, Bill Clinton, George W. Bush, and Barack Obama, have all more or less admitted to smoking weed. Bush’s former budget director and former Indiana Gov. Mitch Daniels, was arrested for possession of marijuana and other drugs while a student at Princeton. On Friday, two major newspaper columnists, Ruth Marcus of the Washington Post and David Brooks of the New York Times, admitted to using marijuana. Yet just as all three of the above presidents presided over a criminal justice system that imposes harsh punishments for marijuana use, Marcus and Brooks argue against Colorado and Washington’s marijuana initiatives, on the basis that marijuana is bad for you. 

The real question with marijuana legalization, however, is not whether it’s not healthy to consume, but whether our current system of prohibition makes things worse. Excessive consumption of alcohol is not healthy, either, but after more than a decade of banning alcohol, Americans decided the collateral consequences of prohibition were far worse than the consequences of letting people have a drink. 

Presumably, Brooks and Marcus don’t think of themselves as criminals who should have gone to jail for their drug use, any more than our three past presidents. Marcus all but acknowledges as much, saying that “[t]hrowing people in jail for smoking pot is dumb and wasteful.” But that’s what marijuana being illegal means in most states and under federal law: It means people go to jail.

It almost never means, however, that people like Brooks and Marcus go to jail. Anyone who wants to see what a de facto legalization environment looks like can visit an elite college campus, where both the trade and use of marijuana are highly visible, and where those who get caught face the relatively minor sanctions associated with breaking campus rules rather than the lifelong consequences of breaking the law. What we basically have now is a system where marijuana is practically legal for the wealthy and white and illegal for everyone else. 

Although marijuana use among blacks and whites is about the same, according to a 2013 report by the ACLU, blacks are almost four times more likely than whites to be arrested for marijuana possession. This startling graph from the report tells the story:

According to the report, there were more than 8 million marijuana-related arrests between 2001 and 2010, almost 90% of which were for possession. Marijuana arrests now make up more than half of all drug arrests in the country. And use of the drug has only increased over time. 

As Michelle Alexander notes in her book, The New Jim Crow, the consequences of being convicted of felony marijuana possession can be far more dire than the sentence itself. Former offenders can find themselves deprived of professional or driver’s licenses, educational aid, food stamps, public housing, their right to vote, and they may find themselves fired and unable to find new employment, having been marked by society as little more than a criminal. For blacks caught up in the system it can compound the already considerable effects of ongoing racial discrimination. 

Yet with their own acknowledged marijuana use our current and former presidents, as well as our well-respected newspaper columnists, are implicitly acknowledging that mere marijuana use does not make you a criminal who should be driven from society and made a pariah, it does not consign you to fate as an addlepated dunce or violent sociopath incapable of holding a job or raising a family. Would that the worst consequence for marijuana use was something resembling Brooks’ recollection of a botched attempt to deliver a presentation in English class while stoned. Legalization means that other people, not just elites and their children, can have the opportunity to shrug off past drug use as a youthful dalliance rather than a life sentence.

There are degrees of decriminalization between marijuana prohibition and legalization–and people can argue how far the government should go. What’s clear is that under the current system, the wealthy and connected are almost entirely shielded from the consequences of marijuana being illegal, while the poor and non-white face catastrophe for engaging in the same behavior. 

Brooks writes that legalizing marijuana means “nurturing a moral ecology in which it is a bit harder to be the sort of person most of us want to be.” The current system already can make it not a bit harder, but insurmountably hard to become the sort of person most of us want to be. But not for Brooks or Marcus, not for any of our recent presidents, and probably not for their children.