My husband was a vegetarian for a while in high school. Whenever he talks about his experience, he always remarks that the most challenging part of being a vegetarian is not your diet, it's everyone else.
I think much is the same with Paleo eating. Food is so closely connected to our culture, tradition, and beliefs. People hold their food near and dear to their hearts. This is why it can be uncomfortable to get into a debate with someone about their food choices. Our government (via food pyramid) has inundated society with differing ideas for decades. Modern research is starting to prove much of that wrong. If you've read In Defense of Food by Michael Pollan, then you know what I'm talking about. Many of the FDA's decision are motivated by the big corporations in the food industry. This has helped to promote misinformation to the general public. I'm not a big food conspiracy theorist, but think much of the status quo's knowledge of nutrition is flawed.
I think that the most difficult part of primal eating is not reading the ingredients on items, going without the tortilla on your burrito, or trying to live without cheese; it is by far trying to live without criticism, skepticism, or condemnation.