It literally refers to woodworking and carpentry. Wood has a pattern, the direction in which the wood fibers grow, called the "grain." When you plane or cut wood "with the grain," it is smooth and easy and you get a clean, smooth cut. If you try to cut "against the grain," the tool binds and chatters and produces a rough, uneven cut. So, "going with the grain" means "going in the direction that makes things go smoothly and easily."

It means to do something that is different to what is considered normal, or the accepted thing.

It means that you do something that is the opposite of what you usually do. Ex. You wear pink every Monday. Yet, one Monday you wear black. It goes against what you normally do. You can also look at it in a social context. Ex. Everyone wears black, and you wear pink.