I don't know why you hate classes not part of your major. For me, I actually like those classes, they gave me many insights into this world. They are nice to take a break for all the science I have to learn for my major, but they actually relate in some way to my major. I took anthropology (made me respect the different cultures around the world), psychology (learn about the working of the human psychological mind), gang class (learn about gangs and thing I never knew before), music (how music is related to society), philosophy (makes me puzzle about this world), writing (helps me think critically). All is related to my major (biology) somehow, must know society if you are going to be a scientist/doctor and how to write research papers. Gangs class, we learn about drugs and how people get high and tolerated, guess what that enhanced my learning in biology.
I have had hard classes, most of them from my major, not general ed classes. College is hard, but if you work hard, you'll find it rewarding in the end.