The first few classes (almost the first year's worth) of classes were weeder classes. There were a lot of people in CS at the time who "liked to play videogames" and that was why they became CS majors. Those people got swatted by the giant flyswatter that is CS. We went from 4 lectures of hundreds of people each (first quarter) down to 1 lecture of maybe 100-150 people by the end of the first year (I went to school during the big influx of students from the tech boom).
I will tell you a little secret though, if you can get through the 1st year of classes, the rest of CS is MUCH easier --- especially upper division classes. But those classes build upon the first year classes, they just go into way more depth. If you don't understand the first year classes, you will be hosed. In my opinion, it is the upper level math classes that will kill you, not the CS classes themselves. For me, the hardest CS class was probably the series on Algorithm Analysis. I just really didnt get it as well as everything else in CS.
Just to give you some perspective though, I have been playing with computers ever since I was like 6 or 7. I knew computers were my destiny, but I wasn't quite sure exactly how. I thought software engineering was alright but it wasn't my passion. But after my CS undergrad (and taking those upper division courses), I know my place in life is in Networking/Telecom. I am two years into grad school and I work for a big tech company doing IPTV!
It helps immensely to make friends in class and to help others. I helped my best friend in college pass the first class of college in CS. Later, that same friend started helping me with all the algorithm analysis and upper division math classes that I was having trouble with. There is nothing I remember more than getting a beer with my friends at the on campus pub! (BTW - that same friend is now going to law school to do Intellectual Property law -- he didn't want to do programming either!)