Is Java really that hard to learn? Should I just drop, and take it next semester, after I've studied on my own a bit?
IMHO, Java is pretty easy to learn as far as programming languages go. It does a lot to prevent you from coding yourself into a dark corner. I also think Java code just reads easier to new people (less cryptic). There is a reason many schools teach it as a first language, and it's not to be sadistic.
That said, I walked into Java having taught myself C++ some years before. Considering that syntacticly Java and C++ have a crap ton of similarities, my situation wasn't quite the same as yours. I walked out of the class with a +100%.
THAT said, I also saw some fairly dim people do well in the class, walking in with absolutely no coding experience at all. Judging by your posts over the years, I doubt you'll have any trouble with the knowledge. I (and probably several others on this board) can help if needed.
Workload wise, I just can't say. My Java class was very easy to me. Maybe 3 hours a week of homework/reading on average. Keep in mind, too, that later assignments often can reuse code you've written, at least in part.
EDIT: Just to be clear, is this course described as an "intro to programming" course? Or is it a "Java for programmers" kind of course?