I played through both games on Normal for starters.
The boss that was the most irritating for me was the bat boss, from Ys 1 (Vaguillon, or something like that, wasn't it?) Eventually, I got a good feel for the fight though and can beat it without too much trouble. Just, it's definitely the longest boss fight, since it all depends on how quickly he'll transform. Also makes it suck for boss rush mode, if you are trying to get a quick time, since it's all luck with that guy...
I actually thought Dark Fact was pretty easy, once I figured out that the Silver equipment works much better on him than the 'best' equipment.
Darm I thought was pretty easy, if you use your 'Shield' magic, like they give you on boss rush. But again, I played through the game on Normal. I'll probably give it a go at a higher difficulty sometime.
Oath in Felghana's bosses were pretty tough overall, with some being pushovers. But, overall, harder. I couldn't actually beat the boss rush for Normal, there were two bosses that were just giving me too hard a time. I think I was a level higher than they give you on the boss rush, since I definitely remember dealing more damage than that... After spending almost two hours on the boss rush mode, I was getting rather tired, and just gave up. =/