Is Engineering hard at Cal Poly Pomona, CA??

You should expect to be challenged more than you have been in the past. Will physics be a problem? That depends on you. It wouldn't be a problem for me, but we are different people. Personally, I'd ask students that are already there.