Currently in 3rd yr optometry..hopefully you see this.
Anyway, you'll survive optometry without doing any sciences (I did). HSC Biology would be a big help for the physiology and biology subjects, but like others have said, you'll learn all the basics you need in 1st yr. If you have at least 2 units of maths, then physics should be okay. 3 and 4 unit maths gives you a big advantage in physics (especially in the motion topics).
If you're worried about the optics courses, then they just require very basic maths. All the optics courses are just deriving equations, memorising equations, a lot of similar triangles/trig work, and punching in the numbers. The only tricky bit is figuring out how to draw diagrams to figure out what to do - the rest follows on logically.