I've been using the Cambridge books for the past 4 years and I can assure you that the Cambridge books are definitely not the hardest, but they are really excellent.
Hard tier books: Terry Lee [Advanced Mathematics/etc], Bob Aus & Bernie Fitzpatrick [New Senior - Fun Fact: Bob Aus was the head math teacher of math when I was in year 7/8 at Merewether High], the rest of the hard tier books double as uni texts [eg stewarts/anton/spivak/strange].
Middle tier books: Pender et Al [Cambridge], Howard [Howard HSC math].
Low tier books: Grove [Focus], excel, oxford, there are quite a few low tier books.
The Cambridge book is really good, but definitely not the hardest.
If you plan on going into a field concerning combinatorics [theoretical computing, cryptography, number theory, quantitative chemistry] then the types of questions you will be exposed to in these fields are absolutely mind boggling compared to anything you will have ever experienced. So, what the books try to do is ratchet the difficulty up a little by making the examples in the latter part of each section more challenging, to expose people to slightly harder applications of the concepts, so as to make sure people have at least seen some of the kinds of things they may be doing later. The harder questions for fact/comb/perm/binomial can get rather wild, so don't worry too much if you are struggling with the harder examples.