I just did the HSC at TAFE. Generally, you can expect much smaller classes than school with the average age of the students being much higher. The kind of people you meet there varies with year to year. I met a handful of people doing limited UAIs, like you - 23, 24 year olds. One of them got a UAI of 96. You're generally surrounded by people who are dedicated and mature and people who may have had similar problems with the whole school situation to you. Others return because their parents are forcing them to do something, drop-outs and druggies etc. and are generally younger. The environment at TAFE is much more mature and free, though, and these people tend to just stop showing up, leaving only the focussed students by term three.
There's no mini-society, like at school - no excursions, assemblies and carnivals, obviously. Much less personal. It's a fasttrack, no-bullshit HSC without the social politics of the playground, but you're also more "on your own" in that regard. You're on first name basis with your teachers and they seem to be on your side. There's no teacher-student superiority complex going on that you might remember from school (I did have one teacher like this but he was the exception to the rule).
Simultaneously, there's no hand-holding either and you need to be personally comitted. Nobody will care or stop you from just dropping out. Teachers won't bend over to give you extensions for overdue assessments. While it will be certainly less stressful than doing six subjects, remember, you'll spend the first term doing "year 11". So the HSC year at TAFE starts with Term 2, as opposed to a school HSC year which begins in Term 4 of the previous year while they, off course, end at the same time. The syllabus gets rushed through pretty fast as a result and you'll need to do a lot of work at home to do well (like anything). Teachers are generally more competent than your typical state school, though. Classes are three hours long.
About the info session, find out the phone number of the HSC Co-ordinator at the TAFE you want to go to and give him/her a call. Check out the stickied threads about it all here, too, if you haven't already.