Actually, season 2 wasn't all that bad. But the end of season 3 was kind of blah. The episode (a couple back) where the governor chased down Andrea was probably the best one ever.
I too was kind of surprised though that they gave up on the nice town (where they have some electricity, clean and safe buildings, etc) for the prison. Especially since the guard tower has not been blown up, their gates are continually getting knocked down or blown off, they still have uncharted areas with zombies in it, and a hole in the back where Tierese originally got in. All in all, the prison doesn't seem too good IMO. Sure they had to continually protect the fence at the town but weren't they mostly guarding against other people, not the zombies. They could just let the zombies pile up outside like they do at the prison.
So all in all a really good season. Just not much of a finale. Especially since they chose the prison over the town.