I lived there for 3 and a half years and I certainly didn't scratch the surface as to everything there is to see and do.
Off the top of my head, if you have the time between work I'd recommend taking a trip to Niagra Falls.
Otherwise you can't go wrong going up either Bloor or Yonge street...