I don't think there's no justification. Quebec City has over 800 000 people in its metro area with the total catchment area of the station probably including around a million, plus you could add Trois-Rivieres as another station.
I do think phasing may have been the better approach though. An initial Toronto-Ottawa-Montreal section, with possible stops in Kingston and maybe an East GTA stop like Oshawa or Pickering.
Then extend it to Quebec and Windsor / Detroit in later phases.
I could see this ultimately being what happens.
Honestly though, the southern extension, which could hypothetically also include London would probably be much more viable than the northern one to Quebec. denser population in general with hypothetical connections to Detroit and other US cities, in addition to easier, flatter geography.
My guess is this was chosen to appease Quebec so they serve both big cities in both Ontario and Quebec.