Immigration officials have the right to refuse entry to people even if they hold a visa, the High Court has been told.