An Invasion Day march due to culminate at Victoria Square has been impacted by an alternative group of protesters.