how is publicly' an exception of adverb as mentioned in the lesson, isn't it an adverb?

Yes it is. It explained the rules to add ly at the end of a word. When a word end with ic, in that case we add 'ally '. Eg. Fanatic = fanatically. But in public we add ly instead of ally. So it is an exception.