According to the writers of Questions.org:
Jesus condemned the Pharisees’ self-righteous hypocrisy because it blinded them from seeing their need for repentance and a Savior.
Many Pharisees prided themselves in their strict avoidance of obvious, outward sin. But they refused to look inside themselves and acknowledge the presence of inner sin that didn’t fall within the boundaries of their man-made rules. Jesus knew that in spite of their obsession with outward perfection, they willfully resisted consciousness of their inner corruption and need for grace.
Jesus didn’t associate with “known sinners” like tax collectors because He minimized their sin ( Luke 19:1-10 ). He freely associated with them because He knew that they were more open to repentance.
“Known sinners” weren’t full of self-righteous pride, deliberately concealing their hidden sins behind a legalistic façade of “righteousness.” Jesus was keenly ironic when He said, “I didn’t come to call the righteous to repentance.” He knew that the Pharisees weren’t righteous, but their pretense of righteousness kept them from accepting the only remedy for their condition — repentance and faith in Him. The obvious sins of “public sinners” made them more likely to repent and look to Jesus for the answers they needed.
We are all sinners, both inwardly and outwardly. Although we may not be notorious “public sinners,” we all share a fallen nature and are often controlled by the “flesh” — the “sin principle” — within us (Romans 8). Jesus’ stern warnings to the hypocritical Pharisees make it clear that sin we ignore and deny is no less serious in its effects than the sin of the public sinner.
Matthew, Chapter 23 clearly expresses Jesus’ anger towards the pharisees.