Not sure what kind of brush you have. A PVA foam brush requires removal of the wheel. The PVA should be wet and pliable. You can cut off or slide off the old one then slide on the new, damp one.
Other brushes may be held on with a retaining rod that can be tapped out.