Elbow grease, a Hammer, and a chisel, or you can use an SDS drill with a chisel bit (you can rent it at most tool places) just make sure you don't damage the brick if you don't have experience but this is the fastest way.

Keep in mind this a very messy job, so put plastic up to protect the surrounding/adjoining area, be sure to wear eye protection, etc have good ventilation while doing this and also a good idea to rent an air-scrubber.

Good luck