It would be stronger to drill into the brick

You can use those blue Tapcon screws to attach to masonary which you can get at any big box hardware store. You'll need a 5/32 masonary bit. You can use a regular drill for those screws but a hammer drill is better. However be careful and look at the max weight they can hold and compare it with the box you're trying to support. For window boxes I would suggest securing brackets into the birck and attaching the window boxes to the brackets.

actually its best to drill in the mortar and not the brick, you dont want to crack or weaken your bricks. go to your hardware store and tell them what youre doing. they will want to know the weight and type of material of the box.