Contrary to popular wisdom, rocks (or other coarse stuff) doesn't help drainage.

And using garden soil in a container makes drainage worse.

Further, don't throw in tree needles.

Always use a potting mix in a container of any size.
Because you re-potted it just 5 to 6 weeks ago, I suggest you consider re-doing it the correct way w/ potting mix -- now.

Unless the potting mix is specifcally designed for azaleas and rhododendrons, I wouldn't use it, either. The most important characteristics of a mix for the use you intend are an acid pH, coarse texture and, especially, rapid drainage. Unless you live in an area of the country where mixes for rhododendrons/azaleas are sold, most of the readily available potting mixes do not have these characteristics. The Container Forum receipe involving peat, bark and granite grit is far better than what's available from the big box stores and giant producers of bagged mixes.

The addition of the conifer needles is not, imho, a bad idea at all. They break down slowly and help to keep a mix open and free draining. Just don't overdo it.