Well, first it depends on the plant and how deep (if any) that it is supposed to be. A water lily could (should) sit right on the bottom in a regular big plastic pot like 5 gal or so or it can go in a similar pot with little holes all over that is designed for ponds. Depending on the medium, they do sometimes need a couple rocks on top. You don't normally use top heavy containers and they don't have to come all the way to the surface. However many other plants would only want to be damp or slightly under water and would therefore need to be held up on something. There are also floating rings that can hold plants while floating. You could make something with styrofoam or whatever.

In one of my small ponds I've got concrete blocks of various sizes. The potted plants sit on those. Some only just have the bottom of the pot in water, and others right up to only just having the top of the pot above water. I keep the plants at the level they're happiest with. The concrete blocks are hollow so the fish swim right through them. None of the pots are tall and narrow and I've never had any trouble with any falling over.

I love the concrete block idea. They styrofoam floating rings I don't find so aesthetically pleasing. Thanks for the responses!

Well I tried several methods -- which mostly failed. The pots tipped over no matter how I tried to secure them, weigh them down with bricks, etc. Finally my husband went in there and sort of leaned them against the sides of the pond. If there's a high wind at some point, uh oh. Of course, when it comes to ponds, it's all worth it. The result so far: