I'm still working on my own garden, so I can't tell you what works for me. :) But, in my area (Baltimore city, pretty urban) what I've seen that impresses me most is one larger plant/tree/shrub with much smaller plantings to accompany it. I'd consider picking a few larger things to be the "bones" of your garden, then you can dress it up with smaller perennials and annuals to be the "skin".

How small is "small"? And no matter what, I think scale is very important. Also important is intended purpose of the space (ie lounging, fruit orchard, miscellaneous etc).

In general, I think that if you are gonna have a handful of plants, then one or two biggish trees (like Crape Myrtle) might look ok; however if you want to have more plants, then it is better to keep them dwarfs.

Also, to draw the eye up and create the sense of more space, try to go with trees having vertical growth such as a birch tree.

Just my 2c.

Thank you both for your responses (even though I'm a year late in acknowledging yours, Akimbo!). Homey bird, the total space in the yard is about 14' x 28', and 10' x 14' of that is taken up with a concrete patio. The house is a 2-story pueblo-ish style (flat roof, stucco), and the yard is surrounded by a 5' high block wall. I decided to play up all that geometry a little bit, so I put in a central bed that's a diamond with concave sides (this house need curves somewhere!) and surrounded it with 4 beds that are convex arches. A 4' x 14' rectangular bed runs along one side of the patio.

It's more or less a potager or kitchen garden--I have garden vegetables in a micro-intensive box unit and in various containers on the patio, the central bed is an herb bed, and I've filled the others with things like currants and sand cherries that will feed either me or the birds that are doing such a nice job of keeping the insect pests down. I've planted some flowers that attract beneficial insects, and some that I just like. I also have 3 small trees that will eventually help filter out some of that New Mexico sunshine.

What seems to work with that number of beds is to have one large thing in each and to fill in the rest with ground cover or subtle texture plants--just like you suggested, Akimbo. The ones that I've tried to make look like traditional borders are waaaay too cluttered looking. I think if I had divided the space into fewer, larger beds--maybe one continuous, irregularly shaped bed around the perimeter--that fussiness/clutter would be less of an issue.

I like what you're saying about scale, homey bird. Maybe that's what I'm trying to pinpoint--how to choose plants that work with the total size of the space but also with the way the space is divided.

Thanks again for your ideas, both of you!

I have a deck in my backyard that about is the same size that you described, and I am wondering what I would do with that space as total yard space. I do understand your instinct :-)

You seem to suggest that having big trees with low shrubs around them is not working for you. Can you imagine trees with semi-dwarf size? They typically go 8'x8' size -- and really ideal for small garden. Fruit trees as well as some kinds of Japanese Maples come into these sizes.

Also, your complaint about cluttered look may have little to do with the composition. Some trees inherently have a jungle look whereas some have very defined shapes. To avoid clutter, you need to have plants with more defined shapes like Lavender, for example, or sculptural plants like Aloe. A groundcover is ok however you must give it definition via other plants. This is just my observation (I am not a professional) -- based on some jungly gardens that I have seen.

Furthermore, if your biggish trees are in the middle of the yard rather than towards the periphery, they will most likely make your yard look smaller. Also, if there are two many textural variations in the yard (e.g. stone leading to groundcover leading to pathway leading to another bed etc.) -- that breakup actually makes the yard look smaller and overwhelms the space, no matter what the size of plantings.

Thanks for your reply, homey-bird. Actually, I don't think I made myself clear. The trees and shrubs are working well for me; it was all the perennial "fillers" in the beds that got a little overwhelming. The trees are New Mexico privet (forestiera neomexicana) and should get to 15-18' tall and 10-12' wide, so they're small trees but not dwarf size. Since they're young, they're only about half that height at the moment, but I don't think they'll get too overwhelming as they mature. With the proper pruning, they can have a nice, airy structure--not "jungly" at all. The trees and shrubs are all in the peripheral "arch" beds and give those beds some nice shape.

The perennials I put in those arch beds, though, grew MUCH faster than I expected them to, and much taller than I ever thought they would--3', say, instead of 1 1/2'. (I'm new to zone 7 gardening, having moved here from zone 4 New England, so I'm still adjusting to lots of differences.) I think the problem is that those taller perennials distracted too much from the central herb bed, which is really the focal point, and they obscured the structure of the trees and shrubs to boot. So in those peripheral beds, I'm now letting the trees and shrubs stand more or less alone with perhaps some small mounding plants or evergreen ground covers next to the paths--much more satisfactory.

I'll probably end up with some dwarf fruit trees in pots on the patio sometime soon, though. :)