More details would be helpful.

Are the stems getting larger or smaller each year? Does the foliage go yellow and then brown from the base upwards? When does this happen? What are the other lilies that are doing well - asiatics, orientals or something else?

Hansonii is reasonably easy to grow if conditions suit it but it does need, for example, better drainage than an asiatic.

Right now, three have turned brown and disappeared, one is turning brown (one that bloomed) and one is still green - they don't seem to go from teh stem up. They got bigger the second year, but not bigger this year. Other lilies I grow - Oriental, Trumpet, Citronella, Henryi, tiger. They are growing in a bed with Oriental and Trumpets that are doing great.

This sounds like they may have been simply too dry or too hot. If you can work out what changed between the first year and the next it may help, but otherwise I'd suggest a little more shade.