Once upon a time, your horse-head (if original) was a hitching post. Hence the ring; it had a practical purpose. That doesn’t stop it being decorative as well, of course. I’m sorry I can’t help with the rust.

I know this will sound crazy, but ask an auto body/painting shop if they can refinish it for you. You would have to bring it in of coursee.

Hey thanks amystroller for the lesson about the horse! I did a bit of research and you are right about origin and purpose. A piece well worth saving.

Covering rust DOES NOT stop it's progression. The first thing you should do is decide if head is ok as is and only base needs attention or if the whole piece needs restoration. Next remove everything covering rust on portion you want to refinish. Once rust is exposed, wire brush away as much debris and rust as you can (both sides of base plate). The base may very well be so thin and pitted it can't support head. If you think that's the case, a welder can cut it off and put on a replacement. If you want to do the best with what's there, follow directions on rust converter product. Some good converters are Corroseal Converter, POR-15 , Ospho and Jasco.

Once treatment is complete, prime and/or paint as suggested on rust treatment container. NOW! We get to your question as how to fasten head to wall. Any adhesive capable of securing head will dissolve finish negating all the efforts above and allow rust to start over. I suggest securing head to fence with masonry anchors. If you are uncomfortable working with masonry anchors, here's an alternate. Secure head to a piece of treated wood with screws the use construction adhesive to secure wood to wall.