Hi, Liz!

As a native San Franciscan, think you're very smart to tread carefully at the outset. While you sort out a siding replacement for the future, it might be easier to slay the street level rock situation first. I'd level out that old, painted fieldstone look with a plasterer and consider a brick grid effect that's balanced to set a solid look for your piers.

Also, how committed are you to the wrought iron gating your entryway? It's not an uncommon feature, but one that always says, "we're afraid to live here!" something I'm sure isn't the case at your location. Cheers!

Drive around several neighborhoods, and take lots of photos of the colors, garage doors and entryways that you like - you might find it easier in the selection process.

Take a deep breath. Breathe out. Now...come to terms with the fact that you are in a long term relationship here. Don't immediately rule out removing the siding. Being in an experienced contractor. Have them assess the condition of what lies beneath the vinyl siding. Also have them discover what is beneath the stone. Then start getting estimates on removal and restoration. Take your time. Your house has beautiful bones. It is your responsibility, yes...responsibility, to reveal and preserve them. One step at a time, as your budget permits. I suspect that in your market, you will be able to recoup every dollar that you invest in restoration (unlike many of us who do this and never expect any kind of return except the knowledge of doing the right thing by our old homes.). Good luck! I envy you the years (yes, years!) ahead as you get to know your gem of a home.

Don't go anything until you are ready for the full project, which will include exploratory demo (removing the vinyl and 'stone'), assessing what is there, then coming up with a restoration scheme.

the houses to the right and left dont look any better. embarrased? its not what others think but how you feel. if you can live with it (even though you detest it), bare with it until you have money for lots of change.
otherwise little tweaks and paint can do some drastic wonders.

I would agree , do not do anything drastic to add to the mess. I have never seen vinyl siding painted but lots of vinyl outdoor furniture that looks hideous the next year so I would not waste the money there. Those rocks have to go but not before you have a plan and IMO the gates are just plain ugly.

I'm not an expert, but just read up about vinyl siding - perhaps some paint experts could comment or you could visit a reputable paint store. Years ago it seems it was a no-no, but paint has come a long way. You do have to use a light shade, as darker colors retain heat and leave the siding vulnerable to premature warping and it requires a latex urethane paint formulated for exterior use. In regards to the stone, until you are able to assess/budget, thought the article below may be of some interest. http://homeguides.sfgate.com/paint-over-exterior-rock-facade-house-32396.html. I wouldn't paint the entire siding though - see house on far right in the picture below for trim/garage colors and if you decide to paint the stone, use the same color as the garage door (waterspout the same color as the siding). I would remove the grilled gate and add a large urn/planting between garage and front door, which will act as a nice visual instead of the stone.

I'm kind of disappointed that there's no quick & dirty fix for this problem, but I appreciate everyone weighing in!

Can any of you suggest how to find the kind of professional that would come evaluate the difficulty of removing the rocks and assess the siding for possible removal? An architect? Designer? Contractor?

Granted you have to fight the urge to rip off the siding and rock. I would find a sample of paint the same color as your siding. Paint a test rock in as inconspicuous place as you can find, but close enough to the siding. See what you think.

Also, have you checked the Register of Deeds office or the online database, even old Google street view to see if there is an earlier photo of your house to see how it originally looked? Maybe there is something in the Inspections records that says the exterior was in bad shape. Might help.

And yes, contact an architect for advice down the road.

I'm curious, what year was this house built?

Exploratory demo will be done by a contractor familiar with historic preservation. Depending on what you find will determine the next step (meaning if there is lots of evidence of what was there you can just put it back, if it has been really remuddled you need an architect to put it back)

I doubt that your home will ever become a Painted Lady but, it really isn't that bad. Wait until you have a big chunk of $$$ because "life happens"; job opportunity, kids, etc. In time you may surprise yourself and decide on a very modern remodel.

House was built in 1913.

Ok, I was almost thinking it was a newer house built in a old style. The windows are unfortunate too. I have a 1910 house that has a 70s exterior, so I know the frustration.

We've decided to bite the bullet and take the siding off to see what's going on underneath. We pulled some siding back and there's some stucco, so hopefully it will be in good enough condition to make something beautiful out of. The rocks are difficult to remove, says at least one contractor. He recommends we just paint them a dark uniform color.

I don’t know what this rock is called but we just bought a house where they put this on an entire wall with a fireplace. It took a few hours to remove. I call it “flinstones” rock because I have a yaba daba doo time when I look at it.

all white (stone and siding) black trim, black door...

I have used this company to hide ugly stone, http://spraystoneusa.com, maybe you can get a stone mason to apply stucco over the existing pattern to smooth it out then spray a granite texture over it.

Your house doesn't look nearly as bad as you seem to think it does.

Is that door on the right to the garage and the stairs on the left your actual entry?

Removing the stone around the door on the right w/utility meter/wires shouldn't even be begun unless your budget isn't an issue. IF, however, the surface is entirely clean, newer paints should adhere to the stone and cement well enough to provide a temporary change worth considering.

If the left gate/steps is your primary entry,

remove the gate;

paint all the stone ground floor front exterior in the indented section in front of the garage a true black -- something easily matched so you can do touchup and/or repaint w/o trying to match some odd color -- including painting the door on the right to make it visually disappear so the actual front entry will be the focal point of the front of your home.

Because the steps appear to reach all the way to the driveway -- no space for a door to open inward, you'll need to either create a front door that opens outward or (driveway/parking), space permitting, you can add a 6'x6' entry in front of the steps;

add an attractive front entry door with a large fixed window in the opening above it with really nice wood trim all around in that new entry. Use the same roof slope as the gables on your house for either the wood trim over what you build -- either just a door that opens outward or a new entry.

https://www.angieslist.com/articles/replacement-windows-vs-new-construction-windows.htm

When deciding the siding on your new entry, keep in mind both what you already have and what you want later to make it meld well with both.