Take them down and leave them down. They make the front of your home look cluttered.

I would take them down and store them, wrapped up in clean paper, for a year. If you miss them, paint them and put them back up. If not, keep them for the next owner. Unlike many shutters, these look like they were properly sized for the windows. Cutouts come and go in popularity. These give a personality to the house that may or may not be unusual for the neighborhood.

Take them down. They're cool but they don't match the style of your house.

Wondering if you live in The Kingsway in Toronto? This is a very typical house for that area and the link below has pictures of homes in that neighbourhood. Most homes of your style do not have shutters and I find yours are a bit too country/cottage feeling and take over from the other lovely classic features. Hopefully the brick will clean up well. I like the color of your trim, but if wanting a change, I would use a deeper grey and then a light grey for the front door, so it feels brighter in the porch area. Perhaps the pictures below may help with color combinations and you could look at other neighbours' color choices (I also look for inspirations on how people are doing their urns seasonally).

https://www.google.com/search?q=homes+in+the+kingsway+toronto&biw=1366&bih=599&tbm=isch&tbo=u&source=univ&sa=X&ei=ymyiVc3MKMTaU_axgdAC&ved=0CDgQsAQ#tbm=isch&tbs=rimg%3ACZtcrofpQYooIjgkstQkU6N9m-r_1NdTgySfvAlomiSSzCIj_1ROXrD3rKZ6xXyMNIdFyEIkpyWleAoQduJfAyU4ipbioSCSSy1CRTo32bEUgeQDAgkdqJKhIJ6v811ODJJ-8RqH78lSKMBl0qEgkCWiaJJLMIiBEIOACOEh1AKioSCf9E5esPespnEcRHdJbzUJI3KhIJrFfIw0h0XIQR4To_1pDXpJ7sqEgkiSnJaV4ChBxHQYL4_1ND29uSoSCW4l8DJTiKluEYQCks43mTuS&q=homes%20in%20the%20kingsway%20toronto

I like the house without the shutters.

Much nicer without the shutters! Pretty home!