Japanese maples do not fair very well when grown indoors - they are not houseplants. Would assume the indoor conditions are just not close to what it wants with regards to light, temperatures and humidity.

Is there a reason it cannot be grown outside?

I didn't want to plant it in garden as I don't want a big tree so pottin is was better for me so it will stay small

It doesn't need to be planted in the ground to be outside. You can grow the tree in a container - many, many folks do so with JM's. It just cannot live indoors.

Ok thank you . When does it have to come indoors then after summers gone

It may not have to, depending on where you are located. JM's in containers remain outdoors year round in my climate :-) But if you are in a colder area, you want to wait until the tree is dormant (dropped all its leaves) and then move to a location where it will stay cool (below 45F) but above freezing. Some place like a garage or unheated basement. Light is not an issue when dormant but some watering is necessary just to keep the roots hydrated.

But don't move it into the living areas indoors - they are far too warm.