Your tree looks pretty young, I suggest taking all the fruit off this year and letting it concentrate on growing. If you can't mange that (I know I couldn't the first year) I'd just leave one lemon. Your tree is so small if you leave too many they won't be able to grow full size and be dry and sub par in quality. My tree was 3x the size of yours last year and wasn't able to grow its first two lemons to full size. They ended up the size of limes and were very dry inside. The later lemons that season were no better and even smaller. This year it gets to carry none until it grows up some more.

I agree with Sammers

Removing all the fruit to encourage growth is a good idea. Having said that, research conducted by Perdue University, found that leaving one (1) fruit on a young citrus tree does not reduce any growth of the tree.

Keep one if you like, just for grins; select the one that is on the strongest branch, nearest the trunk.

I apologize to the creator of the thread, but I would be very much interested in more info about Purdue study. Can you write down the exact title of the study (authors, date,...)? I am refering to Silica's post of course.