Yes, it looks off. Here is one picture with a full length cabinet on the far right corner. It has two knobs.

I prefer all pulls on pantry, because the doors are so tall it looks more proportional.

Thankfully the bittom knob can be changed to a pull, since it's just 1 hole.

The doors still need to be painted so we can fill any holes.

With all pulls, I think the 4” is too small on the tops and the 6” is too big on the bottom. I will probably go with all knobs if this combo is odd looking.

Because the hardware is so close together, I think it will look better to use knobs. Here is another example.

In this example they used pulls, which is not to my taste. I would have used knobs on the upper cabinets and placed the pulls for the lower cabinet at waist level. I think it would be easier to use - no reaching to open the lower portion and more attractive.
Here they used pulls on the upper and lower - all at a convenient height. The upper is a large bi-fold door, so I think they needed to use pulls to distribute the weight.

May I ask what the name/make of your pulls are? I'm currently looking for pulls. Thanks.