Could it be costly? Sure. flooring, electrical, mechanical, etc...
It doesn't reach the ceiling, so, load bearing in the area that doesn't touch the ceiling isn't likely an issue. Where are the bearing points?
sometimes, people take down walls, and then have major quandaries about lifhting, outlets, storage, and... furniture placement in the new open arrangement.
do yourself a favor, and draw everything to scale. plot our furniture, lighting, outlets, tables and lamps.
and, kitchen cabinets and storage. make sure you gain more than you lose.
and, the demo and reconstruction process is messy. dusty.
think it through. especially if you are already living in your home, and will live through the reno.

Would you suggest just eliminating part of the wall to where it’ll be level with the counter too?

what is the goal? Is the sink moving? Where is the fridge going? Do you want to regroup lower cabs and try to fit an island?

We need more info to answer your last question.

Please explain your proposed floor plan. Expenses can increase when you need to move hvac, gas, electrical, plumbing. plumbing might be an issue with the dishwasher and fridge along the wall you are taking down. Obviously the fridge will need to find a new location.....how would the new kitchen layout look? Also, how will this change the furniture arrangement in the living room?

My take on open floor plans is that if you want one, the easiest and best way to get it is to buy a house with the floor plan you like and move.

Removing walls that your home's original architect designed in requires some major (read: expensive) structural changes.

Your situation is probably a little different, since the wall you want to remove isn't load bearing. But it still involves some significant life disruption and expense, as others have mentioned above. Those are often a strain on a marriage or relationship. It's not worth it, IMO, but that's just me.

@ratontoo The idea is to place the fridge just opposite of where it is now. So next to the cabinets on the opposite wall. The sink would stay in place and the upper cabinet next to where the fridge is now would be eliminated. As mentioned earlier, maybe considering to only knock half of the wall to extend the top counter top to align with the bottom and still have that open feeling if only a top portion of the wall would be eliminated. Thanks everyone for your advice and suggestions!

Just adding this picture as the idea to only knock half of the wall and extend the top counter to be symmetrical with the bottom.

There are likely electrical wiring lines in the wall servicing outlets and switches on the kitchen and the living room sides of the wall. Dealing with those and where you want replacements if any can involve how much access you have from the level below(basement for example). It's less likely that water lines are in that wall. But the supply line for your frig comes from somewhere. It seems unlikely hvac ducts are there. Check from below if you have access.

I'd suggest you measure your space and draw a scaled layout without the wall on graph paper' You can cut out some pieces of paper to represent furniture and an island. Play with those to see if you'll get a layout you really like. Draw that up and post it for some feedback. You may want to do some revisions. Then you can put together a projected budget and timeline.

I can understand wanting to open the kitchen up to the adjacent room. It does feel a little closed off. OTOH, the furniture layout in the living room works well with that wall there. I also like the idea of dropping that down to a halfwall or making it a big peninsula of cabinetry.

I have a half wall in my house that I'm taking down currently (more precise - I'm paying awesome people to take it down for me). In my case it is allowing me to change from "medium sized island placed too close to cabinets on the wall" to "large island with adequate distance to cabinets on the wall". But it will make the adjacent room (my dining room) a tad smaller. And I had to figure out how/where to transition the flooring.

I think it would not be a huge cost to remove that wall or shorten it considerably, but I'd be more concerned with how it impacts the living room.

"...However, I want to know is there a reason why the wall is not extended all the way to the ceiling...."

The builder didn't want to build it to a cathedral ceiling. More work & more material. This is often the case. It's definitely not a supporting wall since it's not going to the ceiling. However, that doesn't mean you shouldn't consult a contractor about the work to be done.

As others have mentioned, there's electrical and who knows what else in that wall. Stuff we can't see in the picture anyway. You're the only one who can say if it's worth the cost. What cost? You need to find that out. It looks simple enough but there's always something unseen.

Once you know what you have to move, post a measured drawing so we can help with placement of items.

Great, thanks so much everyone for the feedback. I will get a contractor come out and check out the wall. I’ll be posting additional updates once I have them.

I actually like the way the ceiling glows from the light of the kitchen side. It could be that it was intentionally built that way. I'm wondering if your house's style is soft contemporary.

probably didn't go all the way up to let some light in too. Doesn't seem like there's any windows/natural light on the sitting room side. the lamp and the fridge are plugged into something so there's definitely wiring in there that will need to be relocated. the cost is associated with how you plan to finish the space when the demo is done. Demo is easy.

I would leave it alone. Both rooms, now look very nice. You have plenty of cabinet space, in the kitchen, and plenty of furniture space in the living room. Removing that wall, will make a shortage of both, besides that, I have never figured out, why anyone would want to view their kitchen from the living room. Mine is open to the dining room. House was built that way, and I really dislike it. I dont do a lot of entertaining, anymore, and when I do, it is just family, but, I still hate that those at the table have to view the dishes, using for cooking the meal, and a sink with pots and pans in it. If there was a good way to close mine off, it would have already been done.