So...cutting a new hole?

Really depends on the exterior, structure, permitting if required, utilities in the wall, and on...and on.

Impossible to give you a price on that without additional notes and details.

Here is the rough cost for your estimate, window only, excluding demolition and installation.

vinyl double hung window

estimated size: 36" X 65"

7/8" insulated glass unit.

$310 per unit

Hopefully it helps you to get the rough cost for replacement.

The op said small window to go above the bookshelf. How is a 36" x 65" double hung considered small and able to fit above the book shelf? And the op did not ask about replacement, but about a new window.