No, you definitely need a header as do all doors.

If the walls are open and the framing exposed, you can insert the 1x flush with the studs, drywall, and install the door. Doors do not compress drywall.

I've done one the way you want. Install blocking in the wall and then drywall.

Once the support bracket locations are determined, cut the drywall away and replace with something solid -- small piece of wood, metal etc that will be hidden by the bracket.

There may also be a problem with spacing. When mounting directly to the drywall, the hanging rail may not be far enough from the wall to allow the door to clear any trim.