is there a special way to get this back panel back in without breaking it,...  seems hard. is there a trick?????

No tricks, Looks like the hatch strut and bracket is in the way.

Remove the hatch strut from the bracket, prop the  hatch open. The bracket is welded in place, leave it alone. Get the foot of the panel in place and very gently bend the panel to get it past the bracket.

In addition to what @jfa.series1mentioned, I have found that it helps to have the panel around the quarter window out before installing the rear, due to the overlap of the two panels. In your picture I can't tell if the quarter window panel is installed.

Panel around small window is not out.  I figured that would be another pain to get out.  These are a pain

No question that you have to be careful when flexing these old panels. It can be done if you are careful without removing the quarter window panel, but it does help. On mine it was necessary because of the change from vinyl to carpet on the shock towers. Good luck with what ever method you use. At least the weather is warm, which helps with flexing the panels as compared to winter temps.

They put carpet in mine to. Makes it way higher.  Should I take carpet off,

In my case the carpet was glued down, so it was better to remove the quarter window panel. Or at least the fasteners and seat belt shoulder strap as shown in blue here, that method also worked for me as long as I was careful.

And then what. Just pry up window part a bit and slide big piece in behind ?? Not fully remove window piece?

That's the the idea, carefully flex the window piece away to get a gap on the rear vertical edge to slide the rear panel in behind. Like I said before my preferred method is with the window panel out.