The Borderers looked as though they were down and out until substitute Ian Little struck in the third minute of injury-time.
He was on hand to prod the ball home when Dean Keenan cleared off the line from Mark Cowan, who was first to meet a lofted cross from Andy Seaton.
Berwick took the lead after 11 minutes when Pat Clarke picked up a mis-hit shot from Mark McGarty. He tried to lob Craig Coyle, who managed to get a touch to the ball, but then rounded the keeper to net.
A Seaton corner was deflected against an upright before Gordon Connelly shaved a post.
But Morton hit back after 35 minutes courtesy of Marco Maisano, when Gary O'Connor could only get a hand to his hard drive and the ball crossed the line via an upright.
As the Greenock side started to stamp their authority on the game Chris Millar sent a flashing shot across the goal face but there was no one to get a touch.
He got his reward, however, after 77 minutes with a blistering shot from the edge of the box which gave O'Connor no chance.
Then, in a pulsating finale, Little saved the day for Rangers.