Chasing 181, England were 139-2 with 31 balls remaining and seemingly cruising towards victory before they lost 5-10 in 18 balls.
The fourth game of the five-match series is in Napier at 05:00 GMT on Friday.
With England looking set for victory, captain Eoin Morgan was caught at deep mid-wicket off the final ball of the 14th over.
Sam Billings was then run out second ball by a brilliant piece of fielding from Colin Munro before James Vince picked out mid-off to fall for 49.
England in New Zealand – fixtures, results & squads
England slump after top order impresses
England’s chase started perfectly as Tom Banton, on debut, and Dawid Malan took 25 off the first two overs.
Banton played a glorious off-drive before lofting Ferguson over mid-wicket for a mighty six. The 20-year-old Somerset opener was soon bowled, attempting to ramp, but his 10-ball 18 was a glimpse of his undeniable talent.
Malan batted beautifully for a 34-ball 55. He was strong all round the wicket, with a number of superb cover drives, before he clothed a filthy full toss from Ish Sodhi to Martin Guptill at deep mid-wicket.
Malan’s dismissal gave Vince impetus as he was struggling at less than a run a ball before three fours and a six in four balls kick-started his innings and England’s chase.
They looked to be cruising towards their target after partnerships of 63 and 49 respectively between Malan and Vince, and then Vince and Morgan.
However, the collapse that followed ended England’s hopes of taking a series lead.
Guptill took a liking to Saqib Mahmood, who struggled with his line on his second appearance, in a blistering 17-ball 33 before Tom Curran took a fine catch over his shoulder at deep extra cover in the swirling wind.
The imposing all-rounder did struggle against the leg-spin of Parkinson, despite hitting one glorious inside-out cover drive for six.
But Ross Taylor hit 27 off 24 balls, Jimmy Neesham 20 off 15 and Mitchell Santner 15 off nine as New Zealand posted a target beyond England for the second successive game.
‘That was a capitulation’ – what they said
England captain Eoin Morgan: “That’s one that got away from us. We were in control in the whole chase before we went three or four down.
“The lack of experience may have cost us. The game plan remains the same – everything we do is about positive, smart, aggressive cricket. It’s a must-win game in Napier.”
New Zealand captain Tim Southee: “As long as Morgan was in they had a sniff, especially with James Vince in and playing well, but we knew if we kept taking wickets it would get tough.
“There are still areas we need to improve on but it’s nice to win games like that, that can go either way at the halfway point and during the second innings.”
England bowler Steven Finn on BBC Test Match Special: “Those five wickets in three overs ripped the heart out of England’s reply.
“The outfielding was good, the bowling performance was good. Up until the point where they lost those wickets, they were in control. But it was a bit of a capitulation at the end.”
BBC cricket correspondent Jonathan Agnew: “England were cruising at 139-2 with Vince and Morgan going well. England will say, we were well set up to win that game and we shouldn’t let it slip away like that.”