Top seed Murray, fancied to end his run of near misses in Melbourne following the early exit of main rival Novak Djokovic, lost 7-5, 5-7, 6-2, 6-4 to world No50 Zverev.
The defeat – his biggest upset at a grand slam since 2006 – came as fellow Briton Dan Evans also crashed out against French 12th seed Jo-Wilfried Tsonga.
“I have had tough losses in my career in the past. I’ve come back from them. This is a tough one. I’m sure I’ll come back okay from it. But right now I’m obviously very down,” said Murray.
“Credit to him. He came up with great shots and played a really good match. You always finish matches you lose with things you maybe could have done a bit better, but he played some really good stuff.”
Zverev – the lowest ranked opponent to beat Murray at a grand slam for more than 10 years – hampered the Scot’s rhythm with an effective serve-and-volley approach.
He broke Murray’s serve five times in the first two sets and took control of the match with a sequence of five straight games that gave him the third set and a break in the fourth.
Zverev faces Roger Federer in the quarter-finals after the rejuvenated 17-time grand slam winner prevailed in a five-set marathon against fifth seed Kei Nishikori.
“Honestly, I was in a little coma, just serve and volleying my way through. There were some points I don’t know how I won, but somehow I made it,” said Zverev.
“Everything is new to me, everything feels unreal. Playing Roger would be a dream to me. He inspires me and he was my favourite player growing up.”
Evans fairytale ends
World No51 Evans, meanwhile, reached the end of his best run at a grand slam, but not before going toe to toe with Tsonga.
The Englishman took the first set before the powerful Tsonga made his athleticism count 6-7 (4-7), 6-2, 6-4, 6-4 to reach a quarter-final with fourth seed Stan Wawrinka.
“He was just a bit too strong for me. To win the first set took too much out of me,” said Evans, who is set to rise to a career-high ranking of 45.
“I was flagging pretty much after the first set. I did feel that. My body was sore. Maybe that’s something I can improve on a bit. But I have still come a long way from where I was last year.”