The former All Blacks forward was thought to be among the front-runners to replace Steve Hansen, who departed after their Rugby World Cup defence ended in semi-final defeat to England.
However, Joseph joins the likes of Warren Gatland, Joe Schmidt and John Mitchell in opting not to pursue arguably the most coveted job in world rugby.
Joseph has instead extended his contract with Japan through to December 2023, having guided the Cherry Blossoms to an unprecedented World Cup quarter-final on home soil.
"I have great expectations for rugby in Japan and I'm very honoured that I can lead the team towards the next World Cup," Joseph said via a statement on the Japan Rugby Football Union's official website.
"We achieved the goal of being among the top eight countries in the World Cup, but there are still more issues to be tackled.
"To that end, I chose the path to challenge with the Japanese national team again. I want to strengthen the team."
Crusaders coach Scott Robertson, Ian Foster - who served as assistant to Hansen - and Glasgow Warriors boss Dave Rennie are thought to be on the shortlist to replace Hansen, who led New Zealand to World Cup glory in 2011 and 2015.