Zhitnik was drafted 81st overall by the Los Angeles Kings in the 1991 NHL Entry Draft after playing for Sokil Kyiv in the Soviet Championship League for two years. After another year, he came to the NHL for the 1992–93 NHL season.
He made his NHL debut on October 6, 1992 against the Calgary Flames and scored his first goal nine nights later, also against Calgary. In his rookie season with the Kings, he finished second among rookie defenceman in points with 48. He was an important part of the Kings' playoff run to the Stanley Cup Final, where they lost to the Montreal Canadiens.
After another year of playing with the Kings and Wayne Gretzky, he was traded on February 14, 1995, along with Robb Stauber, Charlie Huddy and a fifth-round draft pick, to the Buffalo Sabres for Grant Fuhr, Denis Tsygurov and Philippe Boucher. He became one of the team's best players, helping the Sabres win the Northeast Division in the 1996–97 NHL season, his second full year with the team. In the 1997–98 NHL season, he led all defensemen in shorthanded goals (3). He also helped the Sabres reach the finals in the 1998–99 NHL season and stayed on the team until the 2004–05 NHL lockout when he returned to the Russian Super League.
Following the lockout, he signed a four-year contract with the New York Islanders. He became an effective force on the team, and even though he missed the last 18 games of the season with a fractured ankle, he finished second among team defenceman in scoring.
On December 16, 2006, Zhitnik was traded to the Philadelphia Flyers for Freddy Meyer and a conditional third round draft pick. However, he was traded two months later on February 24, 2007 to the Atlanta Thrashers to add experience for playoffs, in exchange for Braydon Coburn.