Kramer pitched 16 seasons from 1936 to 1959, twelve in the major leagues and six in the minors. He entered the majors in 1939 with the St. Louis Browns, playing for them three years before joining the U.S. Navy Seabees during World War II. Following his service discharge, he rejoined the Browns in the 1943 midseason and later was demoted to the Toledo Mud Hens of the American Association. On September 11, he pitched a 5–0 no-hitter against the Louisville Colonels. He struggled with his control in his first four years (201 walks in 345.0 IP), but received a fifth chance in part to the World War II player shortage. He responded with a heroic effort that culminated in the Browns only World Series appearance.
In 1944, Kramer led the Browns to their 8th straight Opening Day victory‚ beating Dizzy Trout and Detroit‚ 2–1, at Tiger Stadium. A week later, he extended the Browns season-opening winning streak to six games defeating the White Sox, 5–2, aiding his cause with a two-run home run. In his next start, he pitches the Browns to their American League record 9th straight win to start the season with a 3–1 victory over the Indians, which gave his team a solid 31⁄2 game lead in the AL standings. Kramer finished with a 17–13 record and a 2.49 ERA, including a brilliant one-hitter shutout over the White Sox in September that gave St. Louis a half game lead in front of the Yankees. The Browns finished with an 89–65 record and faced the Cardinals in the historic All-St.Louis World Series. Kramer added a complete game victory in Game Three, allowing just two unearned runs on seven hits and 10 strikeouts. This would be the last time the hapless Browns won a post-season game.
From 1945 to 1947, Kramer averaged 11 wins per season and made the American League All-Star team three times. In the 1946 Game he tossed three no-hit, shutout innings to earn a save. In addition, he went one-for-one and scored a run. In November 1947, he was sent along Vern Stephens to the Boston Red Sox in exchange for Pete Layden, Joe Ostrowski, Roy Partee, Eddie Pellagrini, Al Widmar, Jim Wilson and cash.
In 1948, Kramer went 18–5 for Boston, including a 12-game winning streak from June 3 to August 10 and five victories over the Yankees. His .783 won-loss percentage led the American League pitchers. He faded to 6–8 in 1949 and was sold the New York Giants for $25‚000. Eventually, Kramer will charge the Red Sox with railroading him out of the league because of his differences with manager Joe McCarthy. He went 1–3 for the Giants (1950–51) and 1–3 with the Yankees (1951). Although he pitched with both pennant-winners in 1951, he did not last to the World Series with either.
In a 12-season career, Kramer posted a 95–103 record with 613 strikeouts and a 4.24 ERA in 322 appearances, including 215 starts, 88 complete games, 14 shutouts, seven saves and 16371⁄3 innings of work. He also helped himself with the bat, hitting .144 (72-for-501) with five home runs and 39 RBI.
Kramer died in Metairie, Louisiana at age 77.