Gehlen became a professional soldier in 1920 during the Weimar Republic. In 1942, he became chief of FHO, the German Army's military intelligence unit on the Eastern Front (1941–45). He achieved the rank of major general before being sacked by Adolf Hitler because of the FHO’s pessimistically accurate intelligence reports about Red Army superiority. In late 1945, at the start of the Cold War, the U.S. military (G-2 Intelligence) recruited him to establish an espionage network against the Soviet Union, the Gehlen Organisation, which employed former military officers of the Wehrmacht and former Nazis from the Schutzstaffel (SS) and the Sicherheitsdienst (SD).
Gehlen was the first president of the Federal Intelligence Service of West Germany from 1956 to 1968. While this was a civilian office, he was also a lieutenant-general in the Reserve forces of the Bundeswehr, the highest-ranking reserve-officer in the military of West Germany.
Gehlen was born to a Roman Catholic family; his father was a book-seller in Erfurt. In 1920, he joined the Reichswehr; he then attended the German Staff College, and was graduated in 1935, after which he was promoted to captain and assigned to the German General Staff.
Gehlen served on the General Staff until 1936, and was promoted to major in 1939; at the time of the German attack on Poland (1 September 1939) he was a staff officer in an infantry division. In 1940, he became liaison officer to Field Marshal Walther von Brauchitsch, Army Commander-in-Chief; and later was transferred to the staff of General Franz Halder, the Chief of the German General Staff. In July 1941, he received a promotion to lieutenant-colonel and was sent to the Eastern Front, where he was assigned as senior intelligence-officer to the Fremde Heere Ost (FHO) section of the Staff.
In spring of 1942, Gehlen assumed command of the FHO from Colonel Eberhard Kinzel. Before the Wehrmacht disasters in the Battle of Stalingrad (23 August 1942 – 2 February 1943), a year into the German war against Soviet Russia, Gehlen understood that the FHO required fundamental re-organisation, and secured a staff of army linguists and geographers, anthropologists, lawyers, and junior military officers who would improve the FHO as a military-intelligence organisation despite the Nazi ideology of Slavic inferiority.