A Small Town in Germany occurs in the late 1960s, in Bonn, the capital of West Germany. From London, Alan Turner, of the British Foreign Office, arrives to investigate the disappearance of Leo Harting, a minor British Embassy officer; moreover, secret files have disappeared with him. The embassy's security chief, Rawley Bradfield, is hostile to Turner's investigation. Despite that, he is dinner party host to Turner and Ludwig Siebkron, head of the German Interior Ministry; the latter is close to industrialist Klaus Karfeld, who is successfully building his new political party.
Initially, Turner suspects Leo Harting is a spy, but comes to grasp that Harting was secretly investigating Karfeld's Nazi career — as the war-time administrator of a laboratory that poisoned thirty-one half-Jews. In fact, Harting is hiding from Siebkron, and might assassinate Karfeld. To Turner's chagrin, Bradfield is unsympathetic to Harting's circumstance and uninterested in protecting him, because he considers him a criminal and a political embarrassment.