Vladimir Lipski was born the middle child of Lech and Desiree Lipski, in Brooklyn, New York City. When he was four his family moved to Bound Brook, New Jersey, where his father was in the building and lumber industry. When Vladimir was in his early teens, his mother died, and his father passed away shortly thereafter.
After high school graduation he enrolled at Lehigh University, where he majored in economics and was elected into Phi Beta Kappa, an honorary scholastic society. Vladimir’s interest in jazz resulted from a food poisoning incident in 1936. The day before the end of his sophomore year, Lipski’s fraternity house served undercooked pork, which caused him and several of his classmates to come down with trichinosis. While bedridden over the summer, Vladimir was visited frequently by his high school buddy Ben Martle, a classical pianist and an ardent jazz fan. Martle collected international music magazines in which he read that jazz was America's greatest contribution to the arts. He began to share his interest with Lipski, often bringing along Louis Armstrong and Duke Ellington records when he visited. Lipski was quickly transformed from a Guy Lombardo fan into a jazz buff.