Despite the security and amenities available to them in Canada, the transition to a new place, lifestyle, and culture was difficult for Esther and Ann. Initially, they spoke only Yiddish and felt like strangers in their new home, where the people they met referred to them derogatorily as "greenhorns." Years later, Saul would describe London as a wealthy city with a predominantly white, Christian population, where the Holiffs were the only Jewish family in their neighbourhood, and "anti-Semitism was palpable." He and his brother were verbally harassed as they delivered newspapers after school, and his mother once had a bag of flour thrown in her face on Halloween as she was called "a dirty Jew." (Listen to Saul Holiff discuss this in the audio clip below.)

When the Depression hit, the Holiffs also faced economic challenges. Esther and Joel owned a milliner's shop on Dundas Street that, near bankruptcy, they had to shut down in 1933. To help their family, Morris and Saul (who dropped out of school) purchased a produce business and began delivering fruits and vegetables around the city.

It was never a profitable venture, however; they had no refrigeration for leftover product, so much of it went to waste. Their father took over the business and delivery truck in 1942--which he would eventually transform into a clothing company called Holiff's Personalized Shopping Service--so the brothers could pursue other career paths. Morris enlisted in the Royal Canadian Air Force, while Saul worked a variety of jobs (including menswear salesman, steel puddler at the Steel Company of Canada, and fish delivery driver) before following suit and enlisting in the RCAF in 1943. Saul was posted to New Brunswick, where he became a tail gunner on Lancaster bombers. He was honourably discharged in 1945 without ever having seen combat.