Like the Muir Inn across the street, Lavigne House catered to the passengers arriving and departing by steamboat at Graham's Wharf. With more modest accomodation and prices, it was used by lumbermen, labourers on the boats, construction workers and others with more modest means. At one point in its career, it also served as a cheese factory.