The company had a loyal specialty store following for its luxurious cashmere sweaters.