“It was as awful as you can imagine,” the footwear mogul shared in a new interview.