I'd worry about velvet showing the former stitches, but consult with your tailor; s/he will be far more expert than I.