You can. It isn't wrong. But you may lose work if you do.
I know models who do charge deposits, because people will pay them. I also know models that state if a shoot is canceled within a certain amount of time they will bill for half the cost of the shoot anyway.
I personally wouldn't though.