Once a week to keep them clean and hygienic.  Use either a specific brush cleaner or baby shampoo, rinse thoroughly and leave them flat to dry completely.