Rhubarb, in the form of a traditional Chinese medicine, is used in the treatment of chronic kidney disease (CKD). Previous studies have demonstrated that Rhubarb possesses a good nephroprotective effect, which primarily protects the kidneys from fibrosis, oxidation, inflammation, autophagy, and apoptosis. However, studies have shown that the long-term inappropriate use of Rhubarb may cause damage to renal function. Therefore, how to correctly understand and scientifically evaluate the pharmacodynamics and toxicity of Rhubarb with regard to CKD is a scientific question that urgently needs to be answered. In this review, we explain and illustrate how Rhubarb exerts its nephroprotective effect against CKD. We also describe the mechanisms of action that may cause its nephrotoxicity. Valuable and practical clinical guidance is proposed with regard to methods for mitigating the nephrotoxicity of Rhubarb.