Glucocorticoids (GCs) remain regularly used drugs in patients with chronic inflammatory rheumatic diseases. As long-term intake at high dosages is associated with harm, it is generally advised that GCs be tapered and stopped. However, most recommendations concerning tapering have been eminence- or consensus-based. In this narrative review, we present novel data from recent studies (SEMIRA, CORTICOLUP, and GiACTA) shedding light from different angles on the effects of tapering GCs in patients with rheumatoid arthritis (RA), systemic lupus erythematosus (SLE), and giant cell arteritis (GCA). In RA and SLE, our main findings comprise that (a) the majority of RA and SLE patients can successfully taper their GC, but that (b) tapering increases the risk of flare. In GCA, tocilizumab was shown to be a potent GC-sparing agent. Finally, we also present exemplary tapering schemes for RA, SLE, and GCA, although different tapering regimens have not yet been sufficiently compared in randomized trials.