Anticoagulation in patients with atrial fibrillation (AF) should be guided by considerations of the risk of thromboembolism, stroke, and bleeding as well as the patient’s preference. Well-recognized scores have been developed to help the clinician in daily risk assessment, but there are several special patient populations for whom scores are not developed or validated. Furthermore, these patients were not adequately represented in the pivotal randomized trials for non-vitamin K antagonist oral anticoagulants (NOACs). In patients with cancer, the intrinsic hypercoagulable state has to be balanced against an increased risk of bleeding, and a dynamic concept should be applied, taking into account the cancer type, current disease state, therapeutic strategy, and patient-related factors, with NOACs playing an increasingly larger role. In women with planned pregnancy or already pregnant, NOACs should be avoided. However, accidental exposure during pregnancy should not lead to recommendations for pregnancy termination in view of current observational data. Whether patients on dialysis with AF benefit from anticoagulation at all is questionable. But if the decision for anticoagulation is made, NOACs may contribute to a more favorable risk–benefit profile than vitamin- K antagonists. Finally, patients on the ward deserve special considerations regarding periprocedural management of anticoagulation. Although for the majority of procedures a short discontinuation of oral anticoagulation seems appropriate, there are some low-bleeding-risk procedures that do not require cessation. The aim of the present review is to discuss the major particularities of these four patient subgroups and thus to facilitate the clinical decision-making.