Identifying desired interactions with a target receptor is often the first step when designing new active compounds. However, attention should also be focused on contacts with other proteins that result in either selective or polypharmacological compounds. Here, the search for the structural determinants of selectivity between selected serotonin receptor subtypes was carried out. Special attention was focused on 5-HT7R and the cross-interactions between its ligands and the 5-HT1AR, 5-HT1BR, 5-HT2AR, 5-HT2BR, and 5-HT6R subtypes. Selective and non-selective compounds for each pair of 5-HT7/5-HTx receptors were docked to the respective 5-HTR homology models and 5-HT1B/5-HT2BR crystal structures. The contacts present in the ligand-receptor complexes obtained by docking were characterized by the structural interaction fingerprint and statistically analyzed in terms of their frequency. The results allowed for the identification of amino acids that discriminate between selective and non-selective compounds for each 5-HT7/5-HTx receptor pair, which was further compared with available mutagenesis data. Interaction pattern characteristics for compounds with particular activity profiles can constitute the basis for the coherent selectivity theory within a considered set of proteins, supporting the ongoing development of new ligands targeting these receptors. The in silico results were used to analyze prospective virtual screening results towards the 5-HT7 receptor in which compounds of different chemotypes to known 5-HT7R ligands, with micromolar level activities were identified. The findings in this study not only confirm the legitimacy of the approach but also constitute a great starting point for further studies on 5-HT7R ligands selectivity.