Dynamical processes occurring on top of complex networks have become an exciting area of research. Quenched disorder plays a relevant role in general dynamical processes and phase transitions, but the effect of topological quenched disorder on the dynamics of complex networks has not been systematically studied so far. Here, we provide heuristic and numerical analyses of the contact process defined on some complex networks with topological disorder. We report on Griffiths phases and other rare region effects, leading rather generically to anomalously slow relaxation in generalized small-world networks. In particular, it is illustrated that Griffiths phases can emerge as the consequence of pure topological heterogeneity if the topological dimension of the network is finite.