Which 4-manifold invariants can detect the Gluck twist? And, which 3-manifold invariants can detect the difference between surgeries on mutant knots? What is the most powerful topological quantum field theory (TQFT)? Guided by questions like these, we will look for new invariants of 3-manifolds and smooth 4-manifolds. Traditionally, a construction of many such invariants and TQFTs involves a choice of certain algebraic structure, so that one can talk about "invariants for SU(2)" or a "TQFT defined by a given Frobenius algebra." Surprisingly, recent developments lead to an opposite phenomenon, where algebraic structures are labeled by 3-manifolds and 4-manifolds, so that one can speak of VOA-valued invariants of 4-manifolds or MTC-valued invariants of 3-manifolds. Explaining these intriguing connections between topology and algebra will be the main goal of this talk.