This talk shows how forms of gauge theory, Hamiltonian mechanics, quantum mechanics and genreal relativity arise from a non-commutative framework for calculus and differential geometry. Discrete calculus is seen to fit into this pattern by a reformulation in terms of commutators. Differential geometry begins here, not with the concept of parallel translation, but with the concept of a physical trajectory and algebra related to the Jacobi identity that governs that trajectory. We discuss how Jacobi identity controls the Levi-Connection in this context. We generalize the Feynman-Dyson derivation of electormagnetism in a non-commutative context, and we show how natural constraints on non-commutative derivations give rise to fourth-order, generalizations of Einstein's equations for general relativity (joint work with Anthony Deakin and Clive Kilmister).