Numerical simulation of physical phenomena has long been of interest in the area of mechanical engineering, physics, and in recent years, computer graphics. Recent advances of computation hardware have opened up new opportunities for improving numerical simulations in terms of both scale and performance. However, the heterogeneity of modern hardware has imposed unique challenges that limit the utilization of computation hardware using the traditional programming paradigm. This dissertation investigates the design of efficient Poisson and linear elasticity solvers for modern hardware to demonstrate design practices and principles for utilizing modern hardware in the context of numerical solvers.