For one thing, a hyperbola has two "asymptotes", lines that the hyperbola approaches as you go further and further out the hyperbola. A parabola does not. That means that, as you go further out the hyperbola, the curvature goes to 0. That is not true for a parabola.
Also the hyperbola and parabola (as well as the circle and ellipse) are "conic sections". If you take a true cone (so it has two "napps" that join at there vertices) and slice through it parallel to one side (so your slice crosses only one nappe) the edge is a parabola. If you slice non-parallel (so your slice crosses both nappes) the edges are a hyperbola.
The "eccentricty" of any parabola is exactly 1. A hyperbola can have any eccentricty greater than 1.