I have tried to teach myself both Hamiltonian and Lagrangian classical mechanics and there’s one mental hurdle I have not been able to get over. The problems are generally set up with starting position and momentum known, and ending position and momentum known, and then the math tells you the path taken along the way. But what if the ending position and momentum is unknown? How does one use these formulations of mechanics to predict the future and not just postdict the past? Is this just how beginner problems tend to be set up?
Hm, well typically in Lagrangian formulations, you're right that you define a functional called an "action", and this involves some integral from an initial to a final position in configuration space. Then the idea is that the system will evolve in such a way as to minimize this action. A stationary point (like a maximum or minimum) has gradient 0. So a condition for this "principle of least action" to be realized is that the "gradient" (called the variation here) of the action is 0 for the real path. That is, if you take a path the system carves out through the configuration space and perturb that path, and you compute the total effect on the action from that small perturbation, you find it should be 0.
You do this by actually taking this derivative and you find that you can guarantee that the differential of the action is 0 if the system takes a path which is the solution to a set of differential equations, and you can generally find the solution to those differential equations only with information about the origin, ie you don't need both the start and end conditions to find a unique solution.
So you're right, it's a bit weird conceptually. You sort of start saying "the system obeys a path that minimizes the action between it's initial and final positions" and then find that this produces a set of conditions which form a system of diff eqs that you can find general solutions for and select out a unique solution just with the initial conditions, no need for the final condition.
The answer is, unfortunately, boring. You solve equations to find the unknowns. If there are too many unknowns there are too many solutions and the whole thing is useless.
