Here's a video from AimPoint Golf showing an example of uphill putts having an extremely consistent roll using the 'Perfect Putter.'
We have to remember a few things here:
Optimal speed is roughly 2-3 revolutions per second as it reaches the hole. This is often referred to as 'capture speed.'
As the ball starts to slow down it is more likely to wobble, much like a front bicycle wheel would wobble if a person is struggling to get the bike up a hill.
With uphill putts, the speed of the ball has to be faster as it collides to with the putter and rolls on the green before it reaches its capture speed.
So...the faster the ball speed, the surface imperfections have less influence on the ball.
Conversely, if we are to roll a downhill putt...the optimal capture speed is still 2-3 revolutions per second. But since it is going downhill, the initial speed of the putt will be slower (than it would be for the speed on an uphill putt). And thus the surface imperfections come into play more.
This is a good reason why it's very difficult to putt on aerified greens. If you're left with a downhill putt, it's almost pure guesswork because the slower ball speed will react to imperfections more. Thus, if you catch yourself on aerified or bumpy greens, it's an imperative to try and keep the ball below the cup.