The controller keeps its time very well between NTP updates, the drift is only several seconds per day.
With older controller software version, the phones updated themselves constantly but only showed the last full minute that had passed, so they were up to 59 seconds behind.
Since v8, I've noticed the handsets keep accurate time, so they must be updating themselves to the second now.