The battery in the handset is probably just dying... These rechargeable batteries only have a real-world life of about 2-3 years, so that could be the issue.
There could also be some connectivity issues with the contacts, try cleaning the contacts with some 90%+ pure isopropyl alcohol or even rubbing the contacts with some printer paper or very fine emery cloth.
I have not trying ripping these open to replace the batteries, but that may be the next step as they are discontinued by Mitel.