A study found that those who got less than five hours a night found it difficult to function effectively during the day - with people forgetting to carry out tasks, struggling to remember where things were, and forgetting to do something they had set out to do such as post a letter or take medication.
Ensuring people get adequate sleep should be a public health priority in the way that encouraging exercise and controlling weight gain are, they argue.
The study, by academic psychologists Dr Anna Weighall and Dr Ian Kellar from the University of Leeds, looked at data from a survey of the sleeping habits of more than 1,000 UK adults aged 18 to 80, conducted in collaboration with UK bed manufacturer, Silentnight.
Dr Weighall, revealing the findings at a meeting of the European Society of Cognitive Psychology in Postdam, Germany, said: "A lot of previous sleep research has been based on lab studies - this is the first time we have surveyed people in their everyday lives.
"What is emerging is the debilitating impact of poor patterns of sleep. People who are not getting enough sleep are at risk of experiencing a much lower quality of life and it hinders their ability to function effectively when they are awake."
Scientists have recognized that sleep is important for laying down new memories - and in re-processing what is already "stored" in the brain, selecting what needs to be retained and what can be forgotten.
This study looked at the relationship between quality and quantity of sleep and the cognitive processes around memory and recall, as well wider indicators of physical and mental wel-being.
It involved asking volunteers to fill out a questionnaire about their sleep patterns, memory performance, mental wellbeing and quality of life.
An analysis of the responses found a statistically significant relationship between poor sleep and reduced mental wellbeing, and a highly significant relationship between lack of sleep and an increase is everyday memory problems.
These relationships were even stronger in those who habitually sleep for less than five hours a night.
Dr Weighall said: "There is now a very compelling case to say there is a strong relationship between getting a good night's sleep and experiencing better health, wellbeing and memory function."
The findings indicate that many UK adults are sleep deprived and that this presents a real issue for public health.
Dr Kellar, a health psychologist, said sleep needed to be seen as a public health priority in just the same way as maintaining a healthy weight, eating a healthy diet, and engaging in physical activity.
The NHS recommendation is that adults should aim to get between seven and eight hours of sleep a night.
Silentnight estimates that one in four adults gets less than five hours sleep a night.