Four in 10 adults (41%) did not know the State Pension is taxable, according to analysis from Royal London.
Seven in 10 retired people (68%) paid tax on their pension income last year.
One in four did not pay tax and 7% did not know.
Most over State Pension age who paid tax (88%) expected to do so, but 12% did not.
The average tax paid was over £4,500, while 66% did not know how much tax they paid or could not remember.
Over one in four who paid tax, 27%, paid more than £5,000.
Around half of adults under State Pension age had never checked their State Pension forecast.
Three quarters (73%) had never found gaps in their National Insurance record.
Sarah Pennells, consumer finance specialist at Royal London, said: “The fact that approximately 4 in 10 adults do not know the State Pension is taxable is not surprising as it’s paid without tax being taken off.
“However, from April, the full new State Pension will be less than £30 below the personal allowance, so it’s more important than ever that people understand what tax they may have to pay.
“Our research shows that almost 7 in 10 – 68% – of those who are retired and not working paid tax on their pension income, with the average amount of tax paid standing at over £4,500.”
Pennells added: “However, two thirds of those who’d paid tax didn’t know how much they’d paid or couldn’t remember.
“Some pensioners who built up a larger State Pension under the old system, thanks to the State Earnings Related Pension Scheme (SERPS), will already be paying tax even if they have no other income in retirement.
“If your total income in retirement, including any workplace or private pensions, is more than the personal allowance, you will be taxed automatically or sent a tax bill.”
She said: “To avoid an unexpected bill, work out how much your income is going to be. If you have a defined benefit pension, your pension scheme should tell you each year how much your payments are going to be.
“If you are taking income from a personal or workplace pension, you could vary the amounts you take to reduce the tax you pay or even avoid paying tax altogether.
“There’s a useful tool on Gov.uk which will tell you whether you’re likely to pay tax.”