Queen Camilla's son reveals he hasn't joined the royals for Christmas at Sandringham

While it was reported last year that Queen Camilla had invited her children and grandchildren to celebrate Christmas at Sandringham for the first time, Her Majesty's son Tom Parker Bowles has revealed to PEOPLE that he's yet to join the royals for their traditional holiday gathering.

“We are married in, we are not the royal family – I’ve never for one second said that,” he told PEOPLE. “I find everyone incredibly nice and friendly and we’ve know them all for a long time. It does seem natural.”

“That’s not to say myself and the children spend running around palaces,” Tom added. “I have my own job and the children have school. I haven’t been to Christmas yet for various reasons.”

Tom and his sister Laura Lopes are the Queen's children from her first marriage to Andrew Parker Bowles. “My mother and father are still great friends and we will go down on December 27 and my mother will sometimes come,” Tom shared. “It is something to work towards. It’s a maturity that is a great thing if — despite a marriage sometimes not working –you can keep it together for the sake of the children and the sake of each other then it’s a very good thing.”

Tom's mother has been married to King Charles since April 2005. The Cooking and the Crown: Royal Recipes from Queen Victoria to King Charles III author revealed to PEOPLE that his kids, Lola and Freddy, adore their grandmother and the King, whom them call “Gaga and Uppa." "He has been a wonderful step-grandfather," Tom said of His Majesty.

“We blend in and blend,” the King's stepson continued. “Of course there’s the boys [William and Harry], and talking of the wider family, Peter Phillips is a little bit younger than me and I’ve grown up with him, and Zara [Tindall]. The York girls [Princess Beatrice and Princess Eugenie] are two of the nicest women you’d hope to meet.”