He was booked on suspicion of trespassing on a protected site and possessing an offensive weapon and was taken to a London police station.
According to officials at Buckingham Palace, the Queen was not at home at the time.
The police said in a statement that the knife was found after the man was searched, adding that no-one was injured during the incident.
The incident comes in the wake of several recent security breaches at the Palace.
Read more at http://www.bbc.co.uk/news/uk-24524832.