This part of the machine is found near a garden, and Wickerbottom claims it's neither fully organic nor inorganic, suggesting it may have been grown there (or someone tried to grow it). Wilson mentions that the metal potato contains great and fearful power.
The Metal potato thing is actually a robotic Maxwell's head. Calling it a potato may just be making fun of the game's antagonist, though being near a farm make the potato similarities more obvious.
The Metal potato thing is the reason the Wooden Thing, when fully constructed, is more commonly referred to as the 'Teleportato'.
This may be a reference to the game Portal 2, in which the main antagonist Wheatley puts GLaDOS into a potato battery. This is reinforced by WX-78's quote, seen above.