In Middle English, "pygg" referred to a type of clay used for making various household objects such as jars. People often saved money in kitchen pots and jars made of pygg, called "pygg jars". By the 18th century, the spelling of "pygg" had changed and the term "pygg jar" had evolved to "pig bank." Once the meaning had transferred from the substance to the shape, piggy banks began to be made from other substances, including glass, plaster, and plastic.
An alternative theory, and the one usually given for the origin in German sources, is that in Germany and surrounding countries the pig is a symbol of good luck or good fortune hence keeping money in a piggy bank will bring good fortune.
...and you have to break them, to get your savings out!