These are left over from concrete ties which are used to hold the formwork together ready for pouring concrete. They are usually called tie holes.
After the concrete is cured and forms need to be removed their ends protruding out are bent and snapped.